Page 1 of 2
Threads - whats wrong?
Posted: Tue Feb 24, 2004 3:51 am
by Pantcho!!
i need to program a popup account checker for 100 accounts
if i do it in the oldschool it takes allot of time and its way slow...
so here comes the threads!
my main goal is the set totalnames(numaccounts..) and number of threads to run ... (i wont run 100 at once..)
and wait for each one to finish and run another...
basicly this code runs in 1 sec and disapper even when debugger is on
the debugger hide itself
the delay in the procedure simulate a connection, since a certin connection takes diffrent time in each connection...
been trying to manipulate ... nothing... here is my code
PLEASE help.
thank u...
Code: Select all
Totalnames.l = 25 ; total account
namechkpos.l = 0 ; position of the current account
numthreads.l = 5 ; number of threads to run
Dim threadmonitor.l(numthreads)
Dim threadID.l(numthreads)
Global namechkpos.l
Procedure LaunchCheck(clippos.l)
a = Random(5000)
Delay(3000 + a) ; the delay wont work
Debug "finished"
threadmonitor(clippos) = 0
EndProcedure
;####[START]
For clear = 1 To numthreads
threadmonitor(clear) = 0
threadID(clear) = 0
Next clear
finish = 0
Delay(1000)
Repeat
For x = 1 To numthreads ; running....
If threadmonitor(x) = 0 ; empty!
If threadID(x) > 0
; kill thread
; Debug "thread finish - " + Str(threadID(x))
KillThread(threadID(x))
EndIf
namechkpos = namechkpos + 1
threadID(x) = CreateThread(@LaunchCheck(),x)
If namechkpos = Totalnames ; this is the last thread since we got all account threads
finish = 1
Break
EndIf
Else
; do nothing thread is working...
EndIf
Next x
Until finish = 1
KillThread(threadID(x)) ; kill the last one... altough its not needed
End
Re: Threads - whats wrong?
Posted: Tue Feb 24, 2004 5:44 am
by NoahPhense
It appears that your threads are still running after the app is closed..
Also, I only caught 4 finishing..
I've added two Debug statements to help..
Code: Select all
Totalnames.l = 25 ; total account
namechkpos.l = 0 ; position of the current account
numthreads.l = 5 ; number of threads to run
Dim threadmonitor.l(numthreads)
Dim threadID.l(numthreads)
Global namechkpos.l
Procedure LaunchCheck(clippos.l)
a = Random(5000)
Delay(3000 + a) ; the delay wont work
Debug "finished"
threadmonitor(clippos) = 0
EndProcedure
;####[START]
For clear = 1 To numthreads
threadmonitor(clear) = 0
threadID(clear) = 0
Next clear
finish = 0
Delay(1000)
Debug "aaa"
Repeat
For x = 1 To numthreads ; running....
If threadmonitor(x) = 0 ; empty!
If threadID(x) > 0
; kill thread
; Debug "thread finish - " + Str(threadID(x))
KillThread(threadID(x))
EndIf
namechkpos = namechkpos + 1
threadID(x) = CreateThread(@LaunchCheck(),x)
If namechkpos = Totalnames ; this is the last thread since we got all account threads
finish = 1
Break
EndIf
Else
; do nothing thread is working...
EndIf
Next x
Until finish = 1
KillThread(threadID(x)) ; kill the last one... altough its not needed
Debug "bbb"
End
Posted: Tue Feb 24, 2004 10:42 am
by Iria
COMMENT: Also, I only caught 4 finishing..
Code: Select all
For x = 1 To numthreads ; running....
Change this to:
Remember arrays begin at 0 not 1
If you add a delay to the end of your program (just before the End) for say 10000 ms you will see all your processes finish. What you want to do it write a routine that at the end of your code that will scan the monitor array and wont exit the program until it sees all monitor entries are set to 0. This should delay the program long enough to run all your threads I think only quickly scanned the code tbh.
Threding in PB is a little scary atm, remember not to use strings in your threads as these are currently not thread safe in PB and will cause you a lot of pain
IRIA
Posted: Tue Feb 24, 2004 11:04 am
by DarkDragon
You are using a string
And you are creating the thread always
Try this for some ideas
Posted: Tue Feb 24, 2004 12:40 pm
by Iria
OK had a think about this and knocked this code up...Ive not checked for race conditions or non thread safe data corruption, although Im pretty sure my array of unique data slots should avoid the issue...Be nice to see if it works init
Code: Select all
#TotalTasks = 25 ; total number of tasks to action
#ThreadPoolSize = 5 ; Max size of the thread pool so 6 slots for us here
#INUSE = 1
#EMPTY = 0
Global TaskIndex.l
TaskIndex = 0 ; position of the last task executed
Structure ThreadInfo
ThreadInUse.b
ThreadID.l
ServerName.s
Number.l
ChangedName.s
EndStructure
Dim ThreadPool.ThreadInfo(#ThreadPoolSize) ; tracks usage of my thread pool slot and data
Procedure LaunchCheck(ThreadPoolSlotID.l)
; Thread that executes a task using ThreadPoolSlotID slot in the pool
a.l = Random(5000)
Delay(3000 + a) ; the delay wont work
ThreadPool(ThreadPoolSlotID)\ServerName = "ServerName = "+Str(ThreadPoolSlotID)
ThreadPool(ThreadPoolSlotID)\Number = a
ThreadPool(ThreadPoolSlotID)\ChangedName = "ChangedName = "+Str(ThreadPoolSlotID)
; Debug "Thread"+Str(ThreadPoolSlotID)+" just finished"
ThreadPool(ThreadPoolSlotID)\ThreadInUse = #EMPTY
EndProcedure
Procedure InitialiseThreadInfo(ThreadInfoID.l)
ThreadPool(ThreadInfoID)\ThreadInUse = #EMPTY ; 0 = empty not in use, non zero = in use
ThreadPool(ThreadInfoID)\ThreadID = 0 ; thread ID returned from starting thread (in case we need to kill it)
ThreadPool(ThreadInfoID)\ServerName = "" ; string to check our non thread safe data
ThreadPool(ThreadInfoID)\Number = 0 ; number to add spice to life
ThreadPool(ThreadInfoID)\ChangedName = "" ; another string in a different position!
EndProcedure
Procedure InitialiseThreadPool()
; Clear our thread pool
For ClearThreadPoolID = 0 To #ThreadPoolSize
InitialiseThreadInfo(ClearThreadPoolID)
Next ClearThreadPoolID
EndProcedure
; Main Program
exit = #False
;Delay(1000)
Debug "****-> Start of main program"
; start from task 0
TaskIndex = 0
Repeat
For ThreadPoolID = 0 To #ThreadPoolSize ; check every slot in the thread pool for an openeing
If ThreadPool(ThreadInfoID)\ThreadInUse = #EMPTY ; empty slot so we can Start a task using this slot
If TaskIndex = #TotalTasks ; We've completed all the scheduled tasks skip out
exit = #True
Break
Else
; for debuging purposed print out what the thread did to its data
; before we re-use it
Debug "========================================================================="
Debug "Thread Pool Slot: " + Str(ThreadPoolID)
Debug "Array " + Str(X) + " ThreadID: " + Str(ThreadPool(ThreadPoolID)\ThreadID)
Debug "Array " + Str(X) + " ServerName: " + ThreadPool(ThreadPoolID)\ServerName
Debug "Array " + Str(X) + " Number: " + Str(ThreadPool(ThreadPoolID)\Number)
Debug "Array " + Str(X) + " ChangedName: " + ThreadPool(ThreadPoolID)\ChangedName
; Clean out the junk from the last thread before we use the slot
InitialiseThreadInfo(ThreadPoolID)
; kick off the next thread task
ThreadPool(ThreadPoolID)\ThreadInUse = #INUSE ; reserve our slot and only the thread can unreserve its own slot
ThreadPool(ThreadPoolID)\ThreadID = CreateThread(@LaunchCheck(),ThreadPoolID)
; finally if the thread contains a non zero value then increment the number of tasks initiated
TaskIndex + 1
Debug "**=====> Task "+Str(TaskIndex)+" is now being processed"
EndIf
Else
; do nothing thread pool slot is busy
Delay(100)
EndIf
Next ThreadPoolID
Until exit = #True
Debug "****-> End of Main Program"
just noticed why your program dies suddenly
Posted: Tue Feb 24, 2004 1:30 pm
by Iria
Just noticed your program has an END statement at the end.
eek!
This means your program starts your threads, and immediately the parent thread is killed with the END statement. That kills the child threads.
Try this modified
version of my code and see what I mean, NOTE no END!
Code: Select all
#TotalTasks = 10 ; total number of tasks to action
#ThreadPoolSize = 10 ; Max size of the thread pool so 6 slots for us here
#INUSE = 1
#EMPTY = 0
Global TaskIndex.l
TaskIndex = 0 ; position of the last task executed
Structure ThreadInfo
ThreadInUse.b
ThreadID.l
ServerName.s
Number.l
ChangedName.s
TaskID.l
EndStructure
Dim ThreadPool.ThreadInfo(#ThreadPoolSize) ; tracks usage of my thread pool slot and data
Procedure DoTask(ThreadPoolSlotID.l)
; Thread that executes a task using ThreadPoolSlotID slot in the pool
a.l = Random(5000)
Delay(3000 + a) ; the delay wont work
ThreadPool(ThreadPoolSlotID)\ServerName = "ServerName = "+Str(ThreadPoolSlotID)
ThreadPool(ThreadPoolSlotID)\Number = a
ThreadPool(ThreadPoolSlotID)\ChangedName = "ChangedName = "+Str(ThreadPoolSlotID)
Debug "Thread "+Str(ThreadPoolSlotID)+" running task "+Str(ThreadPool(ThreadPoolSlotID)\TaskID)+" just finished"
ThreadPool(ThreadPoolSlotID)\ThreadInUse = #EMPTY
EndProcedure
Procedure InitialiseThreadInfo(ThreadInfoID.l)
ThreadPool(ThreadInfoID)\ThreadInUse = #EMPTY ; 0 = empty not in use, non zero = in use
ThreadPool(ThreadInfoID)\ThreadID = 0 ; thread ID returned from starting thread (in case we need to kill it)
ThreadPool(ThreadInfoID)\ServerName = "" ; string to check our non thread safe data
ThreadPool(ThreadInfoID)\Number = 0 ; number to add spice to life
ThreadPool(ThreadInfoID)\ChangedName = "" ; another string in a different position!
ThreadPool(ThreadInfoID)\TaskID = 0
EndProcedure
Procedure InitialiseThreadPool()
; Clear our thread pool
For ClearThreadPoolID = 0 To #ThreadPoolSize
InitialiseThreadInfo(ClearThreadPoolID)
Next ClearThreadPoolID
EndProcedure
; Main Program
exit = #False
;Delay(1000)
Debug "****-> Start of main program"
; start from task 0
TaskIndex = 0
; initialise the thread pool for use
InitialiseThreadPool()
Repeat
For ThreadPoolID = 0 To #ThreadPoolSize ; check every slot in the thread pool for an openeing
If ThreadPool(ThreadInfoID)\ThreadInUse = #EMPTY ; empty slot so we can Start a task using this slot
If TaskIndex = #TotalTasks ; We've completed all the scheduled tasks skip out
exit = #True
Break
Else
; for debuging purposed print out what the thread did to its data
; before we re-use it
; Debug "========================================================================="
; Debug "Thread Pool Slot: " + Str(ThreadPoolID)
; Debug "Array " + Str(X) + " ThreadID: " + Str(ThreadPool(ThreadPoolID)\ThreadID)
; Debug "Array " + Str(X) + " ServerName: " + ThreadPool(ThreadPoolID)\ServerName
; Debug "Array " + Str(X) + " Number: " + Str(ThreadPool(ThreadPoolID)\Number)
; Debug "Array " + Str(X) + " ChangedName: " + ThreadPool(ThreadPoolID)\ChangedName
; Clean out the junk from the last thread before we use the slot
InitialiseThreadInfo(ThreadPoolID)
; kick off the next thread task
ThreadPool(ThreadPoolID)\ThreadInUse = #INUSE ; reserve our slot and only the thread can unreserve its own slot
ThreadPool(ThreadPoolID)\TaskID = TaskIndex
ThreadPool(ThreadPoolID)\ThreadID = CreateThread(@DoTask(),ThreadPoolID)
; simple error checking to make sure we dont waste slots on threads that didnt start
If ThreadPool(ThreadPoolID)\ThreadID <> 0
; note no guarantee that the task will complete successfully just that we scheduled
; it and its running!
Debug "**=====> Task "+Str(TaskIndex)+" is now being processed"
; finally if the thread contains a non zero value then increment
; the Number of tasks initiated
TaskIndex + 1
Else
; issue starting the thread for whatever reason
; clear the slot and it will be retried next time round
InitialiseThreadInfo(ThreadPoolID)
EndIf
EndIf
Else
; do nothing thread pool slot is busy
Delay(100)
EndIf
Next ThreadPoolID
Until exit = #True
Debug "****-> End of Main Program"
You could always loop at the end of the code until a condition is set by the last thread i.e. taskindex = 25 and condition is lastthreadfinished = #true. If you need to wait and see what happens to all the threads.
My thread pool code would allow you to start everything off in one go i.e. pool size = number of tasks to complete. Or will round robin loop through the tasks pools size at a time until all the tasks are complete.
Ive had a chance to add some checking code to ensure we dont waste pool slots, although the round robin loop Im using could mean at worst poolsize - 1 is the number of idle slots, Im sure you could improve on this if necessay.
Regards
IRIA
thanks!
Posted: Tue Feb 24, 2004 2:37 pm
by Pantcho!!
thanks iria! and u ppl!
at least someone understodo my goal here
i will check the code and modify it as needed
look great.
OK looked at my code again and spotted a bug!
Posted: Tue Feb 24, 2004 3:29 pm
by Iria
Yeh spotted some nasty bugs, firstly the PB debuger is the least thread safe debugger I know
no offence intended!
Best to take all the debug statements out. Second of all I spotted a nasty bug in my code which Ive corrected.
Ive changed the code to show a more graphical and human friendly output of the threads working.
Lastly try setting the number of tasks to a hundred and the pool size to a 100 and run without debugging you will almost certainly get a race condition (read/write memory lock) and crach the application. This is down to the lack of thread safety in the application and the underlying PureBasic code. Just be aware of it, you can hide this 'feature' by working on timings and possibly using some asm code to lock the strucutre byte when you read and set it.
Code: Select all
#TotalTasks = 100 ; total number of tasks to action
#ThreadPoolSize = 10 ; Max size of the thread pool so 6 slots for us here
#INUSE = 1
#EMPTY = 0
#PROGRESS_TIME = 350
Global TaskIndex.l
TaskIndex = 0 ; position of the last task executed
Structure ThreadInfo
ThreadInUse.b
ThreadID.l
ServerName.s
Number.l
ChangedName.s
TaskID.l
EndStructure
Dim ThreadPool.ThreadInfo(#ThreadPoolSize) ; tracks usage of my thread pool slot and data
Procedure DoTask(ThreadPoolSlotID.l)
RandomTaskTime.l = Random(400)+#PROGRESS_TIME
UniqueGadgetWindowID = ThreadPool(ThreadPoolSlotID)\TaskID + 1
OpenWindow(UniqueGadgetWindowID, 200+Random(400), 200+Random(400), 300, 100, #PB_Window_SystemMenu , "Thread No: "+Str(ThreadPoolSlotID)+" Task "+Str(ThreadPool(ThreadPoolSlotID)\TaskID))
CreateGadgetList(WindowID(UniqueGadgetWindowID))
ProgressBarGadget(UniqueGadgetWindowID, 10, 10, 280, 50, 0, RandomTaskTime, #PB_ProgressBar_Smooth)
exit = #False
Repeat
;UseWindow(ThreadPool(ThreadPoolSlotID)\TaskID)
eventid = WindowEvent()
If GetGadgetState(UniqueGadgetWindowID) < RandomTaskTime
SetGadgetState(UniqueGadgetWindowID, GetGadgetState(UniqueGadgetWindowID) + 1)
Else
CloseWindow(UniqueGadgetWindowID)
exit = #True
Break
EndIf
Delay(1)
Until exit = #True
; must be the very last thing we do before we exit
ThreadPool(ThreadPoolSlotID)\ThreadInUse = #EMPTY
EndProcedure
Procedure InitialiseThreadInfo(ThreadInfoID.l)
ThreadPool(ThreadInfoID)\ThreadInUse = #EMPTY ; 0 = empty not in use, non zero = in use
ThreadPool(ThreadInfoID)\ThreadID = 0 ; thread ID returned from starting thread (in case we need to kill it)
ThreadPool(ThreadInfoID)\ServerName = "" ; string to check our non thread safe data
ThreadPool(ThreadInfoID)\Number = 0 ; number to add spice to life
ThreadPool(ThreadInfoID)\ChangedName = "" ; another string in a different position!
ThreadPool(ThreadInfoID)\TaskID = 0
EndProcedure
Procedure InitialiseThreadPool()
; Clear our thread pool
For ClearThreadPoolID = 0 To #ThreadPoolSize
InitialiseThreadInfo(ClearThreadPoolID)
Next ClearThreadPoolID
EndProcedure
; Main Program
exit = #False
;Delay(1000)
Debug "****-> Start of main program"
; start from task 0
TaskIndex = 0
; initialise the thread pool for use
InitialiseThreadPool()
Repeat
For ThreadPoolID = 0 To #ThreadPoolSize ; check every slot in the thread pool for an openeing
If ThreadPool(ThreadPoolID)\ThreadInUse = #EMPTY ; empty slot so we can Start a task using this slot
If TaskIndex = #TotalTasks ; We've completed all the scheduled tasks skip out
exit = #True
Break
Else
; Clean out the junk from the last thread before we use the slot
InitialiseThreadInfo(ThreadPoolID)
; kick off the next thread task
ThreadPool(ThreadPoolID)\ThreadID = CreateThread(@DoTask(),ThreadPoolID)
; simple error checking to make sure we dont waste slots on threads that didnt start
If ThreadPool(ThreadPoolID)\ThreadID <> 0
; very first thing we do before anything else
ThreadPool(ThreadPoolID)\ThreadInUse = #INUSE ; reserve our slot and only the thread can unreserve its own slot
ThreadPool(ThreadPoolID)\TaskID = TaskIndex
; note no guarantee that the task will complete successfully just that we scheduled
; it and its running!
; finally if the thread contains a non zero value then increment
; the Number of tasks initiated
TaskIndex + 1
Else
; issue starting the thread for whatever reason
; clear the slot and it will be retried next time round
InitialiseThreadInfo(ThreadPoolID)
EndIf
EndIf
Else
; do nothing thread pool slot is busy
Delay(200)
EndIf
Next ThreadPoolID
Until exit = #True
Debug "****-> End of Main Program"
Hope this helps you and others...
IRIA
Re: OK looked at my code again and spotted a bug!
Posted: Tue Feb 24, 2004 3:49 pm
by NoahPhense
OMG IRIA! That was super.. I have to go change my underwear now.
That needs to be added to the release samples, and to the code archive.
- john
Posted: Tue Feb 24, 2004 4:23 pm
by Dare2
lol, NoahPhense.
Potent code, Iria. Thanks.
Posted: Tue Feb 24, 2004 4:29 pm
by NoahPhense
Dare2 wrote:lol, NoahPhense.
Potent code, Iria. Thanks.
We're heading to a Multithreading Convention, and Iria is driving.
lol
Bug
Posted: Tue Feb 24, 2004 7:04 pm
by Pantcho!!
it seems IRIA that in the procedure there is a bug and the program wont let me have more then 10 threads (windows...) and stack in the
creategadagetlist...
i found this as a solution
Code: Select all
Procedure DoTask(ThreadPoolSlotID.l)
RandomTaskTime.l = Random(400)+#PROGRESS_TIME
UniqueGadgetWindowID = ThreadPool(ThreadPoolSlotID)\TaskID + 1
ab = OpenWindow(UniqueGadgetWindowID, 200+Random(400), 200+Random(400), 300, 100, #PB_Window_SystemMenu , "Thread No: "+Str(ThreadPoolSlotID)+" Task "+Str(ThreadPool(ThreadPoolSlotID)\TaskID))
CreateGadgetList(ab)
ProgressBarGadget(UniqueGadgetWindowID, 10, 10, 280, 50, 0, RandomTaskTime, #PB_ProgressBar_Smooth)
this works fine
Re: Bug
Posted: Tue Feb 24, 2004 7:19 pm
by NoahPhense
Pantcho!! wrote:it seems IRIA that in the procedure there is a bug and the program wont let me have more then 10 threads (windows...) and stack in the
creategadagetlist...
i found this as a solution
Code: Select all
Procedure DoTask(ThreadPoolSlotID.l)
RandomTaskTime.l = Random(400)+#PROGRESS_TIME
UniqueGadgetWindowID = ThreadPool(ThreadPoolSlotID)\TaskID + 1
ab = OpenWindow(UniqueGadgetWindowID, 200+Random(400), 200+Random(400), 300, 100, #PB_Window_SystemMenu , "Thread No: "+Str(ThreadPoolSlotID)+" Task "+Str(ThreadPool(ThreadPoolSlotID)\TaskID))
CreateGadgetList(ab)
ProgressBarGadget(UniqueGadgetWindowID, 10, 10, 280, 50, 0, RandomTaskTime, #PB_ProgressBar_Smooth)
this works fine
Heres a quicker solution... rofl
Just change the poolsize... (still laughing)
.. he coded it so that you could do that already ..
- np
Beware
Posted: Tue Feb 24, 2004 8:13 pm
by Iria
Yes sorry the key variables are # constants nr the top of the code. That just the way I do things.
PLEASE NOTE:
Just beware, the code I've shown you here is STILL FLAWED and will never be absolutely safe until PB becomes thread safe. Remember if you set the threadpool and the tasks to be say 200 each you will see what I mean.
Its likely (but it may not!) to cause a contention as more than one thread at a time tries to read or write at exactly the same time (at a guess I would imagine that it will happen in the ThreadPool(ThreadInfoID)\ThreadInUse byte).
Im playing with the idea of using something like LOCK CMPXCHG to wait for the byte to be clear for a thread to access. Im not that familiar with it though....
Ah well heed my words
IRIA
Posted: Tue Feb 24, 2004 8:40 pm
by El_Choni
Apart from the lock trick Iria mentions, you can also use these APIs when dealing with threads: TlsAlloc_(), TlsSetValue_(), TlsGetValue_(), TlsFree_() and InitializeCriticalSection_(), EnterCriticalSection_(), LeaveCriticalSection_(), DeleteCriticalSection_(). Look them up in the WIN32.HLP file or in the platform SDK.