List map threadsafe read

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
Rinzwind
Enthusiast
Enthusiast
Posts: 636
Joined: Wed Mar 11, 2009 4:06 pm
Location: NL

List map threadsafe read

Post by Rinzwind »

Can the pointer that's used internally to keep track of current element be specified "threaded" so each tread has its own copy and reading a list or map becomes threadsafe?

Btw would be nice to be able to specify data type of key too instead of forced string...
User avatar
jacdelad
Addict
Addict
Posts: 1431
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: List map threadsafe read

Post by jacdelad »

You can save the pointer for each thread by yourself. You can only have one item being specified as the selected item any given time, so if two threads access the same list simultaneously it won't work. Instead you can use either maps (if every thread accesses a different item) or manipulate the data from the saved pointer (which I don't know if it's that easy). Otherwise use locks or whatever to be sure only one thread accesses the list any given time, which slows the threads down. PS: I have a complex program running 26 threads on startup filling a map with lots of data. Each thread has its own structured map element with lots of of subitems, including other maps and lists. Works like a charm.

I hope I gave you the correct information, otherwise someone may correct me.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Rinzwind
Enthusiast
Enthusiast
Posts: 636
Joined: Wed Mar 11, 2009 4:06 pm
Location: NL

Re: List map threadsafe read

Post by Rinzwind »

I know of some workarounds. Still won't fix multiple simultaneous foreach loops. Only works if the internal pointer would be per-thread. Curious that's not the case. Would make mutexes for readonly access unnecessary, and therefore a non-blocking execution.
User avatar
jacdelad
Addict
Addict
Posts: 1431
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: List map threadsafe read

Post by jacdelad »

What kind of workarounds do you know?
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Rinzwind
Enthusiast
Enthusiast
Posts: 636
Joined: Wed Mar 11, 2009 4:06 pm
Location: NL

Re: List map threadsafe read

Post by Rinzwind »

Low hanging fruit improvement, no?
infratec
Always Here
Always Here
Posts: 6817
Joined: Sun Sep 07, 2008 12:45 pm
Location: Germany

Re: List map threadsafe read

Post by infratec »

And for what is a Mutex?

Lists can never be threadsave by itself, because elements can be deleted and added.
How do you compensate this in a different thread?
You are standing on an element which is just deleted by an other thread for example.
Last edited by infratec on Mon Apr 11, 2022 11:30 am, edited 1 time in total.
Fred
Administrator
Administrator
Posts: 16619
Joined: Fri May 17, 2002 4:39 pm
Location: France
Contact:

Re: List map threadsafe read

Post by Fred »

There is no thread protection on list/map in PB, so you need to do it yourself with mutexes. We could add such in threadmode, but it will probably impact the performances for all lists/maps so I don't know if it worths it.
Rinzwind
Enthusiast
Enthusiast
Posts: 636
Joined: Wed Mar 11, 2009 4:06 pm
Location: NL

Re: List map threadsafe read

Post by Rinzwind »

infratec wrote: Mon Apr 11, 2022 7:31 am And for what is a Mutex?

Lists can never be threadsave by itself, because elements can be deleted and added.
How do you compensate this in a different thread?
You are standing on an element which is just deleted by an other thread for example.
I am talking about concurrent read access so you can do a read-foreach in multiple threads simultaneously. Which can be threadsafe by default (and is in many other languages afaik). When you want to modify it you ofcourse use a mutex to get exclusive access, which will also block read access elsewhere. Maybe I oversimplify things.. maybe reading the map can set a atomic counter for threaded readonly access until it is done after which mutexed modify action is allowed again.

Well, thanks for thinking it over.
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: List map threadsafe read

Post by skywalk »

Why not use MapSize() and a regular loop without ForEach..Next?
Concurrent reads would be safe as long as you used semaphore|mutex.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
helpy
Enthusiast
Enthusiast
Posts: 552
Joined: Sat Jun 28, 2003 12:01 am

Re: List map threadsafe read

Post by helpy »

skywalk wrote: Mon Apr 11, 2022 6:43 pmWhy not use MapSize() and a regular loop without ForEach..Next?
How should this work?
It is not possible to use item index:

Code: Select all

mSize = MapSize(theMap())
For itemIndex = 1 To mSize
	*item = ????
Next itemIndex
It is only possible to use ForEach or While (with ResetMap and NextMapElement) to iterate through map elements.
Both methodes are not threadsafe.

Did I missed something?
I am always ready to learn new things.
Windows 10 / Windows 7
PB Last Final / Last Beta Testing
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: List map threadsafe read

Post by skywalk »

Yes, the While, but you increment a self counter.
And use Semaphore|Mutex.
ForEach macro scares me. :evil:
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Rinzwind
Enthusiast
Enthusiast
Posts: 636
Joined: Wed Mar 11, 2009 4:06 pm
Location: NL

Re: List map threadsafe read

Post by Rinzwind »

ps. even if you, as the programmer, know for sure you will not modify the list once populated, you still cannot read from multiple threads simultaneously because the current element pointer is not "threaded" but shared, so every ForEach will mess with each other. Which is why I requested this in the first place. At least make the read pointer threaded (as in keyword Threaded in PB). Which is an easy first step to improve I think?
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: List map threadsafe read

Post by mk-soft »

To edit an existing list over two threads, I also use two lists. One with the data and one with the pointers to the data.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
helpy
Enthusiast
Enthusiast
Posts: 552
Joined: Sat Jun 28, 2003 12:01 am

Re: List map threadsafe read

Post by helpy »

mk-soft wrote: Tue Apr 12, 2022 11:38 am To edit an existing list over two threads, I also use two lists. One with the data and one with the pointers to the data.
That is one way.

You could also iterate through the list in one thread and pass the items to several threads to work on the items in parallel running threads.
Windows 10 / Windows 7
PB Last Final / Last Beta Testing
Post Reply