Thread affinity OSX!

[Keya]

hello! i am trying to get my worker thread to run only on one CPU core on OSX.

I've put the following demo together based on my understanding of what im supposed to be doing (perhaps im not on the right track i dont know as this is my first year with OSX), but it's not quite working, whereas the Windows equivalent works perfectly.

TO TEST that my code is locked onto a particular core i call CPUID to retrieve the APIC ID unique to each logical core, which is simply:

Code: Select all

  ! xor ecx, ecx
  ! mov eax, 0xB  ;cpu topology leaf
  ! cpuid
  mov APICID, edx

I found out about that at http://en.wikipedia.org/wiki/CPUID#EAX. ... e_topology

Unlike most other CPUID leaves, leaf Bh will return different values in EDX depending on which logical processor the CPUID instruction runs; the value returned in EDX is actually the x2APIC id of the logical processor.

ANYWAY! I tested on Windows (using winapi) and YES, as expected it returns a unique id for each core - just a low number like 1, 2 etc.

But then I tried it on OSX (using pthread api), and it's not quite working as i would expect... the SET and GET affinity functions are both successful, yet when retrieving the CPU APIC ID on my 4-core system i get results like "0,0,0,1", "3,3,3,3", "0,0,0,0", "1,1,1,2" etc (it should be returning something like "0,1,2,3", maybe not in order but each unique), so it does sort of seem to be switching, but just not exactly to the requirement!

Anyway ive spent the last two days on this but im stuck so must ask for help! Does anyone have any ideas?

Code: Select all

EnableExplicit

#THREAD_AFFINITY_POLICY = 4
#THREAD_AFFINITY_POLICY_COUNT = 1

ImportC ""
  thread_policy_get.i (machthread, flavor, *policy, *count, *getdefault)
  thread_policy_set.i (machthread, flavor, *policy, count)    
  pthread_self.i()
  pthread_mach_thread_np.i(thread)    
  sysctlbyname(sysname.i, *count, *countlen, other, other2)
EndImport 


Global CPUCnt 

Procedure.i GetCPUCount() 
  Protected count, count_len = SizeOf(count), *bufname  
  *bufname = AllocateMemory(512)    
  PokeS(*bufname, "hw.logicalcpu", -1, #PB_Ascii)
  If sysctlbyname(*bufname, @count, @count_len, #Null, 0) <> 0: count = 1: EndIf
  If count = 0: count = 1: EndIf
  ProcedureReturn count
EndProcedure


Procedure.l SingleCoreAffinityMask(corenum)   ;first corenum is 0 which returns 1, core 1 returns 2, core 2 returns 4, etc
  ProcedureReturn 1 << corenum
EndProcedure


Procedure SetThreadCPUAffinity(hThread, CPUCore)
  Protected cpubits = SingleCoreAffinityMask(CPUCore)  ;set new core
  If thread_policy_set(hThread, #THREAD_AFFINITY_POLICY, @cpubits, #THREAD_AFFINITY_POLICY_COUNT) = 0
    ProcedureReturn cpubits
  Else
    ProcedureReturn 0
  EndIf
EndProcedure  


Procedure.l GetThreadCPUAffinity(hThread)
  Protected count, cpubits, getdefault
  getdefault = #False: count = 1
  If thread_policy_get(hThread, #THREAD_AFFINITY_POLICY, @cpubits, @count, @getdefault) = 0
    ProcedureReturn cpubits
  Else
    ProcedureReturn 0
  EndIf
EndProcedure  


;Return the unique APIC ID that each logical CPU has
Procedure.i GetAPICID()
DisableDebugger
  EnableASM
  Protected result.l
  ! xor ecx, ecx
  ! mov eax, 0xB
  ! cpuid
  mov result, edx
  DisableASM
  ProcedureReturn result
EnableDebugger
EndProcedure


;The main thread which sets its affinity to each CPU one at a time
Procedure CPUThread(threadval.i)
  Protected CPU.i, hThread = pthread_mach_thread_np(pthread_self())  
  For CPU = 0 To CPUCnt-1
    Debug "Trying logical core #" + Str(CPU) + "... (Set/Get = 0 on error)"
    Debug " SETAFFINITY=" + Str( SetThreadCPUAffinity(hThread, CPU) )  
    Debug " GETAFFINITY=" + Str(GetThreadCPUAffinity(hThread))
    Debug " CPU APIC ID=" + Str(GetAPICID()) + " (can be 0 but should be unique on each core)"
  Next CPU  
EndProcedure


CPUCnt = GetCPUCount()
Debug "Logical CPU Count = " + Str(CPUCnt)

CreateThread(@CPUThread(), #Null)

MessageRequester("OK","Done")

[wilbert]

I get results like this with all ID values equal.

Logical CPU Count = 4
Trying logical core #0... (Set/Get = 0 on error)
SETAFFINITY=1
GETAFFINITY=1
CPU APIC ID=2 (can be 0 but should be unique on each core)
Trying logical core #1... (Set/Get = 0 on error)
SETAFFINITY=2
GETAFFINITY=2
CPU APIC ID=2 (can be 0 but should be unique on each core)
Trying logical core #2... (Set/Get = 0 on error)
SETAFFINITY=4
GETAFFINITY=4
CPU APIC ID=2 (can be 0 but should be unique on each core)
Trying logical core #3... (Set/Get = 0 on error)
SETAFFINITY=8
GETAFFINITY=8
CPU APIC ID=2 (can be 0 but should be unique on each core)

[Keya]

I usually get all the same APIC ID too, but if i run it say 10 times usually 1 or 2 of those will have different IDs

[wilbert]

Does this help ?
https://developer.apple.com/library/mac ... finityAPI/
Reading it I get the impression that all threads with the same affinity are tried to run on the same core but you are only running one thread
Is there any need for the OS to switch to a different core in this case ?

[Keya]

Is there any need for the OS to switch to a different core in this case ?

well i would like to say "yes, because i set the policy to tell the OS which core to run it on, and the OS told me that was successful" hopefully the OS isn't saying "i'll keep that request in mind, but things seem fine as they are for now so i'll leave things alone, anyway just pretend it was Successful"

As another option that just sprung to mind when you mentioned "but you are only running one thread" I can create a thread for every core, maybe that'll help force it instead of the current single thread, but I'm disappointed the API that seems to be for this job aren't doing their job! the OS is telling me "Success!" ... but not behaving successfully! or im missing part of the puzzle

[Keya]

Trying it with one thread per core, very simple mod, i get SLIGHTLY better results... 1,1,2,3 etc more often than just 1,1,1,1... but STILL not always on the one core, grr!

Code: Select all

EnableExplicit

#THREAD_AFFINITY_POLICY = 4
#THREAD_AFFINITY_POLICY_COUNT = 1

ImportC ""
  thread_policy_get.i (machthread, flavor, *policy, *countx, *getdefault)
  thread_policy_set.i (machthread, flavor, *policy, countx)    
  pthread_self.i()
  pthread_mach_thread_np.i(thread)    
  sysctlbyname(sysname.i, *count, *countlen, other, other2)
EndImport 


Global CPUCnt 

Procedure.i GetCPUCount() 
  Protected count, count_len = SizeOf(count), *bufname  
  *bufname = AllocateMemory(512)    
  PokeS(*bufname, "hw.logicalcpu", -1, #PB_Ascii)
  If sysctlbyname(*bufname, @count, @count_len, #Null, 0) <> 0: count = 1: EndIf
  If count = 0: count = 1: EndIf
  ProcedureReturn count
EndProcedure


Procedure.l SingleCoreAffinityMask(corenum)   ;first corenum is 0 which returns 1, core 1 returns 2, core 2 returns 4, core 3 returns 8, core 4 returns 16, etc
  ProcedureReturn 1 << corenum
EndProcedure


Procedure SetThreadCPUAffinity(hThread, CPUCore)
  Protected cpubits = SingleCoreAffinityMask(CPUCore)  ;set new core
  If thread_policy_set(hThread, #THREAD_AFFINITY_POLICY, @cpubits, #THREAD_AFFINITY_POLICY_COUNT) = 0
    ProcedureReturn cpubits
  Else
    ProcedureReturn -1
  EndIf
EndProcedure  


Procedure.l GetThreadCPUAffinity(hThread)
  Protected count, cpubits, getdefault
  getdefault = #False: count = 1
  If thread_policy_get(hThread, #THREAD_AFFINITY_POLICY, @cpubits, @count, @getdefault) = 0
    ProcedureReturn cpubits
  Else
    ProcedureReturn -1
  EndIf
EndProcedure  



;Return the unique APIC ID that each logical CPU has
Procedure.i GetAPICID()
DisableDebugger
  EnableASM
  Protected result.l
  ! xor ecx, ecx
  ! mov eax, 0xB
  ! cpuid
  mov result, edx
  DisableASM
  EnableDebugger
  ProcedureReturn result
;EnableDebugger
EndProcedure


;The main thread which sets its affinity to each CPU one at a time
Procedure CPUThread(threadval.i)
  Protected CPU.i, hThread = pthread_mach_thread_np(pthread_self())  
  CPU = threadval
    Debug "SETAFFINITY[" + Str(CPU) + "] =" + Str( SetThreadCPUAffinity(hThread, CPU) )  
    Debug "GETAFFINITY=" + Str(GetThreadCPUAffinity(hThread))
    Debug "CPU APIC ID=" + Str(GetAPICID())    
    Delay(10000)
EndProcedure


CPUCnt = GetCPUCount()
Debug "Logical CPU Count = " + Str(CPUCnt)

Define i
For i = 0 To CPUCnt
  CreateThread(@CPUThread(), i)
  Delay(1000)
Next i

MessageRequester("OK","Done")

[wilbert]

well i would like to say "yes, because i set the policy to tell the OS which core to run it on, and the OS told me that was successful"

Where did you tell it to run on what core ?
I didn't see any api function to do so.

[Keya]

thread_policy_set is the api

Code: Select all

Procedure SetThreadCPUAffinity(hThread, CPUCore)
  Protected cpubits = SingleCoreAffinityMask(CPUCore)  ;set new core
  If thread_policy_set(hThread, #THREAD_AFFINITY_POLICY, @cpubits, #THREAD_AFFINITY_POLICY_COUNT) = 0
    ProcedureReturn cpubits
  Else
    ProcedureReturn -1
  EndIf
EndProcedure

[wilbert]

I see that you are using that function Keya but where in the Apple documentation does it say you can set a specific core this way.
As far as I understand the number you can set has no relation to a specific core and the only things that matters if you run multiple threads is if these numbers are equal or different compared to the other threads.

[Keya]

that was just my (mis?!)understanding from what ive read from various places, for example https://developer.apple.com/library/mac ... duler.html

If you need real-time behavior, you must use the Mach thread_policy_set call. This is described in Using the Mach Thread API to Influence Scheduling.

Using the Mach Thread API to Influence Scheduling

This API is frequently used in multimedia applications to obtain real-time priority. It is also useful in other situations when the pthread scheduling API cannot be used or does not provide the needed functionality.

The API consists of two functions, thread_policy_set and thread_policy_get.

I dont need to change priority just affinity but they both seem to be in the same api

Also interesting post #7 at http://superuser.com/questions/149312/h ... ty-on-os-x which also says since 10.5 OSX has supported this:

An application that wants to place a thread on every available processor would do the following:
Obtain the number of processors on the system using sysctl(3).
Create that number of threads.
Set each thread with a distinct affinity tag.
Start all threads.
Threads with default affinity policy will be scheduled more freely on any processor. These threads will be preferentially migrated to run on an idle processor. Threads with affinity tags will tend to remain in place.

[wilbert]

/*
* THREAD_AFFINITY_POLICY:
*
* This policy is experimental.
* This may be used to express affinity relationships
* between threads in the task. Threads with the same affinity tag will
* be scheduled to share an L2 cache if possible. That is, affinity tags
* are a hint to the scheduler for thread placement.
*
* The namespace of affinity tags is generally local to one task. However,
* a child task created after the assignment of affinity tags by its parent
* will share that namespace. In particular, a family of forked processes
* may be created with a shared affinity namespace.
*
* Parameters:
* tag: The affinity set identifier.
*/

[Keya]

im gonna cry if i can only do it on Linux and Windows but OSX wont let me!