Callback function returns always "0"? (GnuTLS) [solved]

Linux specific forum
auser
Enthusiast
Enthusiast
Posts: 195
Joined: Wed Sep 06, 2006 6:59 am

Callback function returns always "0"? (GnuTLS) [solved]

Post by auser »

Currently I'm trying to figure out why my handshake is always "ok" on linux (even if it is not ok because the certificate does not match) while it works like expected on windows. Basically I provide a simple function that have to return "0" or a negative value if finished. The function is used on both (windows and linux) but no matter what I try to return on linux it seems "gnutls_handshake" is always happy with it.

To get a clue what I'm talking about - Scheme:

Code: Select all

ProcedureC test(session.i)
  If good 
    ProcedureReturn(0) 
  Else 
    ProcedureReturn(-43) 
  Endif 
EndProcedure

; Do some cert init stuff...
gnutls_certificate_set_verify_function(xcred,@test()) ; here i specify gnutls my PB function as verify-function

; Init TLS-session, connect, set transport pointer to socket...
gnutls_handshake(session) ; here test() is called... if test() returns 0 gnutls_handshake should be happy else it should return failure

The problem at this point is. gnutls_handshake is calling the PB function as expected (I get all my debug messages and so on) but it does not care what I return him there at the end.


Edit: Found the solution myself. I've compiled on linux 64bit while it was just 32bit on windows. The returnvalue of gnutls_handhake is C int. However using int in PB means 64bit size type. So I did not get 0 or < 0 as documented on GnuTLS docs and expected and checked by me but a value > 0 instead. Changing my variable that carries the return-value from PB int to PB long solved the problem.


Greetings,
auser