PureBasic Forums - English

Posted: **Tue Nov 06, 2012 2:55 am**

I'm not sure this is a bug but I've spent a lot of frustrating time on it and it may be.

I'm getting invalid memory access errors part of the time when calling DLL functions in Windows 7. The test code below is an example.

vx=OpenLibrary(#PB_Any,"vfile.dll")
filename$="text.xxg"
DeleteFile(filename$)
vfilehandle=CallFunction(vx,"vOpen",15,@filename$)
CallFunction(vx,"vClose",vfhandle)
CloseLibrary(vx)

"Vfile.dll" provides a set of database and indexing functions. I have used it for years.
If I run the above code on Windows XP, it works fine every time.
If I run the same code, with the same version of PureBasic, on a 32-bit Windows 7 system, about 50% of the time it returns "Invalid memory access. (read error at address 4)" when it hits the first CallFunction line. The other 50% it seems to work. I can sit here and press F5 to compile and run over and over and it'll crash about half the time with the same error. I have tried it with both Debug and Purifier on and off and threadsafe and Vista administrator modes, with no affect. I tried putting the filename into a memory block (Pokes(*mem,filename$)) instead but still had the problem.

I was alerted to this behaviour back in August when I began to get seemingly random invalid memory access errors after we had been changed to Win 7 at work.

I thought maybe it was the dll but then I began to experience the same problem with another dll which I've used for years to handle GIF images, Image323.

To solve this problem, I tried using Prototypes and the error then happened consistently every time; it was no longer random. I tried Import after using Polib.exe to create a lib file from the dll. That also produced errors every time, although it did seem to work for the other Image323.dll (prototypes didn't). CallFunction is the only function that works part of the time.

So I don't know if this is a bug or what. It is fine in XP but not W7.

Have you heard of anything like this?

Posted: **Tue Nov 06, 2012 4:22 am**

DeanH wrote:...I'm getting invalid memory access errors part of the time when calling DLL functions in Windows 7. The test code below is an example.
Code: Select all
vx=OpenLibrary(#PB_Any,"vfile.dll")
filename$="text.xxg"
DeleteFile(filename$)
vfilehandle=CallFunction(vx,"vOpen",15,@filename$)
CallFunction(vx,"vClose",vfhandle)
CloseLibrary(vx)

Hi DeanH. Besides the small typo (vfhandle --> vfilehandle) at the vOpen & vClose function calls, the calling method seems quite regular, although it is difficult to determine without actually knowing what's going on in the DLL itself.

For example, how does the function read filename$ if it is deleted, or is it being created? And what is the 15 value in the vOpen function call? Since you're dealing with 32bit Windows 7, you can rule out variable-length errors, but still WinXP file processing might differ from that of Win7, which could be causing the read error.

Perhaps you might have a little more information about the vOpen function. Thank you.

Posted: **Tue Nov 06, 2012 3:36 pm**

Couldn't it be the fact that the memory allocated isn't enough?

Code: Select all

filename$="text.xxg"
DeleteFile(filename$)

; Allocate more than enough space...
filename$ = space(200)

vx=OpenLibrary(#PB_Any,"vfile.dll")
vfilehandle=CallFunction(vx,"vOpen",15,@filename$)
CallFunction(vx,"vClose",vfilehandle)
CloseLibrary(vx)

Posted: **Tue Nov 06, 2012 3:51 pm**

jassing wrote:Couldn't it be the fact that the memory allocated isn't enough?

I think that call is for providing the filename, not to receive it.

@DeanH

You should provide a link to the dll, the documentation of the dll, a working snippet of code demonstrating the problem and all of this in coding questions. I'm sure someone will be able to help you this way.

Anyway based on what you are saying could be useful if you could try the same code on a clean windows 7 machine. Maybe inside a virtual machine.

Posted: **Wed Nov 07, 2012 3:16 am**

I posted this as I'd hoped someone else may have run into something similar. Perhaps not. I first thought it was a problem with the DLL itself -- and I have tired this on three different Win7 machines (one 64-bit, the other two 32-bit), one of which was pretty clean -- all with the same results. I eliminated as many variables as possible and tried every approach I could think of. I think I have coded it properly in PureBasic as it works perfectly under Win XP with the same verion of PureBasic (4.61). But when I saw the same error with a completely different DLL, I began to wonder if it was something else and there exists a workaround or something, and that's why I posted this query.

The DLL was written in C++ years ago by a friend who is a very experienced professional programmer. The functions are wrappers for the Villa part of the QDBM database system, which can be found on the net. I contacted him recently about this problem and he's completely stumped, has no idea.

My test program is:
vx=OpenLibrary(#PB_Any,"vfile.dll") ;open DLL
Debug vx
vfilehandle=CallFunction(vx,"vOpen",7,@filename$) ;open the file
Debug vfilehandle
CallFunction(vx,"vClose",vfilehandle) ;close the file
CloseLibrary(vx) ;close library

The library definitely opens as debug shows a value for vx. If it works, Debug shows a value for the open file. If it doesn't the program crashes in the editor with the first CallFunction line highlighted and the reported invalid memory access error.

The function vOpen opens a file. The parameters are a value indicating the mode with which to open the file (7 is the sume of a set of constants that mean open for read/write and create file if it doesn't exist); the next parameter is a pointer to a string containing the filename to be opened. The string is being passed to the function via its address, it is not passed back. Without the @, a bad parameter error is reported. The error is not a space problem.

The function vClose simply closes the open file.

As I said before, it works sometimes and not others in Win7 and it works all the time in XP. Does not fail. Same is true of the other DLL, too. I am not sure this is a PureBasic problem even, it feels like a Windoz 7 issue but I'm simply unsure. I have been using these DLLs for several years and hundreds of users have not had a problem with them.

Here is a URL with a zip file containing the test program VFILETEST2.pb, VFILE.DLL, the original documentation Vfiles.doc, and another test program in which I tried unsuccessfully using Import along with the .lib file. Note: Vfile.dll is freeware and public domain. When it works it is a nice little indexing system for data. It automatically handles file sharing for concurrent multiple users, allows variable length records and uses a B-tree structure so data is retrieved in alphabetical order.

http://www.bookmark.sa.edu.au/files/sec ... letest.zip

Posted: **Wed Nov 07, 2012 9:02 am**

It crashes inside vOpen() here (Win7 64bit) and does not create the file:

Code: Select all

;---[ Try Catch EndTry ]----------------------------------------------------
;
; Structured Exception Handling for WinXP+, PureBasic/Windows 32bit + 64bit
;
; by Danilo, May 2012
;
Import "kernel32.lib"
    ;AddVectoredExceptionHandler(FirstHandler.l, *VectoredHandler)
    ;RemoveVectoredExceptionHandler(*Handler)
EndImport

Structure ExceptionHandlerStack
    catchAddress.i
    oldHandler.i
    StackPointer.i
EndStructure

Threaded NewList ExceptionHandlerList.ExceptionHandlerStack()
Threaded CurrentExceptionInfo.EXCEPTION_POINTERS

Procedure.l UnhandledExceptionFilter(*ExceptionInfo.EXCEPTION_POINTERS)

    CopyStructure(*ExceptionInfo,@CurrentExceptionInfo,EXCEPTION_POINTERS)

    CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
        *ExceptionInfo\ContextRecord\Eip = ExceptionHandlerList()\catchAddress
        *ExceptionInfo\ContextRecord\Esp = ExceptionHandlerList()\StackPointer
    CompilerElse
        *ExceptionInfo\ContextRecord\Rip = ExceptionHandlerList()\catchAddress
        *ExceptionInfo\ContextRecord\Rsp = ExceptionHandlerList()\StackPointer
    CompilerEndIf

    SetUnhandledExceptionFilter_( ExceptionHandlerList()\oldHandler )
    ;RemoveVectoredExceptionHandler( ExceptionHandlerList()\oldHandler )

    DeleteElement( ExceptionHandlerList() )
    ProcedureReturn $ffffffff

EndProcedure

Procedure AddExceptionHandler(*catch,StackPointer.i)
    If AddElement( ExceptionHandlerList() )
        ExceptionHandlerList()\oldHandler   = SetUnhandledExceptionFilter_(@UnhandledExceptionFilter()) ; Win2000pro+ 32bit
        ;ExceptionHandlerList()\oldHandler   = AddVectoredExceptionHandler(1,@UnhandledExceptionFilter()) ; XP+
        ExceptionHandlerList()\catchAddress = *catch
        ExceptionHandlerList()\StackPointer = StackPointer
    EndIf
EndProcedure

Global AddExceptionHandler.i = @AddExceptionHandler()

! macro __try {
!    local label1, label2, label3
!    handler_label equ label1
!    handler_end_label equ label2
!    end_try_label equ label3
! }

Macro Try
    CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
    !__try
    ! push esp
    ! push handler_label
    ! call [v_AddExceptionHandler]
    CompilerElse
    !__try
    ! mov qword rdx, rsp
    ! mov qword rcx, handler_label
    ! call qword [v_AddExceptionHandler]
    CompilerEndIf
EndMacro

Macro Catch
    !jmp end_try_label
    !handler_label:
EndMacro

Macro EndTry
    !end_try_label:
    ! restore handler_label
    ! restore end_try_label
    ! restore handler_end_label
EndMacro

;---------------------------------------------------------------------------

;XIncludeFile "TryCatch.pbi"

;QDBM Villa constants
#VLOREADER=1
#VLOWRITER=2
#VLOCREAT=4
#VLOTRUNC=8
#VLONOLCK=16
#VLREADWRITE=7

;vfile.lib produced by POLIB.EXE vfile.dll OUT:vfile.lib
;Adding underscores to the alias names causes compile POLINK errors.
Import "vfile.lib"
	vOpen(uMode.l, lpFilename.i)  As "vOpen"
	vClose(pVilla.l) As "vClose"
	vCloseAll() As "vCloseAll"
	vPut(pVilla,lpKey$,lpValue$,dwMode.l) As "vPut"
	vDelete(pVilla,lpKey$,lpValue$) As "vDelete"
	vDelAny(pVilla,lpKey$,lpValue$) As "vDelAny"
	vGetValueLen(pVilla) As "vGetValueLen"
	vGetValue(pVilla,lpValue.l) As "vGetValue"
	vGetKeyLen(pVilla) As "vGetKeyLen"
	vGetKey(pVilla,lpKey.l) As "vGetKey"
	vFind(pVilla,lpKey$) As "vFind"
	vNear(pVilla,lpKey$) As "vNear"
	vFirst(pVilla) As "vFirst"
	vLast(pVilla) As "vLast"
	vNext(pVilla) As "vNext"
	vNextDup(pVilla) As "vNextDup"
	vNextNonDup(pVilla,lpKey$) As "vNextNonDup"
	vPrev(pVilla) As "vPrev"
	vPrevDup(pVilla) As "vPrevDup"
	vPrevNonDup(pVilla,lpkey$) As "vPrevNonDup"
	vMark(pVilla,dwMark.l) As "vMark"
	vReset(pVilla,dwMark) As "vReset"
	vDups(pVilla,lpKey$) As "vDups"
	vRecs(pVilla) As "vRecs"
	vClean(pVilla) As "vClean"
EndImport


mode.l=#VLOREADER | #VLOWRITER | #VLOCREAT ;| #VLONOLCK
Debug mode

filename$="text.xxg"
DeleteFile(filename$)
Delay(200)

Try
  vfhandle=vOpen(mode,@filename$)
Catch
  MessageRequester("EXCEPTION","Catched Exception in vOpen")
EndTry

Try
  If vfhandle
    vClose(vfhandle)
    ;vCloseAll()
  EndIf
Catch
  MessageRequester("EXCEPTION","Catched Exception in vClose")
EndTry

End

It looks to me that vOpen() can not create the file and/or some memory allocation fails, and then it is writing
to a "NULL-pointer + 24".

The interesting things begin, if you out-comment the delay() in the code above and disable the debugger.

Sometimes, it creates a file with 127k or 128k that contains the signature "[depot]", and it is crashing in vClose().
Sometimes it crashes in vOpen().
With the delay() after DeleteFile(), it always crashes in vOpen() here. Without the delay after DeleteFile(), I get random crashes in vOpen() and vClose().

Maybe the C++ exception handling collides with PB debugger exception handling?
Maybe there are some checks missing within the DLL (after memory allocation, open/create files)?
Does the DLL use the PureBasic process heap for memory allocation?
Is the DLL compiled with VC++?

Posted: **Wed Nov 07, 2012 10:48 pm**

Thanks for that interesting detective work. The version I was mainly working with was NOT the Import version but the CallFunction version. Can you test that one?

I am not sure which version of C in which it was written. I think it would have been VC++ but I'll have to check. I have no idea whether it is using the PureBasic heap.

I did note in my own attempts that using delays changed things. I also suspected the debugger may be an issue but have not verified it. When it works it should create a 131k file with the signature "[depot]".

As I said before IT WORKS IN WINDOWS XP WITHOUT A PROBLEM. (The CallFunction version below, not the Import one.) This is what is baffling me. Also baffling is that another completely different DLL can crash almost randomly, too, in a similar manner. This is why I'm concerned it may have something to do with PB. I just don't know.
vx=OpenLibrary(#PB_Any,"vfile.dll") : Debug vx
IF vx
vfilehandle=CallFunction(vx,"vOpen",7,@filename$) : Debug vfilehandle
CallFunction(vx,"vClose",vfilehandle)
CloseLibrary(vx)
EndIf

Posted: **Thu Nov 08, 2012 12:38 am**

The zip does not contain a .doc

I tested VFILETEST2.pb under Win7 (x64) and the problems are the same as in the version using import.

Your VFILETEST2 is

Code: Select all

;QDBM Villa constants
#VLOREADER=1
#VLOWRITER=2
#VLOCREAT=4
#VLOTRUNC=8
#VLONOLCK=16
#VLREADWRITE=7

mode.l=#VLOREADER | #VLOWRITER | #VLOCREAT

vx.l=OpenLibrary(#PB_Any,"vfile.dll")
filename$="text.xxg"
DeleteFile(filename$)
vfilehandle=CallFunction(vx,"vOpen",mode,@filename$)
CallFunction(vx,"vClose",vfhandle)
CloseLibrary(vx)

I get the IMA 9 times out of 10.

But if I change it to:

Code: Select all

;QDBM Villa constants
#VLOREADER=1
#VLOWRITER=2
#VLOCREAT=4
#VLOTRUNC=8
#VLONOLCK=16
#VLREADWRITE=7

mode.l=#VLOREADER | #VLOWRITER | #VLOCREAT

filename$="text.xxg"
DeleteFile(filename$)

vx.l=OpenLibrary(#PB_Any,"vfile.dll")
vfilehandle=CallFunction(vx,"vOpen",mode,@filename$)
CallFunction(vx,"vClose",vfhandle)
CloseLibrary(vx)

... I tried it a lot of times with/without the file already present and I never got an IMA, with/without the debugger.

The created file is 128 KB (131.821 bytes)

BTW: shouldn't be

CallFunction(vx,"vClose",vfilehandle) ; <- EnableExplicit prevent these things

?

Posted: **Thu Nov 08, 2012 1:12 am**

Sorry about the variable error. I'm doing this while at work amd am rushing.

So, what you're saying me is that the order of the code seems to matter. With this particular sample, how would that matter?

I just tried the change you made, putting DeleteFile before OpenLibrary. It worked 4 times for me but on the 5th time it crashed with Invalid memory access. (read error at address 24). So the change of order helps but it does not completely eliminate the problem.

I do not understand why shifting DeleteFile or the variable allocation makes any difference. I cannot see why it should.

Anywy, I just did some further testing. If I delay compile/run to about 3 seconds between tries it works more often, although it still can crash. Not sure what that means.

Could the PB program be continuing to run while the DLL functions are still processing? In parallel and not sequential? Is it crashing because the library needs more time to open? That doesn't sound right.

I've used this DLL in a very large, complex program for years that works completely fine in Windows XP. It is only in Win 7 (maybe Vista, not sure, haven't checked) where I've run into this problem. In the big program, one task I do is to convert a large slab of text data (about 15 Mb worth) located in many files into a single file using this DLL's functions. In XP it does this perfectly every time. In Win7, the program gets well into that task, calling the functions (there are many more than in my little test) thousands of times before it finally crashes with the invalid memory error. I tried slowing it down and that did not help. I tried closing the library and re-opening many times. That only changed where it crashed. Shifting lines of code changed where it would crash but it still would. I keep coming back to this point: it works in XP, just not in Win7. I don't think it is PureBasic (I hope not), I feel as if it is something Win7 is doing differently. I was hoping someone might have run into something like thie before.

So, bottom line, either the DLL or PureBasic behaves differently in Win7 than in XP. Either memory is exposed and being overwritten, or something is shifted, or something else. We can all see there's a problem but there's no solution in site yet.

Posted: **Thu Nov 08, 2012 1:22 am**

Here doesn't crash with the instructions shifted but I only tried running it many times, if it crashes for you probably would happen here too in a real program with many calls.
Well, the error make me think the problem is in DLL, don't know why and why it doesn't happen with XP, but the DLL seems to write to an offset starting from 0.

The problem happens in the final program too right ? Not only running from the ide + debugger ?

Maybe if you could have the dll rebuilt with a recent version of the C compiler, or better yet if you could have access to the sources.... can't you ?

Posted: **Thu Nov 08, 2012 1:40 am**

"The problem happens in the final program too right ? Not only running from the ide + debugger ?"

Just checked that. Yes, it happens. I compiled the program and ran it straight. It did not report an error but when I checked, data was missing in the file from the point where it crashed in the IDE.

I'll ask the guy who wrote it if he could recompile it.

Posted: **Thu Nov 08, 2012 8:01 am**

DeanH wrote:I do not understand why shifting DeleteFile or the variable allocation makes any difference. I cannot see why it should.

I've seen such strange, random errors years ago. If shifting/re-ordering variables makes a difference, it is most likely that
something gets overwritten and corrupted (data section, stack, heap memory). It happens if a function writes to out-of-bounds memory.

For example, if it allocates/re-allocates memory from the heap (the DLL imports many heap functions) and overwrites some data there,
where it should not write, it would overwrite/corrupt other data and variables on the heap.
Memory and heap management can change internally between Windows versions, so sometimes you don't get the error on one OS (just luck/accident),
but it crashes in other environments.

Reading from or writing to address 24 is most likely a NULL-Pointer-issue, after a function call failed and returned 0 instead a valid memory location
and you write to that pointer without checking.
But it could also happen if a valid pointer is in a variable (on heap or stack), and the variable gets overwritten with 0 (or 24) by accident (out-of-bounds writing).
Next time you use this pointer-variable (which was valid before), you get a crash.

Posted: **Thu Nov 08, 2012 8:07 am**

luis wrote:...But if I change it to:
Code: Select all
...
filename$="text.xxg"
DeleteFile(filename$)

vx.l=OpenLibrary(#PB_Any,"vfile.dll")
...

Hi luis. Great code sleuthing! Would you happen to know why the string assignment would have caused OpenLibrary() to crash? Could it be that the string was being written into the opened library space?

Posted: **Thu Nov 08, 2012 8:17 am**

Danilo wrote:I've seen such strange, random errors years ago. If shifting/re-ordering variables makes a difference, it is most likely that something gets overwritten and corrupted (data section, stack, heap memory). It happens if a function writes to out-of-bounds memory...

Thanks Danilo. That answers my question; which means that the cause of the crash is the rogue DLL which somehow violates the calling process' memory space upon attachment.

Posted: **Thu Nov 08, 2012 8:40 am**

TI-994A wrote:Hi luis. Great code sleuthing! Would you happen to know why the string assignment would have caused OpenLibrary() to crash? Could it be that the string was being written into the opened library space?

It happens also without PB's OpenLibary() and object management, by using plain WinAPI:

Code: Select all

;QDBM Villa constants
#VLOREADER=1
#VLOWRITER=2
#VLOCREAT=4
#VLOTRUNC=8
#VLONOLCK=16
#VLREADWRITE=7

mode.l=#VLOREADER | #VLOWRITER | #VLOCREAT

Prototype.i proto_vOpen(mode.l, *filename.Ascii)
Prototype   proto_vClose(pVilla.i)


;filename$="text.xxg"     ; define filename$ before next variable (vx) and it doesn't crash (so often)
;DeleteFile_(filename$)

vx.l=LoadLibrary_("vfile.dll")
Debug vx

filename$="text.xxg"    ; define filename$ here for the first time, and it crashes most of the time
DeleteFile_(filename$)


If vx
    vOpen.proto_vOpen = GetProcAddress_(vx, @"vOpen")
    If vOpen
        vfilehandle = vOpen(mode,@filename$)
        Debug vfilehandle

        If vfilehandle

            vClose.proto_vClose = GetProcAddress_(vx, @"vClose")
            If vClose
                vClose(vfilehandle)
            EndIf
        EndIf
    EndIf
    
    FreeLibrary_(vx)
EndIf

(compile in ASCII mode)

PureBasic Forums - English

Possible problem calling DLL functions in Windows 7

Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7

Re: Possible problem calling DLL functions in Windows 7