Help troubleshoot "Overflow in a dynamically allocated mem"?

Just starting out? Need help? Post your questions and find answers here.
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Help troubleshoot "Overflow in a dynamically allocated mem"?

Post by skywalk »

I recently see this error pop up in my main event loop of a threaded app running with the IDE.
Any idea how to trouble shoot?
The app resumes fine with [F7], but I would like to understand the source of this error?
Could this be the Purifier and if so, how do I know which memory is overflown?

Code: Select all

[12:17:21] Waiting for executable to start...
[12:17:21] Executable type: Windows - x64  (64bit, Unicode, Thread, Purifier)
[12:17:21] Executable started.
[14:36:04] [ERROR] myapp.pb (Line: 8197)
[14:36:04] [ERROR] Overflow in a dynamically allocated memory block.
[14:39:19] Execution continued.

8196    Repeat
8197      evWW = WaitWindowEvent(10)  ;<-- Red error here but why?
8198      If evWW
Last edited by skywalk on Fri Sep 22, 2017 7:59 pm, edited 1 time in total.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by mk-soft »

Memory leak at other procedure?!
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Ok, but how do I know which procedure? This is a big app. :?:
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
nco2k
Addict
Addict
Posts: 1344
Joined: Mon Sep 15, 2003 5:55 am

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by nco2k »

if you know roughly when its happening, you could use CallDebugger or breakpoints and go step by step. also have you enabled the purifier? its probably just something stupid like allocating memory for a string in character size instead of byte size, or missing room for the null terminator.

c ya,
nco2k
If OSVersion() = #PB_OS_Windows_ME : End : EndIf
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Well, the ide debugger is stopping at WaitWindowEvent(10)?
I can't really step through the code since the worker function is in another thread.
I did not get this error when running that worker without threads?

So, my question is, what triggered the debugger to stop?
And if I resume and no errors, then what is the real problem?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by mk-soft »

Are used threads without thread safe?
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
nco2k
Addict
Addict
Posts: 1344
Joined: Mon Sep 15, 2003 5:55 am

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by nco2k »

>> Well, the ide debugger is stopping at WaitWindowEvent(10)?
it doesnt matter where it stopped. purebasic cant really precisely tell you where the overflow happened, when you are using threads. so dont rely on that line.

>> I can't really step through the code since the worker function is in another thread.
of course you can.

>> I did not get this error when running that worker without threads?
then you know where to look. :)

>> So, my question is, what triggered the debugger to stop?
an overflow in a dynamically allocated memory block. :D

>> And if I resume and no errors, then what is the real problem?
something you do in your thread. could be many things. could be something you do with strings or even your gui. hard to guess without knowing the worker thread. are you using threadsafe? are you using mutexes to synchronise access to shared objects between the worker thread and main thread? how are you letting your main thread know that the worker thread finished? are you using KillThread()?

c ya,
nco2k
If OSVersion() = #PB_OS_Windows_ME : End : EndIf
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Ok, I'll keep hunting. :oops:

Yes, the app is compiled with these:

Code: Select all

; =================================================
; COMPILER OPTIONS:
;   [ ] Use Compiler:   PureBasic 5.61 (x64)
;   [x] Use Icon:       C:\myapp\my_16x16.ico
;   [ ] Enable inline ASM support
;   [x] Create threadsafe executable
;   [x] Enable OnError lines support
;   [x] Enable XP skin support
;   [ ] Request Administrator mode for Windows Vista
;   [ ] Request User mode for Windows Vista (no virtualization)
;   Library Subsystem:
;   Executable Format:  Windows  ;|Console|Shared DLL
;   CPU:                All      ;|Dynamic|w/MMX|w/3DNOW|w/SSE|w/SSE2
;   File Format:        UTF-8
nco2k wrote:>> Well, the ide debugger is stopping at WaitWindowEvent(10)?
it doesnt matter where it stopped. purebasic cant really precisely tell you where the overflow happened, when you are using threads. so dont rely on that line.
Surely some variable or start/end size is known so I could limit my search? I mean, when I look at my variable list, I see nothing wrong?
nco2k wrote:>> I can't really step through the code since the worker function is in another thread.
of course you can.
Like I said, this is a random error. Once I step forward, I am in the main loop, all is good. I have to put a breakpoint in the worker thread since it is cranking along.
nco2k wrote:>> I did not get this error when running that worker without threads?
then you know where to look. :)
True, I did add a shared structured array for the thread and main loop to process once it receives a postevent().

Code: Select all

Structure myData_INFO
  nthPt.l
  nthPtgui.l
  Array dat$(10000)       ; Pre-fill or ReDim later based on specific data
EndStructure
Global myD.myData_INFO
Although, my worker thread will never come close to 10,000 operations? I'll check this element with next random error.
nco2k wrote:>> So, my question is, what triggered the debugger to stop?
an overflow in a dynamically allocated memory block. :D
haha
nco2k wrote:>> And if I resume and no errors, then what is the real problem?
something you do in your thread. could be many things. could be something you do with strings or even your gui. hard to guess without knowing the worker thread. are you using threadsafe? are you using mutexes to synchronise access to shared objects between the worker thread and main thread? how are you letting your main thread know that the worker thread finished? are you using KillThread()?
The worker thread postevent()'s to the main loop and then immediately pausethread()'s. The main loop acts on the postevent to update a status line and then issues signalsemaphore() and resumethread(). KillThread()'s are only on exit.

I modeled the app after the code in this post using mk-soft's code.
I'm still in development, so I cannot go fully compiled, but I'm hoping this is not a false error induced by the IDE debugger? Already running the app for 4 hours without the error?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by mk-soft »

Code: Select all

Structure myData_INFO
  nthPt.l
  nthPtgui.l
  Array dat$(10000)       ; Pre-fill or ReDim later based on specific data
EndStructure
Global myD.myData_INFO
nthPt.l <- Index or Pointer ?

Perhaps forget to change this from Long (32bit) to Integer (64bit)
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Those are only used as the index to the array myd\dat$(), which is 0 to 10,000.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
Lunasole
Addict
Addict
Posts: 1091
Joined: Mon Oct 26, 2015 2:55 am
Location: UA
Contact:

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by Lunasole »

Seems to be caused by some thread. If I'm not wrong I once also had same thing with threads. It's really easy to shot own leg in multi-threading ^^
Nothing to do, try to disable different threads functional/interaction with global data (replace it with dummy code doing nothing, or completely exclude thread if it's possible) and see if that stops error.
Also sometimes enabling Purifier with PurifierGranularity(1, 1, 1, 1) can help, and surely threaded runtime is strongly recommended, without it it's easy to have some silent error with many default PB functions.
"W̷i̷s̷h̷i̷n̷g o̷n a s̷t̷a̷r"
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Yes, I had the Purifier enabled and it did not complain.
Given it is random, I'll have to debug in more detail on next occurrence.
Unfortunately, this app demands threading because the worker function is time consuming and I must update/respond to the gui while it's operating.
Still confused why the overflow is known to the debugger, but not its relation to a PB variable?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
Lunasole
Addict
Addict
Posts: 1091
Joined: Mon Oct 26, 2015 2:55 am
Location: UA
Contact:

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by Lunasole »

skywalk wrote: Still confused why the overflow is known to the debugger, but not its relation to a PB variable?
It's complicated, it may be not regular overflow of some your data - instead some corruption occurs on some PB function and exposes on WaitWindowEvent later. At least I don't know better explanation ^^
skywalk wrote: Unfortunately, this app demands threading because the worker function is time consuming and I must update/respond to the gui while it's operating.
Anyway you can try to move threaded code to separated DLLs for example (to isolate threads from main stuff and simplify it's debug).
"W̷i̷s̷h̷i̷n̷g o̷n a s̷t̷a̷r"
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by skywalk »

Ok, the Overflow error is gone with Purifier disabled?
But, with the Purifier on, I can continue the app after the error appears?
Why is that allowed if it does not say "warning"?
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
nco2k
Addict
Addict
Posts: 1344
Joined: Mon Sep 15, 2003 5:55 am

Re: Help troubleshoot "Overflow in a dynamically allocated m

Post by nco2k »

check your settings: Preferences -> Debugger -> [X] Kill Program after an Error.

the error is "gone" when the purifier is disabled, because of the padding. the allocated memory area is always multiple of x. so when you allocate 100 bytes, you dont just get 100 bytes, you get more. accessing that area may not cause a crash on one pc, but could on another. such bugs are very hard to track down, if you disable the purifier, since you have no way of knowing that you are doing something wrong. so you should take this error/warning very seriously.

take this code for example. if you disable the purifier, everything seems fine. but when you enable it, boom:

Code: Select all

*mem = AllocateMemory(1)
PokeB(*mem + 1, 1)
c ya,
nco2k
If OSVersion() = #PB_OS_Windows_ME : End : EndIf
Post Reply