Invalid memory access on DisableDebugger line when using very large numbers with CreateImage

Just starting out? Need help? Post your questions and find answers here.
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Invalid memory access on DisableDebugger line when using very large numbers with CreateImage

Post by Mistrel »

When using CreateImage() with an image number returned by #PB_Any, the debugger must be disabled to prevent execution from halting. This works fine in x86 versions of PureBasic but causes an invalid memory access with x64 versions. Tested as failing on both PureBasic 5.71 (x64) and 6.00 Alpha 4 (x64).

Code: Select all

image=CreateImage(#PB_Any,1,1)

DisableDebugger ; <- [ERROR] Invalid memory access
CreateImage(image,1,1)
It can also be triggered by explicitly passing a very large number:

Code: Select all

DisableDebugger ; <- [ERROR] Invalid memory access
CreateImage(23745977676672,1,1)
The reason for using very large numbers is a result of using the number provided by #PB_Any also being very large. I have a procedure which creates and returns an image number for use elsewhere and this number cannot change. It's faster to discard an existing image and create a new one with the same number than it is to erase it and also means I don't have to write any additional code to update any references elsewhere to this number or hard-code what it is to start with.
Last edited by Mistrel on Thu Sep 16, 2021 7:27 pm, edited 2 times in total.
User avatar
jacdelad
Addict
Addict
Posts: 1431
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by jacdelad »

AFAIK when using a fixed number, and not #PB_Any, you are limited to lower numbers (and that's why the returned number from #PB_Any are so high).
Anyway, I wouldn't consider mixing #PB_Any and fixed IDs a good programming style
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Mistrel »

jacdelad wrote: Thu Sep 16, 2021 4:08 pm AFAIK when using a fixed number, and not #PB_Any, you are limited to lower numbers (and that's why the returned number from #PB_Any are so high).
Anyway, I wouldn't consider mixing #PB_Any and fixed IDs a good programming style
That's not what the debugger is telling me; it's providing a warning in the event that this is a bug and can be explicitly disabled. The code will compile fine and also run fine on PureBasic compiled for 32-bit. The bug occurs when compiling for 64-bit and the invalid memory access does not provide any information as to the cause and is indicated on a line separate from where the number is provided.

This report indicates an issue with the debugger and I would hope will also fix what is normal, functional behavior with PureBasic x86.
jacdelad wrote: Thu Sep 16, 2021 4:08 pm Anyway, I wouldn't consider mixing #PB_Any and fixed IDs a good programming style
The only workaround is to either used fixed IDs or write a lot of extra code to allocate, track, and free my own list of dynamic IDs. I shouldn't have to do either of these things because #PB_Any does this for my already. Are either of those options a better programming style?

Also see the second example where the memory access occurs simply by providing a very large number. This has nothing to do with dynamic IDs at all.
#NULL
Addict
Addict
Posts: 1440
Joined: Thu Aug 30, 2007 11:54 pm
Location: right here

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by #NULL »

I don't think it's a bug. You should not reuse dynamic IDs where a new ID for a newly created PB Object is expected.
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Mistrel »

See my second example. I'm not reusing an ID.

Code: Select all

DisableDebugger ; <- [ERROR] Invalid memory access
CreateImage(23745977676672,1,1)
The first example describes a valid use case and works fine in PureBasic (x86).
#NULL wrote: Thu Sep 16, 2021 5:34 pm You should not reuse dynamic IDs..
There is no other way to do this without introducing performance penalties or adding a significant amount of workaround to reimplement #PB_Any.
#NULL wrote: Thu Sep 16, 2021 5:34 pm ..where a new ID for a newly created PB Object is expected.
You don't need to specify an unused ID when calling CreateImage().

The bug is caused by specifying a very large number when calling CreateImage(). Please see the title of the bug report and the example which does not use #PB_Any.
Last edited by Mistrel on Thu Sep 16, 2021 5:56 pm, edited 1 time in total.
#NULL
Addict
Addict
Posts: 1440
Joined: Thu Aug 30, 2007 11:54 pm
Location: right here

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by #NULL »

Large numbers, including PB_Any-generated dynamic IDs are unexpected input causing undefined behaviour, unless that's supposed to be different for CreateImage, Fred has to tell.
If you use dynamic stuff, then you have to update your datastructures and their dependencies when stuff changes or gets recreated, that's normal. Freeing the old object and get a new one with pb_any, how does that affect performance?
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Mistrel »

Mistrel wrote: Thu Sep 16, 2021 6:15 pm Large numbers, including PB_Any-generated dynamic IDs are unexpected input causing undefined behaviour..
I think you misunderstand. This is a bug. A very cursory look reveals even more problems.

PureBasic (x86):

Code: Select all

; [ERROR] #Image object number is very high (over 100000), are You sure of that ?
CreateImage(1000000000,1,1)

PureBasic (x64):

Code: Select all

; Works fine
CreateImage(1000000000,1,1)

Code: Select all

DisableDebugger  ; <- [ERROR] Invalid memory access
CreateImage(10000000000000,1,1)
The documentation does not say that I am only allowed to create 100000 images. It's the debugger warning me that it thinks there is an error because of the large number. This warning can be disabled because I want to specify a number larger than 100000 which is fine, normal, valid, and in no way undefined.

In PureBasic (x64) there is no warning. I can even use numbers that the debugger didn't like in the x86 version.
#NULL wrote: Thu Sep 16, 2021 5:56 pm If you use dynamic stuff, then you have to update your datastructures and their dependencies when stuff changes or gets recreated, that's normal.
No. It's not normal. It's completely unexpected and causes the program to crash and an error to highlight on a completely different line.
#NULL wrote: Thu Sep 16, 2021 5:56 pm Freeing the old object and get a new one with pb_any, how does that affect performance?
There is an example linked in the original post.

There is nothing in the documentation that tells me using large numbers cause undefined behavior, sometimes, depending on how you compile. I spent over an hour trying to locate this bug when recompiling for 64-bit code which worked fine previously because the debugger was crashing in a completely different location.

Please stop offering me advice and overanalyzing the problem. This is a bug.
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by mk-soft »

Unfortunately Fred did not tell us until when an ID (image, gadgets, etc) is interpreted as an array element and from when it is interpreted as a dynamic object.

As is known, dynamic object IDs cannot be overwritten, but can only be deleted with FreeXYZ and created again with #PB_Any.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Mistrel »

mk-soft wrote: Thu Sep 16, 2021 6:30 pm Unfortunately Fred did not tell us until when an ID (image, gadgets, etc) is interpreted as an array element and from when it is interpreted as a dynamic object.
What does this have to do with my bug report?

The debugger is accessing invalid memory in PureBasic (x64) but NOT PureBasic (x86).
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Mistrel »

I've reproduced this bug in PureBasic (x86). I thought it was immune but it's possible to force the invalid memory access:

Code: Select all

DisableDebugger  ; <- [ERROR] Invalid memory access
Define id=1

Repeat
  CreateImage(id,1,1)
  FreeImage(id)
  id*2
  
  ; This bug is not a result of a negative number
  If id<0
    id=1
  EndIf
ForEver
User avatar
jacdelad
Addict
Addict
Posts: 1431
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by jacdelad »

Maybe there's a bug, but there's a reason why nobody encountered that before. Even with huge projects, when do you need more than 10.000 objects of a kind? 100.000? That's why you can use #PB_Any. If you need a large or at coding unknown amount, that's the best choice. Otherwise proper enumerating is no problem.

I, for one, always use #PB_Any, unless it's a minimal example. But that's a personal choice.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
User avatar
Demivec
Addict
Addict
Posts: 4085
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: PB x64 - Invalid memory access when disabling debugger before using very large numbers with CreateImage

Post by Demivec »

help file wrote: Indexed numbering

The static, indexed way, allows you to reference an object by a predefined numeric value. The first available index number is 0 and subsequent indexes are allocated sequentially. This means that if you use the number 0 and then the number 1000, 1001 indexes will be allocated and 999 (from 1 to 999) will be unused, which is not an efficient way to use indexed objects. If you need a more flexible method, use the dynamic way of allocating objects, as described in section II. The indexed way offers several advantages:

- Easier handling, since no variables or arrays are required.
- 'Group' processing, without the need to use an intermediate array.
- Use the object in procedures without declaring anything in global (if using a constant or a number).
- An object that is associated with an index is automatically freed when reusing that index.
The maximum index number is limited to an upper bound, depending of the object type (usually from 5000 to 60000). Enumerations are strongly recommended if you plan to use sequential constants to identify your objects (which is also recommended).
Looking at the help file you are running out of memory because you are using static indexing when creating your images. When a high index number is used PureBasic had to reserve all of the indexes from zero to that index number (i.e 23745977676672 as in your example code). I think you could predict that would cause problems and that it is not a bug and this behavior is documented in the help file.


help file wrote:II. Dynamic numbering

Sometimes, indexed numbering isn't very handy to handle dynamic situations where we need to deal with an unknown number of objects. PureBasic provides an easy and complementary way to create objects in a dynamic manner. Both methods (indexed and dynamic) can be used together at the same time without any conflict. To create a dynamic object, you just have to specify the #PB_Any constant instead of the indexed number, and the dynamic number will be returned as result of the function. Then just use this number with the other object functions in the place where you would use an indexed number (except to create a new object). This way of object handling can be very useful when used in combination with a list, which is also a dynamic way of storage.
Numbers returned by using #PB_Any are not meant to be reused for object creation a second time. If they were then it would be static indexing that would be used and the problem of wasted indexes would occur as already mentioned.

In either case you are causing your own problems by using improper indexing with both dynamic and static indexes.
Last edited by Demivec on Thu Sep 16, 2021 7:41 pm, edited 1 time in total.
User avatar
jacdelad
Addict
Addict
Posts: 1431
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: Invalid memory access on DisableDebugger line when using very large numbers with CreateImage

Post by jacdelad »

Case closed.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: Invalid memory access on DisableDebugger line when using very large numbers with CreateImage

Post by Mistrel »

jacdelad wrote: Thu Sep 16, 2021 7:40 pm Case closed.
No, it's not. What does the title of the topic say?

Thanks, Demivec. This was very helpful and informative. I was looking at "Handles and Numbers" in the documentation and didn't see the page you referenced.

This worked without issue in PureBasic (x86) but this may have been luck.
Post Reply