Limits for DataSection?

Just starting out? Need help? Post your questions and find answers here.
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Limits for DataSection?

Post by Lord »

Are there any limits stated in regards to a DataSection?

I only found here a statement that an Exe can only have 64MB:
http://www.purebasic.fr/english/viewtop ... tion+limit

Is this still true? Does this include the count for a DataSection?

I tried to generate an Exe with a large Datasection and hit a 128MB limit:

Code: Select all

DataSection

  IncludePath "...Use your path here.."

  Block1:
  IncludeBinary "64MB.Block"
  Block2:
  IncludeBinary "32MB.Block"
  Block3:
  IncludeBinary "16MB.Block"
  Block4:
  IncludeBinary "08MB.Block"
  Block5:
  IncludeBinary "04MB.Block"
  Block6:
  IncludeBinary "02MB.Block"
  Block7:
  IncludeBinary "01MB.Block"
  Block8:
  IncludeBinary "01MB.Block"
EndDataSection
I used this code to generate suiteable blocks of data:

Code: Select all

Pfad.s="...Use your path here.."

For Block=0 To 6
  Size.i=(1<<Block)*1024*1024
  *mem=AllocateMemory(Size)
  If *mem
    Datei.s=Pfad+RSet(Str(1<<Block), 2, "0")+"MB.Block"
    If CreateFile(1, Datei)
      Anz.i=WriteData(1, *mem,Size)
      If Anz
        Debug Anz
      Else
        Debug "Not written"
      EndIf
      CloseFile(1)
    Else
      Debug "File not reated"
    EndIf
    FreeMemory(*mem)
  Else
    Debug "Memory not allocated"
  EndIf
Next
If I compile this code above I get an Polink Error.
If I comment out the IncludeBinary at Block8 the
Polink Error doesn't show up.

How can I circumvent this limit of 128MB and generate an
Exe with a larger DataSection?
Image
User avatar
ts-soft
Always Here
Always Here
Posts: 5756
Joined: Thu Jun 24, 2004 2:44 pm
Location: Berlin - Germany

Re: Limits for DataSection?

Post by ts-soft »

Use a windows resource like RCDATA
PureBasic 5.73 | SpiderBasic 2.30 | Windows 10 Pro (x64) | Linux Mint 20.1 (x64)
Old bugs good, new bugs bad! Updates are evil: might fix old bugs and introduce no new ones.
Image
User_Russian
Addict
Addict
Posts: 1516
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Limits for DataSection?

Post by User_Russian »

ts-soft wrote:Use a windows resource like RCDATA
:shock: :shock:
PB stores data in section ".data".

Code: Select all

section '.data' data readable writeable
l_x:
file "file.txt"
l_y:
SYS_EndDataSection:
User avatar
ts-soft
Always Here
Always Here
Posts: 5756
Joined: Thu Jun 24, 2004 2:44 pm
Location: Berlin - Germany

Re: Limits for DataSection?

Post by ts-soft »

This is not the same. win32 supports a resource-section for icons, strings and so on. You can add any datas to this
section. This section is not automatic loaded with the executable.

You have to write a rc (Resource Script) and add to your exe. For using you can use a code like this:

Code: Select all

Procedure GetResource(ResNr.l, ResName.s, ResType.l, hModule.i = 0, *ResSize.long = 0)
  Protected hFind.i, hLoad.i
  If Not hModule  : hModule = GetModuleHandle_(0) : EndIf
  If ResNr : ResName = "#" + Str(ResNr) : EndIf
  hFind = FindResource_(hModule, ResName,ResType)
  If hFind
    hLoad = LoadResource_(hModule, hFind)
    If *ResSize
      *ResSize\l = SizeofResource_(hModule, hFind)
    EndIf
    ProcedureReturn LockResource_(hLoad)
  EndIf
EndProcedure
PureBasic 5.73 | SpiderBasic 2.30 | Windows 10 Pro (x64) | Linux Mint 20.1 (x64)
Old bugs good, new bugs bad! Updates are evil: might fix old bugs and introduce no new ones.
Image
User avatar
ts-soft
Always Here
Always Here
Posts: 5756
Joined: Thu Jun 24, 2004 2:44 pm
Location: Berlin - Germany

Re: Limits for DataSection?

Post by ts-soft »

Here is a module to create a *.res file
(to import like a lib) and how to use.
PureBasic 5.73 | SpiderBasic 2.30 | Windows 10 Pro (x64) | Linux Mint 20.1 (x64)
Old bugs good, new bugs bad! Updates are evil: might fix old bugs and introduce no new ones.
Image
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Ok, Ok, back to the roots.

So there is no way to have more than 127MB in a DataSection, right?

As I'm not familiar with resources, res-files and rc-files I tried this for
the above created files in a "Test.rc":
Test.rc wrote:IMAGE RCDATA "64MB.Block"
IMAGE RCDATA "32MB.Block"
IMAGE RCDATA "16MB.Block"
I made the entry in compiler options and run it with a dummy program:

Code: Select all

For i=1 To 1000
; do nothing  
Next
It compiled without any complains.
So far so good.

But how do I access these data blocks?
I tried your above posted Procedure "GetResource()":

Code: Select all

Procedure GetResource(ResNr.l, ResName.s, ResType.l, hModule.i = 0, *ResSize.long = 0)
  Protected hFind.i, hLoad.i
  If Not hModule  : hModule = GetModuleHandle_(0) : EndIf
  If ResNr : ResName = "#" + Str(ResNr) : EndIf
  hFind = FindResource_(hModule, ResName,ResType)
  If hFind
    hLoad = LoadResource_(hModule, hFind)
    If *ResSize
      *ResSize\l = SizeofResource_(hModule, hFind)
    EndIf
    ProcedureReturn LockResource_(hLoad)
  EndIf
EndProcedure

Debug GetResource(0, "64MB.Block", #RT_RCDATA)
But I always get a 0 as result. I changed resNr to 1 with no effect.
It doesn't seem to bee the right way and it doesnt matter if I use
real image data instead of a *.Block.
So the question stands: How do I access a "Block" of this resource?
How has a working Test.rc to be?
Image
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Me again. :shock:

The good news: I found out that my Test.rc is wrong.
It schould read
Test.rc wrote:IMAGE01 RCDATA "64MB.Block"
IMAGE02 RCDATA "32MB.Block"
IMAGE03 RCDATA "16MB.Block"
The bad news is, that when I use this
Test.rc wrote:IMAGE01 RCDATA "64MB.Block"
IMAGE02 RCDATA "32MB.Block"
IMAGE03 RCDATA "32MB.Block"
there is this ugly message again:
PB wrote:---------------------------
PureBasic - Linker error
---------------------------
POLINK: fatal error: Internal error: write_executable_image.


---------------------------
OK
---------------------------
Does this mean that there ist also a 128MB limit for resources?
Is this a Windows limit or is it a PureBasic limit?
Image
User avatar
Droopy
Enthusiast
Enthusiast
Posts: 658
Joined: Thu Sep 16, 2004 9:50 pm
Location: France
Contact:

Re: Limits for DataSection?

Post by Droopy »

it's a compiler limit, i use updateresource_ function or reshacker to add resource bigger than 128mb
Thorium
Addict
Addict
Posts: 1305
Joined: Sat Aug 15, 2009 6:59 pm

Re: Limits for DataSection?

Post by Thorium »

Yes, it's a linker limit.
Sections in a PE file (.exe) can be theoreticly up to 4GB in size. On 32bit Windows the practical limit is lower but still way over 128MB.
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Hi Droopy!
Droopy wrote:it's a compiler limit, i use updateresource_ function or reshacker to add resource bigger than 128mb
That is a very valuable reply!
A first test with Resource Hacker yielded a good result.
I already could include data bigger than 160 MB.
Thanks for this information.
Image
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Hi Thorium!
Thorium wrote:Yes, it's a linker limit.
Sections in a PE file (.exe) can be theoreticly up to 4GB in size. On 32bit Windows the practical limit is lower but still way over 128MB.
Thank you for this verification.
Limits like this should be mentioned somewhere
in the Help file or, if possible, lifted or raised.
Image
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Hi!

Another question on this topic:
How many items can be attached with a Resource file?
Are there any limits?
Image
User avatar
ts-soft
Always Here
Always Here
Posts: 5756
Joined: Thu Jun 24, 2004 2:44 pm
Location: Berlin - Germany

Re: Limits for DataSection?

Post by ts-soft »

I think, 32767 items.
PureBasic 5.73 | SpiderBasic 2.30 | Windows 10 Pro (x64) | Linux Mint 20.1 (x64)
Old bugs good, new bugs bad! Updates are evil: might fix old bugs and introduce no new ones.
Image
User avatar
Lord
Addict
Addict
Posts: 900
Joined: Tue May 26, 2009 2:11 pm

Re: Limits for DataSection?

Post by Lord »

Hi ts-soft!
ts-soft wrote:I think, 32767 items.
Thank you for this info.

Another question:
Is your provided Procedure GetResource() bulletproof?
Do I have to "free" or "unlock" something after calling
the procedure?
Image
Post Reply