Page 1 of 1

[Implemented] Aligned structures

Posted: Sun Jun 07, 2009 9:34 am
by mback2k
It would be great if PB had a new built-in keyword for C-aligned structures, e.g.:

Code: Select all

Structure Test
  a.b ; 1 byte
  b.l ; 4 byte at offset 1
  c.w ; 2 byte at offset 5
  d.q ; 8 byte at offset 7
  e.l ; 4 byte at offset 15
EndStructure

StructureC Test
  a.b ; 1 byte
      ; 3 byte padding
  b.l ; 4 byte at offset 4 (= 4*1)
  c.w ; 2 byte at offset 8
      ; 6 byte padding
  d.q ; 8 byte at offset 16 (= 8*2)
  e.l ; 4 byte at offset 24
      ; 4 byte padding to fill the structure to size 32 (= 8*4)
EndStructureC
This would make the work with x64 structures a lot easier.

More information: http://en.wikipedia.org/wiki/Data_struc ... cts_on_x86

Posted: Sun Jun 07, 2009 9:56 am
by mback2k
Example for annoying x64 padding:

Code: Select all

Enumeration
  #MAX_ADAPTER_ADDRESS_LENGTH = 8
  #MAX_ADAPTER_DESCRIPTION_LENGTH = 128
  #MAX_ADAPTER_NAME = 128
  #MAX_ADAPTER_NAME_LENGTH = 256
EndEnumeration

Structure IP_ADDR_STRING
  *pNext
  IpAddress.b[16]
  IpMask.b[16]
  Context.l
EndStructure

Structure IP_ADAPTER_INFO
  *pNext
  ComboIndex.l
  AdapterName.b[#MAX_ADAPTER_NAME_LENGTH+4]
  Description.b[#MAX_ADAPTER_DESCRIPTION_LENGTH+4]
  AddressLength.l
  Address.b[#MAX_ADAPTER_ADDRESS_LENGTH]
  Index.l
  Type.l
  DhcpEnabled.l
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  *CurrentIpAddress.IP_ADDR_STRING
  IpAddressList.IP_ADDR_STRING
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[4]
  CompilerEndIf
  GatewayList.IP_ADDR_STRING
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment3.b[4]
  CompilerEndIf
  DhcpServer.IP_ADDR_STRING
  HaveWins.l
  PrimaryWinsServer.IP_ADDR_STRING
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment4.b[4]
  CompilerEndIf
  SecondaryWinsServer.IP_ADDR_STRING
  LeaseObtained.l
  LeaseExpires.l
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment5.b[4]
  CompilerEndIf
EndStructure
The paddings are required because of the substructures, so this should also be supported by StructureC.

And another weird case:

Code: Select all

Structure IP4_ARRAY
  AddrCount.l
  AddrArray.l[1]
EndStructure

Structure IP6_ADDRESS
  IP6Dword.l[4]
EndStructure

Structure DNS_A_DATA
  IpAddress.l
EndStructure

Structure DNS_PTR_DATA
  *pNameHost
EndStructure

Structure DNS_TXT_DATA
  dwStringCount.l
  *pStringArray[0]
EndStructure

Structure DNS_AAAA_DATA
  Ip6Address.IP6_ADDRESS
EndStructure

Structure DNS_RECORD
  *pNext
  *pName
  wType.w
  wDataLength.w
  StructureUnion
    DW.l
    S.i
  EndStructureUnion
  dwTtl.l
  dwReserved.l
  StructureUnion
    A.DNS_A_DATA
    PTR.DNS_PTR_DATA
    CNAME.DNS_PTR_DATA
    TXT.DNS_TXT_DATA
    AAAA.DNS_AAAA_DATA
  EndStructureUnion
EndStructure
How would you align the StructureUnion substructures here?

Re: Aligned structures

Posted: Tue Jun 01, 2010 11:38 pm
by Josh
i searched for the idea StructureC and found this topic. would be fine and it would make many things easier

Re: Aligned structures

Posted: Sat Jun 05, 2010 3:25 am
by kenmo
I'm not 100% sure what you guys want...

but if I understand correctly, you can accomplish it with macros (like nearly everything it seems).

Code: Select all

Macro pad(number,bytes)
  _#number#.s{bytes}
EndMacro

Structure Test
  a.b       ; 1 byte
  pad(1,3)  ; 3 byte padding
  b.l       ; 4 byte at offset 4 (= 4*1)
  c.w       ; 2 byte at offset 8
  pad(2,6)  ; 6 byte padding
  d.q       ; 8 byte at offset 16 (= 8*2)
  e.l       ; 4 byte at offset 24
  pad(3,4)  ; 4 byte padding to fill the structure to size 32 (= 8*4)
EndStructure

Debug "Structure Size: " + Str(SizeOf(Test))
Debug ""
Debug "Offset of a: " + Str(OffsetOf(Test\a))
Debug "Offset of b: " + Str(OffsetOf(Test\b))
Debug "Offset of c: " + Str(OffsetOf(Test\c))
Debug "Offset of d: " + Str(OffsetOf(Test\d))
Debug "Offset of e: " + Str(OffsetOf(Test\e))
EDIT - oh, I think you guys want the compiler to pad certain structures automatically.... I see.

Re: Aligned structures

Posted: Sat Jun 05, 2010 5:19 am
by Josh
@kenmo
it's not so simple, that you can use a macro

this structure you can use with c-structures

Code: Select all

Structure Test
  a.l
  b.w
  c.w
  d.l
Structure
in this case, you have to pad with an additional w

Code: Select all

Structure Test
  a.l
  b.w
  d.l
Structure
kenmo wrote:EDIT - oh, I think you guys want the compiler to pad certain structures automatically.... I see.
the compiler shouldn't fill in any padding variables, it should be use the correct offsets and sizes of structures to put and get structured data, written in other languages then pb.

i think many pb-users already failed, because they didn't know, that they can't simply use pb-structures for exchanging datas with other applications

Re: Aligned structures

Posted: Wed Dec 01, 2010 10:01 pm
by inSANE
Any comments on this from the PB-Team?

Maybe you plan to implement an "auto-padding" version of the Structure command?

Re: Aligned structures

Posted: Fri Sep 14, 2012 3:10 pm
by BorisTheOld
I believe that any sort of auto-padding of structures is a bad idea.

Firstly, vendors of hardware and software all have their own ideas about this topic, so catering to one "standard" could create a nightmare for PB programmers.

Secondly, perhaps auto-padding is not appropriate for the situation. Having an undefined and undocumented field in a structure creates the potential for problems.

Thirdly, I believe that programmers should be more attuned to what's happening in their code. Therefore, they should be the ones adding any required padding to their structures. And they should document why they are doing it.

Re: Aligned structures

Posted: Tue Oct 02, 2012 6:11 pm
by Thorium
BorisTheOld wrote:I believe that any sort of auto-padding of structures is a bad idea.

Firstly, vendors of hardware and software all have their own ideas about this topic, so catering to one "standard" could create a nightmare for PB programmers.

Secondly, perhaps auto-padding is not appropriate for the situation. Having an undefined and undocumented field in a structure creates the potential for problems.

Thirdly, I believe that programmers should be more attuned to what's happening in their code. Therefore, they should be the ones adding any required padding to their structures. And they should document why they are doing it.
On x64 all WinAPI structures are aligned automaticly. Thats microsofts (VC++) standard. If you just copy a structure from MSDN it can lead to crashes in PB, because PB does not align the structure members.
There may be some software that still dont allign structure members, even if it's adviced to do so by Intel and AMD. (Variables should be aligned to a memory address dividable by the length of it's type. word = 2, long = 4, quad = 8 bytes)

To keep compatiblity with software interfaces that dont follow that rule there could be a keyword to create unaligned structures.

Re: Aligned structures

Posted: Tue Oct 02, 2012 6:44 pm
by Josh
Thorium wrote:To keep compatiblity with software interfaces that dont follow that rule there could be a keyword to create unaligned structures.
To keep backward compatibility to old PB code it would be better, that 'Structure' is what it is. If wished an aligned structure, there should be a other keyword, like StructureC or something like this.

Re: Aligned structures

Posted: Tue Oct 02, 2012 8:06 pm
by BorisTheOld
I agree - the default should be "no alignment".

PB is a cross-platform language, therefore it should not default to the quirks of a particular platform or programming language.

And remember, not everyone interfaces with MS code. There are lots of other "standards".

Re: Aligned structures

Posted: Wed Oct 03, 2012 1:54 am
by leonhardt
Do you mean the record field allignment? that's the default compiler settings for both Delphi or VB,
type TRec=record
a:byte;
b:integer;
end;
then sizeof(TRec)=8,not 5,it would be greate if PB suport this as it will speed apps up,though you can do it manually.

Code: Select all

structure TRec
a.b
Empty1.w
Empty2.b
b.i

Re: Aligned structures

Posted: Wed Oct 03, 2012 5:10 pm
by BorisTheOld
leonhardt wrote:...it would be greate if PB suport this as it will speed apps up,though you can do it manually.
Support it, yes, but do not make it the default.

Re: Aligned structures

Posted: Tue Oct 16, 2012 9:35 pm
by Thorium
BorisTheOld wrote: And remember, not everyone interfaces with MS code. There are lots of other "standards".
It's not realy MS standard. It's Intel and AMD's standard and MS is following it. There are not a lot of standards if it comes to structure member alignment. You align or you dont align, thats it.