LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Keya · Post by **Keya** » Thu Jul 21, 2016 11:50 am

LibUnsigned - Provides unsigned operations for 32bit dword and 64bit quad.
All OS supported - Windows, OSX, Linux. x86 + x64. (Ascii+Unicode but not relevant).
Public domain. No warranty. Use at your own risk. I'm not liable. No credits requested.
Static libs pre-compiled for all OS, and full source provided.
---

WHY? I really need high confidence in operations with unsigned 32bit and 64bit integers. Unfortunately Purebasic doesn't support them yet (it has 8bit and 16bit unsigned support but not 32/64), i don't really trust myself with bit hacks or things like knowing where or where not to throw in "& $FFFFFFFF" here and there, not all existing solutions are cross-OS, and tricks like using Quad instead of Long for 32bit unsigned ops don't extend to 64bit unsigned ops, which had me especially stumped today! But unsigned integers aren't going away and they must be operated on properly or it's Miscalculation City if using signed ops on them.

SO! I ended up making a little C library to provide those operators, which to my surprise turned out very straightforward as there aren't many operators and they're all so fundamentally simple themselves! Some that aren't required are provided as macros for set completeness. All operators are provided/supported (Arithmetic, Logical, Relational, Bitwise, and even a few Misc).

Highly modular by design, each function in the .a library is its own .o object file, so only the functions you use get included in your executable! (and most functions are tiny!) And even if you use every function you're still only adding ~3kb, so it's very little overhead, but each operator is essentially a Call to a small Procedure, so it's slightly less efficient than if PB supported it inline natively, but nonetheless the trust/confidence in the calculations/operations should be high and the libs are compiled with gcc's highest optimization. Hopefully a future version of PB will make this lib redundant, but until then...!

USAGE is very simple - just IncludeFile the PBI, then all the u32/u64 functions from the lib are available (and again, only the functions you use (and actually use - not just declare) get compiled into your executable).
Simply use PB's .l long variables for 32bit unsigned dwords, and .q quad for 64bit unsigned quads. (The lib doesn't automatically change or cast all variables to all-unsigned, no... we simply use .l and .q as the placeholders, and you're then free to use both signed and/or unsigned ops on them).
There's a Demo.pb included in the zip for a quick intro, as well as TestAll32.pb and TestAll64.pb for more thorough tests.

Each C procedure is essentially nothing more than the single operation, eg:

Code: Select all

int u64isgreater (unsigned long long int quad1, unsigned long long int quad2) {
 if (quad1 > quad2) { return 1; } else { return 0; }
}

When all your u32/u64 calculations, comparisons etc are done PB's native StrU(var, #PB_Long/#PB_Quad) can be used to display the unsigned value.

DOWNLOAD: (v1.2) http://www.mediafire.com/download/wlvns ... d-v1.2.zip
(if that doesn't exist try this Backup thanks to RSBasic!)
__________________________________________________________________________

LIBUNSIGNED.PBI (excerpt of the main functions)

Code: Select all

ImportC "libunsigned.a"
  ;### ARITHMETIC: returns calculation result
  ;### 32bit DWORD
  Macro u32add (dword1, dword2): (dword1 + dword2) : EndMacro
  u32addq.q (dword1.l, dword2.l) As #prefix$+"u32addq"
  Macro u32sub (dword1, dword2): (dword1 - dword2) : EndMacro
  u32subq.q (dword1.l, dword2.l) As #prefix$+"u32subq"
  u32div.l (dword1.l, dword2.l) As #prefix$+"u32div"
  u32divq.q (dword1.l, dword2.l) As #prefix$+"u32divq"
  u32mod.l (dword1.l, dword2.l) As #prefix$+"u32mod"
  Macro u32mul (dword1, dword2): (dword1 * dword2) : EndMacro
  u32mulq.q (dword1.l, dword2.l) As #prefix$+"u32mulq"
  ;### 64bit QUAD
  Macro u64add (uquad1, uquad2): (uquad1 + uquad2) : EndMacro
  Macro u64sub (uquad1, uquad2): (uquad1 - uquad2) : EndMacro
  u64div.q (uquad1.q, uquad2.q) As #prefix$+"u64div"
  u64divd.d (uquad1.q, uquad2.q) As #prefix$+"u64divd"
  u64mod.q (uquad1.q, uquad2.q) As #prefix$+"u64mod"
  Macro u64mul (uquad1, uquad2): (uquad1 * uquad2) : EndMacro
  
  ;### RELATIONAL: returns 0 false or 1 true
  ;### 32bit DWORD
  u32isgreater.i (dword1.l, dword2.l) As #prefix$+"u32isgreater"
  u32isgreaterorequal.i (dword1.l, dword2.l) As #prefix$+"u32isgreaterorequal"
  u32isless.i (dword1.l, dword2.l) As #prefix$+"u32isless"
  u32islessorequal.i (dword1.l, dword2.l) As #prefix$+"u32islessorequal"
  Macro u32isequal (dword1, dword2): (dword1 = dword2) : EndMacro
  Macro u32notequal (dword1, dword2): Not (dword1 = dword2) : EndMacro
  ;### 64bit QUAD
  u64isgreater.i (uquad1.q, uquad2.q) As #prefix$+"u64isgreater"
  u64isgreaterorequal.i (uquad1.q, uquad2.q) As #prefix$+"u64isgreaterorequal"
  u64isless.i (uquad1.q, uquad2.q) As #prefix$+"u64isless"
  u64islessorequal.i (uquad1.q, uquad2.q) As #prefix$+"u64islessorequal"
  Macro u64isequal (uquad1, uquad2): (uquad1 = uquad2) : EndMacro
  Macro u64notequal (uquad1, uquad2): Not (uquad1 = uquad2) : EndMacro 
  
  ;### BITWISE: returns calculation result
  ;### 32bit DWORD
  u32shr.l (dword1.l, shiftbytes.l) As #prefix$+"u32shr"
  Macro u32shl (dword1, shiftbytes): (dword1 << shiftbytes) : EndMacro
  Macro u32and (dword1, dword2): (dword1 & dword2) : EndMacro
  Macro u32or (dword1, dword2): (dword1 | dword2) : EndMacro
  Macro u32xor (dword1, dword2): (dword1 ! dword2) : EndMacro
  Macro u32not (dword1): (~dword1) : EndMacro
  ;### 64bit QUAD
  u64shr.q (uquad1.q, shiftbytes.l) As #prefix$+"u64shr"
  Macro u64shl (uquad1, shiftbytes): (uquad1 << shiftbytes) : EndMacro
  Macro u64and (uquad1, uquad2): (uquad1 & uquad2) : EndMacro
  Macro u64or (uquad1, uquad2): (uquad1 | uquad2) : EndMacro
  Macro u64xor (uquad1, uquad2): (uquad1 ! uquad2) : EndMacro
  Macro u64not (uquad1): (~uquad1) : EndMacro
  
  ;### LOGICAL: returns 0 false or 1 true. (no Logical functions are required as PB's functions handle them fine)
  ;### 32bit DWORD
  Macro u32_And (dword1, dword2): (dword1 And dword2) : EndMacro
  Macro u32_Or (dword1, dword2): (dword1 Or dword2) : EndMacro
  Macro u32_Not (dword1): Not (dword1) : EndMacro
  ;### 64bit QUAD
  Macro u64_And (uquad1, uquad2): (uquad1 And uquad2) : EndMacro
  Macro u64_Or (uquad1, uquad2): (uquad1 Or uquad2) : EndMacro
  Macro u64_Not (uquad1): Not (uquad1) : EndMacro
  
  ;### TYPECAST
  ;### 32bit DWORD
  u32tofloat.f (dword1.l) As #prefix$+"u32tofloat"
  u32todouble.d (dword1.l) As #prefix$+"u32todouble"
  u32fromfloat.l (float1.f) As #prefix$+"u32fromfloat"
  u64fromu32.q (dword1.l) As #prefix$+"u64fromu32"
  ;### 64bit QUAD
  u64todouble.d (uquad1.q) As #prefix$+"u64todouble"
  u64fromdouble.q (double1.d) As #prefix$+"u64fromdouble"
    
  ;### MISC (Non-fundamental) #################################################
  ;Min/Max
  u32min.l (dword1.l, dword2.l) As #prefix$+"u32min"
  u32max.l (dword1.l, dword2.l) As #prefix$+"u32max"
  u64min.q (uquad1.q, uquad2.q) As #prefix$+"u64min"
  u64max.q (uquad1.q, uquad2.q) As #prefix$+"u64max"
  ;MinOf3/MaxOf3
  u32minof3.l (dword1.l, dword2.l, dword3.l) As #prefix$+"u32minof3"
  u32maxof3.l (dword1.l, dword2.l, dword3.l) As #prefix$+"u32maxof3"  
  u64minof3.q (uquad1.q, uquad2.q, uquad3.q) As #prefix$+"u64minof3"
  u64maxof3.q (uquad1.q, uquad2.q, uquad3.q) As #prefix$+"u64maxof3"
  ;Mean
  u32mean.l (dword1.l, dword2.l) As #prefix$+"u32mean"
  u32meanf.f (dword1.l, dword2.l) As #prefix$+"u32meanf"
  u64mean.q (uquad1.q, uquad2.q) As #prefix$+"u64mean"
  u64meand.d (uquad1.q, uquad2.q) As #prefix$+"u64meand"
  ;Pow
  u32pow.l (dwbase.l, dwexponent.l) As #prefix$+"u32pow"
  u32powq.q (dwbase.l, dwexponent.l) As #prefix$+"u32powq"
  u32fpow.f (dwbase.l, fexponent.f) As #prefix$+"u32fpow"
  u32fpowq.d (dwbase.l, fexponent.f) As #prefix$+"u32fpowq"  
  u64pow.q (qbase.q, qexponent.q) As #prefix$+"u64pow"
  u64fpow.d (qbase.q, dexponent.d) As #prefix$+"u64fpow"
  ;IsPow2
  u32ispow2.i (dword1.l) As #prefix$+"u32ispow2"
  u64ispow2.i (uquad1.q) As #prefix$+"u64ispow2"

Keya · Post by **Keya** » Thu Jul 21, 2016 11:57 am

[deleted] it's no longer practical to post the C source of the library now it's modular, please see /src/ in the zip

But here's a simple 64bit quad demo, first showing a signed comparison using the native PB operators, then an unsigned version:

Code: Select all

XIncludeFile("LibUnsigned.PBI")

quad1.q = $B000000000000000    ;  -5764607523034234880 signed, 12682136550675316736 unsigned
quad2.q = $4000000000000000    ;   4611686018427387904 signed,  4611686018427387904 unsigned

;64bit SIGNED QUAD   (native PB function)
If quad1 > quad2   ;// If -5764607523034234880 > 4611686018427387904 (which it isn't)
  Debug "Yes, signed quad " + Str(quad1) + " is > " + Str(quad2) + " = WRONG"
Else
  Debug "No, signed quad " + Str(quad1) + " isnt > " + Str(quad2) + " = CORRECT"
EndIf

;64bit UNSIGNED QUAD  (only possible with LibUnsigned function)
If u64isgreater(quad1, quad2)    ;// If 12,682136550675316736 > 4,611686018427387904 (which it is)
  Debug "Yes, unsigned quad " + StrU(quad1,#PB_Quad) + " is > " + StrU(quad2,#PB_Quad) + " = CORRECT"
Else
  Debug "No, unsigned quad " + StrU(quad1,#PB_Quad) + " isnt > " StrU(quad2,#PB_Quad) + " = WRONG"
EndIf

wilbert · Post by **wilbert** » Thu Jul 21, 2016 5:34 pm

Keya wrote:I'm not 100% sure at this stage, but I think the following can be removed as they seem to be sign-irrelevant (so native PB operations can be used):
Logical: And, Or, Not
Bitwise: And, Or, Xor, Not
I'm also still not 100% sure if add/sub are needed...
So, the only ones that are sign-sensitive (and therefore needed) would be:
Arithmetic: Div, Mod, Mul (add? sub?)
Relational: >, >=, <, <=
Logical: (none)
Bitwise: Shift-left, Shift-right
anyone know?

As far as I know, if the output variable is the same number of bits as the input, the sign should also be irrelevant for
- shift left
- multiply

Lunasole · Post by **Lunasole** » Thu Jul 21, 2016 5:47 pm

Pretty simple but really nicely done ^^
Just should be better to uncomment equal/not equal too, to have "full set". I did it building testing lib for myself.

Btw don't know what prevented Freak of Fred to add same wrappers to a math lib, for example, at least as temporary solution. That would resolve UINT lack, as they not needed often and it is not a big problem to use wrappers for calculations

Keya · Post by **Keya** » Thu Jul 21, 2016 10:40 pm

wilbert wrote:As far as I know, if the output variable is the same number of bits as the input, the sign should also be irrelevant for
- shift left
- multiply

just in regards to multiply i thought sign would be relevant, seeing as there's mul and imul instructs?
I did some tests with shift left and shift right, and my tests agree with you that shift left doesnt matter but shift right does. This doc also seems to confirm that:

The shift arithmetic left (SAL) and shift logical left (SHL) instructions perform the same operation; they shift the bits in the destination operand to the left (toward more significant bit locations).

The shift arithmetic right (SAR) and shift logical right (SHR) instructions shift the bits of the destination operand to the right (toward less significant bit locations). For each shift count, the least significant bit of the destination operand is shifted into the CF flag, and the most significant bit is either set or cleared depending on the instruction type. The SHR instruction clears the most significant bit (see Figure 7-8 in the IA-32 Intel Architecture Software Developer's Manual, Volume 1); the SAR instruction sets or clears the most significant bit to correspond to the sign (most significant bit) of the original value in the destination operand. In effect, the SAR instruction fills the empty bit position's shifted value with the sign of the unshifted value (see Figure 7-9 in the IA-32 Intel Architecture Software Developer's Manual, Volume 1).

Lunasole wrote:Just should be better to uncomment equal/not equal too, to have "full set".

Or i thought about using Macros for full set completeness, that way they both visually tell the user where native PB ops are suitable (no need for lib version) without adding any extra code, yet if they are used (as a way to keep the calls uniform) then being macros they simply revert back to native PB ops (so no speed loss either)? something along the lines of this for the Bitwise ones for example:

Code: Select all

    ;### BITWISE: returns calculation result
    u32shl.l (dword1.l, shiftbytes.l) As #APIPrefix$+"u32shl"
    u32shr.l (dword1.l, shiftbytes.l) As #APIPrefix$+"u32shr"
    Macro u32andbits(dword1, dword2): (dword1 & dword2) : EndMacro
    Macro u32orbits(dword1, dword2): (dword1 | dword2) : EndMacro
    Macro u32xorbits(dword1, dword2): (dword1 ! dword2) : EndMacro
    Macro u32notbits(dword1): ~(dword1) :  EndMacro

I'll upload v1.1 sometime in the next 24h

wilbert · Post by **wilbert** » Fri Jul 22, 2016 6:37 am

Keya wrote:just in regards to multiply i thought sign would be relevant, seeing as there's mul and imul instructs?

The difference is in the upper half of the result.
If you multiply two 32 bit values, the upper 32 bits from the 64 bit multiplication result are different for signed versus unsigned.
The same for multiplying two 64 bit values, the upper 64 bits from the 128 bit multiplication result are different for signed versus unsigned.
Since you are discarding the upper half in your procedure, there is no difference in the result.

Keya · Post by **Keya** » Fri Jul 22, 2016 5:14 pm

v1.12 is uploaded, please see first post for updated code and download link!

- ive merged them into a single PBI include for easier use, while still keeping the choice of 32 and/or 64 as compiler options
- added Macros for set completeness of functions where the lib isn't required
- commented-out all unrequired functions (they exist as Macros)
- added functions that return 64bit as a result of operations on two 32bit dwords, which are both cast as 64bit
- added two comprehensive test programs
- changed differentiation of logical vs bitwise ops... eg. logical: u32_And, bitwise: u32and (instead of previous u32andbits)

I uploaded v1.1 but quickly found a minor but unexpected difference between Linux and Windows gcc, which was resolved simply by some casting, so v1.12 quickly replaced it. Tested fine on my Windows, Linux and OSX. I think everything seems done now, so it just needs some testing and peer review!

Tenaja · Post by **Tenaja** » Fri Jul 22, 2016 9:12 pm

Surely Wilber will chime in with a PB asm file, evenually...?

wilbert · Post by **wilbert** » Sat Jul 23, 2016 5:22 am

Tenaja wrote:Surely Wilbert will chime in with a PB asm file, evenually...?

Not this time

I don't think there would be any speed improvement in this case.

Post by **idle** » Sat Jul 23, 2016 5:56 am

Tenaja wrote:Surely Wilber will chime in with a PB asm file, evenually...?

There is my "dirty" fasm preprocessor hack but it's inconvenient, probably safer to use keya's lib.

If the asm PB generates changes in later versions from the pattern the fasm pre processor expects, it won't work.
plus it won't work on mac unless yasm supports similar pre processor directives
http://www.purebasic.fr/english/viewtop ... 12&t=65312

Thanks for sharing keya.

Keya · Post by **Keya** » Sat Jul 23, 2016 6:09 am

yes i think it would be tricky and probably a waste of time to try to improve on speed because the operations are so fundamental and relatively small! Shift Right for example effectively just boils down to a single call to the shr instruction, so there's probably not much room for improvement!?! and I've provided libs compiled with every level of gcc optimization, so it's pretty efficient and hard to beat anyway (unless you're wilbert haha). So to improve on speed would really require native support so the operations can be assembled as inline asm rather than the overhead of procedure call/ret, but a quick forum search reveals that requests for native unsigned 32bit/64bit support in Purebasic has sadly fallen on deaf ears since basically forever, I don't know or understand why as unsigned 8bit/16bit are supported and signed/unsigned is such a fundamental part of the system, so choices in the meantime remain very limited - use hacks and hope you get it all right in every location, or use signed operations and invite miscalculations. Neither are realistic options for anyone who needs accurate calculations (all of us). This lib hopefully does provide full unsigned 32bit/64bit support without limitation so it's given me that confidence in operations i needed - its only real issue i can see is essentially that speed hit from procedure call/ret, and lack of uniformity from calling out to lib functions rather than being able to use native PB functions, but at least now that hurdle is reduced to one of efficiency and no longer one of calculation accuracy, and my customers can afford to wait an extra second if it means they're shown the correct values

[edit] idle just posted his as i clicked Post, lol

absolutely brilliant work as anyone can see but unfortunately like he mentioned doesn't support yasm so no OSX support, and also might break if PB's asm generation changes enough which led me to creating this lib!

wilbert · Post by **wilbert** » Sat Jul 23, 2016 8:23 am

@Keya, maybe you could also add min and max procedures for both 32 and 64 bit.
Something like

Code: Select all

unsigned long int u32max (unsigned long int dword1, unsigned long int dword2) {
 if (dword1 > dword2) { return dword1; } else { return dword2; }
}

Keya · Post by **Keya** » Sat Jul 23, 2016 8:28 am

too easy

will add to next update in next day or two

I was also thinking it'd be good to add the Min/MaxOf3 versions, but as that's diverting away from the fundamental operators i wondered if it should be in a separate "U32/64Misc.lib" (to avoid making executables bigger for no reason) or if it wouldn't matter.

I'm a bit newb-confused by what gets compiled in the executable when you use a static lib (used functions only, or entire object file?), because im pretty sure (but uncertain now!) that in the past I've used for example a lib that might be 100kb, but because im only using a couple of its functions the resulting exe is only 50kb!? I could be wrong, or maybe that's just debug metadata not being included, and i dont know if results are different with different lib compilers. Anyway I just did a test then ... I added a function which is just 1000xNop, and even if it's not included as an ImportC declaration (so not used or referenced at all) it still gets compiled into the executable, so yes i think it'd be best to add them in their own Misc lib accessible with #USE_U32MISC=1, although probably not worth being separate if Min/Max is the only misc function

(even after looking through PB's Maths helpfile section i can't really think of any other functions off the top of my head!)

wilbert · Post by **wilbert** » Sat Jul 23, 2016 9:31 am

A lib can consist of multiple object files.
Each object file that is required is linked so if you want small executables you can split your code so it compiles into multiple object files and combine those into a single lib.

Keya · Post by **Keya** » Thu Jul 28, 2016 5:47 pm

v1.2 uploaded, now in "SUPERMODULAR" form!!! what wilbert said above made a very audible click in my brain, lol
It's sooo much better in this format ... (although it did require me learning how to write my first ever Makefile, another bucketlist ✓, lol)

With the earlier versions there was simply one .c file which compiled to one .o file, so even if you only used one function the entire .o file would be compiled into your executable.

But starting with v1.2 each function is now in its own .c (and thus .o) file, and then all the resulting .o files are compiled into the one .a library. So now we link to the .a instead of the .o. This means that only the functions you use will be compiled into your executable

It also means there's no need for the couple of compiler directives i was using, making it even easier to use.

I also found out that C compilers support functions declared as "static inline" so the compiler can compile them inline rather than as a call, and yes you can even do this in libs, but unfortunately it seems that the fasm assembler doesn't support them (and i'm guessing neither does yasm).

All tests looked good on my Win/Linux/OSX x86 + x64 systems, please run TestAll32.pb and TestAll64.pb and you should see "All tests passed" at the bottom of Debug window output.

PureBasic Forums - English

LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad

Re: LibUnsigned: unsigned ops for 32-bit dword & 64-bit quad