Misbehaving with RGB() and RGBA()

[Demivec]

I just notice some problems with some sloppy code of mine that I was using for testing purposes and thought I would make a post describing what happened.

I was generating some colors with a formula and they didn't seem to match up with what I was expecting when I tested them on an image that was produced using them.

Here's some sample code demonstrating what I found:

Code: Select all

Debug RSet(Hex(RGB($999, $2, $3)), 6, "0")
Debug RSet(Hex(RGB($1, $999, $3)), 6, "0")
Debug RSet(Hex(RGB($1, $2, $999)), 6, "0")

Debug "==============================="
Debug RSet(Hex(RGBA($999, $2, $3, $4)), 8, "0")
Debug RSet(Hex(RGBA($1, $999, $3, $4)), 8, "0")
Debug RSet(Hex(RGBA($1, $2, $999, $4)), 8, "0")
Debug RSet(Hex(RGBA($1, $2, $3, $999)), 8, "0")

The code produces the following output:

Code: Select all

030B99
0B9901
999020
===============================
04030B99
040B9901
0D990201
FFFFFFFF

Now it should be obvious that I used a value that was out of range as a parameter for each of these functions. The documentation states that all of the parameters must be between 0 and 255 (or $FF). I kind of missed that point and thought that the parameters would be automatically squeezed to fit the required range either by truncation or use a modular function. Never-the-less I am glad they are not because that would slow things down instead of being responsible.

It can be shown that the parameters are simply combined, possibly by using shifting and addition. This means if you have a parameter that is too large it will overflow into a neighboring parameter. If the parameter is negative it will max out the values (make them all 255).

In either case looking at the output showed some inconsistent results that aren't produced by simply shifting and addition.


Hopefully you learn from my silly misstep. In my case a just substituted the parameter with a modded value (i.e. RGB(x % 256, y% 256, z%256)).

[wilbert]

Here's also an example of clipped RGB and RGBA procedures

Code: Select all

Procedure.l ClippedRGB(Red.l, Green.l, Blue.l); Requires MMX
  !movd mm0, [p.v_Red]
  !movd mm1, [p.v_Blue]
  !punpckldq mm0, [p.v_Green]
  !packssdw mm0, mm1
  !packuswb mm0, mm0
  !movd eax, mm0
  !emms
  ProcedureReturn
EndProcedure

Procedure.l ClippedRGBA(Red.l, Green.l, Blue.l, Alpha.l); Requires MMX
  !movd mm0, [p.v_Red]
  !movd mm1, [p.v_Blue]
  !punpckldq mm0, [p.v_Green]
  !punpckldq mm1, [p.v_Alpha]
  !packssdw mm0, mm1
  !packuswb mm0, mm0
  !movd eax, mm0
  !emms
  ProcedureReturn
EndProcedure

Values below 0 are clipped to 0.
Values above 255 are clipped to 255.

Example

Code: Select all

Debug RSet(Hex(ClippedRGB($999, $2, $3)), 6, "0")
Debug RSet(Hex(ClippedRGB($1, $999, $3)), 6, "0")
Debug RSet(Hex(ClippedRGB($1, $2, $999)), 6, "0")

Debug "==============================="
Debug RSet(Hex(ClippedRGBA($999, $2, $3, $4)), 8, "0")
Debug RSet(Hex(ClippedRGBA($1, $999, $3, $4)), 8, "0")
Debug RSet(Hex(ClippedRGBA($1, $2, $999, $4)), 8, "0")
Debug RSet(Hex(ClippedRGBA($1, $2, $3, $999)), 8, "0")

Output

Code: Select all

0302FF
03FF01
FF0201
===============================
040302FF
0403FF01
04FF0201
FF030201