a new keyword to return the address of a string handle, etc

[BorisTheOld]

We've added a fully functional ByRef feature to our PB code and it handles all data types including strings. We've done this using a few special macros and structures.

A ByRef feature requires that the address of the data, or its handle if it's a complex type like maps or strings, be passed to the called procedure. Except for strings, PB already passes complex types by reference, and the @ prefix can be used to create the address of a non-string data type. However, our implementation of ByRef is not as slick as it could be because there is no easy way to obtain the address of the string handle that points to the string data.

We use different techniques to get the address of the string handle, depending on where the string is located in the program. As a last resort we even create a dummy integer variable adjacent to the string and use @dummy+4 to calculate the address of the string handle.

A simple keyword such as VarPtr() would be extremely useful.

Other BASIC languages have VarPtr() and StrPtr() keywords to perform the same function as PB's @ prefix.

VarPtr(variable-name) returns the address of the specified variable, including structure variables and array elements. However when referencing dynamic items such as maps, dynamic arrays and strings, etc, the address returned points to the handle or descriptor block.

StrPtr(variable-name) returns the address of a string's contents. It's not really required in PB, since the @ prefix performs the same function, and a simple StrPtr macro could be used.

[Tenaja]

Not trying to knock your request, but since you have created so many macros for ByRef etc., why not create one for VarPtr()?

[BorisTheOld]

Not trying to knock your request, but since you have created so many macros for ByRef etc., why not create one for VarPtr()?

The problem is not with creating the macros, but with being able to use a simple command to get the address of the string handle.

If the string is in a structure then OffsetOf() can be used:

Code: Select all

Macro VarPtr(instance, structure, variable)
  @instance + OffsetOf(structure\variable)
EndMacro

However, if the string is a global variable or is a local variable in a procedure, then the above technique won't work. Instead, the address of the string handle has to be inferred from something else. Labels don't work because they are in the code space, not the data space.

I've found the best approach is to define a dummy integer variable immediately before the string. The address of this integer, plus 4 bytes, is the address of the string handle. Because we use code generators to create our code, and make heavy use of macros to improve code readability, I've found that the benefits of a ByRef feature far outweigh a few extra K in program size.

Here's some code to show how we do it. We currently use about 500+ macros to implement OOP, custom gadgets, ByRef, and various other "missing" language features.

PureBasic is amazingly flexible. And being an Assembler programmer from way back, that's why I chose it over other languages like PowerBasic, FreeBasic, FreePascal, and OpenCOBOL.

Code: Select all

Macro Me
  *Self
EndMacro

Macro typString
  s
EndMacro

Macro typInt32
  i
EndMacro

Macro PrivateInt32 (bvsDataParmName)
  bvsDataParmName.typInt32
EndMacro

Macro PrivateString (bvsDataParmName)
  bvsDataParmName#_.typInt32
  bvsDataParmName.typString
EndMacro

Macro ByRef (bvsDataParmName, bvsDataParmType)
  *bvsDataParmName.udtByRef#bvsDataParmType
EndMacro

Macro Value (bvsDataParmName)
  *bvsDataParmName\QQType
EndMacro

Macro Ref (bvsDataParmName)
  *bvsDataParmName
EndMacro

Macro StrRef (bvsDataParmName)
  @bvsDataParmName#_ + 4
EndMacro

Macro VarRef (bvsDataParmName)
  @bvsDataParmName
EndMacro

Structure udtByRefs
  QQType.s
EndStructure

Structure udtByRefi
  QQType.i
EndStructure

Structure udtTest
  PrivateString(prsVar1)
  PrivateString(prsVar2)
  PrivateInt32(priVar3)
  PrivateInt32(priVar4)
  PrivateString(prsVar5)
EndStructure

Procedure TestByRef2(ByRef(brsVar, typString), ByRef(briVar, typInt32))
  debug Value(brsVar)
  debug Value(briVar)
  Value(brsVar) = "123"
  Value(briVar) = 456
EndProcedure

Procedure TestByRef(ByRef(brsVar, typString), ByRef(briVar, typInt32))
  TestByRef2(Ref(brsVar), Ref(briVar))
EndProcedure

Define Me.udtTest

Me = AllocateMemory(SizeOf(udtTest))

Me\prsVar2 = "abc"
Me\priVar4 = 999

TestByRef(StrRef(Me\prsVar2), VarRef(Me\priVar4))

debug Me\prsVar2
debug Me\priVar4

[Tenaja]

Looks good.

The address of this integer, plus 4 bytes

I'm sure you use Sizeof Int, and not 4...

[BorisTheOld]

I'm sure you use Sizeof Int, and not 4...

Probably

[netmaestro]

However, if the string is a global variable or is a local variable in a procedure, then the above technique won't work

Code: Select all

Structure NULL
  VAR.i
EndStructure

Macro VarPtr(instance, Struct=NULL, var=VAR)
  @instance + OffsetOf(struct\var)
EndMacro

Guaranteed to work everywhere or your money cheerfully refunded If the var isn't structured it just adds 0 to the address.

[luis]

@netmaestro

That returns the base address of the string in case of a global/local variable and returns the handle (the pointer to the base address of a string) if the string it's a field.

Code: Select all

Structure NULL
  VAR.i
EndStructure

Macro VarPtr(instance, Struct=NULL, var=VAR)
  @instance + OffsetOf(struct\var)
EndMacro


a$ = "hello"

Structure t
 b$
EndStructure : Define test.t : test\b$ = "world"


Debug VarPtr(a$) ; base address

Debug PeekS(VarPtr(a$)) ; string at that base address

Debug VarPtr(test, t, b$) ; handle

Debug PeekI(VarPtr(test, t, b$)) ; base address from the handle

Debug PeekS(PeekI(VarPtr(test, t, b$))) ; string at that base address

That's why BtO mentioned "a dummy integer variable immediately before the string" to be used as a marker, he wants the handle.

I didn't understand what use he makes of that BTW and why the base pointer to the string is not enough

EDIT: still don't

[netmaestro]

I can't believe the dummy var declared just before the string would be safe 100% of the time. Inside a structure should be fine but for unstructured vars the compiler might do something you didn't expect when allocating the memory for the string and then you have a bug that could be hard to trace. Before implementing that in production software I'd want to hear what the team has to say.

[luis]

Possibly, but that didn't sound cheerful.

[BorisTheOld]

I can't believe the dummy var declared just before the string would be safe 100% of the time

That's why I would like a traditional VarPtr() added to PB. However, so far, after months of use, our workaround seems to be a valid solution.

That's why BtO mentioned "a dummy integer variable immediately before the string" to be used as a marker, he wants the handle.
I didn't understand what use he makes of that BTW and why the base pointer to the string is not enough

Because, for ByRef to work, the string handle address must be passed. That way, each procedure in the calling stack is working with the original string handle, which is what calling by reference is all about. It's the same concept as passing the address of an array descriptor block, so that each procedure in the stack is working with the original array.

Passing the base address of the string causes a memory fault, because, with a ByRef call, the called procedure thinks it's working with a string handle. It would be attempting to use the first few bytes of the string as a pointer to the memory that's allocated to the string.

Take a close look at all the examples on these forums of the "structure trick" for passing standard variable types by reference. Non-strings just wrap the variable, but for strings it's the string handle that's wrapped in the structure. However, this simple approach requires the original string to be embedded in a structure, whereas my approach can work with any string configuration.

[BorisTheOld]

That's why BtO mentioned "a dummy integer variable immediately before the string" to be used as a marker, he wants the handle.
I didn't understand what use he makes of that BTW and why the base pointer to the string is not enough

EDIT: still don't

Strings 101

a) when a dynamic string is defined, a string handle is created

b) when the string is created, memory is allocated on the heap and its address is placed in the string handle

c) if the string is resized, the old memory is de-allocated and the address of the newly allocated memory is placed in the string handle

d) a ByVal call allows a procedure to read a variable but not change it

e) a ByRef call allows a procedure to access and change a variable that's allocated in a different part of the code

f) for fixed length data types, the ByRef pointer points directly to the data since no dynamic memory allocation is involved

g) for dynamic strings, the only way for the string libraries to allocate and de-allocate memory is by means of the original string handle that was created when the string variable was defined -- a copy of the handle can't be used, since the original handle will never be altered to match the new memory allocation, ultimately creating a memory fault

h) ergo: when passing strings ByRef it's the address of the original string handle that must be propagated through the call stack -- not the address contained in the string handle, which points to the current location of the string

i) that's why PB uses the "structure trick" to simulate RyRef calls -- the address of the structure is also the address of the single data field it contains, which for strings is the string handle

j) however, there is currently no direct way to obtain the address of string handles that are not part of a structure, hence the need for a VarPtr() command

[luis]

EDIT: still don't

Wow you spotted it, I'm impressed

Believe it or not, I knew about a-g, but nothing wrong in writing about it !

I understand why the string library has to use the handle, the handle must remain constant for the user of the library etc.
What I don't understand it's why you want to access it, since you are not the string library and I suppose you just use the strings, don't allocate/reallocate them substituting yourself to the string library, and when you use @ you are actually passing the string by reference just not the handle of it.

Uhm... I'll try to see the code you posted above to see if I understand why you need it.

Nope, I don't get it why there is the need of all that convoluted stuff instead of using directly a String structure, beyond the fact you may like that particular syntax and so you have to jump a little around with pointers to land to the same place.
But it's irrelevant, thanks and don't waste more time on me, good luck with your request, an added feature is always better than a missing feature, who know I couldn't use it too for some reason I can't imagine now

[BorisTheOld]

What I don't understand it's why you want to access it..........but it's irrelevant, thanks and don't waste time on me

I'm always happy to waste time.

As you correctly point out, I don't make any use of the string libaries -- and therein lies the cleverness of my strategy.

You will notice that the called procedure uses the ByRef macro to tell the compiler that the "structure trick" is being used. So the compiler creates ByRef code to accept a structure with a single data element, which can be a string or a non-string item.

Under normal circumstances, in the calling code, the compiler will notice that a structure is being passed as a parameter and will put the structure's address on the stack. It doesn't care what is actually in the structure, but we of course know that the structure's address is also the address of the single data item in the structure. And in the case of a string it will be the address of the string handle.

But putting a single data item in its own structure, just so it can be passed by reference, is tedious to do and hard to manage, especially in large applications with thousands of data elements. So what to do?

The solution is to not put the data element in a structure, but just use its address (@data-name) as a parameter in the procedure call. However, using @data-name with a string creates an address that points to the actual string data, not to the string's handle. So the answer is to pass the address of the handle instead.

And that's why I need the address of the string's handle.

.....and when you use @ you are actually passing the string by reference just not the handle of it.

No!!!!!! -- using @data-name with a string is NOT passing ByRef. All you're doing is passing a pointer to where the string is stored in the heap. You would have no ability to manipulate the string via the string library.

ByRef is a specific feature which allows one to reach back into the calling code and change the data. And to do that one needs access to the string library via the string handle.

[luis]

No!!!!!!

OMG what happened !?!?! Jeez I can hear you...

Well, you are right. Technically calling it passing by reference is incorrect. I'm coming from C and at the time most C programmers just used the expressions "byval" and "byref" probably because true byref was not available (came later with C++). Meaning with that you were passing a pointer to the var, but that would be "passing A reference", not really "passing BY reference". A subtle difference but it's technically important.

In reality you can talk of Byref only when the language syntactically supports it, if the language needs to deference the variable because it's not a true alias then it's just 'A' reference through a pointer.

See what I mean ?

http://www.tutorialspoint.com/cprogramm ... erence.htm

Strictly speaking C language can pass only by value, thanks for correcting me, it's a bad habit I should drop.

[BorisTheOld]

OMG what happened !?!?! Jeez I can hear you...

Not to worry.

I just wanted to be sure that those not privy to the black arts weren't confused by your brain fart.