PureBasic Forums - English

http://www.purebasic.com
https://www.purebasic.fr/english/

Are strings fixed in memory or can they change address?

https://www.purebasic.fr/english/viewtopic.php?t=86101

Page 1 of 1

Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 12:34 am
by DeanH
I have been trying to get Index Data's YAZ dll to work from a Pure Basic program. I have had very little luck with this, and even recently resorted to the dreaded ChatGPT. One of the things it objected to is the way strings are passed via CallFunction. I thought strings were fixed in memory in a PB program but ChatGPT insists they are not.

An example of my code was:
YAZ_DLL$="D:\BMV10\yaz5.dll" ;64-bit version
Server$ = "z3950.loc.gov:7090/voyager" ; Connection address
YAZ = OpenLibrary(#PB_Any, YAZ_DLL$)
ServerPtr = Ascii(Server$).
Connection = CallFunction(YAZ, "ZOOM_connection_new", ServerPtr, Port)

ChatGPT said:
"While this can work for a quick reference to a string's memory address, it may lead to issues if the string changes or is not null-terminated. It is safer to allocate memory explicitly for such pointers when interfacing with a DLL."
It insisted I use AllocateMemory + PokeS
; Convert the server string to ASCII and allocate memory
Server$ = "z3950.loc.gov:7090/voyager"
ServerPtr = AllocateMemory(StringByteLength(Server$, #PB_Ascii) + 1)
If ServerPtr
PokeS(ServerPtr, Server$, -1, #PB_Ascii)
EndIf

Again, ChatGPT objected with the same reason.

I thought strings were fixed memory and zero-byte delimited.

Is this correct or not?

For anyone curious, the entire test program's code, mostly generated by ChatGPT, is:
(The 64-bit version of yaz5.dll is needed from IndexData.)

Code: Select all

; Define constants and variables

;PWD$=#DQUOTE$+"set user 548880/bookmark"+#DQUOTE$+" "
;Connect$=#DQUOTE$+"connect z3950.loc.gov:7090/voyager"+#DQUOTE$+" "
;Format$=#DQUOTE$+"set preferredRecordSyntax USMARC"+#DQUOTE$+" "
;Search$=#DQUOTE$+"search "+#DQUOTE$+#DQUOTE$+"ringworld"+#DQUOTE$+#DQUOTE$+" "+#DQUOTE$+" "
;Show$=#DQUOTE$+"show 0 1"+#DQUOTE$+" "
;Close$=#DQUOTE$+"close"+#DQUOTE$+" "
;Quit$=#DQUOTE$+"quit"+#DQUOTE$
;P$="C:\BMV10\zoomsh.exe"
;AF$ = Connect$ + Format$ + Search$ + Show$ + Close$ + Quit$

;"connect z3950.loc.gov:7090/voyager" "set preferredRecordSyntax USMARC" "search ""the exploration of mars"" " "close" "quit"
; AF$=#DQUOTE$+"connect z3950.loc.gov:7090/voyager"+DQUOTE$
; AF$+" "
; AF$+DQUOTE$+"set preferredRecordSyntax USMARC"+DQUOTE$
; AF$+" "
; AF$+#DQUOTE$+"search "+#DQUOTE$+DQUOTE$+"ringworld"+#DQUOTE$+DQUOTE$
; AF$+" "
; AF$+#DQUOTE$+"close"+#DQUOTE$
; AF$+#DQUOTE$+"quit"+#DQUOTE$
; LOC$=GetCurrentDirectory()
; P$=LOC$+"zoomsh.exe"
; C=RunProgram(P$,AF$,LOC$,#PB_Program_Open | #PB_Program_Read|#PB_Program_Hide)
; Debug C
; If C
; 	While ProgramRunning(C)
; 		If AvailableProgramOutput(C)
; 			Debug ReadProgramString(C)
; 		EndIf
; 	Wend
; EndIf
; CloseProgram(C)
; End

; Define constants and variables
YAZ_DLL$ = "\BMV10\yaz5.dll"
Server$ = "z3950.loc.gov:7090/voyager"
Query$ = "ringworld"
Syntax$ = "USMARC"
preferredRecordSyntaxPtr$="preferredRecordSyntax"
loglevel$="log-level"

; Allocate memory for strings

;ServerPtr = AllocateMemory(StringByteLength(Server$) + SizeOf(Character))
;PokeS(ServerPtr, Server$)
;ServerPtr = Ascii(Server$)
; Convert the server string to ASCII and allocate memory
ServerPtr = AllocateMemory(Len(Server$) + 1)
If ServerPtr
    PokeS(ServerPtr, Server$, -1, #PB_Ascii)
EndIf

; Convert the preferred record syntax to ASCII
;SyntaxPtr = AllocateMemory(StringByteLength(Syntax$) + SizeOf(Character))
;PokeS(SyntaxPtr, Syntax$)
SyntaxPtr = Ascii(Syntax$)
;SyntaxPtr = AllocateMemory(Len(Syntax$) + 1)
;If SyntaxPtr
;    PokeS(SyntaxPtr, Syntax$, -1, #PB_Ascii)
;EndIf

; Convert the query string to ASCII
;QueryPtr = AllocateMemory(StringByteLength(Query$) + SizeOf(Character))
;PokeS(QueryPtr, Query$)
QueryPtr = Ascii(Query$)
;QueryPtr = AllocateMemory(Len(Query$) + 1)
;If QueryPtr
;    PokeS(QueryPtr, Query$, -1, #PB_Ascii)
;EndIf

preferredRecordSyntaxPtr=Ascii("preferredRecordSyntax")
;preferredRecordSyntaxPtr=AllocateMemory(Len(preferredRecordSyntax$)+1)
;If preferredRecordSyntaxPtr
;	PokeS(preferredRecordSyntaxPtr, preferredRecordSyntax$,-1,#PB_Ascii)
;EndIf

loglevelPtr=Ascii("log-level")
;loglevelPtr=AllocateMemory(Len(loglevel$)+1)
;If loglevelPtr
;	PokeS(loglevelPtr, loglevel$,-1,#PB_Ascii)
;EndIf


; Open the YAZ DLL
YAZ = OpenLibrary(#PB_Any, YAZ_DLL$)
If YAZ
    Debug "YAZ DLL loaded successfully."

    ; Create a connection to the Z39.50 server
    Connection = CallFunction(YAZ, "ZOOM_connection_new", ServerPtr) ; Explicitly pass port
    If Connection
        Debug "Connected to server: " + Server$

        ; Set the preferred record syntax
        Debug "Setting preferredRecordSyntax to: " + Syntax$
        
        CallFunction(YAZ, "ZOOM_connection_option_set", Connection, preferredRecordSyntaxPtr, SyntaxPtr)

        ; Enable logging
        LogLevelPtr = AllocateMemory(StringByteLength("debug",#PB_Ascii) + 1)
        PokeS(LogLevelPtr, "debug", #PB_Ascii)
        CallFunction(YAZ, "ZOOM_connection_option_set", Connection, loglevelPtr, LogLevelPtr)

        ; Create a query
        Query = CallFunction(YAZ, "ZOOM_query_create")
        If Query
            Debug "Query created successfully."

            ; Set the query string
            Debug "Query String: " + Query$
            CallFunction(YAZ, "ZOOM_query_prefix", Query, QueryPtr)

            ; Execute the search
            Debug "Executing search..."
            ResultSet = CallFunction(YAZ, "ZOOM_connection_search", Connection, Query)
            If ResultSet
                Debug "Search executed successfully."

                ; Get the number of results
                ResultCount = CallFunction(YAZ, "ZOOM_resultset_size", ResultSet)
                Debug "Number of results: " + Str(ResultCount)

                ; Retrieve error message
                ErrorPtr = CallFunction(YAZ, "ZOOM_connection_errmsg", Connection)
                If ErrorPtr
                    ErrorMessage$ = PeekS(ErrorPtr, -1, #PB_Ascii)
                    Debug "Error Message: " + ErrorMessage$
                Else
                    Debug "No error message reported."
                EndIf

                ; Debug diagnostic code
                Diag = CallFunction(YAZ, "ZOOM_connection_diag", Connection)
                Debug "Diagnostic Code: " + Str(Diag)

                ; Destroy the result set
                CallFunction(YAZ, "ZOOM_resultset_destroy", ResultSet)
            Else
                Debug "Search failed."
            EndIf

            ; Destroy the query
            CallFunction(YAZ, "ZOOM_query_destroy", Query)
        Else
            Debug "Failed to create query."
        EndIf

        ; Destroy the connection
        CallFunction(YAZ, "ZOOM_connection_destroy", Connection)
    Else
        Debug "Failed to connect to server."
    EndIf

    ; Close the YAZ DLL
    CloseLibrary(YAZ)
Else
    Debug "Failed to load YAZ DLL."
EndIf

; Free allocated memory
FreeMemory(ServerPtr)
FreeMemory(SyntaxPtr)
FreeMemory(QueryPtr)
FreeMemory(preferredRecordSyntaxPtr)
FreeMemory(loglevelPtr)

Re: Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 12:41 am
by DeanH
One more bit:

I understand if a string is changed, the address can change, but in this case, the strings won't change and the variables re-used.

Re: Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 2:46 am
by idle
it would help if you could link to the lib documentation and I can't find the header to the lib

A literal string address in PB is fixed.
I would also use utf8() and use prototypes which are a better option as you can just pass PB strings and have them converted to utf8 with p-utf8

Code: Select all

PrototypeC prtZOOM_connection_new(server.p-utf8,Port) 
Global ZOOM_connection_new.prtZOOM_connection_new  


YAZ_DLL$="D:\BMV10\yaz5.dll" ;64-bit version
Server$ = "z3950.loc.gov:7090/voyager" ; Connection address

YAZ = OpenLibrary(#PB_Any, YAZ_DLL$)
If YAZ 
  
  ZOOM_connection_new = GetFunctionEntry(YAZ,"ZOOM_connection_new") 
  
Else 
  ;error
  ;end 
EndIf   

If Yaz 
  result = ZOOM_connection_new("z3950.loc.gov:7090/voyager",port) 
EndIf

Re: Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 4:12 am
by DeanH
I don't have a lib, only the DLL.

The YAZ system provides z39.50 protocol handling for sending and retrieving library data. Schools in Australia, New Zealand (probably via Access-It) and elsewhere can download cataloguing data from sources such as SCIS, the U.S. Library of Congress (which I'm using as a test), and others.

The Index Data URL is
https://www.indexdata.com/resources/software/yaz/

The YAZ toolkit, which contains the DLL, source code that someone way cleverer than myself can supposedly compile, and other things is at
https://download.indexdata.com/pub/yaz/win64/
I downloaded the latest version, at the bottom of the list, but I have been using previous versions.

The files yaz5.dll, the libxml2.dll, libxslt.dll should be in the same directory as the test program. They're in the bin folder. I also put zoomsh.exe in there. On my system I've add msvcr120.dll as well even though in theory I shouldn't have to.

Code: Select all

;This code runs the Zoomsh.exe program which uses yaz. It is run from a command line. 

Procedure$ FNQ(A$)
	ProcedureReturn #DQUOTE$+A$+#DQUOTE$+" "
EndProcedure

;Zoomsh.exe test...this works
;Example string "connect z3950.loc.gov:7090/voyager" "set preferredRecordSyntax USMARC" "search ""ringworld"" " "close" "quit"
AF$=FNQ("connect z3950.loc.gov:7090/voyager")		;database
AF$+FNQ("set preferredRecordSyntax USMARC")		;format of the results, USMARC21.
S$=FNQ("ringworld")									;book title
AF$+FNQ("search "+S$+" ")
AF$+FNQ("close")
AF$+FNQ("quit")
Debug AF$
LOC$=GetCurrentDirectory()
P$=LOC$+"zoomsh.exe"
C=RunProgram(P$,AF$,LOC$,#PB_Program_Open | #PB_Program_Read|#PB_Program_Hide)
If C
	While ProgramRunning(C)
		If AvailableProgramOutput(C)
			Debug ReadProgramString(C)
 		EndIf
	Wend
EndIf
CloseProgram(C)
End
Zoomsh.exe can be executed on its own. Then each parameter is entered on a separate line. I have been using the above for years successfully but I'd like to run it from the dll so it can then be accessed from a webpage.

I tried using UTF8 at first but it definitely did not work past opening the library. I got better results with ASCII strings.

I can put all of this into a zip file that can be downloaded from my website, if that helps.

Have to say I'm not entirely clear on how to use prototypes.

Re: Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 6:09 am
by DeanH
I DID IT!!!

Perseverance paid.

I finally got it work work. Had to make a few changes to the way the server is accessed, and to make sure all strings are ASCII format, including instructions like "log-level". It returns 19 records, which is what the Zoomsh.exe approach also returns. The code is a trial version only, not fully developed and refined.

Code: Select all

; Path to the YAZ DLL
YAZ_DLL$ = "C:\BMV10\yaz5-64.dll"
; Server details
Server$ = "z3950.loc.gov" ; Server address with database
Database$ = "voyager" ; Explicit database name
Port = 7090
Syntax$ = "USMARC" ; Preferred record syntax
Query$ = "ringworld" ; Search term

; Allocate ASCII memory for all strings
ServerPtr = Ascii(Server$)
DatabasePtr = Ascii(Database$)
SyntaxPtr = Ascii(Syntax$)
QueryPtr = Ascii(Query$)
databasenamePtr=Ascii("databaseName")
preferredRecordSyntaxPtr=Ascii("preferredRecordSyntax")
loglevelPtr=Ascii("log-level")
debugPtr=Ascii("debug")

; Open the YAZ DLL
YAZ = OpenLibrary(#PB_Any, YAZ_DLL$)
If YAZ
    Debug "YAZ DLL loaded successfully."

    ; Create a connection to the Z39.50 server
    Connection = CallFunction(YAZ, "ZOOM_connection_new", ServerPtr, Port)
    If Connection
        Debug "Connected to server: " + Server$

        ; Set the database name
        CallFunction(YAZ, "ZOOM_connection_option_set", Connection, databasenamePtr, DatabasePtr)
        Debug "Database set to: " + Database$

        ; Set the preferred record syntax
        CallFunction(YAZ, "ZOOM_connection_option_set", Connection, preferredRecordSyntaxPtr, SyntaxPtr)
        Debug "Preferred record syntax set to: " + Syntax$

        ; Enable logging for debugging
        CallFunction(YAZ, "ZOOM_connection_option_set", Connection, loglevelPtr, debugPtr)

        ; Create a query
        QueryHandle = CallFunction(YAZ, "ZOOM_query_create")
        If QueryHandle
            Debug "Query created successfully."

            ; Set the query string
            CallFunction(YAZ, "ZOOM_query_prefix", QueryHandle, QueryPtr)
            Debug "Query String: " + Query$

            ; Execute the search
            Debug "Executing search..."
            ResultSet = CallFunction(YAZ, "ZOOM_connection_search", Connection, QueryHandle)

            ; Check for diagnostics
            Diag = CallFunction(YAZ, "ZOOM_connection_diag", Connection)
            If Diag <> 0
                ErrorPtr = CallFunction(YAZ, "ZOOM_connection_errmsg", Connection)
                If ErrorPtr
                    ErrorMessage$ = PeekS(ErrorPtr, -1, #PB_Ascii)
                    Debug "Error Message: " + ErrorMessage$
                EndIf
                Debug "Diagnostic Code: " + Str(Diag)
            EndIf

            If ResultSet
                Debug "Search executed successfully."

                ; Get the number of results
                ResultCount = CallFunction(YAZ, "ZOOM_resultset_size", ResultSet)
                Debug "Number of results: " + Str(ResultCount)

                ; Retrieve the first record if available
                If ResultCount > 0
                    Record = CallFunction(YAZ, "ZOOM_resultset_record", ResultSet, 0)
                    If Record
                        RecordDataPtr = CallFunction(YAZ, "ZOOM_record_get", Record, Ascii("raw"), #Null)
                        If RecordDataPtr
                            RecordData$ = PeekS(RecordDataPtr, -1, #PB_Ascii)
                            Debug "Record Data: " + RecordData$
                        Else
                            Debug "Failed to retrieve the record."
                        EndIf
                    Else
                        Debug "Failed to retrieve the record."
                    EndIf
                Else
                    Debug "No results found."
                EndIf

                ; Destroy the result set
                CallFunction(YAZ, "ZOOM_resultset_destroy", ResultSet)
            Else
                Debug "Search failed."
            EndIf

            ; Destroy the query
            CallFunction(YAZ, "ZOOM_query_destroy", QueryHandle)
        Else
            Debug "Failed to create query."
        EndIf

        ; Destroy the connection
        CallFunction(YAZ, "ZOOM_connection_destroy", Connection)
    Else
        Debug "Failed to connect to server."
    EndIf

    ; Close the YAZ DLL
    CloseLibrary(YAZ)
Else
    Debug "Failed to load YAZ DLL."
EndIf

; Free memory allocated by Ascii()
If ServerPtr : FreeMemory(ServerPtr) : EndIf
If DatabasePtr : FreeMemory(DatabasePtr) : EndIf
If SyntaxPtr : FreeMemory(SyntaxPtr) : EndIf
If QueryPtr : FreeMemory(QueryPtr) : EndIf
If databasenamePtr : FreeMemory(databasenamePtr) : EndIf
If preferredRecordSyntaxPtr : FreeMemory(preferredRecordSyntaxPtr) : EndIf
If loglevelPtr : FreeMemory(loglevelPtr) : EndIf
If debugPtr : FreeMemory(debugPtr) : EndIf
The result:
YAZ DLL loaded successfully.
Connected to server: z3950.loc.gov:7090/voyager
Database set to: voyager
Preferred record syntax set to: USMARC
Query created successfully.
Query String: ringworld
Executing search...
Search executed successfully.
Number of results: 19

Re: Are strings fixed in memory or can they change address?

Posted: Fri Jan 17, 2025 11:21 pm
by Paul
If you want to use the .lib file found in the "YAZ/lib" folder you can make things a little simpler with Import...

Code: Select all

Import "yaz5.lib"
  ZOOM_connection_new(Host.p-ascii, portnum.i)
  ZOOM_connection_option_set(ZOOM_connection.i, DatabasenamePtr.p-ascii, DatabasePtr.p-ascii)
  ZOOM_query_create()
  ZOOM_query_prefix(ZOOM_query.i,QueryPtr.p-ascii)
  ZOOM_connection_search(ZOOM_connection.i,ZOOM_query.i)
  ZOOM_resultset_size(ZOOM_query.i)  
  ZOOM_resultset_record(ZOOM_query.i,size.i)
  ZOOM_record_get(ZOOM_record.i,RecordPtr.p-ascii,size.i)  
  ZOOM_connection_destroy(ZOOM_connection.i)
  ZOOM_resultset_destroy(ZOOM_query.i)
  ZOOM_query_destroy(ZOOM_query.i)
EndImport

 

Server$   ="z3950.loc.gov"
Database$ ="voyager"
Port      =0
Syntax$   ="USMARC" 
Query$    ="ringworld"

ZOOM_connection=ZOOM_connection_new(Server$,Port)
If ZOOM_connection
  ZOOM_connection_option_set(ZOOM_connection,"databaseName",Database$)
  ZOOM_connection_option_set(ZOOM_connection,"preferredRecordSyntax",Syntax$)
  ZOOM_connection_option_set(ZOOM_connection,"log-level","debug")
  
  ZOOM_query=ZOOM_query_create()
  If ZOOM_query
    ZOOM_query_prefix(ZOOM_query,Query$)
    ZOOM_resultset=ZOOM_connection_search(ZOOM_connection,ZOOM_query)
    If ZOOM_resultset
      ResultCount=ZOOM_resultset_size(ZOOM_resultset)
      Debug "Number of results: " + Str(ResultCount)
      
      If ResultCount
        For rec=1 To ResultCount
          ZOOM_record=ZOOM_resultset_record(ZOOM_resultset,rec-1)
          If ZOOM_record
            RecordDataPtr=ZOOM_record_get(ZOOM_record,"raw",@size)
            If RecordDataPtr
              RecordData$=PeekS(RecordDataPtr,-1,#PB_Ascii)
              Debug "Record Size: "+PeekI(@size)
              Debug "Record Data: "+RecordData$

              Else
              Debug "Failed to retrieve the record."
            EndIf
            Else
            Debug "Failed to retrieve the record."
          EndIf
        Next
        
        Else
        Debug "No results found."
      EndIf      
      ZOOM_resultset_destroy(ZOOM_resultset) 
           
      Else
      Debug "Search failed."
    EndIf
    ZOOM_query_destroy(ZOOM_query)
    
    Else
    Debug "Failed to create query."
  EndIf
  ZOOM_connection_destroy(ZOOM_connection)
  
  Else
  Debug "Failed to load YAZ DLL."
EndIf

Re: Are strings fixed in memory or can they change address?

Posted: Sun Jan 19, 2025 12:14 am
by DeanH
Thanks, Paul. That works even better.

I ran into another problem. The code worked fine on my work laptop (Win 11 23H2) but the DLL refused to load on my Win 10 PC at home and even even on my wife's Win 11 laptop. Reported the DLL failed to load. I'm guessing something is present on the laptop but not the others. But your code works on my home PC. Thank you for digging into this and providing a better solution.
All times are UTC+01:00
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited