Seek a Line in a File

Peter_DevRes · Post by **Peter_DevRes** » Thu Apr 02, 2009 11:58 pm

Hi,

I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.

I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.

I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.

The text below shows essentially what I would put in the loop, just struggling to find the right PureBasic commands.

Hope someone can help me.

...
...
If Readfile(0,"C:\whateverfile.txt")
length=lof(0)
*MemoryID=AllocateMemory(length)
whileEof(0)=0
<break the line of text in to substrings and then insert the substrings into a SQL statement>
wend
Else
MessageRequester(" Information ","Couldn't retrieve the text""
EndIf
...
...

Thanks,
Peter.

Demivec · Post by **Demivec** » Fri Apr 03, 2009 3:49 am

Peter_DevRes wrote:I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.

I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.

I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.

Look in the manual under the General Libraries for "File" and "String".

More specifically, "FileSeek()", "ReadString()", "CountString()", "FindString()", "StringField()".

I will post an example that may be helpful to you. I will post it at the earliest opportunity. Synchronize your watch, it will be in 10 hours (from the time of this message), mark.

sverson · Post by **sverson** » Fri Apr 03, 2009 10:22 am

this may help a little

Code: Select all

MyString$ = ""
LineNo.l = 0
FileNo.l = ReadFile(#PB_Any,"C:\whateverfile.txt")
If FileNo
  While Not Eof(FileNo)
    MyString$ = ReadString(FileNo)
    If MyString$<>""
      If LineNo=0
        SQLf$ = "INSERT INTO MyTable ("
        Elements = CountString(MyString$,";")
        For Element = 1 To Elements
          SQLf$ + StringField(MyString$,Element,";")
          If Element<Elements
            SQLf$ + ", "
          EndIf
        Next
        SQLf$ + ") VALUES("
      Else
        SQLv$ = ""
        Elements = CountString(MyString$,";")
        For Element = 1 To Elements
        SQLv$ + "'" + StringField(MyString$,Element,";") + "'"
        If Element<Elements
        SQLv$ + ", "
        EndIf
        Next
        SQLv$ + ") "
        SQL$ = ReplaceString(SQLf$+SQLv$,"'",#DOUBLEQUOTE$)
        Debug SQL$
      EndIf
      LineNo + 1
    EndIf
  Wend
  CloseFile(FileNo)
EndIf

Peter_DevRes · Post by **Peter_DevRes** » Fri Apr 03, 2009 12:58 pm

Hi,

Thanks for taking the time to go in that level detail, appreciated your assistance. Looks like an interesting approach.

Thanks & Regards,

Peter.

Peter_DevRes · Post by **Peter_DevRes** » Fri Apr 03, 2009 1:00 pm

Demivec wrote:
Peter_DevRes wrote:I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.

I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.

I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.
Look in the manual under the General Libraries for "File" and "String".

More specifically, "FileSeek()", "ReadString()", "CountString()", "FindString()", "StringField()".

I will post an example that may be helpful to you. I will post it at the earliest opportunity. Synchronize your watch, it will be in 10 hours (from the time of this message), mark.

Great will be interested in how you approach this problem, looking forward to it.

Thanks,
Peter

sverson · Post by **sverson** » Fri Apr 03, 2009 1:15 pm

sorry i forgot the textfile

C:\whateverfile.txt:

Code: Select all

name;road;city;
Smith;Old London Rd.;Hell;
Marx;New London Rd;Heaven;

Peter_DevRes · Post by **Peter_DevRes** » Fri Apr 03, 2009 1:43 pm

Right, great.

Thanks,
Peter.

Demivec · Post by **Demivec** » Fri Apr 03, 2009 3:55 pm

Here's a portion of one of my programs. I had a feeling you might need something along these lines. If I've guessed wrong, you might still find some usefulness out of it.

It does demonstrate reading and parsing text from a file.

Code: Select all

;Description: subroutines produce a list of games and their descriptions from a MAME executible
;Author: Demivec (Jared)
;Date: 11/13/2008
;Using: PureBasic v4.30

Enumeration ;Windows
   #Progress_win
EndEnumeration

Enumeration
  #PbarDescription_txt
  #ProgressBar_pbr
EndEnumeration

Structure gameData
  name$
  description$
EndStructure

Global programPath$ = GetPathPart(ProgramFilename())
Global gameCount
Global NewList gameList.gameData() ;holds all game data (Name,Description)

Procedure.l openWindow_Progress(p_text.s,p_min.l,p_max.l) ;setup Progress window and its gadgets
  If OpenWindow(#Progress_win, 538, 605, 279, 64, "Progress...",  #PB_Window_SystemMenu | #PB_Window_TitleBar | #PB_Window_WindowCentered)
    TextGadget(#PbarDescription_txt, 10, 10, 260, 20, p_text,#PB_Text_Center)
    StickyWindow(#Progress_win,1)
    ProcedureReturn 1 ;successfully open window
  Else
    ProcedureReturn -1 ;couldn't open window
  EndIf 
EndProcedure

Procedure closeWindow_Progress() ;close Progress window
  StickyWindow(#Progress_win,0)
  CloseWindow(#Progress_win)
EndProcedure

Procedure.l fillList(p_sourcePath$) ;add new elements from file 'listfull.txt'
  Protected a$,b$,c$,fileNum.l,beginning.l,middle.l,finish.l,length.l,i.l,j.l,event.l
  CallDebugger
  SetGadgetText(#PbarDescription_txt,"Parsing names and adding them to list...")
  While WindowEvent():Wend ;update window
  Delay(1)
  fileNum = OpenFile(#PB_Any,p_sourcePath$ + "listfull.txt")
  a$ = ReadString(fileNum)
  CloseFile(fileNum)
  length = Len(a$)
  beginning = FindString(a$, ":", 1)
  beginning = FindString(a$, ":", beginning + 1) + 1
  While beginning < length
      b$ = Trim(Mid(a$,beginning,8))
      middle = beginning + 11
      finish = FindString(a$,Chr(34),middle) ;skip over 2 spaces and a quote
      c$ = Mid(a$,middle,finish - middle)
      beginning = finish+1
      AddElement(gameList())
      gameList()\name$ = b$
      gameList()\description$ = c$
      SetGadgetText(#PbarDescription_txt,"Parsed names for " + Str(ListSize(gameList())) + " games.")
      While WindowEvent():Wend ;swallow events and update window
  Wend
  DeleteFile(p_sourcePath$ + "listfull.txt")
  
  ProcedureReturn ListSize(gameList())
EndProcedure

Procedure CreateNewList() ;create 'listfull.txt' from Mame executible
  Protected openFile$, fileNum.l,result.l,a$,event.l
  
  openFile$ = OpenFileRequester("Select MAME executible Source","C:\","Mame FILE | Mame*.exe", 0)
  If openFile$
    ;Repeat:event = WindowEvent():Until event = 0
    filename$ = openFile$
    openWindow_Progress("Retrieving list from MAME executible",0,100)
    While WindowEvent():Wend ;update window
    result = RunProgram(filename$," -ll","",#PB_Program_Open | #PB_Program_Read | #PB_Program_Hide)
    fileNum = OpenFile(#PB_Any,programPath$ + "listfull.txt")
    
    While Not AvailableProgramOutput(result):Wend
    Repeat
      a$ = ReadProgramString(result)
      If a$ <> ""
        WriteString(fileNum,a$)
      EndIf
    Until AvailableProgramOutput(result) = 0
    CloseFile(fileNum)
    CloseProgram(result)
    If FileSize(programPath$ + "listfull.txt") > 0
      gameCount = fillList(programPath$)
    EndIf
    closeWindow_Progress()
  EndIf
EndProcedure

CreateNewList()
CallDebugger
MessageRequester("Info","Parsed data for " + Str(gameCount) + " games")

Peter_DevRes · Post by **Peter_DevRes** » Fri Apr 03, 2009 10:19 pm

Thanks very much, I will go through the code and see what I can make of it.

Best Regards,
Peter

Ollivier · Post by **Ollivier** » Fri Apr 03, 2009 10:47 pm

@Peter

Mh? I can see you don't have headaches, so I give you this code.

For the example, it creates a near 440 megabytes sized text file with near 15 000 000 lines. This record part is between star commented lines.
It is always the three same lines, but you can test other algo.

The main goal is the call of the procedure named LoadStringArray().

Calling this procedure allow you to load all the text lines in a static array could be read or checked, etc...

I am satisfacted by its speed...

[Nota] It's only a method to load a text file and access to each text line. There is not a research algo as I can see in the code of Demivec.

Code: Select all

#FileOpened   = 1
#MemAllocated = 2
#FileLoaded   = 4
#TableCreated = 8

Structure BuildArrayInfo

   FileName.S
   ExecStatus.I

   *SeqBegin
   *SeqEnd
   SeqSize.I ; En octet

   *ArrayTable
   *ArrayTableEnd
   ArrayTableSize.I ; (nombre de pointeurs)

   LineMeanLength.I ; Taille moyenne d'une ligne
   
EndStructure

Structure TextLineInfo
   TextLine.S[1 << 24]
EndStructure

Procedure.L LoadStringArray(*Info.BuildArrayInfo)

   Protected *SeqBegin
   Protected *SeqEnd
   Protected *TextLine
   Protected *TableEnd
   Protected FileHnd.I
   Protected ExecStatus.I
   
   FileHnd = OpenFile(#PB_Any, *Info\FileName)
   If FileHnd
     
      ExecStatus | #FileOpened
      SeqSize.Q = Lof(FileHnd)
      *SeqBegin = AllocateMemory(SeqSize)
     
      If *SeqBegin
     
         ExecStatus | #MemAllocated
         *SeqEnd = *SeqBegin + SeqSize - 1
     
         If ReadData(FileHnd, *SeqBegin, SeqSize)
         
            ExecStatus | #FileLoaded
            CloseFile(FileHnd)
   
            *Info\SeqBegin = *SeqBegin
            *Info\SeqEnd = *SeqEnd
   
            If *Info\LineMeanLength = 0
               *Info\LineMeanLength = 10 ; Moyenne par défaut
            EndIf
   
            *Info\ArrayTableSize = ((*SeqEnd - *SeqBegin) / *Info\LineMeanLength) << 2
            Debug *Info\ArrayTableSize
            If *Info\ArrayTableSize < 1 << 8
               *Info\ArrayTableSize = 1 << 8
            EndIf
   
            ;Debug *Info\ArrayTableSize
            *Info\ArrayTable = AllocateMemory(*Info\ArrayTableSize)
            If *Info\ArrayTable

               ExecStatus | #TableCreated
               *TextLine = *Info\ArrayTable
               ! mov eax, 13           
               ! mov edi, [p.p_SeqBegin]
               ! mov ebp, [p.p_SeqEnd]
               ! mov edx, [p.p_TextLine]
   
               ! mov ecx, ebp ; ecx = EndSeq
               ! sub ecx, edi ;     - BeginSeq
               ! inc ecx      ;     + 1
LoadStringArrayLoop:
               ! mov ebx, edi ; Retient le début de la chaîne
               ! cld          ; Fixe le sens croissant (convention)
               ! repne scasb  ; Recherche le 13
   
               ! mov byte [edi - 1], 0 ; Remplace le 13 par le 0
               ! inc edi      ; Passe le 10
   
               ! mov [edx], ebx ; Copie l'adresse de début de ligne
               ! add edx, 4 ; ... Et passe au pointeur suivant
   
               ! cmp edi, ebp ; Fin de séquence ?
               ! jng l_loadstringarrayloop ; Non, continue
   
               ! mov [p.p_TableEnd], edx
               *Info\ArrayTableEnd = *TableEnd
               *Info\ArrayTableSize = *TableEnd - *TextLine
               ;Debug "***" + Str(*Info\ArrayTableSize)
               *Info\ArrayTable = ReAllocateMemory(*TextLine, *Info\ArrayTableSize)
            EndIf
         EndIf           
      EndIf
   EndIf
   *Info\ExecStatus = ExecStatus
EndProcedure

   Define *TextLine.TextLineInfo
   Define BuildArrayInfo.BuildArrayInfo
   
   ;___________________________________________________
;************************************************
   X.S = "Voici le Bonjour" + Chr(13) + Chr(10) + "Ici, et bien c'est le Au revoir" + Chr(13) + Chr(10) + "Et là, ben c'est simplement A bientôt" + Chr(13) + Chr(10)
   CreateFile(0, "Test.TXT")
   For I = 0 To 5000000
      WriteData(0, @X, Len(X) )
   Next
   CloseFile(0)
   Delay(500)
;***********************************************
   ;¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
   
   BuildArrayInfo\FileName = "Test.TXT"
   LoadStringArray(BuildArrayInfo)

   *TextLine = BuildArrayInfo\ArrayTable
   Debug *TextLine\TextLine[0]
   Debug *TextLine\TextLine[1]
   Debug *TextLine\TextLine[2]
   Debug str(BuildArrayInfo\ArrayTableSize >> 2) + "th line = "
   Debug *TextLine\TextLine[BuildArrayInfo\ArrayTableSize >> 2 - 1]

PureBasic Forums - English

Seek a Line in a File

Seek a Line in a File

Re: Seek a Line in a File

Re: Seek a Line in a File