Page 1 of 1
Seek a Line in a File
Posted: Thu Apr 02, 2009 11:58 pm
by Peter_DevRes
Hi,
I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.
I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.
I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.
The text below shows essentially what I would put in the loop, just struggling to find the right PureBasic commands.
Hope someone can help me.
...
...
If Readfile(0,"C:\whateverfile.txt")
length=lof(0)
*MemoryID=AllocateMemory(length)
whileEof(0)=0
<break the line of text in to substrings and then insert the substrings into a SQL statement>
wend
Else
MessageRequester(" Information ","Couldn't retrieve the text""
EndIf
...
...
Thanks,
Peter.
Re: Seek a Line in a File
Posted: Fri Apr 03, 2009 3:49 am
by Demivec
Peter_DevRes wrote:I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.
I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.
I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.
Look in the manual under the General Libraries for "File" and "String".
More specifically, "FileSeek()", "ReadString()", "CountString()", "FindString()", "StringField()".
I will post an example that may be helpful to you. I will post it at the earliest opportunity. Synchronize your watch, it will be in 10 hours (from the time of this message), mark.
Posted: Fri Apr 03, 2009 10:22 am
by sverson
this may help a little
Code: Select all
MyString$ = ""
LineNo.l = 0
FileNo.l = ReadFile(#PB_Any,"C:\whateverfile.txt")
If FileNo
While Not Eof(FileNo)
MyString$ = ReadString(FileNo)
If MyString$<>""
If LineNo=0
SQLf$ = "INSERT INTO MyTable ("
Elements = CountString(MyString$,";")
For Element = 1 To Elements
SQLf$ + StringField(MyString$,Element,";")
If Element<Elements
SQLf$ + ", "
EndIf
Next
SQLf$ + ") VALUES("
Else
SQLv$ = ""
Elements = CountString(MyString$,";")
For Element = 1 To Elements
SQLv$ + "'" + StringField(MyString$,Element,";") + "'"
If Element<Elements
SQLv$ + ", "
EndIf
Next
SQLv$ + ") "
SQL$ = ReplaceString(SQLf$+SQLv$,"'",#DOUBLEQUOTE$)
Debug SQL$
EndIf
LineNo + 1
EndIf
Wend
CloseFile(FileNo)
EndIf
Posted: Fri Apr 03, 2009 12:58 pm
by Peter_DevRes
Hi,
Thanks for taking the time to go in that level detail, appreciated your assistance. Looks like an interesting approach.
Thanks & Regards,
Peter.
Re: Seek a Line in a File
Posted: Fri Apr 03, 2009 1:00 pm
by Peter_DevRes
Demivec wrote:Peter_DevRes wrote:I am a little new to PureBasic, in the former language I was programming in to seek a line in a file you use a command @line.
I don't seem to find a similar function in PureBasic. Been reading through the manual, is anyone able to help.
I basically want to read a pre-formated file in to a database by splitting the string in to substings and inserting the substrings in to an SQL command, but can't seem to find a very basic command that controls the pointer line number in a loop and how to address the line in the loop.
Look in the manual under the General Libraries for "File" and "String".
More specifically, "FileSeek()", "ReadString()", "CountString()", "FindString()", "StringField()".
I will post an example that may be helpful to you. I will post it at the earliest opportunity. Synchronize your watch, it will be in 10 hours (from the time of this message), mark.
Great will be interested in how you approach this problem, looking forward to it.
Thanks,
Peter
Posted: Fri Apr 03, 2009 1:15 pm
by sverson
sorry i forgot the textfile
C:\whateverfile.txt:
Code: Select all
name;road;city;
Smith;Old London Rd.;Hell;
Marx;New London Rd;Heaven;
Posted: Fri Apr 03, 2009 1:43 pm
by Peter_DevRes
Right, great.
Thanks,
Peter.
Posted: Fri Apr 03, 2009 3:55 pm
by Demivec
Here's a portion of one of my programs. I had a feeling you might need something along these lines. If I've guessed wrong, you might still find some usefulness out of it.
It does demonstrate reading and parsing text from a file.
Code: Select all
;Description: subroutines produce a list of games and their descriptions from a MAME executible
;Author: Demivec (Jared)
;Date: 11/13/2008
;Using: PureBasic v4.30
Enumeration ;Windows
#Progress_win
EndEnumeration
Enumeration
#PbarDescription_txt
#ProgressBar_pbr
EndEnumeration
Structure gameData
name$
description$
EndStructure
Global programPath$ = GetPathPart(ProgramFilename())
Global gameCount
Global NewList gameList.gameData() ;holds all game data (Name,Description)
Procedure.l openWindow_Progress(p_text.s,p_min.l,p_max.l) ;setup Progress window and its gadgets
If OpenWindow(#Progress_win, 538, 605, 279, 64, "Progress...", #PB_Window_SystemMenu | #PB_Window_TitleBar | #PB_Window_WindowCentered)
TextGadget(#PbarDescription_txt, 10, 10, 260, 20, p_text,#PB_Text_Center)
StickyWindow(#Progress_win,1)
ProcedureReturn 1 ;successfully open window
Else
ProcedureReturn -1 ;couldn't open window
EndIf
EndProcedure
Procedure closeWindow_Progress() ;close Progress window
StickyWindow(#Progress_win,0)
CloseWindow(#Progress_win)
EndProcedure
Procedure.l fillList(p_sourcePath$) ;add new elements from file 'listfull.txt'
Protected a$,b$,c$,fileNum.l,beginning.l,middle.l,finish.l,length.l,i.l,j.l,event.l
CallDebugger
SetGadgetText(#PbarDescription_txt,"Parsing names and adding them to list...")
While WindowEvent():Wend ;update window
Delay(1)
fileNum = OpenFile(#PB_Any,p_sourcePath$ + "listfull.txt")
a$ = ReadString(fileNum)
CloseFile(fileNum)
length = Len(a$)
beginning = FindString(a$, ":", 1)
beginning = FindString(a$, ":", beginning + 1) + 1
While beginning < length
b$ = Trim(Mid(a$,beginning,8))
middle = beginning + 11
finish = FindString(a$,Chr(34),middle) ;skip over 2 spaces and a quote
c$ = Mid(a$,middle,finish - middle)
beginning = finish+1
AddElement(gameList())
gameList()\name$ = b$
gameList()\description$ = c$
SetGadgetText(#PbarDescription_txt,"Parsed names for " + Str(ListSize(gameList())) + " games.")
While WindowEvent():Wend ;swallow events and update window
Wend
DeleteFile(p_sourcePath$ + "listfull.txt")
ProcedureReturn ListSize(gameList())
EndProcedure
Procedure CreateNewList() ;create 'listfull.txt' from Mame executible
Protected openFile$, fileNum.l,result.l,a$,event.l
openFile$ = OpenFileRequester("Select MAME executible Source","C:\","Mame FILE | Mame*.exe", 0)
If openFile$
;Repeat:event = WindowEvent():Until event = 0
filename$ = openFile$
openWindow_Progress("Retrieving list from MAME executible",0,100)
While WindowEvent():Wend ;update window
result = RunProgram(filename$," -ll","",#PB_Program_Open | #PB_Program_Read | #PB_Program_Hide)
fileNum = OpenFile(#PB_Any,programPath$ + "listfull.txt")
While Not AvailableProgramOutput(result):Wend
Repeat
a$ = ReadProgramString(result)
If a$ <> ""
WriteString(fileNum,a$)
EndIf
Until AvailableProgramOutput(result) = 0
CloseFile(fileNum)
CloseProgram(result)
If FileSize(programPath$ + "listfull.txt") > 0
gameCount = fillList(programPath$)
EndIf
closeWindow_Progress()
EndIf
EndProcedure
CreateNewList()
CallDebugger
MessageRequester("Info","Parsed data for " + Str(gameCount) + " games")
Posted: Fri Apr 03, 2009 10:19 pm
by Peter_DevRes
Thanks very much, I will go through the code and see what I can make of it.
Best Regards,
Peter
Posted: Fri Apr 03, 2009 10:47 pm
by Ollivier
@Peter
Mh? I can see you don't have headaches, so I give you this code.
For the example, it creates a near 440 megabytes sized text file with near 15 000 000 lines. This record part is between star commented lines.
It is always the three same lines, but you can test other algo.
The main goal is the call of the procedure named LoadStringArray().
Calling this procedure allow you to load all the text lines in a static array could be read or checked, etc...
I am satisfacted by its speed...
[Nota] It's only a method to load a text file and access to each text line. There is not a research algo as I can see in the code of Demivec.
Code: Select all
#FileOpened = 1
#MemAllocated = 2
#FileLoaded = 4
#TableCreated = 8
Structure BuildArrayInfo
FileName.S
ExecStatus.I
*SeqBegin
*SeqEnd
SeqSize.I ; En octet
*ArrayTable
*ArrayTableEnd
ArrayTableSize.I ; (nombre de pointeurs)
LineMeanLength.I ; Taille moyenne d'une ligne
EndStructure
Structure TextLineInfo
TextLine.S[1 << 24]
EndStructure
Procedure.L LoadStringArray(*Info.BuildArrayInfo)
Protected *SeqBegin
Protected *SeqEnd
Protected *TextLine
Protected *TableEnd
Protected FileHnd.I
Protected ExecStatus.I
FileHnd = OpenFile(#PB_Any, *Info\FileName)
If FileHnd
ExecStatus | #FileOpened
SeqSize.Q = Lof(FileHnd)
*SeqBegin = AllocateMemory(SeqSize)
If *SeqBegin
ExecStatus | #MemAllocated
*SeqEnd = *SeqBegin + SeqSize - 1
If ReadData(FileHnd, *SeqBegin, SeqSize)
ExecStatus | #FileLoaded
CloseFile(FileHnd)
*Info\SeqBegin = *SeqBegin
*Info\SeqEnd = *SeqEnd
If *Info\LineMeanLength = 0
*Info\LineMeanLength = 10 ; Moyenne par défaut
EndIf
*Info\ArrayTableSize = ((*SeqEnd - *SeqBegin) / *Info\LineMeanLength) << 2
Debug *Info\ArrayTableSize
If *Info\ArrayTableSize < 1 << 8
*Info\ArrayTableSize = 1 << 8
EndIf
;Debug *Info\ArrayTableSize
*Info\ArrayTable = AllocateMemory(*Info\ArrayTableSize)
If *Info\ArrayTable
ExecStatus | #TableCreated
*TextLine = *Info\ArrayTable
! mov eax, 13
! mov edi, [p.p_SeqBegin]
! mov ebp, [p.p_SeqEnd]
! mov edx, [p.p_TextLine]
! mov ecx, ebp ; ecx = EndSeq
! sub ecx, edi ; - BeginSeq
! inc ecx ; + 1
LoadStringArrayLoop:
! mov ebx, edi ; Retient le début de la chaîne
! cld ; Fixe le sens croissant (convention)
! repne scasb ; Recherche le 13
! mov byte [edi - 1], 0 ; Remplace le 13 par le 0
! inc edi ; Passe le 10
! mov [edx], ebx ; Copie l'adresse de début de ligne
! add edx, 4 ; ... Et passe au pointeur suivant
! cmp edi, ebp ; Fin de séquence ?
! jng l_loadstringarrayloop ; Non, continue
! mov [p.p_TableEnd], edx
*Info\ArrayTableEnd = *TableEnd
*Info\ArrayTableSize = *TableEnd - *TextLine
;Debug "***" + Str(*Info\ArrayTableSize)
*Info\ArrayTable = ReAllocateMemory(*TextLine, *Info\ArrayTableSize)
EndIf
EndIf
EndIf
EndIf
*Info\ExecStatus = ExecStatus
EndProcedure
Define *TextLine.TextLineInfo
Define BuildArrayInfo.BuildArrayInfo
;___________________________________________________
;************************************************
X.S = "Voici le Bonjour" + Chr(13) + Chr(10) + "Ici, et bien c'est le Au revoir" + Chr(13) + Chr(10) + "Et là, ben c'est simplement A bientôt" + Chr(13) + Chr(10)
CreateFile(0, "Test.TXT")
For I = 0 To 5000000
WriteData(0, @X, Len(X) )
Next
CloseFile(0)
Delay(500)
;***********************************************
;¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
BuildArrayInfo\FileName = "Test.TXT"
LoadStringArray(BuildArrayInfo)
*TextLine = BuildArrayInfo\ArrayTable
Debug *TextLine\TextLine[0]
Debug *TextLine\TextLine[1]
Debug *TextLine\TextLine[2]
Debug str(BuildArrayInfo\ArrayTableSize >> 2) + "th line = "
Debug *TextLine\TextLine[BuildArrayInfo\ArrayTableSize >> 2 - 1]