Page 1 of 1
Posted: Fri Oct 19, 2001 11:36 pm
by BackupUser
Restored from previous forum. Originally posted by Franco.
This is code for adding a line in text files.
Like the other Procedures (GetLine,SetLine,MaxLine) it works only if you open a file with ReadFile(), before you use this command. Maybe some day (if Fred will change OpenFile) it will work with OpenFile. At the end of the procedure the file is reopened with ReadFile().
Code: Select all
; (c) 2001 - Franco's template - absolutely freeware
; Adds a line to a text file
; return value 1 = new line added
; return value 0 = function failed
Procedure AddLine(File,FileName$,Line,String$)
UseFile(File)
LineNumber=1
If Line>1
While LineNumber0
Size=SizeORG-Location
*Buffer=AllocateMemoryBank(1,Size,0)
FileSeek(Location)
ReadData(*Buffer,Size)
CloseFile(0)
If OpenFile(0, FileName$)
UseFile(0)
FileSeek(Location)
WriteStringN(String$)
WriteData(*Buffer,Size)
CloseFile(0)
If ReadFile(0, FileName$) And SizeORG<Lof()
ProcedureReturn 1
Else
ProcedureReturn 0
EndIf
EndIf
Else
ExitAddLineWhile:
ProcedureReturn 0
EndIf
Else
ProcedureReturn 0
EndIf
EndProcedure
Have a nice day...
Franco
(ReadMemory and WriteMemory renamed to ReadData and WriteData)
Edited by - Franco on 06 December 2001 23:14:29
Re: AddLine for Text Files
Posted: Wed May 18, 2011 12:27 pm
by IdeasVacuum
I want to do something like this but I'm not sure how this procedure ever worked. Of course the syntax has changed since 2001, but how does the line
work when all 3 vars are unassigned? Indeed, not sure about
either......
Re: AddLine for Text Files
Posted: Wed May 18, 2011 3:49 pm
by kenmo
Hmm. Do you want to insert a line of text (that is, increasing the number of lines) or replace a line of text?
There are a few ways to do both. I started writing an insertion procedure, but realized
(a) you might want a line replace procedure, and
(b) the method of reading/buffering and rewriting the file each time you change a line (like Franco did here) is probably inefficient. Might be better to store and change the text in an array or list, then just write a Flush() procedure that re-builds the text file only when you want.
Re: AddLine for Text Files
Posted: Wed May 18, 2011 3:53 pm
by IdeasVacuum
Hmm, I can insert a new string (new line) into the required location, but the next line in the file is overwritten:
Code: Select all
Procedure InsertLine(iFileID,sFileName.s,iLine.i,sNewLine.s,iMode.i)
;-------------------------------------------------------------------
sLine.s = ""
iSize.l = 0
iInsertPosn.l = 0
qFileLen.q = 0
iLen.i = 0
iExtra = 4
If((iMode = #PB_Ascii) Or (iMode = #PB_UTF8)) : iExtra = 2 : EndIf
If OpenFile(iFileID,sFileName)
For i = 1 To iLine
sLine = ReadString(iFileID,iMode)
iLen = StringByteLength(sLine,iMode)
iInsertPosn = iInsertPosn + iLen + iExtra
Next i
FileSeek(iFileID,iInsertPosn)
WriteStringN(iFileID,sNewLine,iMode)
CloseFile(iFileID)
EndIf
EndProcedure
Edit: I could of course read the entire file, close it, create it, write including new line, close it. However, seems like a sledge hammer to crack a nut:
Code: Select all
Procedure InsertLine(iFileID,sFileName.s,iLine.i,sNewLine.s,iMode.i)
;-------------------------------------------------------------------
sHeader.s = ""
sBody.s = ""
sLine.s = ""
If ReadFile(iFileID,sFileName)
For i = 1 To iLine
sLine = ReadString(iFileID,iMode)
sHeader = sHeader + sLine + #CRLF$
Next i
While Eof(iFileID) = 0
sLine = ReadString(iFileID,iMode)
sBody = sBody + sLine + #CRLF$
Wend
CloseFile(iFileID)
EndIf
If CreateFile(iFileID,sFileName)
WriteString(iFileID,sHeader,iMode)
WriteStringN(iFileID,sNewLine,iMode)
WriteString(iFileID,sBody,iMode)
CloseFile(iFileID)
EndIf
EndProcedure
Works well with a small file (approx 40 chars per line, 500 lines)
Re: AddLine for Text Files
Posted: Wed May 18, 2011 4:17 pm
by IdeasVacuum
Hi kenmo
Yeah, I want to insert a new line of text. For small files, the hammer-crack-nut method works well and for my current project it is fine. Still, there should be a 'best method' for any text file, big or small.
Re: AddLine for Text Files
Posted: Wed May 18, 2011 7:32 pm
by flaith
Hi,
just an idea :
using sqlite in memory import each line of the text file then use the sql command "insert" and then save the new file text ?
Re: AddLine for Text Files
Posted: Thu May 19, 2011 3:15 am
by netmaestro
just an idea :
using sqlite in memory import each line of the text file then use the sql command "insert" and then save the new file text ?
Yes, simple and elegant way to do it without reinventing the wheel. Only one downside: Place the line
UseSQLiteDatabase() anywhere in your code and you add 400k to the size of the executable. Not that that should necessarily bother you, I'm just saying. It costs 400k to avoid tackling the logic to insert a line of text where you want it. There's a term for programmers who feel that's a good deal, they're called .Net coders
p.s. No offense, I'm betting you didn't know about the 400k
Re: AddLine for Text Files
Posted: Thu May 19, 2011 3:34 am
by kenmo
Filesize overhead aside, doesn't using an SQL database for this seem like overkill anyway?
You'd have to read through the file, send Insert/Update/Select queries to the database, loop through them to write it out....
Might as well use PB's linked lists. Less overhead, probably slightly faster, no need for queries and string escaping.
Just use SelectElement() indexed by a line number, then you can InsertElement() or just change lines of text as necessary. Saving is also simple.
Re: AddLine for Text Files
Posted: Thu May 19, 2011 5:57 am
by IdeasVacuum
Using Linked List:
Code: Select all
Procedure InsertLine(iFileID,sFileName.s,iLine.i,sNewLine.s,iMode.i)
;-------------------------------------------------------------------
NewList sData.s()
If ReadFile(iFileID,sFileName)
While Eof(iFileID) = 0
AddElement(sData())
sData() = ReadString(iFileID)
Wend
CloseFile(iFileID)
EndIf
SelectElement(sData(),iLine)
InsertElement(sData())
sData() = sNewLine
If CreateFile(iFileID,sFileName)
FirstElement(sData())
For i = 1 To ListSize(sData())
WriteStringN(iFileID,sData(),iMode)
NextElement(sData())
Next
CloseFile(iFileID)
EndIf
FreeList(sData())
EndProcedure
Edit: The LinkedList method is much faster when processing a large-ish file (40 chars per line x 60,000 lines), but it still takes a while at around 15 seconds.
Re: AddLine for Text Files
Posted: Thu May 19, 2011 6:43 am
by kenmo
What sort of files are you working with? How often will you be writing to them?
When I suggested linked lists I assumed you'd load the file once, change the linked list while you use your program, and eventually save it back to the file.
That would be much faster, if you're making multiple changes to it.
Reading and writing the entire file every time you change one line is probably going to be pretty slow no matter how you do it...
(I would think ReadData and WriteData would be faster than thousands of ReadLines and WriteLines though.)
Re: AddLine for Text Files
Posted: Thu May 19, 2011 11:53 am
by flaith
netmaestro wrote:just an idea :
using sqlite in memory import each line of the text file then use the sql command "insert" and then save the new file text ?
Yes, simple and elegant way to do it without reinventing the wheel. Only one downside: Place the line
UseSQLiteDatabase() anywhere in your code and you add 400k to the size of the executable. Not that that should necessarily bother you, I'm just saying. It costs 400k to avoid tackling the logic to insert a line of text where you want it. There's a term for programmers who feel that's a good deal, they're called .Net coders
p.s. No offense, I'm betting you didn't know about the 400k
hehe, from you, there's no offense, i agree i totally forgot the 400kb
Re: AddLine for Text Files
Posted: Thu May 19, 2011 12:09 pm
by IdeasVacuum
What sort of files are you working with? How often will you be writing to them?
I am working with relatively small ASCII text files, 500 to 2000 lines. The lines (strings) are around 40 chars but vary in length. Both of the hammer-nut-crack Procedures I have posted are perfectly fine for my app, even though they do need to be saved every time a line is inserted (which is good practise in terms of preserving the data).
However, I'm minded that someone else will one day make the same forum search, so it would be good to have a defacto solution. Read/Write data is probably it, but it would seem to require finding the end of each line (CR, CR/LF), which could be messy.......
Re: AddLine for Text Files
Posted: Fri May 20, 2011 1:01 am
by IdeasVacuum
Here is an attempt using ReadData-WriteData:
Code: Select all
Procedure InsertLine(iFileID,sFileName.s,iLine.i,sNewLine.s,iMode.i)
;-------------------------------------------------------------------
sTest.s = ""
iFileLen.l = 0
iReadLen.l = 0
iStrLen.l = 0
iHdrLen.l = 0
If ReadFile(iFileID,sFileName)
iFileLen = Lof(iFileID)
Debug "File Length-->" + Str(iFileLen)
If(iFileLen > 0)
*MemFile = AllocateMemory(iFileLen)
*MemCopy = AllocateMemory(iFileLen)
*MemPosn = *MemFile
If *MemFile
iReadLen = ReadData(iFileID,*MemFile,iFileLen)
EndIf
CloseFile(iFileID)
If(iFileLen = iReadLen)
; Collection of lines ('File Header') before insertion position
For i = 1 To iLine
sTest = PeekS(*MemPosn,-1,iMode)
iStrLen = StringByteLength(sTest,iMode)
Debug "Line " + Str(i) + " Length-->" + Str(iStrLen)
*MemPosn = *MemPosn + iStrLen
iHdrLen = iHdrLen + iStrLen
Next
Debug "Header Length-->" + Str(iHdrLen)
; line to be inserted
iStrLen = StringByteLength(sNewLine,iMode)
*MemNewLine = AllocateMemory(iStrLen)
PokeS(*MemNewLine,sNewLine,iStrLen,iMode)
If CreateFile(iFileID,sFileName)
WriteData(iFileID,*MemFile,iHdrLen)
WriteData(iFileID,*MemNewLine,iStrLen)
WriteData(iFileID,*MemPosn,(iFileLen - iHdrLen))
CloseFile(iFileID)
EndIf
Else
MessageRequester("Memory Issue","File read failed",#PB_MessageRequester_Ok)
EndIf
Else
MessageRequester("File Issue","File empty",#PB_MessageRequester_Ok)
EndIf
EndIf
EndProcedure
;### Test: insert a string @ line 5 ###
InsertLine(0,"C:\TestFile.txt",5,"Inserted Line",#PB_Ascii)
End
....it's no good though, the first PeekS collects the entire file!
The test file "C:\TestFile.txt" contents:
Test Line 1111111111
Test Line 222222222222222
Test Line 33333333333333333333
Test Line 4444444444444444444444444
Test Line 555555555555555555555555555555
Test Line 66666666666666666666666666666666666
Test Line 7777777777777777777777777777777777777777
Test Line 888888888888888888888888888888888888888888888
Test Line 99999999999999999999999999999999999999999999999999
Re: AddLine for Text Files
Posted: Fri May 20, 2011 3:48 am
by idle
you could also use a map and list storing the address from delimited words stored in the map
it'd cut down on memory and allow you to manage large text files and do multiple inserts, deletes and replaces.
If you wanted it to be line based you'd need to store line numbers in the list
Code: Select all
Structure fileIndex
index.i
lineNumber.i
EndStructure
something like this.
Read in data
Code: Select all
While Not Eof(fn)
LineNumber + 1
line = ReadString(fn)
ct = CountString(line,delimiter)
For a = 1 To ct +1
word.s = StringField(line,a,delimiter)
textmap(word)=word
AddElement(index())
index()\index = @textmap(word)
index()\line = LineNumber
;add delimeter to?
Next
Wend
insert data
Code: Select all
FirstElement(index())
For a = 1 To position
NextElement(index())
Next
LineNumber = index()\line
ct = CountString(string,delimiter)
For a = 1 To ct +1
word.s = StringField(string,a,delimiter)
textmap(word)=word
AddElement(index())
index()\index = @textmap(word)
index()\line = LineNumber
;add delimiter
Next
write it out
Code: Select all
FileSeek(fn,0)
ForEach index()
word = PeekS(PeekI(index()\index))
WriteString(fn,word)
Next
CloseFile(fn)
Re: AddLine for Text Files
Posted: Fri May 20, 2011 4:35 am
by IdeasVacuum
Hello Idle
I think reading a large file line-by-line is too slow, as per my earlier example using LinkedList.
See also
http://www.purebasic.fr/english/viewtop ... 13&t=46407