File Manipulation

chen · Post by **chen** » Fri Jan 20, 2006 6:26 am

Hello there,

Could some one of you with some free seconds put a link where
I can read file IO in PureBasic something deep?.....
I have read the manual.... and some examples.....

I have this kind of doubts...
When I open a file ( OpenFile(#File, FileName$) ) could I read and
modify the file..? or Do I need to open a second file and move line by
line from the first one to the second , changing in this process the
lines I requiere? And rename the second file to the first one at the end
of the process

I Appreciate your help

blueznl · Post by **blueznl** » Fri Jan 20, 2006 7:27 am

you can read and modify

Post by **Dare2** » Fri Jan 20, 2006 8:42 am

Hi chen,

PureBasic allows a file to be read from and written to.

As to whether that is workable for you, it depends on what you are doing.

If you are relying on ReadString(), WriteStringN() and the string sizes change, you could have an issue with one file.

eg:

ABC<>DEF<>GHI<>

where the <> represents end of line characters (Chr(13) and Chr(10) for windows). Then

Code: Select all

ptr=Loc()     ; Storing current position in file
w.s=ReadString()  ; Reading string (getting ABC)
If w="ABC"
  w="MYABC"
  FileSeek(ptr)     ; Returning to start of "ABC"
  WriteStringN(w)  ; Overwriting "ABC<>DE" with "MYABC<>"
EndIf

So this would give you

MYABC<>F<>GHI<>

Clobbering DE of DEF

However if all "fields" read in and written out are going to be of the same type (or actually length) when updated, no problems. Just use Loc() to store the current position in the file so you can FileSeek(toIt) for the write. After the write Loc() is the old position plus the length of the info written. Eg, 4 bytes for a long.

If you are not sure and there is no issue with speed, diskspace, etc, then read from one and write to another. Don't forget to use UseFile(fileNum) before each read and write sequence though, so PureBasic knows which file to read from and which to write to.

Sorry if that was too simplistic. To too obscure.

chen · Post by **chen** » Fri Jan 20, 2006 9:51 pm

Dare2, Thanks for your help....

The goal of this program is to modify strings in a file.
Then, the program looks for strings and replace them...

Here is my code(It works!).. Could you make some suggestions to
improve it?.... speed, code style....

Code: Select all

Procedure.l Replace(searchSt.s, replaceSt.s, fileWithPath$)
;Read file1, Change Some Strings, write in file 2
;Delete file1, rename file2 to file1
;Return 1 -> Success
;       0 -> Can't open original File or is not a valid file
;      -1 -> Can't open new file or is not a valid file
;      -2 -> Length original file doesn't match with new length file + changes length
;      -3 -> Couldnt delete original file
;      -4 -> Rename new file to old file Fail.

Enumeration 
  #oriFile ; Original file
  #newFile ; New file
EndEnumeration

success.l = 1
searchStL.l = Len(searchSt)            ; Used at end to compare length of original and new files
replaceStL.l = Len(replaceSt)          ; Used at end to compare length of original and new files
oriFileLength.l = 0
newFileLength.l = 0
numReplacements.l=0                    ; Used at end to compare length of original and new files


If ReadFile(#oriFile, fileWithPath$) 
  If IsFile(#oriFile)
    ;------ Original File Data ------------------
    oriPath$ = GetPathPart(fileWithPath$)
    oriFileName$ = GetFilePart(fileWithPath$)
    oriFileLength = Lof()
    ;----------  New File Data ------------------
    newPath$ = oriPath$
    newFileName$ = "tmpxx_"+oriFileName$
    newFileWithPath$ = newPath$ + newFileName$
    ;---------------------------------------------
    If OpenFile(#newFile, newFileWithPath$) ;Create newFile (includes path)
      If IsFile(#newFile)
        While Eof(#oriFile) = 0
          UseFile(#oriFile)
          Text$ = ReadString()
          String$ = ReplaceString(Text$, searchSt, replaceSt)
          numReplacements = numReplacements + CountString(Text$, searchSt)
          UseFile(#newFile)
          WriteStringN(String$)
        Wend
      Else
        success = -1
      EndIf  
    Else
      success -1
    EndIf 
    CloseFile(#oriFile)
    UseFile(#newFile)
    newFileLength = Lof()                       
    CloseFile(#newFile)
    If oriFileLength = (newFileLength-((replaceStL*numReplacements)-(searchStL*numReplacements)))
      If DeleteFile(fileWithPath$)
        If RenameFile(newFileWithPath$, fileWithPath$) = 0
          success = -4
        EndIf
      Else
        success = -3
      EndIf 
    Else
      succes = -2
    EndIf 
  Else 
    success = 0
  EndIf 
Else 
  success = 0
EndIf 

ProcedureReturn success
EndProcedure 

begin$ = "Who do you know"
ending$ = "ZZZZZZZZZZZZ"
Replace(begin$,ending$,"C:\Documents And Settings\CZarza\Desktop\back\Rapid Belief Change.txt")

thanks  :)

Post by **Dare2** » Sat Jan 21, 2006 2:58 pm

Hi chen,

There are more clued-up coders than I around, so someone else may be able/willing to give you some advanced pointers.

FWIW, I think your code is nice and clean, and it looks okay to me. In fact you do more checking for errors (If OpenFile ... If isFile) than I do.

I didn't run your code, just looked at it, mainly the read/write loop. Only things I noticed are:

1: You might want to move your CloseFiles up a bit, so only closing if successfully opened.
2: You might prefer to use CreateFile rather than OpenFile for your destination file. This will give you a fresh empty file. Otherwise if you overwrite an existing file and the result is smaller, you'll have strays at the end of it.

Eg, with bits chopped out::)

Code: Select all

If ReadFile(#oriFile, fileWithPath$)
  If CreateFile(#newFile, newFileWithPath$)
    While Eof(#oriFile) = 0
      UseFile(#oriFile)
      Text$ = ReadString()
      String$ = ReplaceString(Text$, searchSt, replaceSt)
      UseFile(#newFile)
      WriteStringN(String$)
    Wend
    newFileLength = Lof()                       
    CloseFile(#newFile)
  Else
    success = -1
  EndIf 
  CloseFile(#oriFile)
Else
  success -1
EndIf

As I said earlier, there are many people smarter than I on these boards, so maybe someone else will give feedback as well. (And Trond would prefer you used ReadData

and j/k)

BTW, I just realised this is in General Discussion, these sorts of posts might be better in coding questions.

Trond · Post by **Trond** » Sat Jan 21, 2006 4:00 pm

Yup, ReadData would be faster.

Here's one with ReadData, but the strings must be the same size:

Code: Select all

;Logical not
Procedure.l Not(b.l) 
  !CMP  DWord [ESP], 0
  !JZ   l_cl_0
  !XOR  eax, eax
  ProcedureReturn
  cl_0:
  !MOV  eax, 1
  ProcedureReturn
EndProcedure

;Check a result and ev. abort
Procedure Chk(a.l, s.s)
  ;If a is false, abort.
  ;If a is false and s contains an error message, show it before abort.
  ;If a is not false, return a
  If Not(a)
    If s.s
      MessageRequester("", s.s)
    EndIf
    End
  EndIf
  ProcedureReturn a
EndProcedure

Procedure.l MemoryFileStringReplace(FindStr.s, ReplaceStr.s, Filename.s)
  Protected Lof.l, Memory.l
  Chk( OpenFile(0, Filename), "Couldn't open file " + Filename)
  Lof = Lof()
  Memory = Chk( AllocateMemory(Lof), "Couldn't allocate memory")
  Chk( Len(FindStr)=Len(ReplaceStr) Or 0, "Strings not same length")
  SLen = Len(FindStr)
  ReadData(Memory, Lof)
  Debug SLen
  For I=0 To Lof-SLen Step 1
    If CompareMemory(@FindStr, Memory+I, SLen)
      Debug "equal"
      CopyMemory(@ReplaceStr, Memory+I, SLen)
    EndIf
  Next
  FileSeek(0)
  WriteData(Memory, Lof)
  CloseFile(0)
EndProcedure

MemoryFileStringReplace("rt", "!!", "c:\w00t.txt")

Post by **Dare2** » Sat Jan 21, 2006 4:06 pm

HeHe.

But everything in it's place. I use ReadData a lot. I wouldn't use it here. Because of the string sizing and the memory manipulations that would entail.

Trond · Post by **Trond** » Sat Jan 21, 2006 4:38 pm

Dare2 wrote:HeHe.

But everything in it's place. I use ReadData a lot. I wouldn't use it here. Because of the string sizing and the memory manipulations that would entail.

I'm working on it while trying to keep my code shorter than one page.

Here's version two, the replace with string can now be shorter than the find string (but not the other way around):

Code: Select all


;Logical not
;Copy and paste from above

;Check a result and ev. abort
;Copy and paste from above

Procedure.l MemoryFileStringReplace(FindStr.s, ReplaceStr.s, Filename.s)
  Protected Lof.l, Memory.l, FSLen, RSLen, Exp, Skip
  Chk( OpenFile(0, Filename), "Couldn't open file " + Filename)
  Lof = Lof()
  Chk( Len(FindStr)>=Len(ReplaceStr) Or 0, "Replace string longer than find string")
  Memory = Chk( AllocateMemory(Lof), "Couldn't allocate memory")
  FSLen = Len(FindStr)
  RSLen = Len(ReplaceStr)
  DSLen = FSLen - RSLen
  ReadData(Memory, Lof)
  Exp = Lof-SLen
  For I=0 To Exp Step 1
    If CompareMemory(@FindStr, Memory+I, FSLen)
      CopyMemory(@ReplaceStr, Memory+I, RSLen)
      If DSLen
        CopyMemory(Memory+I+FSLen, Memory+I+RSLen, Lof-I-FSLen)
        Skip + DSLen
        Exp - DSLen
        I + FSLen
      EndIf
    EndIf
  Next
  FileSeek(0)
  WriteData(Memory, Lof-Skip)
  CloseFile(0)
EndProcedure


MemoryFileStringReplace("rt", "!", "c:\w00t.txt"

Post by **Dare2** » Sat Jan 21, 2006 5:41 pm

I thought I might give you a run for your money, but I decided to back you instead. So my money is on you.

Challenge: Can you do the whole thing, including necessary supporting routines, in 35 lines or less of PureBasic code?

And this sort of thing is cheating, he wrote:If exp : do things : EndIf

However, kindly, he wrote:But you can drop comment and blank lines!

Trond · Post by **Trond** » Sat Jan 21, 2006 5:47 pm

Dare2 wrote:

I thought I might give you a run for your money, but I decided to back you instead. So my money is on you.

Challenge: Can you do the whole thing, including necessary supporting routines, in 35 lines or less of PureBasic code?

And this sort of thing is cheating, he wrote:If exp : do things : EndIf

However, kindly, he wrote:But you can drop comment and blank lines!

Using ReadData()? That'll be hard.

Edit: I'll have to de-optimize it to get it that short, so maybe the other one will be faster then...

Edit: With error checks?

Post by **Dare2** » Sat Jan 21, 2006 5:55 pm

Yes.

That was tongue in cheek, sorry. You're around 40-50 lines now.

But going good!

Edit:

Posting (and editing?) at the same time.

Okay then, make it size -v- speed that counts, and forget the error checking (assume file exists, etc). You reckon you can manage 35 or less?

Trond · Post by **Trond** » Sat Jan 21, 2006 6:00 pm

Definetely if I ignore performance altogether, but we'll see if I can make the memory consumption a bit more reasonable.

Post by **Dare2** » Sat Jan 21, 2006 6:41 pm

Ack! No challenge.

I can do it in less than 30! (With a small assumption). So you'll have do better than that!

Edit:

Testdata

ASD ASaaaBC aaa
JKL MNO aaa XYZ
ASD

Hello aaa, and goodbye!

Code: Select all

Procedure ChangeStrings(Filename.s,oldStr.s,newStr.s)
  dif=Len(oldStr)-Len(newStr)
  OpenFile(0, Filename)
  sz=Lof()*2     ; Cheat #1 - assumes file won't grow 2x
  mem=AllocateMemory(sz)
  ReadData(mem,sz)
  ptr=0
  While ptr<=(sz/2)+ctr
    If CompareMemory(@oldStr,mem+ptr,Len(oldStr))
      If dif<0
        CopyMemory(mem+ptr,mem+ptr+(dif * -1),sz-(ptr + (dif * -1)))
        CopyMemory(@newStr,mem+ptr,Len(newStr))
      ElseIf dif>0
        CopyMemory(@newStr,mem+ptr,Len(newStr))
        CopyMemory(mem+ptr+Len(oldStr),mem+ptr+Len(newStr),sz-(ptr + (dif * -1)))
      Else
        CopyMemory(@newStr,mem+ptr,Len(newStr))
      EndIf
      ctr-dif
      ptr+Len(oldStr)+dif
    Else
      ptr+1
    EndIf
  Wend
  FileSeek(0)
  WriteData(mem,Lof()+ctr)
  CloseFile(0)
EndProcedure
;ChangeStrings("test.txt","aaa","bbbb")
;ChangeStrings("test.txt","aaa","bb")
ChangeStrings("test.txt","aaa","bbb")

Assumes file won't grow to double size. Should really get ratio between old and new.

And leaves trailing oldstuff in file if new is shorter than old.

But came in under 30.

Trond · Post by **Trond** » Sat Jan 21, 2006 7:16 pm

Code: Select all

  sz=Lof()*2     ; Cheat #1 - assumes file won't grow 2x

Hey, that's cheating!

I'm still working on mine, it doesn't work yet, but it's not much time left until it does.

Trond · Post by **Trond** » Sat Jan 21, 2006 7:43 pm

Arrrggg, I give up! After getting it to work I realized that I was using an unnecessary extra buffer (an idea that should conserve memory) and that the code wouldn't be short enough!

Here's yours in 24 lines, though:

Code: Select all

Procedure ChangeStrings(Filename.s,oldStr.s,newStr.s) 
  OpenFile(0, Filename) 
  mem=AllocateMemory(Lof()*2) 
  ReadData(mem,Lof()*2) 
  While ptr<=((Lof()*2)/2)+ctr 
    If CompareMemory(@oldStr,mem+ptr,Len(oldStr)) 
      If Len(oldStr)-Len(newStr)<0 
        CopyMemory(mem+ptr,mem+ptr+((Len(oldStr)-Len(newStr)) * -1),(Lof()*2)-(ptr + ((Len(oldStr)-Len(newStr)) * -1))) 
        CopyMemory(@newStr,mem+ptr,Len(newStr)) 
      ElseIf Len(oldStr)-Len(newStr)>0 
        CopyMemory(@newStr,mem+ptr,Len(newStr)) 
        CopyMemory(mem+ptr+Len(oldStr),mem+ptr+Len(newStr),(Lof()*2)-(ptr + ((Len(oldStr)-Len(newStr)) * -1))) 
      Else 
        CopyMemory(@newStr,mem+ptr,Len(newStr)) 
      EndIf 
      ctr-(Len(oldStr)-Len(newStr))
      ptr+(Len(oldStr)+(Len(oldStr)-Len(newStr)))-1
    EndIf
    ptr+1 
  Wend 
  FileSeek(0) 
  WriteData(mem,Lof()+ctr) 
  CloseFile(0) 
EndProcedure