Page 1 of 1
How to get stdout from Linux terminal?
Posted: Wed Mar 03, 2021 3:53 pm
by AZJIO
How to get stdout from Linux terminal? I already broke my brain.
Code: Select all
Procedure.s ExeComStr(Execut$, ComStr$)
Protected tmp, res$
tmp = RunProgram(Execut$, ComStr$, "", #PB_Program_Open | #PB_Program_Read)
; Debug Execut$
; Debug ComStr$
res$ = ""
If tmp
While ProgramRunning(tmp)
If AvailableProgramOutput(tmp)
res$ + ReadProgramString(tmp) + Chr(10)
EndIf
Wend
CloseProgram(tmp)
EndIf
ProcedureReturn res$
EndProcedure
; find '/home/user/' -regex '.+/[A-Za-z]+\.\w\w' -type f
tmp$ = "'/home/user/' -regex '.+/[A-Za-z]+\.\w\w' -type f"
tmp$ = ExeComStr("gnome-terminal", "-- bash -c find " + tmp$ + " 2>&1")
; tmp$ = ExeComStr("gnome-terminal", "-- bash -c " + Chr(34) + "find " + tmp$ + " 2>&1" + Chr(34))
Debug tmp$
Re: How to get stdout from Linux terminal?
Posted: Wed Mar 03, 2021 3:55 pm
by NicTheQuick
Just execute "find" directly without any "gnome-terminal" or "bash". But you need the full path then. I try to make an example for you. Please wait.
Re: How to get stdout from Linux terminal?
Posted: Wed Mar 03, 2021 4:04 pm
by NicTheQuick
Instead of ReadProgramData you also could use ReadProgramString I guess. But I had more fun with ReadProgramData.
Code: Select all
EnableExplicit
#BUFFER_SIZE = 1024
Procedure.s getOutput(command.s, args.s)
Protected hProg.i
Protected stdout.s, stderr.s
Protected avail.i, *buffer = AllocateMemory(#BUFFER_SIZE)
hProg = RunProgram(command, args, "", #PB_Program_Open | #PB_Program_Read | #PB_Program_Error | #PB_Program_UTF8)
If hProg
While ProgramRunning(hProg)
avail = AvailableProgramOutput(hProg)
If avail
If avail > #BUFFER_SIZE
avail = #BUFFER_SIZE
EndIf
avail = ReadProgramData(hProg, *buffer, avail)
stdout + PeekS(*buffer, avail, #PB_ByteLength | #PB_UTF8)
EndIf
stderr + ReadProgramError(hProg)
Wend
CloseProgram(hProg)
EndIf
FreeMemory(*buffer)
If stderr
Debug "======ERROR====="
Debug stderr
Debug "================"
EndIf
ProcedureReturn stdout
EndProcedure
Debug getOutput("/usr/bin/find", "/home/nicolas/tmp -type d")
Debug getOutput("/usr/bin/find", "/home/nicolas/tmp -type r")
Re: How to get stdout from Linux terminal?
Posted: Wed Mar 03, 2021 4:06 pm
by AZJIO
if i use "find" i can't add "|sort"
download:
yandex,
upload.ee
Re: How to get stdout from Linux terminal?
Posted: Wed Mar 03, 2021 4:12 pm
by NicTheQuick
Ah, you want to concatenate commands like bash can do it?
Then just call it like so:
Code: Select all
Debug getOutput("/bin/bash", ~"-c \"find /home/nicolas/tmp -type d | sort\"")
Re: How to get stdout from Linux terminal?
Posted: Sat Mar 06, 2021 11:42 am
by AZJIO
I've done a call count (b). Initially it was b=1500, but when I increased #BUFFER_SIZE 10 times, then the number of calls became 150. Increasing the size even more did not reduce the number of calls.
Code: Select all
EnableExplicit
; #BUFFER_SIZE = 1024
#BUFFER_SIZE = 10240
Procedure getOutput(*Result.string)
Protected NewList Files.s(), *Point, Len = 0
Protected hProg.i;, c=0, b=0
Protected stderr.s, tmp$
Protected avail.i, *buffer = AllocateMemory(#BUFFER_SIZE)
Debug *Result\s
hProg = RunProgram("bash", *Result\s, "", #PB_Program_Open | #PB_Program_Read | #PB_Program_Error | #PB_Program_UTF8)
If hProg
While ProgramRunning(hProg)
; c+1
avail = AvailableProgramOutput(hProg)
If avail
; b+1
If avail > #BUFFER_SIZE
avail = #BUFFER_SIZE
EndIf
avail = ReadProgramData(hProg, *buffer, avail)
If AddElement(Files())
Files() = PeekS(*buffer, avail, #PB_ByteLength | #PB_UTF8)
Len+avail
EndIf
tmp$ = ReadProgramError(hProg)
If tmp$
stderr + tmp$ + Chr(10)
EndIf
EndIf
Wend
CloseProgram(hProg)
EndIf
FreeMemory(*buffer)
Debug Len
*Result\s = Space(Len)
*Point = @*Result\s
ForEach Files()
CopyMemoryString(Files(), @*Point)
Next
ClearList(Files())
; Debug c
; Debug b
If stderr
Debug "======ERROR====="
Debug stderr
Debug "================"
EndIf
EndProcedure
; If you add "|sort -r" then "access denied" is printed to the error stream
Define Result.string
Result\s = "-c " + Chr(34) + "find '/home/user/' -regex '^.*?\.\(png\|jpg\|gif\)$' -type f|sort -r 2>&1" + Chr(34)
getOutput(@Result)
Debug Result\s
I read binary and line by line, then compare the results. Differ considerably. 52653 vs 54748 files. Compare in Meld. In the binary version, it is as if part of the data is cut off at each iteration step.
Re: How to get stdout from Linux terminal?
Posted: Sat Mar 06, 2021 2:09 pm
by NicTheQuick
I've done a call count (b). Initially it was b=1500, but when I increased #BUFFER_SIZE 10 times, then the number of calls became 150. Increasing the size even more did not reduce the number of calls.
I guess the standard buffer size is 8192 bytes (8kB) and because of that you'll see no difference in the ReadProgramData() calls if you the define the #BUFFER_SIZE bigger than that.
I read binary and line by line, then compare the results. Differ considerably. 52653 vs 54748 files. Compare in Meld. In the binary version, it is as if part of the data is cut off at each iteration step.
Well, that's bad. I just tried it out with a bigger directory, consisting of 406390 files and directories but Purebasic is so fucking slow at string concatenation that I have to wait minutes before I see any result. I expect the string to be round about 49 MB in size:
Code: Select all
nicolas@tp-w530:~$ find /home/nicolas/Install | wc -c
49013651 [bytes]
nicolas@tp-w530:~$ find /home/nicolas/Install | wc -l
406390 [lines]
I modified my version a bit and now it takes only 1.3 seconds to get the files (with Debugger on!) and the result is correct.
Code: Select all
EnableExplicit
#BUFFER_SIZE = 8192
Procedure.s getOutput(command.s, args.s)
Protected hProg.i
Protected stdout.s, stderr.s
Protected avail.i, bytesRead.i, bufferSize.i
Protected bufferCapacity.i = #BUFFER_SIZE, *buffer = AllocateMemory(bufferCapacity, #PB_Memory_NoClear), *tBuffer
hProg = RunProgram(command, args, "", #PB_Program_Open | #PB_Program_Read | #PB_Program_Error | #PB_Program_UTF8)
If hProg
While ProgramRunning(hProg)
avail = AvailableProgramOutput(hProg)
If avail
If avail > bufferCapacity - bufferSize
While avail > bufferCapacity - bufferSize
; Double up the buffer size each time it is too small for new data
bufferCapacity * 2
Wend
*tBuffer = ReAllocateMemory(*buffer, bufferCapacity, #PB_Memory_NoClear)
If Not *tBuffer
DebuggerError("Out of memory. Can not allocate " + bufferCapacity + " bytes.")
Break
EndIf
*buffer = *tBuffer
EndIf
Repeat
bytesRead = ReadProgramData(hProg, *buffer + bufferSize, avail)
avail - bytesRead
bufferSize + bytesRead
Until avail = 0
EndIf
stderr + ReadProgramError(hProg)
Wend
CloseProgram(hProg)
EndIf
stdout = PeekS(*buffer, bufferSize, #PB_ByteLength | #PB_UTF8)
FreeMemory(*buffer)
If stderr
Debug "======ERROR====="
Debug stderr
Debug "================"
EndIf
ProcedureReturn stdout
EndProcedure
Define time.i = ElapsedMilliseconds()
Define files.s
files = getOutput("/bin/bash", ~"-c \"find /home/nicolas/Install\"")
Debug "Time: " + Str(ElapsedMilliseconds() - time) + " ms"
Debug "Count: " + Str(CountString(files, #LF$))
Debug "Bytes: " + StringByteLength(files, #PB_UTF8)
And the result is:
Debugger wrote:Time: 1371 ms
Count: 406390
Bytes: 49013649
The bytes length only differs in 2 bytes. I guess that is something `StringByteLength()` does wrong after the string was converted to Unicode in between. If you debug the variable `bufferSize` you will get the exact same amount of data as `wc -c` shows you.
Re: How to get stdout from Linux terminal?
Posted: Sat Mar 06, 2021 8:18 pm
by AZJIO
I guess the standard buffer size is 8192 bytes (8kB)
I substituted 9024, it was not enough
at the moment I am unable to get the list of files using your code
Re: How to get stdout from Linux terminal?
Posted: Sat Mar 06, 2021 9:04 pm
by NicTheQuick
AZJIO wrote:I guess the standard buffer size is 8192 bytes (8kB)
I substituted 9024, it was not enough
at the moment I am unable to get the list of files using your code
That's weird. What are you getting instead?