Linux and pipes

[kake26]

In Linux on occasion one might want to pipe the output of one command to another then do something with the output. However, it seems purebasic via the run command doesn't work as I'd expect. I'm sure there some nuance I'm missing, I just can't figure it. I've resolved my problem with find a single command to do the job and that workeds.

Here is a example of the code that behaves odd.

Code: Select all

fndver = RunProgram("dpkg", "-l discord | awk '/discord/{print $3}'", "", #PB_Program_Open | #PB_Program_Read | #PB_Program_Error | #PB_Program_Hide)
If fndver
  While ProgramRunning(fndver) And (ElapsedMilliseconds() - StartTime < Timeout)
    If AvailableProgramOutput(fndver)
      Output$ + ReadProgramString(fndver)
    EndIf
    If AvailableProgramError(fndver)
      Error$ + ReadProgramError(fndver)
    EndIf
    Delay(10)
  Wend
  If ProgramRunning(fndver)
    KillProgram(fndver)
    Output$ = "Error: Command timed out"
  EndIf
  CloseProgram(fndver)
EndIf
Debug "Error Output: " + Error$

What I expect is to get a output like

$ dpkg -l discord | awk '/discord/{print $3}'
0.0.92

I know its because I'm overlooking something, but what that is it eludes me. Would someone please shed some light on this?

Just for reference the output I get is more like this.

Discord version: Desired=Unknown/Install/Remove/Purge/Hold| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)||/ Name Version Architecture Description+++-==============-============-============-=================================un awk <none> <none> (no description available)ii grep 3.7-1build1 amd64 GNU grep, egrep and fgrep

Yeah totally not the expected.

[benubi]

Perhaps you also need the #PB_Program_Connect flag

[Piero]

AvailableProgramError is not PB…

If you need single/double quotes, pipes etc.:

Code: Select all

Procedure simpleShell(ShellCommand$, wait = #False)
   Protected shell = RunProgram("/bin/sh","","", #PB_Program_Open|#PB_Program_Write)
   If shell
      WriteProgramStringN(shell,ShellCommand$)
      WriteProgramData(shell,#PB_Program_Eof,0)
      If wait
         While ProgramRunning(shell) : Delay(10) : Wend
      EndIf
      CloseProgram(shell)
   EndIf
EndProcedure

Structure daResults : Out.s : Err.s : ExC.w : EndStructure
Global Sh.daResults

Procedure Shell(ShellCommand$, AddLF = #True)
   Protected Err$, tmper$, Output$, shell, Exc.w =-1 ; exit code -1 on failed launch
   shell = RunProgram("/bin/sh","","",
      #PB_Program_Open|#PB_Program_Write|#PB_Program_Read|#PB_Program_Error )
   If shell
      WriteProgramStringN(shell,ShellCommand$)
      WriteProgramData(shell,#PB_Program_Eof,0)
      While ProgramRunning(shell)
         If AvailableProgramOutput(shell)
            Output$ + ReadProgramString(shell)
            If AddLF : Output$ + ~"\n" : EndIf
         EndIf
         tmper$ = ReadProgramError(shell)
         If tmper$ : Err$ + tmper$ + ~"\n" : EndIf
      Wend
      Exc = ProgramExitCode(shell) : CloseProgram(shell)
   EndIf
   Sh\Out = Output$ : Sh\Err = Err$ : Sh\ExC = Exc
EndProcedure

Procedure.s QuoteString(str$) ; useful to quote FILE/FOLDER PATHS
   ProcedureReturn "'" + ReplaceString(str$, "'", "'\''") + "'"
EndProcedure

Shell(~"echo '\"a\" 'b")
Debug Sh\Out

[NicTheQuick]

You need a shell to be able to use pipes. RunProgram does not spawn a shell. It runs a command with arguments/parameters.
In principle it should work like this:

Code: Select all

program = RunProgram("/bin/sh", "-c 'whoami | md5sum'", "", #PB_Program_Open | #PB_Program_Read)
Output$ = ""
If program
	While ProgramRunning(program)
		If AvailableProgramOutput(program)
			Output$ + ReadProgramError(program) + #LF$
			Output$ + ReadProgramString(program) + #LF$
		EndIf
	Wend
	Output$ + #LF$
	Output$ + "Exitcode: " + Str(ProgramExitCode(program))
	
	CloseProgram(program)
EndIf

MessageRequester("Output", Output$)

But somehow I only see Exitcode 2. The code is a modified version of that one from the help page of `RunProgram`.

[Piero]

Code: Select all

RunProgram("/bin/sh", "-c 'whoami | md5sum'"…

Code: Select all

Shell("sh -lc " + QuoteString("whoami | md5sum…") ; QuoteString and Shell posted above

[Axolotl]

Acc. to the Help Pages, there is a small difference between this procedures:

Code: Select all

ReadProgramError()  ; non blocking <=> ... doesn't halt the program flow if no error output is available
ReadProgramString() and ReadProgramData() ; blocking <=> ... waits until there is data available to read from the program 

which means the code should look like this.....

Code: Select all

;....
  Output$ + ReadProgramError(program) + #LF$  
  If AvailableProgramOutput(program)
    Output$ + ReadProgramString(program) + #LF$  
  EndIf 
;.....

[Piero]

Output$ + ReadProgramError(program) + #LF$
If AvailableProgramOutput(program)
Output$ + ReadProgramString(program) + #LF$
EndIf

No: Output$ is not for ProgramError

[NicTheQuick]

Oh, I see. I need double quotes and not single quotes.

Code: Select all

program = RunProgram("/bin/sh", ~"-c \"/bin/whoami | /bin/md5sum\"", "", #PB_Program_Open | #PB_Program_Read)

But why do I get an "Invalid memory access" on `ReadProgramError()`? That looks like a weird bug. Maybe I should check the beta versions if that got fixed already.

[Piero]

Oh, I see. I need double quotes and not single quotes.

Code: Select all

program = RunProgram("/bin/sh", ~"-c \"/bin/whoami | /bin/md5sum\"", "", #PB_Program_Open | #PB_Program_Read)

But why do I get an "Invalid memory access" on `ReadProgramError()`? That looks like a weird bug. Maybe I should check the beta versions if that got fixed already.

You made me remember the 1st time I tried to fight with runprogram on Mac……… forget the "bugs"…

IT'S BECAUSE YOU DON'T USE (the above) SHELL() !!!

Code: Select all

#PB_Program_Open|#PB_Program_Write|#PB_Program_Read|#PB_Program_Error
WriteProgramStringN(shell,ShellCommand$)
WriteProgramData(shell,#PB_Program_Eof,0)

[Piero]

[kake26]

Okay, thank you. I figured it was something like that. No shell spawned go it. Now that I know that I can find a way to make it work. Thanks again.

[Piero]

Code: Select all

ReadProgramError(program) + ~"\n"

may potentially produce a ton of unneeded LineFeeds! (~"\n" = #LF$)

From the Shell() Procedure posted above:
 

Code: Select all

tmper$ = ReadProgramError(shell)
If tmper$ : Err$ + tmper$ + ~"\n" : EndIf ; If tmper$ <> "" …

Note: needs #PB_Program_Error
How to merge out and err:

Code: Select all

Shell("hgfhjf")
Debug Sh\Err
Debug "--------------"
Shell("hgfhjf 2>&1") ; stderr to stdout
Debug Sh\Out
Debug "--------------"

 
This part is to use "pipes and quotes" as you would do on terminal:

Code: Select all

WriteProgramStringN(shell,ShellCommand$)
WriteProgramData(shell,#PB_Program_Eof,0)

Note: needs #PB_Program_Write
 
This will be a login shell like a terminal (e.g., will "see" stuff added to ~/.profile):

Code: Select all

ls$= ~"shopt -q login_shell && echo 'Login shell' || echo 'Not login shell' \n echo End" ; ~"\n" or ; ? ;)
Shell("sh -lc " + QuoteString(ls$) ) ; better ALWAYS use QuoteString for -c
Debug Sh\Out
Debug "--------------"

You can also do this way:

Code: Select all

shell = RunProgram("/bin/sh", "-l", …

PS/NOTES:
Recent MacOS Terminal uses zsh (and ~/.zprofile)
If you need to "send scripts" or use multiple commands,
TEST HOW YOUR SHELL BEHAVES FROM PB! (better ; or \n ?)
Also:

Code: Select all

(cmdA; cmdB) 2>&1