Biostar Beta. Not for public use.
Question: Gnu Parallel Quoting With Awk
1
Entering edit mode

As I try to get familiar with the GNU Parallel tutorial on quoting, I need some help to move faster.

The command I want to quote is

 parallel --dryrun "awk '{ print $0,"\t","{.}" }' {} > {}" ::: *foo.txt

Basically, I want to add the FILENAME as the last column of the tab-delimited file, for each input file.

Problem is that bash interpretes the above command as

awk '{ print /bin/bash,t,foo.txt }' foo.txt > foo.txt

Which doesn't work. So it needs appropriate quoting to work

ADD COMMENTlink 5.9 years ago tayebwajb • 90 • updated 5.5 years ago ole.tange ♦ 3.4k
7
Entering edit mode

I assume what you want to do is given the file foo.bar you want this run:

    awk '{ print $0,"\t","foo" }' foo.bar > foo.bar

But that will not work, as you have foo.bar both as input an output. What will happen is that foo.bar will be 0 bytes long.

So you need a temporary file:

    awk '{ print $0,"\t","foo" }' foo.bar > foo.bar.tmp; mv foo.bar.tmp foo.bar

To do that using parallel it might be easier to define a function:

    doawk() {
       FILE=$1
       BASE=$2
       awk '{ print $0,"\t","'"$BASE"'" }' "$FILE" > "$FILE".tmp; mv "$FILE".tmp "$FILE"
    }
    export -f doawk
    parallel doawk {} {.} ::: *.txt

This way you avoid double quoting the awk script.

If you want to avoid the function we are looking at:

    parallel awk\ \'\{\ print\ \$0,\"\\t\",\"\'{.}\'\"\ \}\'\ {}\ \>\ {}.tmp\;\ mv\ {}.tmp\ {} ::: *.txt
ADD COMMENTlink 5.5 years ago ole.tange ♦ 3.4k
3
Entering edit mode

With no info about the output expected, this is what I think you are trying to do:

parallel "awk '{print \$0, "\t", FILENAME}' {} > {}.tmp; mv {}.tmp {}" ::: *input.txt
ADD COMMENTlink 5.9 years ago dapregi • 50
Entering edit mode
0

I added information about the expected output.

ADD REPLYlink 5.9 years ago
tayebwajb
• 90

Login before adding your answer.

Similar Posts
Loading Similar Posts
Powered by the version 2.0