How can I read line by line from a variable in bash?

  • I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable?

    For example:

    jobs="$(jobs)"
    if [ "$jobs" ]; then
        # read lines from $jobs
    fi
    
  • dogbane

    dogbane Correct answer

    10 years ago

    You can use a while loop with process substitution:

    while read -r line
    do
        echo "$line"
    done < <(jobs)
    

    An optimal way to read a multiline variable is to set a blank IFS variable and printf the variable in with a trailing newline:

    # Printf '%s\n' "$var" is necessary because printf '%s' "$var" on a
    # variable that doesn't end with a newline then the while loop will
    # completely miss the last line of the variable.
    while IFS= read -r line
    do
       echo "$line"
    done < <(printf '%s\n' "$var")
    

    Note: As per shellcheck sc2031, the use of process substition is preferable to a pipe to avoid [subtly] creating an subshell.

    Also, please realize that by naming the variable jobs it may cause confusion since that is also the name of a common shell command.

    If you want to keep all your whitespace, then use `while IFS= read` .... If you want to prevent \ interpretation, then use `read -r`

    I've fixed the points fred.bear mentioned, as well as changed `echo` to `printf %s`, so that your script would work even with non-tame input.

    To read from a multiline variable, a herestring is preferable to piping from printf (see l0b0's answer).

    @ata Though I've heard this "preferable" often enough, it must be noted that a herestring always requires the `/tmp` directory to be writable, as it relies on being able to create a temporary work file. Should you ever find yourself on a restricted system with `/tmp` being read-only (and not changeable by you), you will be happy about the possibility of using an alternate solution, e. g. with the `printf` pipe.

    @syntaxerror many thanks for the info, it's good to know!

    You're welcome. TBH, I've been through this "hell" once before you, to know more see this question.

    In the second example, if the multi-line variable doesn't contain a trailing newline you will loose the last element. Change it to: `printf "%s\n" "$var" | while IFS= read -r line`

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM