Using while loop to ssh to multiple servers

  • I have a file servers.txt, with list of servers:

    server1.mydomain.com
    server2.mydomain.com
    server3.mydomain.com
    

    when I read the file line by line with while and echo each line, all works as expected. All lines are printed.

    $ while read HOST ; do echo $HOST ; done < servers.txt
    server1.mydomain.com
    server2.mydomain.com
    server3.mydomain.com
    

    However, when I want to ssh to all servers and execute a command, suddenly my while loop stops working:

    $ while read HOST ; do ssh $HOST "uname -a" ; done < servers.txt
    Linux server1 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
    

    This only connects to the first server in the list, not to all of them. I don't understand what is happening here. Can somebody please explain?

    This is even stranger, since using for loop works fine:

    $ for HOST in $(cat servers.txt ) ; do ssh $HOST "uname -a" ; done
    Linux server1 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
    Linux server2 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
    Linux server3 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
    

    It must be something specific to ssh, because other commands work fine, such as ping:

    $ while read HOST ; do ping -c 1 $HOST ; done < servers.txt
    
  • derobert

    derobert Correct answer

    7 years ago

    ssh is reading the rest of your standard input.

    while read HOST ; do … ; done < servers.txt
    

    read reads from stdin. The < redirects stdin from a file.

    Unfortunately, the command you're trying to run also reads stdin, so it winds up eating the rest of your file. You can see it clearly with:

    $ while read HOST ; do echo start $HOST end; cat; done < servers.txt 
    start server1.mydomain.com end
    server2.mydomain.com
    server3.mydomain.com
    

    Notice how cat ate (and echoed) the remaining two lines. (Had read done it as expected, each line would have the "start" and "end" around the host.)

    Why does for work?

    Your for line doesn't redirect to stdin. (In fact, it reads the entire contents of the servers.txt file into memory before the first iteration). So ssh continues to read its stdin from the terminal (or possibly nothing, depending on how your script is called).

    Solution

    At least in bash, you can have read use a different file descriptor.

    while read -u10 HOST ; do ssh $HOST "uname -a" ; done 10< servers.txt
    #          ^^^^                                       ^^
    

    ought to work. 10 is just an arbitrary file number I picked. 0, 1, and 2 have defined meanings, and typically opening files will start from the first available number (so 3 is next to be used). 10 is thus high enough to stay out of the way, but low enough to be under the limit in some shells. Plus its a nice round number...

    Alternative Solution 1: -n

    As McNisse points out in his/her answer, the OpenSSH client has an -n option that'll prevent it from reading stdin. This works well in the particular case of ssh, but of course other commands may lack this—the other solutions work regardless of which command is eating your stdin.

    Alternative Solution 2: second redirect

    You can apparently (as in, I tried it, it works in my version of Bash at least...) do a second redirect, which looks something like this:

    while read HOST ; do ssh $HOST "uname -a" < /dev/null; done < servers.txt
    

    You can use this with any command, but it'll be difficult if you actually want terminal input going to the command.

    where does the number `10` come from?

    @MartinVegter I made it up. 0/1/2 are stdin, stdout, and stderr. You can pick any number you like—bash lets you go pretty high, probably up to the OS limit. Other shells may limit you to fewer...

    @MartinVegter I've edited to answer both of your comments.

    Another alternative: `exec 3<&0; while read HOST; do ssh $HOST "uname -a" <&3; done

    The `-u` option is not supported by POSIX, and thus should not be used for `#!/bin/sh` scripts; use `read HOST <&10` instead. Also, POSIX only requires shells to support file descriptors 0 through 9, so `10

    how can you use this with the output of a command? For example, if instead of `servers.txt` you wanted to use `grep "pattern" servers.txt`

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM