[FIXED] How to run for loop inside heredoc while accessing remote machine

Issue

Here is my script in which I use local variable inside a remote machine using heredoc. But the loop under the heredoc takes the first variable value only. The loop runs fine inside the heredoc but with the same values.

#!/bin/bash
prod_web=($(cat /tmp/webip.txt));
new_prod_app_private_ip=($(cat /tmp/ip.txt));
no_n=($(cat /tmp/serial.txt));
ssh -t -o StrictHostKeyChecking=no ubuntu@${prod_web[0]} -p 2345 -v << EOF
set -xv
for (( x = 0; x < '${#no_n[@]}'; x++ ))
do
sudo su
echo '${no_n[x]}'
echo '${new_prod_app_private_ip[x]}'
curl -fIkSs https://'${new_prod_app_private_ip[x]}':9002 | head -n 1 
done
EOF

So, my ip.txt file contains values like:

10.0.1.0
10.0.2.0
10.0.3.0

My serial.txt file:

9
10
11

So, my loop runs for only the first IP (present in /tmp/ip.txt) in the remote machine, three times. I want to run it for all the three IPs. My remote ip is present in the file /tmp/webip.txt.

Got stuck for a long time, any help is appreciated. Is there any other solution that I can go with?

Solution

There are 2 environments. On your local machine and on the remote machine. You need to think how to transfer data/variables/state/objects/handles between these machines. If you set something on your local machine (ie. prod_web=($(cat /tmp/webip.txt));) and then just ssh to remote host (ie. ssh user@host 'echo "${prod_web[@]}"'), the variable will not be visible/exported to the remote machine. You can:

• scp the files {ip,serial}.txt and execute the whole script on the remote machine, then cleanup , ie. remove the {ip,serial}.txt files from the remote machine
• pass the files {ip,serial}.txt somehow merged/joined/pasted to the stdin of the ssh and then read up stdin on the remove machine
• create all the commands to run on your local machine and then pass pre-prepared commands to remote machine, like ssh .... "$(for ...; do; echo curl ...; done)"

I would go with the second option, as I like passing everything using pipes and don't like to cleanup after me - removing temporary files in case of error can be a mess.

My script would probably look like this:

#!/bin/bash 
set -euo pipefail
read -r host _ <webip.txt
paste serial.txt ip.txt | ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu@"$host" '#!/bin/bash
set -euo pipefail
while read -r no_n ip; do
     for ((i = 0; i < no_n; ++i)); do
         printf "%s\n" "$no_n"
         printf "%s\n" "$ip"
         curl -fIkSs https://"$ip":9002 | head -n 1 
      done
done
'

As the remote script would become larger and less qouting friendly, I would save it into another remote_scripts.sh and execute ssh ... -m remote_scripts.sh.
I don't get what you are trying to do with that sudo su, which 100% does not do what you want.
If the no_n magic number is the number of times to execute that curl and you have xargs and you don't really care about errors, you can just do a magic and confusing oneliner:

#!/bin/bash 
set -euo pipefail
read -r host _ <webip.txt
paste serial.txt ip.txt | ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu@"$host" 'xargs -n2 -- sh -c "seq 0 \"\$1\" | xargs -n1 -- sh -c \"curl -fIkSs https://\\\"\\\$1\\\":9002 | head -n 1\" -- \"\$2\"" --'

Preparing all the command to run maybe actually more readable and may save some nasty qouting to resolve. But this really depends on how big serial.txt and ip.txt are and how big are the commands to be executed on the remote machine, as you want to minimize the number of bytes transferred between machines.
Here the commands to run are constructed on local machine (ie. "$(...)" is passed to ssh) and executed on remote machine:

# semi-readable script, not as fast and no xargs
ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu@"$host" "$(paste serial.txt ip.txt | while read -r serial ip; do 
   seq 0 "$serial" | while read -r _; do
         echo "curl -fIkSs \"https://$ip:9002\" | head -n 1"
   done

done)"

HERE-doc does not expand shell commands, so:

$ cat <<EOF
> echo 1
> EOF
echo 1

but you can use command substitution $( ... ):

$ cat <<EOF
> $(echo 1)
> EOF
1

Answered By - KamilCuk

Answer Checked By - David Goodson (PHPFixing Volunteer)