[FIXED] How to check status of URLs from text file using bash shell script

Issue

I have to check the status of 200 http URLs and find out which of these are broken links. The links are present in a simple text file (say URL.txt present in my ~ folder). I am using Ubuntu 14.04 and I am a Linux newbie. But I understand the bash shell is very powerful and could help me achieve what I want.

My exact requirement would be to read the text file which has the list of URLs and automatically check if the links are working and write the response to a new file with the URLs and their corresponding status (working/broken).

Solution

I created a file "checkurls.sh" and placed it in my home directory where the urls.txt file is also located. I gave execute privileges to the file using

$chmod +x checkurls.sh

The contents of checkurls.sh is given below:

#!/bin/bash
while read url
do
    urlstatus=$(curl -o /dev/null --silent --head --write-out '%{http_code}' "$url" )
    echo "$url  $urlstatus" >> urlstatus.txt
done < $1

Finally, I executed it from command line using the following -

$./checkurls.sh urls.txt

Voila! It works.

Answered By - Jayakrishnan GK

Answer Checked By - David Marino (PHPFixing Volunteer)