Emailing a webpage from the command line

There are many situations whey you have information on a web page that needs to be emailed to individuals on a regular basis. Often times all that is needed is some type of script that will grab a URL, put it in an email and send it to the correct person. This article is going to look at some of the tools that can be used to do this. I’m testing it on OS X, but it should work under Linux, Unix and possibly Cygwin as well.

curl

First we need a way to grab the web page. The “copy URL” command or curl works great for this. In its simplest form it will just grab the contents of a URL and dump it to the screen. The following line will just grab the contents of Google and dump it to the screen.
curl http://www.google.com
We can put the information into a file like this:
curl http://www.google.com > file.html

So now we have a way to get the contents of a URL and pull it down to the computer. Now we need a way to send mail.

mail

You can send a simple message using the mail program. Here is a sample interaction:

mark-LT-G4:/ marks$ mail [email protected]
Subject: Test
Test Message
.


On the first line we launch the mail program and tell it we want to send an email to [email protected]. The mail program asks for the subject and then lets you type your message. To end the message just type a period on a line by itself and hit enter.

This is great, but we need some way to create a single line to send a message. Just for the sake of the example lets look at the echo command. The following line with echo the text “Hello World”.

echo "Hello World"

Now we can take the mail program and pipe this information into it so it will send the message. The | character will evaluate the command on the left hand side and use it as the input for the command on the right hand side. So the following code will send the “Hello World” message.

echo "Hello World" | mail [email protected]

Ok now that we have a way to send the output of one program as an email message, we can combine it will curlcurl http://www.google.com | mail [email protected]

The email that gets sent isn’t particularly useful because it is the text of the html. There isn’t a simple way (that I’m aware of) to tell mail to set the message type to html. However, we can encode the web page as a file and send it using mail using a program called uuencode. uuencode takes a binary file and encodes it in a way that can be sent over email. Your email client handles doing this automatically, but since we are working from the command line we’ll have to do it ourselves.

curl http://www.markwshead.com | uuencode page.html | mail [email protected]

The command is in three section. The first section is our call to curl like we’ve seen before. The second section puts the content of the curl command into a file called page.html and sends that file to the mail command. This means our message is blank, but has a file attached called page.html. The file opens in our web browser and shows the contents of the web page we called from curl.

sendmail

Generally when we send an html email we want to have the contents show up in the message. If the user needs to open an attachment to see the message, we’re better off just sending a link. To make the mail clients recognize that our message needs to be shown as html instead of as plain text we’ll need to set the message content type to text/html instead of the default of text/plain.

The following command will do this for us.

( echo "Content-Type: text/html"; curl http://www.markwshead.com) | sendmail [email protected]

The parentheses allow us to execute two commands and concat their output into the next command. The commands are separated by a semicolon. So first we echo the content header that we want to use and then append the web page. All of the preceding is sent to the sendmail program. It would seem that you could do the same thing with mail instead, but mail seems to be hardwired to handle plain text. It displays the header as part of the message body. sendmail is a much more complicated program and can handle many more options including adding the headers. You can use a similar method to change subject and other header information.

final note

Keep in mind that some of the messages may look funny in the email programs because they use relative links to style sheets. You can overcome this problem by making the links absolute or including the style sheet information into the web page itself. If you are sending information from pages that you don’t have the ability to change you might want to write a small program that will replace relative links with absolute links.