APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Introduction to email


© August 2008 Anthony Lawrence

Before we get started, what's the first word you think of when I say "email"?

Was it Outlook or Outlook Express? Maybe AOL? Or Webmail?

Those are things you use to send email, to read email other people send you, but how does it all work? If I send you an email, how does what I write get from my computer to yours?

By the way, some people think Microsoft invented email. Actually, email goes way back to 1965, ten years before Bill Gates ever thought about computers. By 1972, people on the Internet were sending and receiving email using the same "bob@somecomputer" addresses we use today. Microsoft didn't even start connecting to the Internet until the mid-90's and Outlook didn't exist until 1997! I had an Internet email address in 1991.

Sending Mail

Here's how:

Don't panic - yes, it's a liitle geeky, but you don't have to study it or understand what each line means. The stuff your computer sends is in bold, the normal stuff is the answers from the server.

Connected to aplawrence.com.
Escape character is '^]'.
220 vps.pcunix.com ESMTP SMTP Ready; Fri, 8 Aug 2008 15:12:09 GMT
HELO aplawrence.org
250 vps.pcunix.com Hello aplawrence.com [71.184.234.68], pleased to meet you
MAIL FROM: tony@aplawrence.com
250 2.1.0 tony@aplawrence.com... Sender ok
RCPT TO: pcunix@aplawrence.com
250 2.1.5 pcunix@aplawrence.com... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Date: Aug 25, 2008
Subject: Test message

Hi there

.
250 2.0.0 m78FCoC86720 Message accepted for delivery
QUIT
221 2.0.0 vps.pcunix.com closing connection
 

That's a record of an actual email being sent. Every time you send an email to anyone, something very similar to that takes place. Your mail program (Outlook, Aol, Webmail, whatever) makes a connection to another server, introduces itself (the HELO above), tells who the mail is from and who it is going to, and then issues a DATA command followed by the actual email. When it has sent everything, it sends a period (".") on a line by itself, and that's the end of it.. as far as your computer is concerned anyway.

The server you sent the mail to could be the server that is really responsible for the mail. That is, if the mail is going to tony@aplawrence.com, some server in the aplawrence.com domain will be the one that mail ultimately needs to get to. However, you probably use your ISP's server or maybe Aol or Gmail or MSN. In that case, you'd be sending the mail to (for example) Verizon.net or Comcast. That server would "relay" the message to my server at aplawrence.com.

Let's say I need to send you email. You are "joe@verizon.net". So, sooner or later I or someone I get to relay my messages needs to connect to a mailserver at Verizon.net and send the mail just as we did above.

Every email message gets sent the same way. It doesn't matter if you use Outlook, AOL, webmail or whatever: the session I show here is what actually happens.

But wait a minute. Verizon must have thousands of computers. How do we know which one to send it to?

We ask DNS.

You may not know what DNS is, but your computer talks to it all the time. Every time you look up something on the web, your browser asks DNS a question. You want to read news.google.com? Your browser asks DNS where news.google.com is, and DNS says "74.125.47.147, 74.125.47.104, 74.125.47.99 and 74.125.47.103, take your pick". Your browser will probably take the first one,

Why does it need that number? Basically the same reason you need to look up telephone numbers: the telephone system works with phone numbers, the Internet works with IP (Internet Protocol) numbers.

Oh, so that's easy then: we just ask DNS where "mail.google.com" is, right?

Umm, no, not quite.

Let's back up a minute. "mail", "news", "www", "maps" - all of those are things you'll find at Google.

Do the names mean anything?

How many of you think that if I say "Use your browser to go to whatever.org" you HAVE to type "www.whatever.org"?

You don't. You can just type "whatever.org".

And that's true for almost all websites. There are a very few that are set up so that you really do have to put in that "www"; most do not need it.

Every computer on the Internet has a name. Some are named "www", some are named "news", some might be named "fred" or "mary". Some have more than one name: I'm "Tony" to you guys, "Anthony" to the IRS, "Tone" to my daughters.. Computers can have more than one name too.

Even your computer has a name when it is connected to the internet. For example, my computer right now has the name "pool-71-184-234-68.bstnma.fios.verizon.net." (tomorrow it might have a different name; we'll talk about that another time).

The computer that handles mail for Google or Verizon or me might be named "mail", but it doesn't have to be. It could be named "BigOldBob" (and so could a webserver). So there's a special way to find out where the mail should go: you ask DNS for the "MX" or "Mail eXchange" record.

Let's ask DNS about Google's MX records:

;; ANSWER SECTION:
google.com.		6953	IN	MX	10 smtp4.google.com.
google.com.		6953	IN	MX	10 smtp1.google.com.
google.com.		6953	IN	MX	10 smtp2.google.com.
google.com.		6953	IN	MX	10 smtp3.google.com.

;; ADDITIONAL SECTION:
smtp1.google.com.	3420	IN	A	209.85.237.25
smtp2.google.com.	3543	IN	A	64.233.167.25
smtp3.google.com.	3172	IN	A	64.233.183.25
smtp4.google.com.	3504	IN	A	72.14.221.25
 

That means that Google has at least four mailservers and that I should feel free to connect and talk to any one of them if I need to send mail to someone at Google.

Now remember, your Outlook or whatever you use probably doesn't connect to Google. It most likely connects to your ISP and their mailserver is the one that looks up the MX record and gets it to the right place.

Simple so far? But wait.. why doesn't your Outlook just go right to google.com or aplawrence.com or wherever it needs to go to and deliver the mail directly? Why does it go through another server?

Well, it could go directly, but more and more ISP's won't let it. Why? Spam.

We'll get a liitle geeky here so try to stay awake. Remember I said we "connect" to the mailserver? Well, we do, but we connect on a special "port". Specifically, to send mail, we connect on port 25 - that's the SMTP (Simple mail Transfer Protocol) port. Think of ports like telephone extensions. We ask DNS what the MX is for aplawrence.com and it tells us that it is 64.226.42.29. We connect to that address and ask for port 25.

What does it talk to on port 25? A mailserver program. That might be Sendmail, Postfix, Qmail, Kerio (that's what I sell) or one of many others.

You might be able to do this right from your computer. If you are using Windows, do Start->Run and type "telnet aplawrence.com 25". If you have a Mac with OS X, open up Terminal and type the same thing. You might see this:

Trying 64.226.42.29...
Connected to aplawrence.com.
Escape character is '^]'.
220 vps.pcunix.com ESMTP SMTP Ready; Fri, 8 Aug 2008 16:12:10 GMT
 

If you do, you could actually send me mail by typing the same commands we saw at the beginning: HELO, MAIL FROM: RCPT TO:, DATA.

But nowadays many ISP's block port 25 going anywhere EXCEPT to their mailserver. As I said, they do this to help prevent your computer being taken over by a virus and being used to send spam. That virus isn't going to use Outlook to send mail (though it may look in Outlook for people to send to). It will try to send directly because it doesn't want to go through any filtering your ISP may have and it also wants to send messages very quickly - lots and lots of messages. Blocking port 25 can at least slow it down and your ISP may be able to notice your infection more easily.

Nothing. We'll see later that spammers and virus email almost always pretend to be from someone else.

OK, we're all clear on how mail is sent? That's great, but we have to GET the mail too. How does mail get FROM the mail server back to our computer.

Sending Mail

I'm assuming you are using Outlook or Thunderbird or some other MUA (Mail User Agent). I will talk about Webmail eventually, but right now we're going to look at it this way.

(I snuck in that TLA (Three Letter Acronym). Outlook is an MUA, a mailserver is an MTA (Mail Transfer Agent). If you ever had to take a test on how mail works, you'd need to know that. You probably aren't planning on taking that test, right?

POP, IMAP, MAPI

There are three common ways your computer (or your cell phone) can get email from the server that has been storing it: POP, IMAP and MAPI. The first two are "open" protocols, MAPI is Microsoft. Unless you are talking to a Microsoft Exchange Server or something that acts like an Exchange server, you'll be using POP or IMAP. POP stands for "Post Office Protocol" and IMAP is "Internet Message Access Protocol".

Most servers can handle either. That is, you could tell Outlook to use POP (its default) or IMAP and you'd still get your mail. So which is better?

IMAP. Microsoft stupidly puts POP as the default, so that's what most people choose, but most would be happier if they used IMAP. What's the difference?

POP brings messages to your machine, while IMAP ALWAYS leaves messages on the server. IMAP only downloads header information initially. That can be advantageous for slow connections and for being able to access your email from multiple locations/machines. Pop also has the disadvantage of only using one folder (INBOX) while Imap can have multiple server folders.

Webmail uses IMAP, by the way.

If you are accessing Email from two or more computers, you may find that IMAP makes it easier to maintain a consistent view of your mail. However, you can achieve the same thing if you tell POP to leave a copy on the server and delete it when you delete your local copy.

If you mix Webmail with POP, you are going to get confused because Webmail will show you only the messages that have not been removed by POP.

The disadvantage of IMAP is that if you cannot connect to the server, you can't see mail that you've already seen (unless you have specifically copied it to a local folder). With POP, what has reached your inbox is now on your machine: you may not be able to get at new messages, but your old messages are here.

Some IMAP clients allow local storage automatically..

If your local machine doesn't get backed up, having the messages stored on the server can be a life saver. Of course if the server isn't being backed up, using IMAP can leave you with nothing at all after a server crash. It's never a bad idea to copy important email to local folders.

Another problem with POP is the way Outlook handles messages. Outlook and Outlook Express are very stupid programs. What they do is mash all messages together in one giant file with extra information stuck in so that it can be sorted by date, by name and so on. The smart way to do that is to leave the messages as individual files and build a separate index for sorting. Then if abnything happened to the index, you could rebuild it by re-reading the messages.

What do you think you get when something bad happens to Microsoft's big mashed up messages/index file?

That's right, you don't get email. In fact, you might lose ALL your email and although Microsoft has a program (SCANPST.EXE) that can sometimes fix things, more often it fails miserably and leaves you with nothing. All that email from your kids, pictures of grandkids, whatever, all gone..

And does it screw up often? You can bet your bippy on it. One of the most common problems I see is the "2 Gigabyte inbox" problem. Windows XP and all older Windows operating systems can't have a file bigger than 2 Gigabytes. That's a pretty big file, but if you keep saving email for years, you can get there easily. And what happens when you do? Does Outlook say "Gee, I'm sorry but I can't do any more email until you delete some"? You wish.. no, Outlook just crashes and burns - that's why I said it's "stupid". It's stupid because they did it the wrong way to start with and they don't take the trouble to protect you from a problem that WILL happen if you don't delete some mail or move it somewhere else.

And by the way, that Deleted Items folder really isn't another folder. All those messages are still in your Inbox counting toward that 2 Gigabyte limit. The only way to clean that up is to "Purge".

These are just some of the reasons I dislike Microsoft. Don't get me started.

Anyway, back to mail. When you set up Outlook for the first time, you are going to be asked a few questions: your name, your email address, and then "Server Type". What are the choices?

Right, POP or IMAP.

Then it will ask for server information: Outgoing server? That's the server you use to SEND mail, usually your ISP's SMTP server. Incoming Server? The server you GET your mail from, by POP or IMAP.

You can do POP from the command line too. Here's an example:

telnet popserver 110
-server will (or should) respond indicating that it is ready
user username
- server will tell you that a password is required
pass usernames_password
- server will tell you how many messages username has
list
- server will show you size of each message
retr 1
- first message will be displayed
quit 
 

Isn't that neat? No? Well, it is, because if Outlook isn't working you can use this to test whether it is Outlook or your server. You can test IMAP too:

The stuff you'd type is in bold. The "abc1" etc at the begining of each command is just a tag you make up. Supposedly it should be unique for each line; I've found that with most servers you can actually use the same tag over and over again.

telnet 10.1.36.248 143
Trying 10.1.36.248...
Connected to 10.1.36.248 (10.1.36.248).

Escape character is '^]'.
* OK e-smith.pcunixx.com IMAP4rev1 v12.264 server ready
abc1 login linda AFr54Tui8
abc1 OK LOGIN completed
abc2 select inbox
* 1 EXISTS
* 2 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1020929003] UID validity status

* OK [UIDNEXT 1020929444] Predicted next UID
* FLAGS (\Answered \Flagged \Deleted \Draft \Seen)
* OK [PERMANENTFLAGS ()] Permanent flags
* OK [UNSEEN 2] first unseen message in inbox
abc2 OK [READ-WRITE] SELECT completed
abc3 fetch 2 body[text]
* 2 FETCH (BODY[TEXT] {25}

This is just a test


)
* 2 FETCH (FLAGS (\Recent \Seen))
abc3 OK FETCH completed
abc4 close
abc4 OK CLOSE completed
abc5 logout
* BYE e-smith.pcunixx.com IMAP4rev1 server terminating
connection
abc5 OK LOGOUT completed
 

Do I expect you to remember that? Of course not - I don't even remember it. I have to look this stuff up every time I do it. The point is not to remember it, but just to see that it's really relatively simple. A few magic words, and you are talking directly to your mailserver! You'll probably never do it - even I don't do this very often - but it shows you that it can be done.

And of course you can test SMTP as we saw at the beginning.

Webmail

Webmail is just IMAP through a browser. Verizon has Webmail - you can use Outlook, but you can also use Webmail. As I mentioned above, it can be confusing to use Webmail and POP together because by default POP deletes messages from the server and IMAP only shows what is on the server. If you use IMAP instead of POP, there's no confusion.

Gmail is Webmail. I really recommend that people get a Gmail or MSN, Hotmail or something like that. Why? Because if you have "peg@verizon.net now", and then switch to Comcast next year, you'll have to change your email address.. if you have "peg@gmail.com" you'll never have to change it.

My opinion? Gmail is the best right now. If you want to get a free Gmail account, just point your browser at gmail.com. It's easy to sign up.

A Test

Let's have a little test. Nothing we've talked about yet, but which of these addresses would get to me?

pcunix@gmail.com
"Tony Lawrence" <pcunix@gmail.com>
pcunix+hello@gmail.com
pcunix(hi there tony)@gmail.com
 

Answer: Probably all of them. They are all legal addresses according to RFC822 and RFC2822. RFC's are documents that describe how things are supposed to work so that when somebody wants to write a new MUA because they hate Outlook, they can be sure they are doing everything they are supposed to be doing.

So MOST (almost all) MTA's and MUA's will happily deliver "pcunix+anything@gmail.com" just as though it were "pcunix@gmail.com".

There are lots of RFC's on the Internet, most serious, but some are just meant to be funny - especially those published on April 1st of each year. See https://www.faqs.org/rfcs/ if you happen to be a computer geek.

Rules

By the way, have you ever gotten an email where a lot of the lines ended with "=" signs and another character or two? Like

This is a line=20
 

If you look at RFC2045 you can learn what that's all about. But what RFC2045 won't tell you is that this "quoted printable" encoding is often used by spammers to disguise their junk from mail filter rules.

Mail Filter Rules? Sure, Outlook and almost all MUA's have filter rules that can automatically delete messages or move them into special folders based on content.

Remember that "pcunix+hello@gmail.com"? Suppose I said "Use pcunix+ComputerClub@gmail.com"? I could then write a rule that looked for that address and put those in a special folder. I could tell my children to always use "pcunix+dad@gmail.com" and file those somewhere else - automatic email organization.

Is it hard to write rules? No. If you have Outlook, just give it a try - you'll probably have no trouble at all. Most Webmail programs have rule filters too.

Some mail servers (most) also have server side rules for the same kind of thing, though most of that today is for spam and virus filtering.

Viruses and Spam

Ahh, viruses and spam. The curse of email, and particularly so if you run Windows. If you run Mac or Linux you don't worry very much about virus infection; it's almost all aimed at Microsoft. But spam is aimed at everyone..

Remember the reply address Outlook asked you for? What would stop you from putting MY email address in there?

Do you think that spammers use a legitimate reply address? They might if they want you to send them email, but usually they want you to go to a website, so the Reply-To may be bogus. How about email containing a virus? How likely is that to contain a real email address?

Actually, it usually does. Oh, not the address of the person really responsible, but a real address. Here's how it works:

A virus takes over your friend Mary's machine. Mary was very careless, she doesn't have a hardware firewall and she didn't buy any AntiVirus Software. She picked up a nasty virus and it's in her computer.

First thing the virus does is check Mary's Outlook. Oh, good, she uses that "wonderful" Microsoft product. Let's see who's in her address book, shall we? There's Bob, and Ellen, and.. OK, let's see if we can send Bob some spam and we'll say it's from Ellen so he might open it. We won't send any that says it's from Mary because we don't want anybody to trace the origin back to here - the longer we can run undeteced, the better. Maybe we'll send Ellen a copy of ourselves and say it's from Bob..

And off it goes. That's one use for Virus programs; there are others. Your computer might be used as part of an army of 'bots that will be used to annoy and distract a financial web site while another computer tries to break in.. please, use a hardware firewall and use anti-virus software.

Which A/V software? Cousin Joe says McAfee is junk. The computer guy that came to your house says Symantec is horrible and that he uses clamAV.

Truth? The free stuff, like ClamAV and a few others, is a lot better than nothing. The stuff you pay for (McAfee, Norton, et al.) will stop more threats but will also bog down your computer more. The number one reason Windows computers get so frustratingly slow is because of anti-spam and anti-virus software.

Pick your poison. Using SOMETHING is infinitely better than using nothing. I have a customer that scans with four different products: once before the mail server, twice at the mailserver, and again at the desktop. He still gets a virus infection every now and then.

I want to leave some time for questions, but before that I do want to cover some email etiquette.

First, don't send email with blank subject. Some email filters automatically delete anything with no subject. Put SOMETHING in there - "Hey" is better than nothing!

Don't type IN ALL CAPS. It's hard to read and it makes you look a little stupid. Don't do it.

If you are forwarding the latest joke, take a minute to delete out all the unnecessary stuff and remove those leading "<" and ">" signs.

When mailing to a bunch of people, put yourself in the To: box and put all the rest in the BCC: (Blind Carbon Copy) box. This cuts down on accidental Reply-All messages..

Oh, and when forwarding anything that says PopTarts give you cancer or that Obama kissed McCain on the lips, try checking it on Snopes.com first. You could save yourself some embarrassment.

Some common questions

Why can't I open the attachment Mary sent me?

Because Mary has a program on her computer that you don't. For example, I sometimes see people with older computers send out Microsoft Works attachments. Strangely, Microsoft Word (which is what most of you use now) can't open Works documents. There is a conversion program you can get from Microsoft that will let Word read Works files, but you have to download it.

Or: Outlook Express (version 6.0 onwards) has a security setting which prevents attachments being opened. To change this setting, go to the [Tools/Options/Security] dialogue and untick the box labeled "Do not allow attachments to be saved...".

It gets worse: Microsoft Outlook (version 2000 SP3 onwards) absolutely prevents the opening of certain types of attachments. Isn't that nice?

Why did the email I sent to Bob bounce back to me? Or why is it stuck in my Outbox?

Almost always I find that the address was mis-typed. It was supposed to be "bob@verizon.net" but you typed "bob@verizon.com" or "bob,@verizon.net" (comma instead of period).

Sometimes Bob has left Verizon and gone with Comcast.

How do I automatically send an "On vacation" message?

Many Webmail programs have such a setting (Gmail does, for example), but Outlook makes it hard. See https://www.mccordweb.com/e-newsletters/instructions/vacation-message.php for detailed Outlook instructions and https://email.about.com/od/outlookexpresstips/qt/et102806.htm for Outlook Express.

How can I find someone's email address?

There is no sure way. You can try Googling their name - it's easy to find people like me that way, but for most people you'll come up blank or get someone else's email.

I hit "Reply-All" but Outlook says there is something wrong with one of the addresses.

It got munged somehow. All you need is the "xyz@abc.com" part. So strip out everything else ("Joe Blow" <joe@xyz.com> becomes joe@xyz.com).

I have "mary@aol.com", "mary@verizon.net" and "mary@gmail.com".. it's very confusing!

All mail services let you "forward" your email somewhere else. So if you like Gmail, have all your AOL mail forwarded to there and do the same with your Verizon account.

Why do I get rejection messages for mail that I did not send?

Answered above in the section that talks about virus infection.

Do Challenge-Response systems prevent spam?

Yes, at the cost of really annoying everyone else. These are the systems that intercept email and won't send it to you until the sender proves who they are and you accept the proof. These do prevent spam, but are very, very annoying.


Got something to add? Send me email.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

->
-> Basics of Email


Inexpensive and informative Apple related e-books:

Take Control of Pages

Take Control of Preview

Take Control of High Sierra

Are Your Bits Flipped?

Take Control of Automating Your Mac




More Articles by © Anthony Lawrence




Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us


Printer Friendly Version





Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious. (Fred Brooks, The Mythical Man Month)




Linux posts

Troubleshooting posts


This post tagged:

Basics

Mail



Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode





SCO Unix Sales, Support, & Service

Phone:  707-SCO-UNIX (707-726-8649Toll Free: 833-SCO-UNIX (833-726-8649)
www.SCOsales.com