Using Mutt with Gmail (POP3 over SSL)
This page is a guide to using the email client mutt, in conjunction with fetchmail, procmail and msmtp, to send, receive and read email under Linux using a Gmail account as a relay. It is different from the myriad of other "Mutt and Gmail" blogs as it still uses the POP3 over SSL method, and has propounded this technique for many years. If you need to ask why anybody would want to put such an effort into an undertaking like this, and did not simply use the web interface of Gmail or even mutt with IMAP, perhaps this page is not for you. To use another's words:
Mutt is not designed to suit everyone, including those without basic knowledge about the Mail Concept, or those unwilling to perform configuration. mutt is even not designed for the mass of "average users", although it works well for them, too. For those who prefer a client to "just work somehow" rather than give dedicated best performance, other mail programs are probably better suited.
Perhaps you already feel the lure of the incredibly powerful Linux console applications? If so read on and please use the email link at the base of this page to let me know how you profited, or otherwise, from the information on this page.
Contents of the Page
This is a fairly complex page that I believe would benefit from reading from top to bottom but I include some navigation here for those who would like to sample rather than consume:
- Downloading the Mail Receiving mail from Gmail.
- Sending the Mail Setting up the Mail Sending Agent (MSA) msmtp.
- Reading the Mail Setting up the console Mail User Agent (MUA) mutt.
I will not be dealing with the specifics of downloading and installing the required software for this page as there are too many distro-specific issues there. But obviously you will need to have installed OpenSSL, fetchmail, procmail, msmtp and mutt; all fairly standard Linux programs. Whether this is done by using your distro's version or compiling from source the end result should be the same. But before moving on to the actual setup I need to introduce my new friend John.
Introducing John ...
To avoid confusion in editing the many configuration files involved in this setup I will describe the setup of mutt and gmail for my new friend John, who has been created especially for this guide. John's details are as follows:
Gmail Address: firstname.lastname@example.org Gmail Password: rover Computer Username: john
John's details will always be in italics, bold and in red to remind you, Gentle Reader, to substitute your details for his. Hopefully this will lessen the confusion that I have unintentionally created with older versions of this page!
Downloading the Mail
Downloading the mail is perhaps the most complex part of this exercise but take it one step at a time, Gentle Reader, and it becomes quite logical. First step is to download and setup the required SSL certificates, then to setup fetchmail and finally to setup procmail.
Download the SSL Certificates
The debacle of the Gmail "expired certificate" saga of August 2008 has prompted me to completely rewrite this section and to reconsider the method I have advocated in the past. I will demonstrate a method to create your own certificate pack directly from the Mozilla's source tree using Daniel Stenberg's amazing perl script and then extract the required certificates from this certificate pack. This script is part of the newest version of cURL but I give the directions here to download it manually from gitweb as certainly on Slackware at least this marvellous script is not installed by default. To subsequently run this script you will need perl installed as well as the
perl-libwww modules which can be easily picked up by installing
$ mkdir -pv $HOME/.certs $ cd $HOME/.certs $ touch Thawte_Premium_Server_CA.pem $ touch Equifax_Secure_CA.pem $ wget --no-check-certificate \ https://github.com/bagder/curl/raw/master/lib/mk-ca-bundle.pl $ perl mk-ca-bundle.pl
(I last tested this technique on May 8th 2015 and it still runs beautifully!) This will generate a full certificate pack named
ca-bundle.crt. You will need to open this with your favourite text editor and then manually extract two certificates, the first is the "Thawte Premium Server CA" certificate which should copied and pasted into the Thawte pem file that we created with touch. I show the certificate here for any who has trouble with the perl script, although it is always best to generate the certificate yourself:
-----BEGIN CERTIFICATE----- MIIDJzCCApCgAwIBAgIBATANBgkqhkiG9w0BAQQFADCBzjELMAkGA1UEBhMCWkExFTATBgNVBAgT DFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3Vs dGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UE AxMYVGhhd3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNlcnZl ckB0aGF3dGUuY29tMB4XDTk2MDgwMTAwMDAwMFoXDTIwMTIzMTIzNTk1OVowgc4xCzAJBgNVBAYT AlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEdMBsGA1UEChMU VGhhd3RlIENvbnN1bHRpbmcgY2MxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2 aXNpb24xITAfBgNVBAMTGFRoYXd0ZSBQcmVtaXVtIFNlcnZlciBDQTEoMCYGCSqGSIb3DQEJARYZ cHJlbWl1bS1zZXJ2ZXJAdGhhd3RlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0jY2 aovXwlue2oFBYo847kkEVdbQ7xwblRZH7xhINTpS9CtqBo87L+pW46+GjZ4X9560ZXUCTe/LCaIh Udib0GfQug2SBhRz1JPLlyoAnFxODLz6FVL88kRu2hFKbgifLy3j+ao6hnO2RlNYyIkFvYMRuHM/ qgeN9EJN50CdHDcCAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQQFAAOBgQAm SCwWwlj66BZ0DKqqX1Q/8tfJeGBeXm43YyJ3Nn6yF8Q0ufUIhfzJATj/Tb7yFkJD57taRvvBxhEf 8UqwKEbJw8RCfbz6q1lu1bdRiBHjpIUZa4JMpAwSremkrj/xw0llmozFyD4lt5SZu5IycQfwhl7t UCemDaYj+bvLpgcUQg== -----END CERTIFICATE-----
The second is the "Equifax Secure CA" certificate which should copied and pasted into the Equifax pem file that we created with touch. Again I show the certificate here for any who has struggled with the perl script:
-----BEGIN CERTIFICATE----- MIIDIDCCAomgAwIBAgIENd70zzANBgkqhkiG9w0BAQUFADBOMQswCQYDVQQGEwJVUzEQMA4GA1UE ChMHRXF1aWZheDEtMCsGA1UECxMkRXF1aWZheCBTZWN1cmUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5 MB4XDTk4MDgyMjE2NDE1MVoXDTE4MDgyMjE2NDE1MVowTjELMAkGA1UEBhMCVVMxEDAOBgNVBAoT B0VxdWlmYXgxLTArBgNVBAsTJEVxdWlmYXggU2VjdXJlIENlcnRpZmljYXRlIEF1dGhvcml0eTCB nzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAwV2xWGcIYu6gmi0fCG2RFGiYCh7+2gRvE4RiIcPR fM6fBeC4AfBONOziipUEZKzxa1NfBbPLZ4C/QgKO/t0BCezhABRP/PvwDN1Dulsr4R+AcJkVV5MW 8Q+XarfCaCMczE1ZMKxRHjuvK9buY0V7xdlfUNLjUA86iOe/FP3gx7kCAwEAAaOCAQkwggEFMHAG A1UdHwRpMGcwZaBjoGGkXzBdMQswCQYDVQQGEwJVUzEQMA4GA1UEChMHRXF1aWZheDEtMCsGA1UE CxMkRXF1aWZheCBTZWN1cmUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MQ0wCwYDVQQDEwRDUkwxMBoG A1UdEAQTMBGBDzIwMTgwODIyMTY0MTUxWjALBgNVHQ8EBAMCAQYwHwYDVR0jBBgwFoAUSOZo+SvS spXXR9gjIBBPM5iQn9QwHQYDVR0OBBYEFEjmaPkr0rKV10fYIyAQTzOYkJ/UMAwGA1UdEwQFMAMB Af8wGgYJKoZIhvZ9B0EABA0wCxsFVjMuMGMDAgbAMA0GCSqGSIb3DQEBBQUAA4GBAFjOKer89961 zgK5F7WF0bnj4JXMJTENAKaSbn+2kmOeUJXRmm/kEd5jhW6Y7qj/WsjTVbJmcVfewCHrPSqnI0kB BIZCe/zuf6IWUrVnZ9NA2zsmWLIodz2uFHdh1voqZiegDfqnc1zqcPGUIWVEX/r87yloqaKHee95 70+sB3c4 -----END CERTIFICATE-----
And finally to rehash the certificates so Openssl knows how to find them:
$ c_rehash $HOME/.certs/
I should mention as well that many distros have a bundle of CA certificates with a package name something like "ca-certificates" that can also be used in place of these reasonably complex instructions. I shall point out how this bundle can be utilised further on in the guide. This completes the SSL configuration and now onto lighter matters with the required steps to setup fetchmail.
Setting up fetchmail
Fetchmail is fairly easy to use and setup, particularly as the hard yards have already been done with the SSL certificates. You will need to create the file
$HOME/.fetchmailrc and then add the following information for accessing the Gmail server:
poll pop.gmail.com with proto POP3 user 'john.example' there with password 'rover' is 'john' here mda "/usr/bin/procmail -d %T" options no keep ssl sslcertck sslcertpath /home/john/.certs/
It is almost too obvious to point out, Gentle Reader, that you will need to substitute your own details for password, username, email address etc. Those who have installed a CA certificate bundle through their distro's repository will need to change the
sslcertpath setting to somehing like
/etc/ssl/certs according to the installation path of the certificate bundle. There remains a final touch, since the username and password are openly in this file you should make the file readable only by the file owner:
$ chmod 600 ~/.fetchmailrc
Now would be a good to time also to make sure you have POP forwarding enabled in your Gmail account. You will find this in: Settings - Forwarding and POP at the web interface of Gmail. Note as well that you cannot remove mail from Gmail servers via POP3 but you can choose to have your messages archived, kept or deleted once they have been downloaded via POP3. This is a Gmail setting hidden in Settings - Forwarding and POP: "When messages are accessed with POP..."
Setting up procmail
procmail is the final link in the chain for downloading mail. Before setting it up there is a little bit of outstanding business to attend to: setting the
~/.bashrc making the obvious substitution for username:
# Sets the Mail Environment Variable MAIL=/var/spool/mail/john && export MAIL
procmail will now know the location of the default mail spool and will deliver all mail to there that has not been sorted to other locations. But the creation of a
$HOME/.procmailrc is still required and I give an example below. I have also added a sample filtering recipe for the mutt user mailing list to give the very beginning of what can be a complex process:
PATH=/bin:/usr/bin:/usr/local/bin VERBOSE=off DEFAULT=/var/spool/mail/john MAILDIR=$HOME/mail LOGFILE=$HOME/.procmaillog # Recipes below this comment: :0: * ^TOmutt-user mutt
The single filtering recipe means that procmail delivers all email addressed to "mutt-user" directly to
$HOME/mail/mutt. All other mail goes to the default location
/var/spool/mail/username as specified by
~/.bashrc, Gentle Reader, I have demonstrated above how to add this to
~/.procmailrc with the DEFAULT setting. For the definitive guide to further recipes, and guidance on much more complex recipes, don't forget to run
man procmailex, it all starts to make sense after a while.
Sending the Mail
I have formerly been an advocate for the simple MTA ssmtp, which some would call a Mail Sending Agent (MSA). However I believe that development of ssmtp has ceased and I have moved with some regret on to msmtp. A single configuration file is required for msmtp:
$HOME/.msmtprc and the following section gives the required details to access Gmail and reference the required certificate:
account default host smtp.gmail.com port 587 from email@example.com tls on tls_starttls on # tls_trust_file /home/john/.certs/Thawte_Premium_Server_CA.pem tls_trust_file /home/john/.certs/Equifax_Secure_CA.pem auth on user john.example password rover logfile ~/.msmtp.log
I need not mention again, Gentle Reader, that there should be some fairly obvious changes here to substitute your own username, password and email address? For those who have installed a bundle of CA certificates from their distro's repository the
tls_trust_file setting will need to be changed to something like
/etc/ssl/certs/ca-certificates.crt depending on the exact installation path of the certificate bundle. And then the final touch, since the username and password are openly in this file, you should make the file readable only by the file owner:
$ chmod 600 ~/.msmtprc
msmtp is a great program that has many features that quite frankly I am still exploring, feel free to point out anything that I have missed, there is an email link at the base of this page for that purpose. But now finally to Mutt:
Reading the Mail
March 12th 2014 saw the release of a "Development" version of Mutt: 1.5.23 and it is this version that I have used for this page. Although labelled as a pre-release version I have found it very stable and I suspect you will as well. Mutt is driven by the file
$HOME/.muttrc and I spent some time building this file from scratch but for you, Gentle Reader, I include here a more basic version, similar to the one I started from. Some parts of this, such as aliases and colors, are sourced from their own file so don't forget to create these files.
#======================================================# # Boring details set realname = "john" set from = "firstname.lastname@example.org" set use_from = yes set envelope_from ="yes" # Use a signature set signature="~/.signature" # Use msmtp rather than sendmail. Check that # the path is correct for your system: set sendmail="/usr/bin/msmtp" # If not set in ~/.bashrc: set spoolfile = /var/spool/mail/john #======================================================# # Folders set folder="$HOME/mail" # Local mailboxes stored here set record="+sent" # Where to store sent messages set postponed="+postponed" # Where to store draft messages set mbox_type=mbox # Mailbox type set move=no # Don't move mail from spool #======================================================# # Watch these mailboxes for new mail, useful only if # Procmail or Maildrop is used to sort mail. mailboxes ! +slrn +fetchmail +mutt set sort_browser=alpha # Sort mailboxes by alpha(bet) #======================================================# # What to show and order of headers ignore * unignore Date: From: User-Agent: X-Mailer X-Operating-System To: \ Cc: Reply-To: Subject: Mail-Followup-To: hdr_order Date: From: User-Agent: X-Mailer X-Operating-System To: \ Cc: Reply-To: Subject: Mail-Followup-To: #======================================================# # which editor do you want to use? # vim of course! set editor="vim -c 'set tw=70 et' '+/^$' " set edit_headers=yes # See the headers when editing #======================================================# # Aliases set alias_file = ~/mail/mutt_aliases # In their own file source ~/mail/mutt_aliases # Source them set sort_alias=alias # Sort alphabetically #======================================================# # Colours: defaults are a little bleak so experiment! source ~/mutt/mutt_colors # In their own file #======================================================# # Lists: An example using the mutt-users list: lists mutt-users subscribe mutt-users set followup_to=yes # Sets 'Mail-Followup-To' header set honor_followup_to=yes fcc-hook mutt-user +mutt # See your own posts using fcc #======================================================# # Odds and ends set markers # mark wrapped lines of text in the pager with a + set smart_wrap # Don't wrap mid-word set pager_context=5 # Retain 5 lines of previous page when scrolling. set status_on_top # Status bar on top. push <show-version> # Shows mutt version at startup
I have been taken to task somewhat by one reader of this page who felt there should be a little more information about Mutt and colors and so ...
Coloring in Mutt
If your terminal supports color, and I believe it is a rarity these days for this not to be case, you can color almost any aspect of the Mutt window. All the fine details are in the Mutt manual: "Section 3: Configuration 8: Using color and mono video attributes". But I can tell you, Gentle Reader, that the basic usage is:
color object foreground background
This can be made a lot more compilcated but a simple start is a good start. The basic colors are white, black, green, magenta, blue, cyan, yellow, red and default but you can also prefix a foreground color with "bright" to make the color bold. Now I personally use a Terminal with a white background so my own colors are:
#---- Mutt Colors for White Background ------- color hdrdefault black default color quoted red default color signature brightblack default color indicator brightwhite red color attachment black default color error red default color message blue default color search brightwhite magenta color status brightyellow blue color tree red default color normal blue default color tilde green default color bold brightyellow default color markers red default
There should also be a file called
colors.default installed as part of your Mutt installation which is intended for Terminals with white backgrounds, this might also furnish a starting point as it did once for me. Just to balance out the equation a little I will also give the colors for those who prefer a black background. This example is taken directly from another sample file
colors.linux which should be installed along with Mutt:
#---- Mutt Colors for Black Background ------- color hdrdefault blue black color quoted blue black color signature blue black color attachment red black color message brightred black color error brightred black color indicator black red color status brightgreen blue color tree white black color normal white black color markers red black color search white black color tilde brightmagenta black color index blue black ~F color index red black "~N|~O"
If you don't like either of these feel free to experiment a little and come up with your own, most Mutt users will alter these basic colors. I prefer mine simple but if you pull out the manual you will see that you can spend many hours getting it exactly as you want. But now finally to check the mail:
Finally it is reward time as you open Mutt, type
! to open a shell prompt, type
fetchmail -v and start reading the mail! My parting gift to you, Gentle Reader, is a little macro that was written for me by a generous person on the mutt-user mailing list that will actually do this for you when you simply press "I". Place the following in your
macro index,pager I '<shell-escape> fetchmail -v<enter>'
~/.muttrc is intentionally a little basic, although I suspect that it will cover most needs anyway. Don't be afraid however to spend hour after hour painstakingly crafting your own to produce that perfect setup as mutt encourages and rewards such efforts!
And in conclusion...
I wish you all the best with one of the truly great Linux console programs! Please send me an email, using Mutt of course, to let me know if you have found this page at all useful and as well to suggest any corrections that you feel should be made. If you are feeling generous perhaps you could also assist me keep this page alive by assisting with the hosting bills for this site, if not please feel free to utilise this page in any way you see fit and remember: "Have Fun!".