Using Mutt with Gmail (POP3 over SSL)

When I first created this page back in 2008 using the Gmail servers to collect your email with tools such as Mutt or Alpine was considered pretty cool and this page was quite popular. Times have changed somewhat and the rise of the gui email clients, the arrival of IMAP access to Gmail and the lack of interest by many Linux users in digging too deep have all contributed to a drop in interest in a page which still proposes this 'old school' approach!

However I am not in this business to achieve high page rankings, the aim of this page is to show that an old school approach to accessing the Gmail servers with a collection of high quality Linux utilities still has a place. Certainly I have been using variations of the techniques described on this page to collect my email for over a decade now. And now, if you are interested Gentle Reader, I will show you how you can do it too!

All the steps...

In sequence there are four applications that need to be installed and appropriately configured to download the mail, sort it, read it and then send a reply via Gmail. But first some settings will have to be made via the Gmail gui. Log in using the gui interface and ferret out the following settings:

  1. POP Forwarding: You will have to tell Gmail that you want to use POP3 access rather than IMAP. This setting can be found under Settings --> Forwarding and POP/IMAP --> POP Download.
  2. Delete Email: Just underneath the option above is another that gives you a few options with what to do with the email on the remote server after it has been accessed via POP3. I select 'delete GMail's copy'.
  3. Allow 'less secure apps': You will have to allow Gmail to be accessed by 'less secure apps' as this includes Mutt! The relevant link is here...

Now the gui can be shut down and the more satisfying work with configuration files can commence.

Receiving the mail...

For a very long period of time I used Fetchmail to collect my mail from Gmail but now in late 2019 I can see that there has not been a new release of Fetchmail in 6 years, April 23rd 2013. When I looked for alternatives I realised that Charles Cazabon's getmail not only has a fairly amazing vintage with the first release in 1998 but a recent release in early 2019. So I took it for a whirl and I have been very happy with it and as a bonus getmail is installed as part of a full Slackware installation. The configuration file for Gmail is simple, and lives on my sytem in /home/andrew/.getmail/getmailrc:

#----------------------------------#
#  Some simple GMail Settings:     #
#----------------------------------#

[retriever]
type = SimplePOP3SSLRetriever
ca_certs = /etc/ssl/certs/ca-certificates.crt
server = pop.gmail.com
username = xxxxxxxxxxxxxx@gmail.com
port = 995
password = xxxxxxxxxx

[destination]
type = MDA_external
path = /usr/bin/maildrop
unixfrom = True

[options]
verbose = 2
message_log_verbose = true
message_log = ~/.getmail.log
# Add some headers to all messages, specifically:
#   1. Delivered-To: header field (disabled as it does not work here)
#   2. Received: header field (enabled)
delivered_to = false
received = true
# Delete from remote server after download:
delete = true

Note that I have munged my personal details, you will have to place your own details in place. Only other note of importance is that getmail hands off to my 'Mail Delivery Agent' (MDA) rather than delivering directly to the mail spool. Note as well that although I have specified the location of the Slackware ca-certs installation I believe that no checking of the remote certificate takes place. But now to sort out the MDA.

Sorting the mail...

For more than a decade I used procmail as an MDA and procmail is indeed one of the oldest GNU utilities with the first release in 1990. Predating Linux itself! However in 2021 there will be 2 decades with no work on procmail and even an email from the developer himself warning users away so again I have looked for a replacement. I did not have to look hard as the obvious substitute is maildrop and the latest version (released on September 2018) was easily compiled and installed from SBo. Looks like the previous script maintainer is not interested in updating so I will become the maintainer in the next week or so! I cobbled together a relatively straightforward /home/andrew/.mailfilter file:

#-------------------------------------------#
#    Andrew's very basic mailfilters...     #
#-------------------------------------------#

DEFAULT=/var/spool/mail/andrew
MAILBOX="$HOME/mail/mailboxes"          
SENDMAIL="/usr/bin/msmtp"
logfile "$HOME/.mailfilter.log"

#-- Sort out the mailing lists...  --#
if (/^List-Id:.*ffmpeg-user.ffmpeg.org>/)
    to $MAILBOX/ffmpeg

if (/^List-Id:.*slackbuilds-users.slackbuilds.org>/)
 	to $MAILBOX/slackbuilds

if (/^List-Id:.*mutt.org>/)
 	to $MAILBOX/mutt

#-- Some frequent Forums...  --#
if (/^From:.*forum@linuxquestions.org>/)
	to $MAILBOX/linuxquestions

#--    cc syntax ...     --#
if (/^From:.*@iinet.net.au>/)
{
	cc"!xxxxxxxxxxxxx@gmail.com"
}

The cognoscenti may notice that my mail setup is a little idiosyncratic in that I deliver all mail to my spool at /var/spool/mail/andrew and deliver much of it to mailboxes in $HOME/mail/mailboxes. Many variations of this are possible but this one works well for me. I have included a sample of the ability of maildrop to automagically email to another address when correctly filtered, this works beautifully with msmtp. But now to lay hands on mutt itself!

Reading the mail...

Slackware is well served with mutt packages and a full installation of Slackware will always hold the very latest version. In late 2019 this means version 1.12.1 which comes complete with the sidebar, which I confess that I tried to like for some time and then stopped using it. I give a very brief sample $HOME/.muttrc file below but you would be better to 'roll your own' as there are so many choices to make! Have a look, Gentle Reader, in the docs section of your mutt installation and you will see the file sample.muttrc and a scattering of other sample configuration files that will get you well started. But as well here is a simplified and munged version of my own .muttrc file:

######################################################
#                                                    #
#              Andrew's .muttrc                      #
#                                                    #
######################################################

### The basics ###
set realname = "andrew"
set from = "xxxxxxxxxxxxxxxxx@gmail.com"
set use_from = yes
set envelope_from ="yes"
set signature="~/.signature"
set sendmail="/usr/bin/msmtp"         # Use msmtp rather than sendmail
set spoolfile=/var/spool/mail/andrew  # Conventional spool setting

### Mailboxes ###
set mbox_type=mbox                 # Choice of mbox or maildir
set folder="~/mail/mailboxes"      # Contains all the mailboxes
set record="+sent"                 # Where to store sent messages
set postponed="+postponed"         # Where to store draft messages
set move=no                        # Don't move mail from the spool when closing mutt
set sort_browser=alpha             # Sort mailboxes by alpha(bet)

# Watch these mailboxes which will get mail delivered automatically by 
# the maildrop filters, otherwise their mail may be missed!
mailboxes ! +linuxquestions +ffmpeg +slackbuilds +mutt

### Show which headers? ###
ignore *
unignore Date: From: User-Agent: X-Mailer X-Operating-System X-User-Agent: To: \
               Cc: Reply-To: Subject: Mail-Followup-To:
hdr_order Date: From: User-Agent: X-Mailer X-Operating-System X-User-Agent: To: \
               Cc: Reply-To: Subject: Mail-Followup-To: 
               
# Use vim with 70 character width and put cursor in first blank line
set editor="vim -c 'set tw=70 et' '+/^$' " 
set edit_headers=yes                # See the headers when editing
set hostname=`hostname --fqdn`      # Set right hand side of MID

### Taming HTML Messages ###
set mailcap_path="~/mail/mutt/mutt_mailcap"
auto_view text/html            # Automatically parse html messages through w3m
auto_view text/x-vcard         # vcard attachments viewed through mutt.vcard.filter
set implicit_autoview=yes      # Load in pager if setting in .mailcap says 'copiousoutput'.
                                           
### Aliases ###
set sort_alias=alias                 # Sort aliases in alpha order by alias name
set alias_file=~/mail/.mail_aliases	# where I keep my aliases

### Colours ###

# (default, white, black, green, magenta, blue, cyan, yellow, red)
# (bright...)
# (color1,color2,...,colorN-1)
#
#---- --Mutt Colors for Black Background -------
#       Object	    Foreground	    Background
color   hdrdefault   yellow            black
color   quoted       blue              black
color   signature    blue              black
color   attachment   red               black
color   message      brightred         black
color   error        brightred         black
color   indicator    black             yellow
color   status       white             blue
color   tree         white             black
color   normal       white             black
color   markers      red               black
color   search       white             black
color   tilde        brightmagenta     black
color   index        blue              black ~F
color   index        white             black "~N|~O"

### Lists ###
lists slackbuilds-users abcde-users ffmpeg-user mutt-user   
subscribe slackbuilds-users abcde-users ffmpeg-user mutt-user
set followup_to=yes            # Sets the Mail-Followup-To header for lists
set honor_followup_to=yes      # Honours the Mail-Followup-To header in replies

### View my own posts in list's folder using fcc ###
fcc-hook slackbuilds-users +slackbuilds
fcc-hook abcde-user +abcde
fcc-hook mplayer-users +mplayer
fcc-hook ffmpeg-user +ffmpeg

# Macros
macro index,pager I '<shell-escape> getmail <enter>'     # Checks the mail
macro attach s "<save-entry><bol>$HOME/downloads/<eol>"  # Alters 'save' path for attachments

### Odds and ends ###
set markers=no            # mark wrapped lines of text in the pager with a +
set smart_wrap            # Don't wrap mid-word
set pager_context=5       # Retain last line of previous page when scrolling.
set status_on_top         # Status bar on top
push <show-version>       # Shows mutt version at startup
set check_new=yes         # Check for new mail
set sort=threads
source ~/mail/mutt/crypto # Crazy PGP stuff

I have retreated more than a little from a much more ornate configuration file and this one is a little more simplified for presentation here. But it is a usable configuration file and specifically created to match in with the configuration of the other 3 utilities being used in this sequence. And now to the final piece in this jigsaw: msmtp.

Sending the mail...

Although I am somewhat tempted by postfix, in particular as Slackware has removed sendmail as default and placed postfix instead, msmtp is a better choice for the simple task of sending mail via Gmail. A single configuration file is required for msmtp $HOME/.msmtprc and the following section gives the required details to access Gmail and reference the required certificate:

#----------------------------------------------------#
#                                                    #
#  Andrews's msmtprc setup with a nod to the Arch    #
#  wiki, Debian pages, the msmtp man pages etc etc.  #
#                                                    #
#----------------------------------------------------#

# Set default values for all accounts:
defaults
auth            on
tls             on
tls_starttls	on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile        ~/.msmtp.log

#Gmail Account:
account			gmail
host			smtp.gmail.com 
port			587           
from			xxxxxxxxxx@gmail.com     
user			xxxxxxxxxx@gmail.com  
password		xxxxxxxxxx

# Set a default account:
account default : gmail

# Check it all out:
# echo "Hello there" | msmtp --debug xxxxxxxxxxxx@gmail.com

There are no great mysteries in this file which can be found pretty much anywhere on the Internet these days. It has worked robustly for me with both mutt and maildrop but if there are any trouble I have included a little debugging line at the base of the configuration file.

Reward Time!

Finally it is reward time as you open Mutt, type ! to open a shell prompt, type in getmail -v and start reading your mail! My parting gift to you, Gentle Reader, is a little macro that was written for me by a generous person on the mutt-user mailing list that will actually do this for you when you simply press "I". I have already placed this macro in my sample .muttrc file above but here it is again:

macro index,pager I '<shell-escape> getmail <enter>'

I wish you all the best with this, remembering that I have used small variations of this technique for more than a decade now and it has served me very, very well. I hope that you also have a great experience with these marvelous Linux command line tools!