OfflineIMAP
OfflineIMAP is a Python utility to sync mail from IMAP servers. It does not work with the POP3 protocol or mbox, and is usually paired with a MUA such as Mutt.
Contents
Installation
Install offlineimap. For a development version, install offlineimap-gitAUR.
Configuration
Offlineimap is distributed with two default configuration files, which are both located in /usr/share/offlineimap/
. offlineimap.conf
contains every setting and is thorougly documented. Alternatively, offlineimap.conf.minimal
is not commented and only contains a small number of settings (see: Minimal).
Copy one of the default configuration files to ~/.offlineimaprc
.
Minimal
The following file is a commented version of offlineimap.conf.minimal
.
~/.offlineimaprc
[general] # List of accounts to be synced, separated by a comma. accounts = main [Account main] # Identifier for the local repository; e.g. the maildir to be synced via IMAP. localrepository = main-local # Identifier for the remote repository; i.e. the actual IMAP, usually non-local. remoterepository = main-remote [Repository main-local] # OfflineIMAP supports Maildir, GmailMaildir, and IMAP for local repositories. type = Maildir # Where should the mail be placed? localfolders = ~/mail [Repository main-remote] # Remote repos can be IMAP or Gmail, the latter being a preconfigured IMAP. type = IMAP remotehost = host.domain.tld remoteuser = username
Selective folder synchronization
For synchronizing only certain folders, you can use a folderfilter in the remote section of the account in ~/.offlineimaprc
. For example, the following configuration will only synchronize the folders Inbox
and Sent
:
~/.offlineimaprc
[Repository main-remote] # Synchronize only the folders Inbox and Sent: folderfilter = lambda foldername: foldername in ["Inbox", "Sent"] ...
For more options, see the official documentation.
Usage
Before running offlineimap, create any parent directories that were allocated to local repositories:
$ mkdir ~/mail
Now, run the program:
$ offlineimap
Mail accounts will now be synced. If anything goes wrong, take a closer look at the error messages. OfflineIMAP is usually very verbose about problems; partly because the developers did not bother with taking away tracebacks from the final product.
Tips and tricks
Running offlineimap in the background
Most other mail transfer agents assume that the user will be using the tool as a daemon by making the program sync periodically by default. In offlineimap, there are a few settings that control backgrounded tasks.
Confusingly, they are spread thin all-over the configuration file:
~/.offlineimaprc
# In the general section [general] # Controls how many accounts may be synced simultaneously maxsyncaccounts = 1 # In the account identifier [Account main] # Minutes between syncs autorefresh = 0.5 # Quick-syncs do not update if the only changes were to IMAP flags. # autorefresh=0.5 together with quick=10 yields # 10 quick refreshes between each full refresh, with 0.5 minutes between every # refresh, regardless of type. quick = 10 # In the remote repository identifier [Repository main-remote] # Instead of closing the connection once a sync is complete, offlineimap will # send empty data to the server to hold the connection open. A value of 60 # attempts to hold the connection for a minute between syncs (both quick and # autorefresh).This setting has no effect if autorefresh and holdconnectionopen # are not both set. keepalive = 60 # OfflineIMAP normally closes IMAP server connections between refreshes if # the global option autorefresh is specified. If you wish it to keep the # connection open, set this to true. This setting has no effect if autorefresh # is not set. holdconnectionopen = yes
systemd service
When configured to run background jobs, offlineimap can be managed with the following systemd/User service:
/etc/systemd/user/offlineimap.service
[Unit] Description=Start offlineimap as a daemon [Service] ExecStart=/usr/bin/offlineimap [Install] WantedBy=default.target
Automatic mailbox generation for mutt
Mutt cannot be simply pointed to an IMAP or maildir directory and be expected to guess which subdirectories happen to be the mailboxes, yet offlineimap can generate a muttrc fragment containing the mailboxes that it syncs.
~/.offlineimaprc
[mbnames] enabled = yes filename = ~/.mutt/mailboxes header = "mailboxes " peritem = "+%(accountname)s/%(foldername)s" sep = " " footer = "\n"
Then add the following lines to ~/.mutt/muttrc
.
~/.mutt/muttrc
# IMAP: offlineimap set folder = "~/mail" source ~/.mutt/mailboxes set spoolfile = "+account/INBOX" set record = "+account/Sent\ Items" set postponed = "+account/Drafts"
account
is the name you have given to your IMAP account in ~/.offlineimaprc
.
Gmail configuration
This remote repository is configured specifically for Gmail support, substituting folder names in uppercase for lowercase, among other small additions. Keep in mind that this configuration does not sync the All Mail folder, since it is usually unnecessary and skipping it prevents bandwidth costs:
~/.offlineimaprc
[Repository gmail-remote] type = Gmail remoteuser = user@gmail.com remotepass = password nametrans = lambda foldername: re.sub ('^\[gmail\]', 'bak', re.sub ('sent_mail', 'sent', re.sub ('starred', 'flagged', re.sub (' ', '_', foldername.lower())))) folderfilter = lambda foldername: foldername not in ['[Gmail]/All Mail'] # Necessary as of OfflineIMAP 6.5.4 sslcacertfile = /etc/ssl/certs/ca-certificates.crt
Password management
.netrc
Add the following lines to your ~/.netrc
:
machine hostname.tld login [your username] password [your password]
Do not forget to give the file appropriate rights like 600 or 700:
$ chmod 600 ~/.netrc
Using GPG
GNU Privacy Guard can be used for storing a password in an encrypted file. First set up GnuPG and then follow the steps in this section. It is assumed that you can use your GPG private key without entering a password all the time.
First type in the password for the email account in a plain text file. Do this in a secure directory with 700
permissions located on a tmpfs to avoid writing the unencrypted password to the disk. Then encrypt the file with GnuPG setting yourself as the recipient.
Remove the plain text file since it is no longer needed. Move the encrypted file to the final location, e.g. ~/.offlineimappass.gpg
.
Now create a python function that will decrypt the password:
~/.offlineimap.py
#! /usr/bin/env python2 from subprocess import check_output def get_pass(): return check_output("gpg -dq ~/.offlineimappass.gpg", shell=True).strip("\n")
Load this file from ~/.offlineimaprc
and specify the defined function:
~/.offlineimaprc
[general] # Path to file with arbitrary Python code to be loaded pythonfile = ~/.offlineimap.py ... [Repository example] # Decrypt and read the encrypted password remotepasseval = get_pass() ...
Using pass
pass is a simple password manager from the command line based on GPG.
First create a password for your email account(s):
$ pass insert Mail/account
Now create a python function that will decrypt the password:
~/.offlineimap.py
#! /usr/bin/env python2 from subprocess import check_output def get_pass(account): return check_output("pass Mail/" + account, shell=True).splitlines()[0]
This is an example for a multi-account setup. You can customize the argument to pass as defined previously.
Load this file from ~/.offlineimaprc
and specify the defined function:
~/.offlineimaprc
[general] # Path to file with arbitrary Python code to be loaded pythonfile = ~/.offlineimap.py ... [Repository Gmail] # Decrypt and read the encrypted password remotepasseval = get_pass("Gmail") ...
Gnome keyring
In configuration for remote repositories the remoteusereval/remotepasseval fields can be set to custom python code that evaluates to the username/password. The code can be a call to a function defined in a Python script pointed to by 'pythonfile' config field. Create ~/.offlineimap.py
according to either of the two options below and use it in the configuration:
[general] pythonfile = ~/.offlineimap.py [Repository examplerepo] type = IMAP remotehost = mail.example.com remoteusereval = get_username("examplerepo") remotepasseval = get_password("examplerepo")
Option 1: using gnomekeyring Python module
Install python2-gnomekeyring. Then:
~/.offlineimap.py
#! /usr/bin/env python2 import gnomekeyring as gkey def set_credentials(repo, user, pw): KEYRING_NAME = "offlineimap" attrs = { "repo": repo, "user": user } keyring = gkey.get_default_keyring_sync() gkey.item_create_sync(keyring, gkey.ITEM_NETWORK_PASSWORD, KEYRING_NAME, attrs, pw, True) def get_credentials(repo): keyring = gkey.get_default_keyring_sync() attrs = {"repo": repo} items = gkey.find_items_sync(gkey.ITEM_NETWORK_PASSWORD, attrs) return (items[0].attributes["user"], items[0].secret) def get_username(repo): return get_credentials(repo)[0] def get_password(repo): return get_credentials(repo)[1] if __name__ == "__main__": import sys import os import getpass if len(sys.argv) != 3: print "Usage: %s <repository> <username>" \ % (os.path.basename(sys.argv[0])) sys.exit(0) repo, username = sys.argv[1:] password = getpass.getpass("Enter password for user '%s': " % username) password_confirmation = getpass.getpass("Confirm password: ") if password != password_confirmation: print "Error: password confirmation does not match" sys.exit(1) set_credentials(repo, username, password)
To set the credentials, run this script from a shell.
Option 2: using gnome-keyring-queryAUR tool
~/.offlineimap.py
#! /usr/bin/env python2 # executes gnome-keyring-query get passwd # and returns the output import locale from subprocess import Popen, PIPE encoding = locale.getdefaultlocale()[1] def get_password(p): (out, err) = Popen(["gnome-keyring-query", "get", p], stdout=PIPE).communicate() return out.decode(encoding).strip()
python2-keyring
There is a general solution that should work for any keyring. Install python2-keyring from AUR, then change your ~/.offlineimaprc to say something like:
[general] pythonfile = /home/user/offlineimap.py ... [Repository RemoteEmail] remoteuser = username@host.net remotepasseval = keyring.get_password("offlineimap","username@host.net") ...
and somewhere in ~/offlineimap.py add import keyring
. Now all you have to do is set your password, like so:
$ python2 >>> import keyring >>> keyring.set_password("offlineimap","username@host.net", "MYPASSWORD")
and it will grab the password from your (kwallet/gnome-) keyring instead of having to keep it in plaintext or enter it each time.
Emacs EasyPG
See http://www.emacswiki.org/emacs/OfflineIMAP#toc2
KeePass / KeePassX
Install python2-keepass-gitAUR from the AUR, then add the following to your offlineimap.py file:
#! /usr/bin/env python2 import os, getpass from keepass import kpdb def get_keepass_pw(dbpath, title="", username=""): if os.path.isfile(dbpath): db = kpdb.Database(dbpath, getpass.getpass("Master password for '" + dbpath + "': ")) for entry in db.entries: if (entry.title == title) and (entry.username == username): return entry.password else: print "Error: '" + dbpath + "' does not exist." return
(Credit to aki--aki: https://gist.github.com/aki--aki/5180359)
Next, edit your ~/.offlineimaprc:
[general] # VVV Set this path correctly VVV pythonfile = /home/user/offlineimap.py ... [Repository RemoteEmail] remoteuser = username@host.net # Set the DB path as well as the title and username of the specific entry you'd like to use. # This will prompt you on STDIN at runtime for the kdb master password. remotepasseval = get_keepass_pw("/path/to/database.kdb", title="<entry title>", username="<entry username>") ...
Note that as-is, this does not support KDBs with keyfiles, only KDBs with password-only auth.
Kerberos authentication
Install python2-kerberosAUR from AUR and do not specify remotepass in your .offlineimaprc. OfflineImap figure out the reset all if have a valid Kerberos TGT. If you have 'maxconnections', it will fail for some connection. Comment 'maxconnections' out will solve this problem.
Troubleshooting
Overriding UI and autorefresh settings
For the sake of troubleshooting, it is sometimes convenient to launch offlineimap with a more verbose UI, no background syncs and perhaps even a debug level:
$ offlineimap [ -o ] [ -d <debug_type> ] [ -u <ui> ]
- -o
- Disable autorefresh, keepalive, etc.
- -d <debug_type>
- Where <debug_type> is one of
imap
,maildir
orthread
. Debugging imap and maildir are, by far, the most useful.
- -u <ui>
- Where <ui> is one of
CURSES.BLINKENLIGHTS
,TTY.TTYUI
,NONINTERACTIVE.BASIC
,NONINTERACTIVE.QUIET
orMACHINE.MACHINEUI
. TTY.TTYUI is sufficient for debugging purposes.
Folder could not be created
In version 6.5.3, offlineimap gained the ability to create folders in the remote repository, as described here.
This can lead to errors of the following form when using nametrans
on the remote repository:
ERROR: Creating folder bar on repository foo-remote Folder 'bar'[foo-remote] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Duplicate folder name bar (Failure)'])
The solution is to provide an inverse nametrans
lambda for the local repository, e.g.
~/.offlineimaprc
[Repository foo-local] nametrans = lambda foldername: foldername.replace('bar', 'BAR') [Repository foo-remote] nametrans = lambda foldername: foldername.replace('BAR', 'bar')
- For working out the correct inverse mapping. the output of
offlineimap --info
should help. - After updating the mapping, it may be necessary to remove all of the folders under
$HOME/.offlineimap/
for the affected accounts.
SSL fingerprint does not match
ERROR: Server SSL fingerprint 'keykeykey' for hostname 'example.com' does not match configured fingerprint. Please verify and set 'cert_fingerprint' accordingly if not set yet.
To solve this, add to ~/.offlineimaprc
(in the same section as ssl = yes
) one of the following:
- either add
cert_fingerprint
, with the certificate fingerprint of the remote server. This checks whether the remote server certificate matches the given fingerprint.
cert_fingerprint = keykeykey
- or add
sslcacertfile
with the path to the system CA certificates file. Needs ca-certificates installed. This validates the remote ssl certificate chain against the Certification Authorities in that file.
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
Copying message, connection closed
ERROR: Copying message -2 [acc: email] connection closed Folder sent [acc: email]: ERROR: while syncing sent [account email] connection closed
Cause of this can be creation of same message both locally and on server. This happens if your email provider automatically saves sent mails to same folder as your local client. If you encounter this, disable save of sent messages in your local client.
See also
- Official OfflineIMAP mailing list
- Mutt + Gmail + Offlineimap - An outline of brisbin's simple gmail/mutt setup using cron to keep offlineimap syncing.