Hello:
Continuing on my previous email about setting up lei with lore.kernel.org,
here's the next installment. As with the previous part, this will be posted on
people.kernel.org as well as here.
# lore+lei: part 2, now with IMAP
This is the second installment in the series where we're looking at using the
public-inbox `lei` tool for interacting with remote mailing list archives such
as lore.kernel.org. In the previous article we looked at delivering your
search results locally, and today let's look at doing the same, but with
remote IMAP folders.
For our example query today, we'll do some stargazing. The following will show
us all mail sent by Linus Torvalds:
f:torvalds AND rt:1.month.ago..
I'm mostly using it because it's short, but you may want to use something
similar if you have co-maintainer duties and want to automatically receive a
copy of all mail sent by your fellow subsystem maintainers.
## Note on saving credentials
When accessing IMAP folders, `lei` will require a username and password.
Unless you really like typing them in manually every time you run `lei up`,
you will probably want to have them cached on your local system. `Lei` will
defer to `git-credential-helper` for this purpose, so if you haven't already
set this up, you will want to do that now.
The two commonly used credential storage backends on Linux are "libsecret" and
"store":
- *libsecret* is the preferred mechanism, as it will work with your Desktop
Environment's keyring manager to store the credentials in a relatively safe
fashion (encrypted at rest).
- *store* should only be used if you don't have any other option, as it will
record the credentials without any kind of encryption in the
`~/.git-credentials` file. However, if nothing else is working for you and
you are fairly confident in the security of your system, it's something you
can use.
Simply run the following command to configure the credential helper globally
for your environment:
git config --global credential.helper libsecret
For more in-depth information about this topic, see `man git-credential`.
## Getting your IMAP server ready
Before you start, you should get some information about your IMAP server, such
as your login information. For my examples, I'm going to use Gmail, Migadu,
and a generic Dovecot IMAP server installation, which should hopefully cover
enough ground to be useful for the vast majority of cases.
What you will need beforehand:
- the IMAP server hostname (and port, if it's not 993)
- the IMAP username
- the IMAP password
It will also help to know the folder hierarchy. Some IMAP servers create all
subfolders below INBOX, while others don't really care.
### Generic Dovecot
We happen to be running Dovecot on mail.codeaurora.org, so I'm going to use it
as my "generic Dovecot" system and run the following command:
lei q -I https://lore.kernel.org/all/ -d mid \
-o imaps://mail.codeaurora.org/INBOX/torvalds \
<<< 'f:torvalds AND rt:1.month.ago..'
The `<<<` bit above is a Bash-ism, so if you're using a different shell, you
can use the POSIX-compliant heredoc format instead:
lei q -I https://lore.kernel.org/all/ -d mid \
-o imaps://mail.codeaurora.org/INBOX/torvalds <<EOF
f:torvalds AND rt:1.month.ago..
EOF
The first time you run it, you should get a `username:` and `password:`
prompt, but after that the credentials should be cached and no longer required
on each repeated access to the same imaps server.
NOTE: some IMAP servers use the dot `.` instead of the slash `/` for
indicating folder hierarchy, so if `INBOX/torvalds` is not working for you,
try `INBOX.torvalds` instead.
### Refreshing and subscribing to IMAP folders
If the above command succeeded, then you should be able to view the IMAP
folder in your mail client. If you cannot see `torvalds` in your list of
available folders, then you may need to refresh and/or subscribe to the newly
created folder. The process will be different for every mail client, but it
shouldn't be too hard to find.
### The same with Migadu
If you have a linux.dev account (see https://korg.docs.kernel.org/linuxdev.html),
then you probably already know that we ask you not to use your account for
subscribing to busy mailing lists. This is due to Migadu imposing soft limits
on how much incoming email is allowed for each hosted domain -- so using `lei`
+ IMAP is an excellent alternative.
To set this up with your linux.dev account (or any other account hosted on
Migadu), use the following command:
lei q -I https://lore.kernel.org/all/ -d mid \
-o imaps://imap.migadu.com/lei/torvalds \
<<< 'f:torvalds AND rt:1.month.ago..'
Again, you will need to subscribe to the new `lei/torvalds` folder to see it
in your mail client.
### The same with Gmail
If you are a Gmail user and aren't already using IMAP, then you will need to
jump through a few additional hoops before you are able to get going. Google
is attempting to enhance the security of your account by restricting how much
can be done with just your Google username and password, so services like IMAP
are not available without setting up a special "app password" that can only be
used for mail access.
Enabling app passwords requires that you first enable 2-factor authentication,
and then generate a random app password to use with IMAP. Please follow the
process described in the following Google document:
https://support.google.com/mail/answer/185833
Once you have the app password for use with IMAP, you can use `lei` and imaps
just like with any other IMAP server:
lei q -I https://lore.kernel.org/all/ -d mid \
-o imaps://imap.gmail.com/lei/torvalds \
<<< 'f:torvalds AND rt:1.month.ago..'
It requires a browser page reload for the folder to show up in your Gmail web
UI.
## Automating lei up runs
If you're setting up IMAP access, then you probably want IMAP updates to
happen behind the scenes without your direct involvement. All you need to do
is periodically run `lei up --all` (plus `-q` if you don't want non-critical
output).
If you're just getting started, then you can set up a simple `screen` session
with a `watch` command at a 10-minute interval, like so:
watch -n 600 lei up --all
You can then detach from the screen terminal and let that command continue
behind the scenes. The main problem with this approach is that it won't
survive a system reboot, so if everything is working well and you want to make
the command a bit more permanent, you can set up a systemd user timer.
Here's the service file to put in `~/.config/systemd/user/lei-up-all.service`:
[Unit]
Description=lei up --all service
ConditionPathExists=%h/.local/share/lei
[Service]
Type=oneshot
ExecStart=/usr/bin/lei up --all -q
[Install]
WantedBy=mail.target
And the timer file to put in `~/.config/systemd/user/lei-up-all.timer`:
[Unit]
Description=lei up --all timer
ConditionPathExists=%h/.local/share/lei
[Timer]
OnUnitInactiveSec=10m
[Install]
WantedBy=default.target
Enable the timer:
systemctl --user enable --now lei-up-all.timer
You can use `journalctl -xn` to view the latest journal messages and make sure
that the timer is running well.
CAUTION: user timers only run when the user is logged in. This is not actually
that bad, as your keyring is not going to be unlocked unless you are logged
into the desktop session. If you want to run `lei up` as a background process
on some server, you should set up a system-level timer and use a different
git-credential mechanism (e.g. `store`) -- and you probably shouldn't do this
on a shared system where you have to worry about your account credentials
being stolen.
## Coming up next
In the next installment we'll look at some other part of lei and
public-inbox... I haven't yet decided which. :)
Best regards,
-K
