A new take on Email to Blog

ian   June 1, 2016   Comments Off on A new take on Email to Blog
tl;dr I have a new son, Jasper, and now need to share photos with friends and family. The existing tools (social networks, email, and SMS) are too invasive or only work with a handful of people before becoming SPAM so I built my own system using a static site generator and an email server. I think it is a novel configuration that may be interesting to people working on self-hosting projects or small group collaborations. Configuration details to follow in later posts.

Social Sharing

I have spent the past couple of weeks wrestling with a seemingly simple question: how do you share photos without using a social network? The recent birth of my son Jasper means that I have a newfound need to share pictures with family and friends but all the existing solutions are either invasive (social networks) or will only work for a handful of participants before turning your happy news into SPAM (email, SMS). Interestingly, I have found a rather large portion of my friends who are fine with social networks in general but not for pictures of their children. I also have a number of friends who use social networks for everything but feel SPAMed by people’s baby pictures there. I set out to find some better options to address this fairly widespread need.

Defining the need

Like any good technologist I started out by defining what I need from this social sharing tool and then turned those goals into technical requirements. The general goal is to have a tool that lets my wife and I privately share photos and text with family, builds an archive or record of what we post, will work with everyone’s software setups, and is easy enough to use that we actually use it. As I worked through some of the details of these goals I distilled them into these requirements:
  1. Must run on servers I can trust.
  2. Must use software people are already using or very general purpose free software.
  3. Must use authentication for reading and posting but must not require me to manage passwords for each user.
  4. Must not flood people with announcements.
  5. Must have browse-able archive.
  6. Must be able to post from and read on iOS devices.
  7. Cannot be harder to add content to than it is to send an email.
  8. The hosting software must require minimal maintenance and have minimal security risk.

The Candidates

With requirements in hand I started looking at the available technology out there for a solution. Five emerged for consideration: commercial social networks (facebook, instagram, etc), free software social networks (diaspora, frendika, etc), email, mailing lists, and blogs.
Commercial social networks
Facebook, Instagram, etc all run on commercial servers designed to datamine your social interactions and profile all participants so they fail criteria 1: run on servers I can trust.
Free software social networks
Diaspora, frendika, etc could be good fits for this situation, especially because, of all the self-hosted options, these offer the easiest fine-grained permission management. Unfortunately no one in my social circle is using them, including myself, so they fail criteria 2: use software people are already using.
Email
Email actually comes pretty close here, which is probably why it is the main tool my friends use for sharing photos of their children with family. Unfortunately, it does flood people so it fails at criteria 4 and it does not provide an archive for recipients so it fails criteria 5. People can build their own archive by saving all their messages but this only works for original recipients, not someone we add later or might have forgeten to include on some individual messages.
Mailing list
Mailing lists actually solve almost all the issues with email. You get a browse-able archive. People can manage their own subscriptions so they can unsubscribe if they are getting too many messages. People do need new passwords but the listserv software manages storing those and has built-in password reset capabilities. Unfortunately, I know many of my family and friends simply get so much email or have too few tools to effectively filter the email they get that even something as easy to process as a mailing list would be burdensome or get ignored. Ultimately, this still fails criteria 4: don’t flood.
Blog
This also hits a number of the big requirements, especially using a static site generator to achieve the low maintenance and security risk of criteria 8, but some challenges remain. The biggest issues are authentication and ease of posting. Since I will be running the web server I can use whatever authentication I wish but I do not want to have to setup and maintain passwords for all of my friends and family. Posting is simply hard, especially from a mobile device, and that is where my custom work comes in. For authentication I looked into simplifying things with RSS. Many of my family members do use RSS readers for news browsing on their iPads and RSS clients are general enough that I would be comfortable telling family to install one in return for baby photos. I would also feel better about supporting individual passwords for client software than I do for general web pages because I know the websites will be viewed on multiple devices (requiring multiple times the password support). Sadly none of the free software RSS readers for iOS support authenticated feeds so this fails criteria 6: iOS required. I ended up settling on simple authentication with shared credentials for all readers and moving posting authentication to the email submission. If anyone knows of a free software iOS reader that can handle authenticated feeds, I would love to hear about it. The Android spaRSS reader worked wonderfully if anyone is looking. For ease of posting I had a problem. Logging in and posting to a blog is just not easy enough, especially if you are posting a gallery of photos, which is the main purpose of the project. I knew that if things were more complicated than sending an email we would just end up falling back to sending emails and only our parents would ever hear about Jasper. I started looking at existing email-to-blog tools but all of these either require you to trust your blog software with your email login credentials or trust a third party to process your email and post your updates. I decided that email-to-blog was the right approach but that having my blog check email for me would break criteria 8: server software should have low maintenance and security risk. In the end I decided the only blogging software durable enough for me to configure and then ignore were the static site generators, none of which have an email-to-blog submission tool that I could find. So I needed to build my own.

The Plan

What I ended up designing, and am now almost done building, is a specially configured email server (postfix) that is set to only accept email from a couple of people, to pass those messages to a local script that converts them into blog-formated plain text files with accompanying directories of images, and then feeds that into a static site generator (pelican) sitting on my webserver (Apache). I used whitelists and some anti-SPAM tools on the mail server to control who can post based on their existing email addresses, which means my wife and I can post without needing new passwords or to remember to send the post from special accounts. Using this approach it is possible to turn postfix into a sort of file-based application server that may be useful in a number of situations. I think this approach has particular potential for those looking to build a system somewhere between the simplicity of static site generators and the complexity of dynamic tools like WordPress. Currently the mail server configuration and the static site are both complete but the script to move the email messages along is still in process. I will write more detail in the next few posts on how I configure both postfix and pelican for this project and what some of the security considerations are for this using the kind of delegated authentication scheme.