Last weekend, I made this. It was a fun little exercise and I continue to get utility out of having this, so I thought I’d go over what it is and the process of building it.
What’s a Tumble?
Over the years, I’ve used a couple of different services as an easy way to track interesting things I’ve found on the internet. Basically a simple version of del.icio.us (if you remember that) but recorded over time and without categories. One of the things I really like about this is that I can go back over time and see a history of the things I was reading or thinking about. I love looking through it occasionally and going “oh yeah..“. I also love being able to go back and quickly lookup things from a week or a month ago that would otherwise be lost to me. It seems like a simple thing, but it’s gotten a lot of use.
For most of the last ten years I used either Posterous or Tumblr. Posterous shut down after acquisition and I’m just not a fan of Tumblr any more, especially after they shut down my favorite feature: email posts.
For a bunch of different reasons, I prefer sending email to add these links and snippets. I’m on a few different computers throughout the day and it’s just easier to be consistent; I always have email access. So when I got tired of Tumblr and some of their annoyances, I decided to setup and maintain my own.
More History
The whole thing is built on top of GatsbyJS, which I moved my blog over to a few months ago. For the most part I’ve just been using the Gatsby starter template, but I’ve also started building another site with it and have another in the works. I think I’ve used almost everything for my blog at this point. Ages ago, it was just a little wiki site sitting on top of Apache. I tried Wordpress for awhile, got annoyed, and was relieved when Jekyll was introduced and the whole thing felt clean again. The idea of a static site generator always made a lot of sense to me. But then I bailed to try Ghost for a few years, mostly because I liked the live preview idea and thought I’d eventually host it myself.
I never did. So when I learned about Gatsby, I was ready to head back to the land of static site generators. Gatsby had the added benefit of using React on the front end and GraphQL for pulling pages together and running queries - both things I wanted to tinker with. The philosophy behind using GraphQL in particular is interesting; it’s run at compile time (not runtime) and pulls everything you need to build your site and pages together from whatever sources you want, whether it’s markdown files, DBs, or web services. Webpack is run for optimization and the whole thing is designed for blazing speed. Add in live preview (via gatsby develop) and that everything happily sits back in a Git repo again, and life is good.
Export
Back to the tumble. The first step was unlocking all of my data from Tumblr, which meant registering a Developer API key to hit their endpoints and get results. Any old language would do to get the data and do what I wanted with it, so I took the opportunity to mess some more with Elixir, which I’m growing very fond of. I’m still a complete beginner with it, but it’s making me think differently and I like that.
At the same time, I knew some folks who were stumbling trying to use Java and Jackson to deserialize some complex JSON out of an interface. Not only was it infuriating, but the sheer amount of boiler plate and POJO construction/annotation that had to be done was insane. So when I used Poison to pull out the fields I cared about from the fairly verbose Tumblr API, I took a big sigh of relief. It was trivial to walk down the data structure and Enum.map against what I cared about. I know this seems like a silly, simple thing.. and this is really just a longwinded way to say I feel completely done with the Java ecosystem. It’s too verbose and too much hassle. It focuses on constructs I don’t want to care about.
Anyway, I pulled the data and threw it all in a Google Sheet. For the most part, the data lined up fine and I didn’t lose much.
Google Setup
My preferred way of adding to my tumblelog has always been via email, so I wired that up. Google Scripts makes this kind of thing pretty easy these days. Emails that are to and from me with a specific subject line prefix get a label in Gmail, marked as read, and skip the inbox. Then an hourly job comes around (called a trigger in Google Scripts), takes a look at those emails, and adds stuff to the spreadsheet.
//set a time-driven trigger to run this function on the desired frequency
function monitorEmails() {
var label = GmailApp.getUserLabelByName('Links');
//Logger.log(label.getName());
var links = label.getThreads();
//Logger.log("links: " + links.length);
var doc = SpreadsheetApp.openById("putyouridhere");
var sheet = doc.getSheetByName("Links");
for( var i = 0; i < links.length; ++i ) {
//Logger.log("processing an email");
var email = links[i].getMessages()[0];
var body = email.getPlainBody().split("--")[0];
var subj = email.getSubject().split(": ");
//Logger.log(subj[1]);
//Logger.log(body);
sheet.insertRows(2
sheet.getRange("B2").setValue(subj[1]);
sheet.getRange("C2").setValue(body);
links[i].removeLabel(label).moveToTrash()
}
}A second function just adds a standard date format for each new row.
function fillDate() { var sh = SpreadsheetApp.getActiveSheet();
var range = sh.getRange("A2:B"); // assuming A is the date column var values = range.getValues();
for (var i=0; i < values.length; i++) {
if (!values[i][0] && values[i][1]) {
range.getCell(i+1, 1).setValue(Utilities.formatDate(new Date(), 'EST', 'yyyy-MM-dd'));
}
}
}That’s it! I’ve got my long list of miscellanea in one spot, easily updatable and maintainable.
Gatsby Setup
Gatsby pulls data from wherever. Usually this is just flat files, but lots of people use the Wordpress, Drupal, and Contentful plugins to pull existing data from these sites. Smart stuff, since this provides a nice easy migration path beyond exporting.
There was also a Google Sheets plugin available. So I wired that up and pointed it to my new datasource. It took a little bit of work to map the credentials and share the Google sheet correctly, but once done I had access to all the data in Gatsby.
The GraphQL to grab the data is really simple, and using GraphiQL made it a breeze to build. That said, I’m still pretty skeptical of the query syntax. I love the ideas, but the syntax and documentation so far feel pretty shitty. In particular, the nodes and edges syntax feels confusing, and I’ve run into other issues where the filtering syntax is unwieldy.
export const pageQuery = graphql` query TumbleQuery {
allGoogleSheetLinksRow { totalCount
edges {
node {
date
title
content
}
}
}
}
`The rest of the work was just presentation layer, futzing with React components to make something pretty enough for now. Which brings me to some notes on the browser environment.
First of all, React is the best I’ve found so far (I’m hoping to try Elm this year). I can think intuitively in terms of components, and markup in this context is an easy way to think about building a UI.
But I’m sick of using interpolating strings for embedding languages. CSS in JS, JS in JSX, CSS in JSX, JS in CSS.. is it a single brace, two braces, arrows, or something else? It’s just annoying.
I’ve gone back in forth between preferring everything frontend in one file and keeping HTML/CSS/JS as separate concerns. Styled Components is the best fix for this I’ve found yet. It let’s you set your styling in one place per component while also building these up in a hierarchy a la React.
Future Improvements
What I’ve got now works. It’s far from perfect, but it satisfies me as an MVP. I’ve got other little projects I want to mess with right now but when I come back to this, here’s my laundry list:
- Initial page load is slow. The Pinterest style react component doesn’t seem to do great with 1000+ divs. I’ll probably try switching to another component to build on.
- Image loading is also causing height issues, so hopefully that will be fixed with the new component too.
- Scheduling. This thing only gets updated at compile time, which means the live site only gets updated when I redeploy (on surge right now, moving to Netlify probably). I’m just building for myself, so whatever, but it would be nice to just cron the build at some point in the future so it’s up to date daily.