Today’s guest is Simon Glenn-Gregg, News Engineer at The Washington Post. He joins us to talk about using Elixir to build a prototype for a platform the news house recently implemented to visualize the results of the November 2020 elections in real-time.
While the job title of ‘News Engineer’ makes it seem like Simon invents news, this is not the case. He is focused on software related to publishing at The Washington Post, and in particular, has been working on building their election visualization platform for the past two years. Before the final iteration of the platform was built, the software engineering team at The Washington Post were given a period to test different technologies as an experiment to find the best fit, and Simon decided to try his hand at Elixir and Phoenix. He talks about what led him to this choice, and his experiences building out his prototype which he demonstrated on a dataset generated by the North Carolina 3rd Congressional District house race in September. Simon tells us about how the pitch went, what the team at The Washington Post was especially impressed by, and what led to the choice to use Node in the end.
Simon talks about the culture of openness to new technologies at The Washington Post as well as some of the limitations to their adoption. We also hear about how the final version of the visualization platform held up during the elections proper, and Simon’s plans to include Elixir in future stacks due to its amazing abilities as far as concurrency and memory. Tune in today!
Key Points From This Episode:
What kinds of projects Simon works on at The Washington Post as a ‘news engineer’.
The live updating election visualizer that Simon was working on at The Washington Post.
What went into building the infrastructure for this platform and how Elixir was chosen as a candidate.
A blog post about Elixir’s memory management; abilities Elixir has regarding concurrency.
The first steps Simon took toward learning Elixir and Phoenix by building simple projects.
Simon’s process of integrating his Elixir app into the current stack and people at The Washington Post.
Testing the app on the 3rd Congressional District house race in North Carolina, and pitching to the team.
Why it is hard to implement projects in new languages at The Washington Post.
How the voting process went after the pitch and which project won.
What the audience at the pitch loved about Elixir, and what the stumbling blocks were.
The nature of working at a deadline-driven place like The Washington Post when it comes to adopting new technologies.
Sources The Washington Post used to get their election data.
What technologies and methods the team used to handle the size of data some election moments generated.
Which parts about the election visualizer that ended up being built using Node would have been easier to do in Elixir.
The other side of the coin – what made writing the app in Python and JavaScript easier.
What future projects Elixir might be better suited to at The Washington Post.
Simon’s background, education, and how he learned programming in previous jobs
Why Simon stuck programming out and decided it was the right path for him.
The need for software engineers at The Washington Post and why they are recruiting so often.
A deep dive into the tech stack at The Washington Post and how they render their pages and maps.
How Simon feels having reached the end of a successful project that millions of people engaged with.
Future projects at The Washington Post and Simon’s hopes to incorporate more Elixir.
A funny story about having to manually update the votes from rural New Hampshire into the app.
Links Mentioned in Today’s Episode:
SmartLogic — https://smartlogic.io/
Simon Glenn-Gregg on LinkedIn — https://www.linkedin.com/in/simongle/
Simon Glenn-Gregg — http://simonglenngregg.com/
The Washington Post — https://www.washingtonpost.com/
Jason Holt on Twitter — https://twitter.com/offpol
‘Elixir RAM and the Template of Doom’ — https://