Introduction
When Revue, Twitter’s cherished newsletter service, announced its shutdown ↗️, I stood at a digital crossroads.
It was a moment ripe with possibilities — migrate to another platform or embark on a journey in building my in-house newsletter system. I chose the path less traveled.
The decision wasn’t just about finding a new home for my newsletters, it was an opportunity to dive headfirst into a side project that had been in back of my mind from a long time ↗️.
Little did I know that this seemingly whimsical idea would evolve into an exhilarating, knowledge-packed journey.
What I wanted to build?
My vision for this newsletter system was clear: I wanted a solution that seamlessly integrated into my existing website workflows.
This meant writing content in Markdown or MDX, the same format I used for my website, and then effortlessly sending it out to subscribers as newsletters. No more copy-pasting content into email HTML formats; the process had to be automated.
The transition from the idea’s simplicity to its execution, however, proved to be quite the journey.
Energy can neither be created nor destroyed
— Albert Einstein
In this case, the energy (and the headaches along with it) spent on manual copy-pasting was redirected towards automating the entire flow.
Why Re-Invent the Wheel?
Before we dive into my journey of building an in-house newsletter system, you might be wondering, “Why re-invent the wheel?”
Indeed, it’s a valid question. There are countless battle-tested services out there, trusted by professionals and businesses alike. So, why take the road less traveled?
Well, this project was more about learning and adventure than about solving a pressing need. If I were on a mission-critical task or working in a professional capacity, I’d probably choose an established solution. The goal here was to explore, experiment, and expand my horizons.
So, as we go into the tale of my in-house newsletter system, keep in mind that sometimes, it’s the journey, not just the destination, that matters.
With that out of the box, let’s start now!
I started by outlining a high-level product spec — a roadmap, if you will. This spec served as my guide throughout the project. Breaking it down into smaller, manageable tasks made the process less daunting.
Here’s an overview of what I aimed to achieve:
-
Writing Content: My content creation process needed to remain unchanged.
I’d continue writing in Markdown/MDX, following the Astro content collections ↗️ format, which ensured that my content was web-ready.
-
Processing Markdown: I wanted to automate this process by writing scripts to take these Markdown files and process them further.
-
Email HTML: Converting HTML to email-compatible HTML posed a challenge, as email clients are notorious for rendering emails differently.
Finding a solution to this issue became a crucial subtask and also one of the thing which was difficult to automate.
-
Newsletter Distribution: The core purpose of this project was to send newsletters. Therefore, I had to create a mechanism to send them efficiently.
-
Unsubscribe Feature: Adding an unsubscribe feature was a must. While subscribers could sign up, I needed to provide a way for them to opt out of the newsletter if they wished.
With a structured plan in place, I began tackling each of these components, even though my full-time job meant I could only work on this project during weekends and late evenings. Breaking the problem into smaller parts helped me keep a clear focus and steadily make progress.
There were definitely times when I felt overwhelmed, given the sheer number of moving parts. But I kept at it, and eventually, the pieces started falling into place.
The most difficult part was the on and off nature of the project. I would work on it for a few days (or nights), then leave it for a few weeks, and then come back to it again. This made it difficult to keep track of the progress and the next steps.
Fast forward to around 2 months later, I had successfully completed all the necessary flows, merged the pull requests, and sent out my first newsletter using the system I had built. It was a moment of immense satisfaction. 🥳
Seeing those “Email sent to user” messages in the console was incredibly rewarding.
As with any project, continuous improvement followed the initial release. Here are some enhancements made since then:
-
Improved Code Structure: The codebase is now much better organized and maintainable than in the initial iterations.
This helps me optimising some flows even better now and gives overall visibility of all the moving parts.
-
Resolving Edge Cases: Of course the system was not bullet proof and I encountered many edge cases over the time, which are now resolved.
This is an on-going journey and I can only hope it reduces over time
-
[PREVIEW] Newsletters: Implemented a cleaner approach to sending [PREVIEW] newsletters to myself.
This was necessary since testing out the whole flow was very difficult in the start, now its much easier to test and iterate.
-
Enhanced Robustness: The system now keeps track of sent newsletters, retries for failed deliveries, and ensures that users don’t receive duplicate newsletters.
-
Automated Scheduled Emails: I’ve also automated the process of sending out newsletters on a schedule.
This means that I can write a newsletter, schedule it for a future date, and the system will automatically send it out on that date.
All of this is powered by a small Raspberry Pi in my homelab which runs periodically everyday and send’s out the newsletters.
What’s Next?
As I stand here with my in-house newsletter system up and running, I can’t help but wonder what lies ahead. The journey of building this system has been an incredible learning experience, and I know there’s still lots of room for improvement and growth.
One thought that’s crossed my mind is packaging this system into an npm module. It might sound enticing, but it comes with its own set of challenges. Since this system isn’t a one-size-fits-all solution and requires some tweaking to handle various use cases, maintaining a package might be quite demanding. I’d need to ensure it’s flexible enough to accommodate different workflows and edge cases.
Another idea that’s been brewing is transforming this system into a simple, user-friendly service. Imagine a platform where you provide your markdown files (even via APIs), select your audience, and hit send — effortless and efficient.
But whether this becomes a reality largely depends on the interest and demand from fellow content creators and businesses.
If you’re someone who finds the idea of such a service appealing, a tool that could simplify your content distribution process, I’d love to hear from you. Your input could shape the future of this project. Feel free to shoot me a DM or any of my social channels.
As for me, I’ll keep tinkering, learning, and enhancing this system with each iteration. After all, the journey doesn’t end here; it’s an ongoing adventure of discovery and innovation.
Thank you for joining me on this fascinating journey into the world of in-house newsletter systems. Until next time! 🚀