Devlog #003: Killed my backend, long live my backend. Adopting Ubuntu Server as my platform of choice, honing my focus.
Not knowing anything and grinding through technologies is killing my productivity. Automation First Approach.
I need to accept I’m a junior dev.
The Cloud is fucking expensive and I’m looking into ways to self-host
It’s weird how you can be so sure of yourself when you’re supporting OTHER PEOPLE’s code / workloads in an enterprise environment, but then you start building your own projects and you’re responsible for the Frontend, the Backend, the Servers that run your code, and you cloud resources. Of course this isn’t tangible to you when you start, so you decide you’re going to be Rico Studly and do everything from scratch and learn all the things. But I’m not sure that’s how it actually works in real life, espeically when you’re the only one doing everything.
So I think this blog I’m going to start talking about some of my challenges, how I can improve my focus, and what my roadmap will be fore the rest of the year. I kinda feel like a sell out or a cheater, but at the same time I cant think of a more practical way to do things.
Killing my Backend
So lets start talking about my backend.
My backend was originally built with python and Django Rest Framework. I tried to write it in a graph style where my API provided objects with edges and as the objects become more detailed there are more “edges”. For Example:
This quickly started becoming tedious. I was building a lot of models and serializers for each foreign key / edge and I started realizing there was a lot more I would need like Object storage, Signup/Signin workflows etc. So while I was initially pleased that DRF natively provides auth, and can be configured for token based authentication I made the quick and easy decision to look for a new BaaS solution so i could focus on Features/Frontend and Infrastructure work.
Supabase
Typically when I want to learn a new technology. I look for a course that’s, at most, 11 hours long and covers the topics within the domain I’m trying to build a competency in. I had heard about supabase and around the time I started with flutter and it seemed to make sense. Little did I know deploying it is wretchedly convoluted, the documentation is somehow thorough and thoroughly useless, and there is a shockingly sparse amount of tutorials worth a rip.
I gave it about a week and realized the time investment to benefit ratio just wasn’t there. Infrastructure I feel good about handling, but coding, with coding I need to realize I’m a junior dev at best and I need to focus on building mid-tier skills. I need to learn my language and framework of choice, not a package and platform for my language and framework of choice.
Pocketbase
After a couple days of youtube binging I settled on pocketbase as my BaaS. Honestly in one evening I had It deployed, signin/up workflows configured, emailing setup, and intial models configured into the database and there is a great flutter package for it already that is incredibly straightforward to use.
- I think the first two things I plan on coding now that my backend is in place are the:
- Submission of People, URLs to Archive,
- A realtime landing page with a feed of newly archived objects, a
- Social system for simple message posts so people can share their threads / trains of thoughts.
Moving away from RHEL to Ubuntu Server
Originally, I had plan to run this project on Rocky Enterprise Linux but quickly started to discover the strict configuration wasn’t serving me, and that the majority of open source solutions I wanted to use don’t run on enterprise linux. I could use something like docker, but I’m not really sure what container deployment patterns / strategies look like right now and if I would even benefit from adopting them. So for now, Servers all the way. though I do have my eyes on Fedora CoreOS.
I am starting to realize it will probably be impossible to have a homogeneous environment.
Real Quick: What’s the different between General and Enterprise Linux?
- General linux has newer kernels, more frequent updates – Considering my use case, since I’m not doing things like kernel programming, or writing platform codes but rather deploying a web app it’s actually not a problem that Ubuntu moves faster than Rocky and I’m okay with the more frequent releases and newer kernels. Probably doesn’t hurt my security posture either.
- General Linux has fewer advanced and Security Tools – Enterprise have a bunch of weird internally imposed security requirements as well as formal standards like PCI DSS, or FIPS and with more advanced hardware/appliances come more advanced features and requirements to secure them. Tha’s super overkill for me right now, Ubuntu Server will do JUST fine and it will low-key be nice that my servers are the same Distro as my Desktop.
Automation First Approach
SO. I am actually taking a page from Jeff Geerling’s book here and focusing on automating my project as I go. This has resulted in what I’m called the Contegra Ansible Distribution (CAD). The project aims to be a clonable “distribution” of playbooks that stack neatly on top of ansible-core that will:
- Deploy the components of projects (A Server that runs my Baas, a server that runs the research portal, deploy archivebox)
- Deploy Services: Directory server, Backup Server, File Server, etc. whatever I might need
- Configures Features: Resize a disk, manage users, etc.
- Policy Playbooks – that set a baseline configuation for my firewall, SSH Keys, etc.
You can probably see from the screenshot I have a couple interesting playbooks. applypolicydocument.yaml, destroypolicydocument.yaml, executepolicydocument.yaml. What I’ve done is actually come up with a system to deploy a “Policy document” to each server that’s a CSV list of “configuration policies”, defined as ansible playbooks, that should apply to the server. executepolicydocument.yaml runs on a cadence and is responsible for parsing the CSV file and executing the corresponding playbooks.
All this is deployed on a dedicated workbench PC that serves as the hub of my automations and scheduled / regular tasks are handled with Systemd timers. I hope to use this approach and these tools to improve my agility and hopefully handle more things as a solo dev.
Flutter brutality
- I need to re-process dart fundamentals.
- I need to take a Riverpod focused course.
Code is hard to talk about in detail so I’m going to try and high level this part (it’s getting late and ya boy works a fulltime):
- Layout Mastery – I had to figure out responsive layouts. My app now scales/sizes with the display it’s on as a function of the size available, and I’ve been learning how layout views with Columns, Rows, Expanded, Flexible, etc. My UI is consistent but I need to focus on containing widgets-within-widgets and implementing LayoutBuilders (IIRC)
- Dependency Hell – I ended up discovering that one of my dependencies broke my app. What’s interesting is my apps were working JUST fine in dev, but when I built releases everything was futzed. Being a systems guy I was spending my time poking around the server trying to figure out what could possibly be wrong but it ended up I should have been in the browser looking at the console and understanding constraints better.
- State Management – It’s absolutely Ironic. When I was working through my bootcamp I remeber thingking “Who in the world passes state down the app, that seems like such a bad idea” and within the first features my first app I was doing it. Shame on me maybe but, lol. As far as state managemnt I’m using a simple implementation to pass state through my app in a single scope.
- Explore the Widget Catalog – I’m going to do my best to attempt to implement different widgets each sprint cycle where organically possible. I want to master the widiget catalog and using more and different kinds of widgets will expose me to more data types and features of the language.
Improving my Focus
I think now that I’ve gone through a couple initial approaches and am getting a sense of what works I will be able to focus on feature work and moving things forward in the socializing the project / “Getting out there” side of things.
Roadmap
- Sprint 1
- [Infrastructure] Deploy Archivebox Internally
- [Middleware] Write Archiving Microservice responsible for taking web app submissions and archiving them and collating them in the product DB.
- [FrontEnd] Migrate People-submission code to my new BaaS
- [Frontend] Build Realtime feed of newly archived webpages with metadata into an archive landing page.
- Sprint 2
- [Frontend] Social Feed that displays people’s status updates. Probably limited to 750 Characters to start.
- [Backend] Implement modeling for Social feeds with likes, comments, etc.
- [RND] Start Explore Storage through Pocketbase so whitepapers, public domain books, pictures, and videos can be uploaded and made composable in supporting evidences.
- Sprint 3
- TBD
- Custom model generated from ethically sourced data to facilitate study and research.
- Tour Guide: Can I build a model that can direct people through the data the Archive has?
- Tutor: Model that has “learned” what’s in the archive and can speak conversationally about it?
Leave a Reply