Devlog #002: Backend Deployment Automated, Programming and Framework learning wins, Initial App Progress

Backend Deployment Fully Automated

This is Kinda what I do for a living so this wasn’t particularly difficult. But it took longer than I expected since this was my first Django deployment

How it went down…

What the Backend Currently Offers

• Figure out virtualization for an M3 MacBook Pro: I deeply regret, but don’t, buying a MacBook Pro for my sys Eng and development work. Apple is Garbage but this laptop is what a computer is supposed to be. Anyways, originally I was planning on trialing and buying Parallels, but after seeing it’s a yearly subscription I looked a little harder and learned that Broadcom made Fusion Pro 13 and Workstation Pro 17 have been made free for personal use. So I scooped that up and am pleasantly suprised by how many ARM options there are and how little they need in terms of resources to run smoothly, even with a DE on top.
• Targeting RedHat Enterprise Linux – This Will probably seem blasphemous, but yeah, I’m using RHEL to run everything from my backend, databases, and a Trunk instance of my app. I need to learn RHEL for work and I’m also starting to learn a lot of the things I’m learning Sys Admin-wise on RHEL apply to Ubuntu Server as well. In the 2.x sprint series I plan to plan (?) a migration strategy by ensuring existing playbook compatiblity with Rocky Enterprise Linux, and writing another set of playbooks for Debian-based distributions. I’m walking this doing to practice linux sysadmin and Ansible, but containerization is coming.
• Documenting the Manual Deployment – So I started off by manually building a README. that documented Installing system packages, deploying the code, configuring the WSGI server to serve my Backend, Implemented a systemd service file so It can run in the background and be managed, Then configured Nginx to proxy taht local traffic. This was my first experience with SELinux and I ended up having to set 2 differently policy settings around http location connections and local proxying to get my applicaiton to server. Once that was done I built the applicaiton 3 times by hand to verify the readme and moved on to automating the deployment with ansible.
• Ansible Playbook from the Manual Deployment – I’m pleased with this. I can take a base install of RHEL 9.x run this playbook against it and once it reboots I have a functioning instance of my backend running. HOWEVER, I Need to automate my database backups and deployments, and I had a list a couple days go of what sucks about this implementation, like lack of Basic SSL/ Cloudflare WAF Integration. So I’m probably going to go digging for that but for now I have everything I need to configure a CI/CD Pipeline to deploy the latest version of my API as I release it. This is a skill step up for me. DevOps is about to be real.

• Users and Auth
• JWTs – I’m exploring using a JWT Server as a licensing server.
• Content Tickets – To coordinate the capture and processing of audio and video content via a containerized microservice architecture where each collection is handled by an instance of a service written for the platform the media is being collected from, these are called “field agents”. Will be expanded to include documents (spreadsheets, PDFs, etc). Relies on the socials indexed in the People Following section. Currently exploring cluster computing with Libre Computer SBCs How much can I ingest for how little?
• People Following – Originally I had planned to really keep tabs on people, but I’m starting to think it makes more sense to have an Entity model that would cover People, Organizations,Brands and Businesses. But for now this feeds the Add Person form you see in the app below so as people are added to the collection, automated processing can begin.

Programming and Framework Learning Wins

• Let me start by saying I’m super not stoked with how slow this all is going. I’m trying to sit down and really engage with the materials and experiment with the things I’m learning as I’m learning them. I’m darther along than I’ve ever been but I really need to move
• So If I haven’t mentioned it already. I am using Dart/Flutter for my front-end. Im starting off by targeting web, and I will probably target desktops later. I don’t see this being a mobile app, maybe the social aspect and the explore / or notes feature but I think a research app like this just makes sense for Web and Desktop.
• I started off learning the Dart programming language using Kodeco’s Dart Apprentice book. It was a great introduction to the language and I will be making another pass at it after I complete my bootcamp. I will need to refocus on Asynchronous programming, Collections, and Advanced Classes. Collections have stupid useful helper functions in them so I’m looking forward to that structured exposure to them to enhance my programming.
• To Learn the Flutter Framework I’m using the Academind Flutter & Dart – The Complete Guide. I ended up having to completely restart as the app got updated in the middle of my first taking. I’m 60%-ish complete and Im kinda stoked at the possibilities once I’ve learned about using the camera, GPS, etc. Maybe there’s a mobile app in here somewhere afterall….
• I won’t really be sharing much outside status updates but my backend fieldagents are working, coordinated by “dispatch” and I have a modest 3 node cluster up ready to ingest data 24/7

Layouts & Responsive Design

• Learning layouts was satisfying. I ended up stopping for a couple nights to work through some youtube tutorials on layouts and then experiment with a hand full of different layout widgets that make it possible to a partition the screen and layout widgets within those partions. I’m happy to say I’ve only scratched the surface and will be going through the rest of the layout widget catalog as time allows, but for now I just need to layout a POC, I don’t intend to be come a UI/UX Expert.

App-Wide State Management

• This is probably my largest win. I’ve been handling state by passing it up or down the Widget tree with functions. no more. I’ve been learning Riverpod as part of my boot camp it is WILDLY simple to program providers on top of model data and go from there, and it’s much simplier than I thought.
• I will probably take this week to try to run through that section at least 2 more times and build a couple pointless provider apps to make sure I can do it from muscle memory.

Initial App Progress (Since Last Update)

• There’s now a consistent layout to the application across “portals” (what I call a section of the application). I’m gathering lots of bugs into the back log, for example, the thread image in it’s timeline view not covering the entire space alloted for it, it’s a bunch of little bugs that will take my skills from rudimentary to refined as I figure out the little quirks cause my app not to behave as expected.
• The Application is now responsive to adjustment without overflow rendering
• People can be submitted to the project and microservice dispatch will take care of visiting a person’s Youtube, Rumble, or PeerTube and identifying the delta between what’s posted and whats in the archive and create content tickets for what needs to be collected.
• Video field agents now spawn in response to content tickets in the queue, go out to the target site, gather the content, and process it with AI. I’m working on abstracting collections into a “universal content model” so there’s a consistent data frame for any kind of artifact in the archive with useful meta-data that will make it easier for researchers and technologists to find what they’re looking for.
• Web Build validated, MacOS and Linux Apps don’t break so much and minor fixes to accommodate for sizing differences have been implemented. Once the code is written to handle trains of thought (timelines) and different components of the thought (Articles, Books, Videos, Audio, Documents etc) I will probably spend some time living in the desktop client to smoothen it out.

Future Technical Plans

• The month of September is going to be spent finishing any Flutter education I need.
• I will move away from my own Backend and on to a BaaS to simplify development.
• I will begin re-evaluating some certifications. I think I want a couple more AWS Certs and will probably formally certify on Linux and Kubernetes.
• I’m hoping this month to be able to afford a refurbished 3090 ti with 24GB of VRAM so I can begin building AI Workflows with opensource AI Projects to replace AssemblyAI. If anyone is curious the card will pay for itself in about 600 content hours of content processed, Electricity will be the efficiency factor from there.
• By January 1st I want to be in a position to start Codecademy’s NLP Specialty.
• I will likely be returning to college to begin studying Philosophy.

Thanks for your time with this article. I’m trying to keep things in bullet points as I’ve learned technical jargon isn’t everyone’s cup of tea. I’d really love your feedback if you have any.