7 years of development: discipline in software engineering
June 7th 2025 marks 7 long years of development on Sandpolis, an attempt to build the ultimate remote administration/management tool for sysadmins like you and me.
For 7 years I thought about and/or worked on this project almost daily, and yet it's still nowhere near being finished, not even MVP-level. Am I the worst software developer ever to touch a keyboard?
Unlikely :)
Quite a few things happened in the last 7 years: college, a job, a wife, a house, a child, a full rewrite from Java to Rust...
... is what I would have said, but I now realize those are shallow excuses.
What's slowing down this project is a critical quality that's becoming as scarce as good taste in software engineering: discipline.
Discipline in the software engineering discipline
If your project is fueled by necessity, curiosity, or excitement alone, it's unlikely to reach 100%.
That's primarily what motivated these last 7 years of development on Sandpolis. I'd have a satisfying streak of progress in some interesting area until I got stuck. And since solving hard problems is hard, I'd jump to somewhere else and repeat.
Unlike other crafts, in software engineering, it really is possible to build the roof before you finish building the walls. And, as if this was Minecraft, it's also possible to build that roof so it doesn't even eventually line up with the walls.
Nevertheless, at one point in 2019, I had an application that technically worked. The server side was entirely Java (~50K lines) with a cute little iOS frontend app in Swift (~10K lines). Let's admire it for a second:
Back then, I thought Java was quite alright. Modern features made the language decently comfortable. I don't remember exactly what prompted it, but at some point, I decided to rewrite everything in Rust - probably a case of cosmic rays in my brain, much like in my code.
So, while my curiosity took me down an exciting new path with the most hyped language of the decade, I no longer had a working application (a grievous mistake). Rewrites are extremely costly and rarely the right answer. By that point, I was experienced enough to know better, but I let the decision make itself instead of taking a strategic path.
Since this was just a side project, I usually worked on the fun stuff over the hard stuff. After a while, what you're left with is a project full of holes. Parts of it are certainly nicely done, but if it's not possible to unite those pieces into a working whole, you can't get off the launchpad.
Discipline is what unites a project into a working whole. It's what allows you to solve the hard problems. It's what keeps you on a path going forward when temptations arise.
Surely there are some people out there with a side project that they enjoy coding just for the sake of coding, but I always aspired to make my programs actually do something useful. Maybe even something that no one else has ever done before.
AI development tools
Now that AI development tools are here to stay, discipline in software engineering is more important than ever. It's just a classic problem reframed:
- Why remember directions when the GPS is always right?
- Why learn long division when everyone has a calculator within arms reach?
- Why practice handwriting when we type almost everything?
The general reason in all of those cases, and likewise in software engineering, is doing those things leads to understanding (sometimes of things you wouldn't expect). Which is something that AIs don't, and maybe can't, have.
Discipline is what causes you to do the things that lead to long-term understanding, even when a shortcut is right in front of you.
I'm not suggesting you uninstall copilot and stop prompting ChatGPT for code, but I am saying it's extremely easy to mistakenly take it too far.
How to practice discipline when coding
In general, do the highest priority thing until it's 100% done, even when it's not the most enjoyable part.
Don't let problems on the critical path leave your L1 cache
For example, the data model in Sandpolis is critical for everything else to work. When I got stuck on this hard problem early on, I diverted my attention to other aspects that weren't as important like how user passwords are handled to avoid hash shucking attacks.
Sometimes when I'm stuck on a problem, I can work on something else in the meantime and a solution to my original problem suddenly presents itself, as if my unconscious brain was thinking about it the whole time. Once a problem leaves your L1 cache because you haven't thought about it in a while, you're no longer making progress on it. In fact, it's the opposite of progress because you slowly start to lose context which will take effort to regain.
That's why it's best to mostly stay on the critical path than to hop around. It's OK to do some side quests here and there, but don't let them subvert the main questline.
Minimize the length of time the software is broken
To make real improvements in software, you usually have to break things first. The longer it takes to get your application compiling or running again, the more costly the improvement becomes. Once a change starts taking weeks, the full context becomes hard to keep in your L1 cache and the probability of refuctoring increases.
I'm a repeat offender when it comes to this. I've made many fundamental improvements very low in the stack that affects basically everything, but I've failed to propagate that change completely for a long time, usually due to some difficult edge case that I didn't think about initially.
As a corollary, don't get tempted to rewrite so easily. Rewriting is the ultimate form of breaking your software. My favorite discussion on why rewriting is bad is Spolsky's Things You Should Never Do. Part I.
Pay now or pay later
Usually, both in software and in life, it's better to pay now rather than pay later. It's going to cost much more time and energy to change a function prototype after it has hundreds of call sites than when it has none or just a few. In other words, do it right or do it again.
It takes experience to predict what's going to be worth paying your time into in the future and what will be a waste when it becomes quickly irrelevant. Of course, it's still important to have a healthy amount of YAGNI (you ain't gonna need it) to stay working on the things that actually matter.
As an aside, you can combine "pay later" with the witty "later is never" quip to derive: "pay never", which sounds like a sweet deal until you realize that it turns your software into concrete over time - impossible to change without a jackhammer.
Conclusion
Although I could still fail, I now recognize there's nothing else to blame other than myself. The only thing worse than failing is failing and blaming something else for it.