When to develop slow and brittle systems
Why would you ever build a system that is slow and breaks easily? Because optimization and extensibility take time and add complexity. It is needed, but only once we are sure what we are building.
I have spent years trying to avoid this. Telling my product managers that tech debt is evil and fixing it “later” means “never”. That we need to build SOLID right from the start.
With more years behind me and with experience as a product manager, I hope my realization might help you decide when to add that extra effort or not to your projects.
Quality takes time and effort
I think we can all agree that it takes time to make good stuff. Whether it is cars, woodwork or software. It requires research, prototyping, testing, reworking and polishing to make something really good.
We also need to build for reuse and extensibility to create something that will live for long. Remove duplication and optimise. This requires time and makes the system specialized in exactly what it is meant to do.
I want to propose that we need sometimes to avoid doing this. I do not mean that we should rush or ignore code quality. But, I do want to propose that we should not always build for quality.
We must determine when we are building a prototype or a first version. When we are building on assumptions and without knowing for sure exactly what the problem we are solving (which is usually the case).
We do not know how, before we do
Nothing works on the first try
This is my mantra. It might sound negative but really, it’s to remind myself that very seldom, I get it right the first time and that is completely fine! I learn.
You never know something until you do
Equally true is that even if you read a book on a topic or visit a seminar. Until you actually solve a specific problem or apply a specific technique, you don’t actually know how to do it.
The first time we build a feature or application, we assume a lot of things and we often need to change these later.
We need to be aware of this and that we should not optimise or abstract before we know what we are building.
If instead, we build something first that works well enough to serve our purpose we can then see where we went wrong and need to improve.
As an example in an EC application, once we are able to accept orders or create items, we will see how this works in practice and which fields really need to be mandatory or validated.
The above of course would not apply when we are rebuilding some functionality, in which case we supposedly already know how it should work and are rebuilding to improve and stabilise the abstractions we have now realised exists.
Get it working, get it nice
Where it is easy to go wrong and I still do, is to forget to revisit what we build.
When we have found a data model that works, we need to make it smooth to work with, consistent, error-free and extensible.
Building on an unstable ground will eventually slow us down and create a death spiral of mounting fires to put out. Each quick fix creates more issues than they resolve.
We know this as tech debt (although I think there are other types as well, it could be in UX or process just as well as technology).
Once we’ve proved that whatever we are making works, then it is time to take what we have learned and begin abstracting, adding tests and making it SOLID before we add more functionality.
Are there other systems we can repurpose to solve some solutions? Are there parts of what we built we can replace with third party solutions we don’t have to operate or maintain? Are there changes in the core models or abstractions we have found that we need to refactor now to make things easier for the future?
Example of the wrong abstraction
Let me show some specifics. This example is taken from a design of a domain model I did in 2020 as we were rebuilding our app FACY at Styler Inc.
In the previous version of the system, item registration was difficult. Compared to traditional EC sites, in our application, a single shop brand can have multiple shops in different locations. This meant that every item, even if it was the same item, had to be registered once per shop it was sold in. The only difference was the stock information.
When the opportunity came to rebuild I wanted to change this and share items between shops. This was completely right, but I abstracted too much too early.
I sat down, drew out my boxes of all the different states and data needed and called a meeting with our support and sales staff, asking them if what I had come up with was right.
The new model looked something like in the graph below, where an organisation has different shops that can share items.
The business stakeholders shrugged and said “sure”.
This isn’t uncommon and we could have spent more time debating it but I don’t think we would have got a better result anyway. We simply couldn't know at this point. What we could know, is that this core model was unproven and might need to change.
After this, we spent additional months building on this core premise only to find about a half year after the system had been in operation that in fact, a lot of customers (organisations) were in fact owners not only to shops but multiple shops in different brands. I had completely missed one level of abstraction!
This new missing level would allow us to share the same organisation data for several brands, which is how many of our clients work in reality.
In this case, we found this issue pretty early but the mistake I did was to not begin adjusting for it straight away and now it will be difficult to introduce the change across many APIs and apps.
Starting simple then evolve
Next, let me share an example of when I instead evolved the models instead of prematurely creating them.
I have an app called JReader which provides learners of Japanese with my original and other articles combined with a dictionary lookup function.
In the first version of this app, I only relied on external articles, meaning a number of concepts I was planning weren’t needed. This is the original article model:
However, I already then had a mind to add on articles written by myself. Those would require a number of more details to allow controlling when to publish them and who could edit them. These concerns do not apply to the other articles I pull in externally though.
I could have added all of this on from the start but that would have delayed me and likely, I would have got it wrong. My new model for original articles looks like this:
You can see here how the original article model contains an author, publishing and editing dates as well as a state. All of this helps control when it is readable by users and who can edit it.
To join these two models I instead created the second original article model as a separate table and then publish these articles over to the main feed only when needed. I added only the author field to the other articles and could generate a fixed value for this based on the source.
Perhaps partially thanks to the ease of working with events and document-based data in Firebase this was a smooth ride but I also believe that relying on different models for the article feed and evolving it step by step helped make it easy to make these changes.
Plan for the assumptions
To wrap up, I think that we need to consider the context and how many assumptions we are building with. What do we know we need to abstract and what guesses are we making? What should we test first (in real life) before we solidify?
For first time work, we can follow a looping process:
- Getting it to work: proving how it runs and releasing it
- Testing it: running it in practice
- Stabilising and optimizing: going back and stabilising the structures required
I am personally very prone to jumping from feature to feature never leaving step one, only getting it to work.
The problem with this is that we end up drowning in tech and UX debt.
I want to propose instead is to open this conversation between both those that request features (stakeholders) and those that develop them to create an understanding of how we build features from a place of not knowing for sure what we want.
This is mainly a call out to product managers and development teams to be explicit about what assumptions we have or what requirements we know for sure. What the scope and expectations are on what is delivered and if we are working on assumptions, planning for the iteration on the delivery to make it truly final once the assumptions have been proven (or disproven).
Thank you for reading!
