Software Systems (3 blogmarks)

Incredible opening quote from Dijkstra:

Simplicity is a great virtue, but it requires hard work to achieve and education to appreciate. And to make matters worse, complexity sells better. --Edsger Dijkstra

It's easier to make a compelling narrative about a complexly architected, "robust" system that is super scalable. It's harder to have much to say about unrealized complexity avoided by a simpler solution.

Her work was better. But it’s invisible because of how simple she made it look. You can’t write a compelling narrative about the thing you didn’t build. Nobody gets promoted for the complexity they avoided.

Complexity is unavoidable at times. A frequent dichotomy I see is inherent versus accidental complexity. The author gets at a different distinction -- unearned complexity.

The issue isn’t complexity itself. It’s unearned complexity. There’s a difference between “we’re hitting database limits and need to shard” and “we might hit database limits in three years, so let’s shard now.”

Part of the solution here is to be careful about rewarding complexity institutionally as well as publicly and socially:

One more thing: pay attention to what you celebrate publicly. If every shout-out in your team channel is for the big, complex project, that’s what people will optimize for. Start recognizing the engineer who deleted code. The one who said “we don’t need this yet” and was right.

In Martin Fowler's words:

Conway's Law is essentially the observation that the architectures of software systems look remarkably similar to the organization of the development team that built it. It was originally described to me by saying that if a single team writes a compiler, it will be a one-pass compiler, but if the team is divided into two, then it will be a two-pass compiler. Although we usually discuss it with respect to software, the observation applies broadly to systems in general.

From Melvin Conway himself:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

I've read about Conway's Law before and I see it get brought up from time to time in online discourse. Something today made it pop into my brain and as I was thinking about it, I felt that I was looking at it with my head cocked to the side a little, just different enough that it helped me understand it a little better.

I tend to work on small, distributed software teams that work in an async fashion. That means minimal meetings, primarily high-agency independent work, clear and distinct streams of work, and everyone making their own schedule to get their work done.

I had been thinking about the kinds of things you need to have in place in your codebase and software system to make that way of working work well. A monolith is compatible with minimal, async communication because there aren't lots of distributed pieces that need coordinating. As another example, deploying things behind feature flags so that they can be released incrementally on a schedule separate from deployments also lends itself to this way of working.

The way this teams have decided to be organized and to communicate has a direct impact on how we develop the software system and what the system looks like.

any question I might care to ask (about computers) has a comprehensible answer which is accessible with determined exploration and learning.

The “with determined exploration and learning” is the important part here.

There is no magic. There is no layer beyond which we leave the realm of logic and executing instructions and encounter unknowable demons making arbitrary and capricious decisions. Most behaviors in one layer are comprehensible in terms of the concepts of the next layer, and all behaviors can be understood by digging down through enough layers.

Even recognizing and sorting out the different layers can be a great starting point. Then when you are thinking about a concept or issue, you can determine what layer is relevant to help set the context.

The trickiest bugs are often those that span multiple layers, or involve leaky abstraction boundaries between layers. These bugs are often impossible to understand at a single layer of the abstraction stack, and sometimes require the ability to view a behavior from multiple levels of abstractions at once to fully understand.

This tells us a lot about what we should strive for when trying to write clear, understandable code and whether we’ve done a good job when creating an abstraction.

Adopt a mindset of curiosity:

My advice for a practical upshot from this post would be: cultivate a deep sense of curiosity about the systems you work with. Ask questions about how they work, why they work that way, and how they were built.

… build your understanding, and your confidence that you can always understand more tomorrow.

My friend Jake Worth wrote his own version of this post — https://jakeworth.com/posts/computers-can-be-understood/

I think this is a great idea for any blogger. Take an inspiring or intriguing concept and give your own take on it.