Tuesday, March 23, 2021

The folly of microservices

I’m sure I’m not the first to make some of these observations about the microservices fad, but here’s my perspective nonetheless.

Microservices are definitely all the rage, and proffered as an obvious solution to the dreaded monolith. But I'm not so sure it's all that obvious. As another swing at service-orientation, which is really about composability and modularity, it has the same appeal as previous iterations.

What I find disappointing about microservices is the emphasis on technologies over decomposition techniques. Its more concerned with how to run and deploy services than it is about how to identify the proper boundaries and granularity of services. It tells you how to run a service, without telling you what a service should be or do.

Even when it comes to technologies, it's not a no brainer. More services and more technologies (containers, orchestrators) often means more complexity to manage, and a system that is harder to reason about. That complexity management requires yet more technology. All fun and cool for us technology folks, but not clearly necessary.

Applying the technology solutions of microservices without the proper system decomposition will actually lead to some pretty bad outcomes (distributed monolith). There seem to be two class of problems here:

  • Taking non-modular code and distributing that over the network, is unlikely to improve anything.
  • Avoiding decomposition, by choosing extremely granular services, and distributing those over the network can be even worse.

Microservices are a technique to solve a certain type of problem.  They are not a goal in of themselves (unless you're doing resume-based development or a variant thereof). So it's important to understand what you are trying to solve with them.

My impression is that it is a solution to a problem of large teams, with well-defined responsibilities, that would benefit from the increased encapsulation, independence and deployment technologies. They do not seem like a good solution for a small team with broad responsibilities, or with poorly defined system boundaries and/or team responsibilities.

Sometimes modularity itself is enough, without all the tech, but where's the fun in that? And what's the next hiring manager going to say when I don't have years of microservice experience?

Thursday, March 18, 2021

RIF Notes Series Finale

“Things that can’t go on forever don’t” – Stein’s Law

This will prove to be the most shocking episode ever, as it will be the last of the Reading is Fundamental series.

  1. Hackers Breach Thousands of Security Cameras, Exposing Tesla, Jails, Hospitals
  2. Has the Pandemic Transformed the Office Forever? - What’s an office for? Is it a place for newbies to learn from experienced colleagues? A way for bosses to oversee shirkers? A platform for collaboration? A source of friends and social life? A respite from the family? A reason to leave the house?
  3. Gab Has Been Breached
  4. #NoAccountability
  5. Experience Reports: Before and After Shape Up
  6. Clean Coders Planning Poker by Micah Martin
  7. BeckDesignRules
  8. Verica - Security Chaos Engineering Report
  9. Why I Barely Mention Velocity Anymore
  10. America's Most Hated Office Jargon
  11. Russell L Ackoff From Mechanistic to Systemic thinking


Left over quotes

  1. “We are often more frightened than hurt; and we suffer more from imagination than from reality.” – Seneca
  2. "If it's this complicated for Netflix, who essentially do one thing, imagine how it is for a major bank". @cote
  3. “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult” -- C.A.R. Hoare
  4. “80/15/5: spend 80% of your time on guaranteed impact, 15% on big risk/big reward, and 5% on crazy stuff with no foreseeable payoff” - Kent Beck
  5. “The man who can do the average thing when all those around him are going crazy.” Napoleon’s definition of a military genius:
  6. “Software development is a game of insight, and insight comes to the prepared, rested, relaxed mind.” ― @KentBeck
  7. “Developers are drawn to complexity like moths to a flame—frequently with the same result.” -- Neal Ford.
  8. "Only a few know, how much one must know to know how little one knows." -- Werner Heisenberg
  9. “The older I get, the more I realize the biggest problem to solve in tech is to get people to stop making things harder than they have to be.” Chad Fowler
  10. “The goal of software design is to create chunks or slices that fit into a human mind. The software keeps growing but the human mind maxes out, so we have to keep chunking and slicing differently if we want to keep making changes.”- Kent Beck
  11. “Reaching for microservices is like writing convoluted code because a performance bottleneck demands it. It’s not that you WANT to devolve your code to please the performance alter, but that you accept that you must. So to do this without being forced to? Self-flagellation." to "Right. Microservices are a technical debt that you accept because the gained organizational simplicity is worth it in very large companies." –DHH
  12. "This is why microservices can make sense for large companies with many, separate, and isolated teams. And why it just about never makes any sense for small companies where teams can grasp the entire application."
  13. "If your company is producing software too slowly, hiring more "programming talent" will probably not speed you up in any significant way. Instead, remove waste from your existing system. It's way more cost effective. Adding people makes sense only if there's no waste." – Allan Holub
  14. "I was burned out - but not from overwork - from underpurpose." dhh
  15. "Simplicity--the art of maximizing the amount of work not done--is essential."

Sunday, March 14, 2021

The illusion of free will is an illusion

Back and my undergraduate days, I discovered the inescapability of mechanistic determinism. I even wrote my senior thesis in philosophy on the topic or determinism and Laplace's demon. Although determinism appears to be inarguably true, I never took it much further in explaining what the implications were for free will, morality and responsibility.

I had a notion that chaos theory might explain the gap between the truth of determinism and the limits of predictability. That lack of predictability was where free will dwelled, but it no more undermined determinism than the inability to predict the weather indicated consciousness in clouds.

Sam Harris has been putting forth a compelling account of those implications for a while, and his latest podcast is a 45 minute walk-through of his full argument. Well worth the time.

Wednesday, February 10, 2021

The Yin and Yang of the architect

The role of a software architect varies between companies as do the qualifications for the title. Having carried the software architect title for years, I've recently had cause to reflect how I've operated and what that implies about my philosophy on the role.

Much criticism is levied against ivory tower architects who produce "dreamy architectures that are detached from IT, business and budgetary realities" or the title inflation that makes architects indistinguishable from "most senior engineer".

For me, I've found balancing both to be the most effective. The architect needs to be able to develop architecture, designs and plans that are both forward thinking but also constrained by what's practical, and what's truly needed. A good way to know what's practical and required is to spend time in the trenches, likely as the most senior engineer.

Playing both roles creates an evolutionary feedback loop from design to implementation and back. This can result in discovering standards that need updating upon recognition of confusion, it could be finding a need for additional templates or libraries to smooth out the difficulty of a working particular technology, or the need for enforcement mechanisms to keep developers coloring within the lines. You feel their pain. It also establishes the credibility of the architecture, since the architect is the most senior engineer.

There are lots of things that can be discovered in the course of development, using the same tools, processes, templates, libraries as the rest of the team. But you can't spend all of your time there. At some point you need to step back out and build those standards, libraries, prototype the use of new tools, etc, i.e. be the architect.


The image that best represents my thinking is the yin and yang symbol. The architect and the "most senior engineer" are balanced forces, forever cycling with a fuzzy boundary between the roles. Additionally, there are seeds of each within the other. When acting as the architect you carry the seed of the engineer (how will this be applied, used, misunderstood), and when the engineer there is the seed of the architect (where can this be standardized, re-used, baked into a framework, replaced with something better).

Depending on your environment, and the size of your team, it may not always be possible to operate like this, but I find it a compelling model.

Sunday, February 7, 2021

Best in shows

It may have taken me a while to get on the binge watching bandwagon that everyone else seemed to start with the first of the lockdowns. I’ve watched some pretty good shows lately, which reminded me of many of the other shows I’ve really liked. 

Here’s my recommended list, in rough chronological order, of when I saw them not necessarily when they aired.

  • Warrior
  • The Boys
  • The Mandalorian
  • The Expanse
  • The Watchmen
  • Kingdom
  • Vikings
  • The Walking Dead
  • The Last Kingdom
  • Boardwalk Empire
  • Justified
  • Game of Thrones
  • The Wire
  • Entourage
  • True Detective (Season 1)
  • Marco Polo
  • Sons of Anarchy
  • Breaking Bad
  • True Blood
  • Deadwood
  • Battlestar Galactica
  • Farscape
  • Spartacus
  • Rome
  • Firefly
  • The Shield
  • The Sopranos
  • Six Feed Under
  • Babylon 5

Friday, January 29, 2021

Tuesday, January 26, 2021

RIF Notes #65

RIF Notes #64

"A management obsessed with productivity usually has little patience for the quiet time essential to profound creativity." —Gordon MacKenzie

RIF Notes #63

"My code can’t be tidier than my thinking. The purpose of my tidying is to clarify my thinking by manipulating the code. The code ends up better, but because I understand more not because I somehow forced it to be better in spite of my confusion." -Kent Beck

RIF Notes #62

“teams at Amazon are long lived service teams. Amazon does NOT do “projects”. Funding is continuous. This ensures attention to operational quality, reduces tech debt, avoids lock-in to legacy technologies” – Adrian Cockcroft

RIF Notes #60

“With the possible exception of the four #Accelerate metrics, a metrics driven approach to evaluation is rarely effective. As Deming says: It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth." - Allen Holub

RIF Notes #63

"Any code of your own that you haven't looked at for six or more months might as well have been written by someone else." -Eagleson's Law

RIF Notes #62

"My code can’t be tidier than my thinking. The purpose of my tidying is to clarify my thinking by manipulating the code. The code ends up better, but because I understand more not because I somehow forced it to be better in spite of my confusion." -Kent Beck

RIF Notes #61

“teams at Amazon are long lived service teams. Amazon does NOT do “projects”. Funding is continuous. This ensures attention to operational quality, reduces tech debt, avoids lock-in to legacy technologies” – Adrian Cockcroft