Thursday, December 15, 2022

We want easy

Recently, I decided to resurrect and upgrade my 10 year old EasyCache nuget package, although it ended up not being that easy. Codeplex has since shutdown, and Nuget no longer allows sign-in via a username/password account, nor can you claim access to the old accounts via any other means.

Nevertheless, I created a new account, and dredged up the source code to publish a new version. I also published a couple other utilities that I thought might be handy.


Wednesday, April 20, 2022

The Agile hypothesis

At recent software architecture training, there was a lot of negative discussion of Agile, which got me thinking about the what the strongest defense of Agile might look like. And by Agile I think more of the practices of Extreme Programming than rituals of Scrum.

IF you accept a few premises, then I think Agile appears reasonable. Those premises look something like this:

  1. Engineering requires a rigid and highly specified design/design process.
  2. Producing a highly specified design is expensive (time, money, effort, training), but necessary to avoid the even more expensive prospect of wasted building materials.
  3. Building materials (code) are virtually free and infinite in software.
  4. Software is inherently more complex than other engineering disciplines, or at least too new for a body of sufficiently reliable engineering practices to have evolved.
  5. There may yet be something unique about software's lack of physical constraints that defy or modify typical engineering practices.
  6. Demand for software, and the pace of change are too fast to wait for the body of knowledge to develop, or for engineering experts to be trained, if it's even possible.
  7. Software requirements are more volatile than other fields because the of the pace of innovation and the lack of physical constraints.
  8. Demand for the software engineers already far exceeds the supply.
  1. Therefore, since software (being new/unique/complex/volatile) is more expensive to specify but the materials are less expensive, the economics are flipped. For software, we can lower the expertise bar, increasing our engineering capacity, and still deliver systems, by churning the building materials with incremental experiments, and switch directions with less sunk cost, in order to arrive at a good enough systems.

Said another way, if you're fumbling in a dark room where the furniture is moving around, the most effective way to get from point a to point b is probably slow incremental experiments. A kind of software Darwinism, iterating through many quick generations of software versions in order to evolve into a good enough design.

That doesn't discount that likelihood that many of the rituals and ceremonies of practices like scrum, don't in fact ad much value or enable agility.

Some of these premises are likely flawed, but it seems like #3 and #7, #8 are the most prominent.

3) Maybe code is free, but labor isn't. Nor are the downstream costs of excessive resource consumption of poorly written systems. It's at least (Material + Labor) which probably evens the score.

8) Demand for engineers is self-reinforcing. More poorly written code released into the world increases demand for engineers.

On the other hand, within #3 and #5 there is a kernel of mystery there. The lack of physical constraints does seem to change the calculation, and Agile does seem to be working. There is an awful lot of software in the world encroaching on every aspect of our lives, and while often buggy and imperfect, we much prefer it to not having it.

Some of this might be explained by a lack of competition. Rather than a binary choice between success and failure, companies operate on a continuum of effectiveness. As long as everyone is building mediocre software behind schedule, you don't fail by similarly creating mediocre software behind schedule. Further, the impacts of poor design are not often felt immediately, but impact the longevity of a company. Maybe decades later, which could be after a company has already failed for other reasons.. Again from a Darwinian perspective, other techniques haven't demonstrated a competitive advantage over (decently run) Agile yet.

The other part of the answer is that the need for engineering and architecture is rare.

The vast majority of all software engineering work is a feature grind, not architecture or re-architecture. Incrementally modifying the existing software within the context of whatever architecture you're stuck with. XP may not be a bad match for the construction phase, if you're incrementally delivering building blocks.

That's also true in a house metaphor. You build a house once, maybe once a decade you need an architect again for an addition, but otherwise it’s a combination of professionals and unprofessional (homeowners) incrementally performing construction on the house.

The worst forms of Agile, emphasizing rituals, just pulling tickets with no particular plan or design, maximizing rework, aren't helpful. But perhaps the disciplined variety with a good enough plan, and incremental delivery work in software where they wouldn't necessarily in other fields.

Tuesday, March 23, 2021

The folly of microservices

I’m sure I’m not the first to make some of these observations about the microservices fad, but here’s my perspective nonetheless.

Microservices are definitely all the rage, and proffered as an obvious solution to the dreaded monolith. But I'm not so sure it's all that obvious. As another swing at service-orientation, which is really about composability and modularity, it has the same appeal as previous iterations.

What I find disappointing about microservices is the emphasis on technologies over decomposition techniques. Its more concerned with how to run and deploy services than it is about how to identify the proper boundaries and granularity of services. It tells you how to run a service, without telling you what a service should be or do.

Even when it comes to technologies, it's not a no brainer. More services and more technologies (containers, orchestrators) often means more complexity to manage, and a system that is harder to reason about. That complexity management requires yet more technology. All fun and cool for us technology folks, but not clearly necessary.

Applying the technology solutions of microservices without the proper system decomposition will actually lead to some pretty bad outcomes (distributed monolith). There seem to be two class of problems here:

  • Taking non-modular code and distributing that over the network, is unlikely to improve anything.
  • Avoiding decomposition, by choosing extremely granular services, and distributing those over the network can be even worse.

Microservices are a technique to solve a certain type of problem.  They are not a goal in of themselves (unless you're doing resume-based development or a variant thereof). So it's important to understand what you are trying to solve with them.

My impression is that it is a solution to a problem of large teams, with well-defined responsibilities, that would benefit from the increased encapsulation, independence and deployment technologies. They do not seem like a good solution for a small team with broad responsibilities, or with poorly defined system boundaries and/or team responsibilities.

Sometimes modularity itself is enough, without all the tech, but where's the fun in that? And what's the next hiring manager going to say when I don't have years of microservice experience?

Thursday, March 18, 2021

RIF Notes Series Finale

“Things that can’t go on forever don’t” – Stein’s Law

This will prove to be the most shocking episode ever, as it will be the last of the Reading is Fundamental series.

  1. Hackers Breach Thousands of Security Cameras, Exposing Tesla, Jails, Hospitals
  2. Has the Pandemic Transformed the Office Forever? - What’s an office for? Is it a place for newbies to learn from experienced colleagues? A way for bosses to oversee shirkers? A platform for collaboration? A source of friends and social life? A respite from the family? A reason to leave the house?
  3. Gab Has Been Breached
  4. #NoAccountability
  5. Experience Reports: Before and After Shape Up
  6. Clean Coders Planning Poker by Micah Martin
  7. BeckDesignRules
  8. Verica - Security Chaos Engineering Report
  9. Why I Barely Mention Velocity Anymore
  10. America's Most Hated Office Jargon
  11. Russell L Ackoff From Mechanistic to Systemic thinking


Left over quotes

  1. “We are often more frightened than hurt; and we suffer more from imagination than from reality.” – Seneca
  2. "If it's this complicated for Netflix, who essentially do one thing, imagine how it is for a major bank". @cote
  3. “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult” -- C.A.R. Hoare
  4. “80/15/5: spend 80% of your time on guaranteed impact, 15% on big risk/big reward, and 5% on crazy stuff with no foreseeable payoff” - Kent Beck
  5. “The man who can do the average thing when all those around him are going crazy.” Napoleon’s definition of a military genius:
  6. “Software development is a game of insight, and insight comes to the prepared, rested, relaxed mind.” ― @KentBeck
  7. “Developers are drawn to complexity like moths to a flame—frequently with the same result.” -- Neal Ford.
  8. "Only a few know, how much one must know to know how little one knows." -- Werner Heisenberg
  9. “The older I get, the more I realize the biggest problem to solve in tech is to get people to stop making things harder than they have to be.” Chad Fowler
  10. “The goal of software design is to create chunks or slices that fit into a human mind. The software keeps growing but the human mind maxes out, so we have to keep chunking and slicing differently if we want to keep making changes.”- Kent Beck
  11. “Reaching for microservices is like writing convoluted code because a performance bottleneck demands it. It’s not that you WANT to devolve your code to please the performance alter, but that you accept that you must. So to do this without being forced to? Self-flagellation." to "Right. Microservices are a technical debt that you accept because the gained organizational simplicity is worth it in very large companies." –DHH
  12. "This is why microservices can make sense for large companies with many, separate, and isolated teams. And why it just about never makes any sense for small companies where teams can grasp the entire application."
  13. "If your company is producing software too slowly, hiring more "programming talent" will probably not speed you up in any significant way. Instead, remove waste from your existing system. It's way more cost effective. Adding people makes sense only if there's no waste." – Allan Holub
  14. "I was burned out - but not from overwork - from underpurpose." dhh
  15. "Simplicity--the art of maximizing the amount of work not done--is essential."

Sunday, March 14, 2021

The illusion of free will is an illusion

Back and my undergraduate days, I discovered the inescapability of mechanistic determinism. I even wrote my senior thesis in philosophy on the topic or determinism and Laplace's demon. Although determinism appears to be inarguably true, I never took it much further in explaining what the implications were for free will, morality and responsibility.

I had a notion that chaos theory might explain the gap between the truth of determinism and the limits of predictability. That lack of predictability was where free will dwelled, but it no more undermined determinism than the inability to predict the weather indicated consciousness in clouds.

Sam Harris has been putting forth a compelling account of those implications for a while, and his latest podcast is a 45 minute walk-through of his full argument. Well worth the time.