Software is Made of Fickle Loops

You can use the lens of iterative loops to look at software development today. We continuously run tests in CI, we regularly update dependencies, our web apps are loops turning HTTP requests into responses, and user input into UI updates and database writes.

What makes software development hard is that these loops are fickle: they are prone to failure, delays, and uncertainty.

Let’s go through them:

Code

Loop Steps Fickleness
Code compilation is a fickle loop of source code into executable instructions edit → compile → link → run
  • syntax errors
  • type errors
  • missing dependencies
  • configuration mistakes
  • language breaking changes
CI is a fickle loop of flaky tests on a remote machine install → build → test → fail… → fix… → pass
  • flaky tests
  • slow feedback
  • dependency drift
Dependencies are a fickle loop of independent code changes new release → upgrade → fix breakages → git commit
  • transitive dependencies
  • ecosystem churn
Code review is a fickle loop of human opinions pull request → review… → revise… → merge
  • human delays
  • inconsistent standards
  • unclear feedback
Tech debt is a fickle loop of labor applied to a running software system add features → fix bugs → meet goals → hire staff
  • changing requirements
  • staff turnover
  • lack of system understanding

Networks & storage

Loop Steps Fickleness
Databases are a loop of writes with durability guarantees and loops of consistent reads connect → open → write → commit → replicate
  • network partitions
  • crashes
  • conflicts
Web servers are a fickle loop of HTTP requests request → handle → cache → respond
  • timeouts
  • partial failure
  • server load
  • poor Wi-Fi or cellular connections
RPC is a fickle loop of requests over unreliable networks request → timeout → retry/backoff → acknowledged success/fail
  • timeouts
  • network partitions
  • server overload
TCP is a fickle loop of packets over unreliable networks send → ack/loss → retransmit/adjust
  • packet loss
  • network congestion
  • variable latency

Product & users

Loop Steps Fickleness
Lean startup is a fickle loop of validating product-market fit build → measure → learn
  • misleading or lagging proxy metrics
  • shifting market conditions
  • changing user behavior
  • conflicting management incentives
Double-diamond is a fickle loop of exploring & narrowing problem/solution space discover → define → develop → deliver
  • noisy signals from research
  • feature creep
  • conflicting management incentives
  • shifting user behavior
User sessions are a fickle loop of authentication and engagement login → interact → timeout/logout
  • forgotten passwords
  • session expirations
  • engagement drop-offs
Subscriptions are a fickle loop of billing and user retention bill → renew/cancel → plan changes
  • payment failures
  • churn
  • pricing and plan changes
Enterprise contracts are a fickle loop of negotiations and renewals negotiate → sign → deliver → renew
  • changing requirements
  • legal reviews
  • budget cycles
  • feature overload
Accessibility is a fickle loop of design constraints, legal requirements, and user needs design → implement → test with users → verify compliance → revise
  • diverse user needs
  • evolving standards
  • limited QA resources
  • conflicting best practices

Trust

Loop Steps Fickleness
Security is a fickle loop of attack and defense zero-day/attack → detect → mitigate → patch → monitor
  • adversarial inputs
  • supply chain risks
  • human error
  • complex systems
Compliance is a fickle loop of interpreting laws and auditing regulation changes → implement → audit → report
  • evolving regulations
  • ambiguous laws
  • regional differences
  • more reporting requirements
Privacy is a fickle loop of data collection and user consent collect data → anonymize parts → store/analyze → comply with requests & laws → delete data
  • changing regulations
  • data breaches
  • law enforcement requests
  • encryption

Note how these loops are interlinked: one loop can cascade its problems into another. Faster CI loops often improve code review loops. Web server loops rely on TCP loops. Congestion over the network can cause TCP to be austere compounding into a sluggish UX. An onerous login flow can lead to less product usage and even cancellation. Tech debt slows almost all other loops.


Our loops are becoming yet more fickle

Due to more complicated management of software projects these loops are becoming more encumbered with fickleness. Here are some examples:

Lubricants for fickle loops

The industry has tried various ways to fix or streamline these problems.

Making successful software is to triumph over these fickle loops and create value in the daily loops of users.

Agentic coding: lubricant or abrasive?

Where does AI coding fit in? Does it accelerate existing loops or is it a new loop in itself? How can it help and where could it go wrong?

Loop Steps Fickleness
Agentic coding is a fickle loop of using LLMs to write code of all sorts prompt → generate → review → deploy
  • syntax errors
  • remote code execution
  • data breaches
  • supply chain risks
  • incorrect implementation of regulations
  • data loss
  • flaky tests
  • feature overload
  • invisible partial failure

Just like tech debt, AI can affect all other loops. There’s the potential for faster iteration inside existing loops. Loops like CI, dependencies, review, security, testing, design could all increase in efficiency.

But there are also massive risks that fickleness could beamplified through over-reliance on generated code and loss of human oversight and understanding.

I’d argue that understanding all the loops present and sources of fickleness and their failure modes is as important as ever. That’s what senior engineers are on the hook for. This need hasn’t gone away.

We can improve loops by understanding how they work and what they connect to in the real world, making them faster by removing friction, and make failure loud and unambiguous. Ideally we remove unnecessary loops instead of adding more automated loops.

I suggest focusing on loops where humans are involved with agency, such as user experience, security attackers, legal compliance.

The loops haven’t disappeared, they are becoming more abstracted. It’s our duty to see them, understand them, and be accountable no matter how much AI we adopt.