Is 'Done' a Toxic Idea? Aka Rise of Cephalopods and Distributed Consciousness.

Done is the idea that something is finished. It has reached its end state. It has been built, it has reached its teleological end.

Isn’t it strange that something that has only just been built is done? It has gone through its life cycle? It has achieved its purpose?

Does that sound like something that has just been shipped or something that is about to be retired?

Do we really mean Done? I don’t think we do - but others take the meaning of done precisely as ‘no more changes required’, build and plan projects on the back of it, and this is toxic.

This is a problem with scope in that it primarily effects work driven by (non-exhaustive):

  • Statements of Work
  • External non-continuous funding
  • Contracting in capacity

Over the last 4-5 years I have worked primarily in public sector and with consultancies - prior to this I have spent most of 20 years in private companies. This has given me both sides of an ugly fence determined by a legal framework that seems to of been adapted from the trade or insurance of goods rather than an ongoing healthy productive relationship.

There is a fundamental difference in how the economics of these 2 groups behave - and the way we manage this could be found to be an initial cause of a sequence of decisions that produces all kinds of dysfunction.

Some of the failings I see in public sector I have also seen in product companies - and it is nearly always caused by someone with a contractual mentality attempting to define states. It is when we start to transact for deliverable things instead of customer engagement that the problems start.

This, fundamentally, is probably where my beef with how ‘delivery’ people have appeared. I think they are tremendously useful humans (doing things that my brain avoids, and I deeply appreciate that) - but often acting and directed by others in ways that fundamentally undermine everything.

The TLDR is:

we are nurturing and growing a piece of distributed consciousness

With software, we are not building a thing, we are growing a being. I have started to think of it as a cybernetic thing - however the word cybernetic has a body of science behind it that is problematic in some ways, so I do not mean that.

When I say cybernetic I literally mean in the cyberpunk meaning of people being augmented by tech that is networked together to better enable those people to achieve their goals. Collections of these people form organisations that help transact money via servicing some customer need, we call those businesses.

There are 2 ways of using technology, aka automation, we either use it to replace humans with alleged superior automation (which is fine in simple or complicated domains) or we can augment humans making them better at what they do. Both of these carry all kinds of deep and ethical implications. Eg if we go down the replace route then capitalism will eventually start to fall apart.

The ‘thing’ conception of technology heads towards Terminators whereas the ‘augmentation of beings’ conception leads you towards cyberpunk. The 2 ideas represent different ends of the spectrum of how tech and humanity interact, one it extrinsic whereas the other in intrinsic. Both are pretty gritty pictures, but it doesn’t have to be dark with a brutal metal soundtrack (Fear Factory probably). We could have unicorns with technological smiles. Essentially: are we replacing people with machines? or becoming more and more embedded with machines? What we deliver to businesses and customers is largely the latter - crappy chatbots that everyone hates are an example the former (ie replacement).

The system and networks of technology and people being fused together are constantly changing, as they could probably be regarded more as an organism with distributed consciousness. It is our job as engineers, product and delivery to carefully feed, nurture and grow this organism.

Yes, software is a ‘thing’, but that ‘thing’ has no value intrinsically. It only has value and meaning in the context of all parties, customers, external customer needs, business, business owners external needs, employees, employees external needs, the ‘builders’ and their external needs. All these parts with needs and behaviors connect together in an ever changing soup glob trying to survive as it bumps us against other globs.

network diagram of needs and players in business-customer-developer cycle

A pipe in a bathroom has no meaning without the context of the human turning taps on that are connected to a watermain that is connected to mains water managed by other humans feeding water into it that then goes into sewer pipes maintained by humans, back to farms (that then dump shit in the rivers) that eventually feed mains water, closing the loop. The pipe depends on complex human-thing coupling for it to make sense. We just generally don’t have all that in mind - unless your job is to design the system that the pipe is a part of. That is precisely our job also yet thinking of software as a deliverable thing is, all too often, simply focussing on the pipe in abstraction.

The thing is the least important part of all these relationships and organising around the thing wastes billions.

That is why ‘done’ is toxic, it takes a being and turns it into a noun (its too early in the day for existentialism but thats exactly where we are) … but lets look at how this occurs …

Product

Product Company

The contract with a product company and the people doing the work is a job description and a contract around behavior, it is with employees primarily. This is a critical difference to external parties.

People building and changing things are not doing so because of a contract telling them to. This means that what is done can always be changed.

These organisations have a cashflow that they own and then a lot of people relationships backed by tech tying it all together to provide information to decision makers and to automate parts of the process that would otherwise have to be passing pieces of paper about. The work done in these companies in theory should be relatable to cashflow or to proxies.

When we study things like lean or process engineering the assumption is that its this kind of business. Directives set from above, hopefully measures and empowered departments coming up with original strategies to get there! Yes, I joke, product companies are not some panacea, but they are compared to what im going to compare them to! (in this context)

There are a lot of very real organisational problems around enabling distributed decision making that adds up to the whole. This is where my thinking started from 20 years ago, why do we have IT departments? These almost sit outside of the value chain, it lower cohesions of departments because it is a nexus for high cross coupling making organisational evolution harder. IE having separate IT if not exceptionally careful will cause departmental change to become dramatically harder.

These companies still suffer from all the organisational design problems and delivery architecture problems from previous posts - but because they own the money they are empowered to change it.

These companies are constantly changing and tweaking everything - there is a never ending stream of work changing everything as the companies relationship with its customers (usually the other way around!) change.

  • new page layouts to test
  • new colors for conversion
  • alternative flows to test attrition rates
  • better integration of x with y, new payment providers
  • lets add a returns service
  • new warehouse suppliers with different logistic systems
  • yet another promotion system
  • having to build something right now because some idiots decided it was a good idea to leave europe
  • add cookie consents.

There is a need for change all over, constantly, but in good ones also experimentation because the future can always change.

In a product company nothing is done, engineering work gets finished and moved to done, but that is the start of that things life cycle. Observability and user research lead to changes and tweaks all towards a better place.

It fills me with all the good feelings of people being able to try stuff, see if it works and then decide later to invest in it more or try something else. This is what good feels like.

Product companies that to go wrong when they introduce a product planning layer that attempts to tie multiple value streams together in a coordinated way with long sequential plans that take the flexibility of the destination away. This is usually the result of ambition getting ahead of now.

It is confusing arriving at the destination for the being on the journey. It is a form of delaying joy and value now for something allegedly larger ie greed, in a way that reduces ability to make more better decisions as we sense our way to something. The canonical example is ‘the rewrite’ as opposed to the mutation into something better.

What is the other side of the coin? Read on intrepid adventurer - and sorry this is long!

Public Sector / Consultancy

These organisations are fundamentally different - perhaps not all public sector orgs are like this, but every single one I have engaged with is (including charities) over multiple consultancies. What I am about to describe is likely incorrect in its detail, but its the gist that matters here. Consulting into private organisations can exhibit these patterns also.

In these organisations money is awarded to them for them to spend and they then have to show value from it and argue their case to get it. Everything is business case driven because the control is essentially external, options get presented to some kind some kind of committee, some sort of award where scope gets defined. From that point on there is a procurement process where work goes out to tender in some way, people write up how they think they will deliver it (based on hardly any information), some kind of process to evaluate and score these bids by a team (who will be following all kinds of rules that they didn’t define for themselves), then a contract negotiation where all kinds of practices end up being set in stone, and then finally people can turn up and do some work.

All of this gets agreed and contractualised before the delivery team arrive proper.

This is all an effort to make it ‘fair’ by some definition of the word. The reality is that the cost of entry and the process that comes off the back of it is largely impenetrable for small organisations - and the judging criteria seem to massively favour huge organisations.

This long process results in agreement of statements of work - itemised work that has already been agreed to be done in a time period.

We end up with a list vaguely like the following (this is a slightly transplanted, but real example)

  • Month 1: Project setup, inception, CI pipelines
  • Month 2: Build a use create / Login Journey
  • Month 3: Build a shopping cart
  • Month 4: Create a product system to load product catalog
  • Month 5: Create product search system
  • Month 6: Create pages for products so they can be added to basket
  • Month 7: Integrate with payment provider
  • Month 8: Add accessability
  • Month 9: Lets add Welsh! Internationalisation.

It should be noted delivery people perhaps see nothing wrong with this list. I suspect a lot of product people would not. But for me this list is terrible. Where is the usable system that delivers value for someone?

It is not unreasonable to want a list and some kind of timeline. But this is all usually agreed at high levels before the teams that will be building it can arrive - because it is the contract of engagement, the things that will be done because things are easy to measure.

It is the worst kind of done because it has no relevance to whether anyone is getting value out of the work, nor how that work effects the clients ability to exert their agency.

Something like this is necessary - I think the point of this piece is that this is not it. Moreover when you have government bodies attempting to take teams and get them to produce this sort of approach then it begins to uncover some of the reasons why these projects are late, over budget and fail to achieve their potential.

What specifically is wrong here?

In product companies the contract is around peoples behaviour and responsibilities (doing) - but over here the contract is ultimately around deliverables (thing). You can call these things, or outcomes: but the outcome is still an end, a target rather than a set of evolving steering parameters.

Primarily if you have a start and an end defined for a piece of work then there is no budget to come back to it to change it later. This fundamentally undermines all efforts at agility, honesty and integrity. We do not know what we need to build or how users will interact with it, a system that presupposed this is fundamentally corrupt and wll produce deviance - by design.

In this world, Done means DONE. In product world ‘done’ means ‘we shipped an iteration, lets see how customers behave and decide which of our next moves to take’. But that cannot work in a world where a plan has been agreed for a rolling process that could take years that goes from one state to another (all defined way up front before any learning about what actually will happen can take place).

The month 9 example here is internationalisation, often it is observability or customer service.

Internationalisation requires a page navigation structure, some kind of user settings, it requires content that fits the layouts it requires translation for every piece of text now and in the future, it implies that new markets might be opened up, which implies marketing. How do we transition or link to pages. It represents a fundamentally different mode of operation in a market place - but it comes as a ‘fast follow’.

Observability as a deliverable implies that no work until that point needed any kind of feedback to improve it (lets be honest, nothing is in production yet anyway! - which should be unthinkable in a product org), it implies that up till this point ‘best’ was already known. It implies that observability is not a design goal, a first order requirement which if not met implies the failure of the system to meet organisation needs. No organisation wants anything where it cannot see what is happening, this coming ‘later’ is bizarre.

These systems get built and then later retrofitted logging gets bastardised into ‘observability’ rather than simply designing something from the ground up to self validate (and so potentially auto correct) its behavior - because nobody wants to need to support the thing out of hours.

Customer service coming later implies that nothing prior was built understanding how customers behave to failure - see above point

All 3 of these examples point to a fundamental misunderstanding of what is being built. The sorts of deficiencies we see - eg organisations unable to cope with customer phone calls because they were scaled in a way that didn’t design in not requiring this from the start.

Conclusion - Summary

We are not building a thing - we are building some weird technical thing that plugs people and machines into other people. It is not a static thing, its is the growing and education of a being. It is a living changing adaptive thing that never stays the same.

So my question is this:

Why are we trying to define it as a thing rather than a set of behaviors and changes to those behaviors?

It is not ‘done’; it is ‘alive’

That is how businesses manage their operations, through careful migration of people from one process onto another - not through things.

What causes this idea of done and how is it used?

The Procurement, options analysis and business plan processes are not fit for purpose

I do not have better ideas here - just a notional sense of direction. It’s easy to throw stones but this is really not my area of expertise, I am really just a pundit looking at this and seeing how system understanding problems cause billions to be wasted.

Eg I don’t really know the history that led to the current state or the problems that were solved on the way.

My - limited - experience of drawing up options is that the options need to be separate enough to be evaluated on their own. This seems reasonable - when options overlap or depend upon one another the complexity for the group of people who don’t have a clue what is going on goes up and so they cannot make a simple decision, which induces bad decisions. But this points at the design flaw of this system - the coupling across parties that lowers the cohesion of the thinking and meaning. One party knows - yet the other decides.

When we have a lack of cohesion due to coupling across these groups it is an indication of troubled design - really what these people are asking for is a simple statement so that they can apply some rules to reach a decision.

They are asking for the group that wants to do a thing to make it easy for them to agree - IE they don’t want a decision, they want to execute a set of rules to provide an audit trail of why because the next stage goes wrong and they don’t want the buck to stop at them.

My current thoughts are that this means that the people who end up making the decision should be involved in the process of generating the options - not be at arms length in some external body. That way, we can increase the cohesion and lower the coupling between the organisations by moving the parts that need to communicate closer together.

The writing up of how that decision was arrived at can come later and follow a framework - the framework should not be the decision making process.

So, until that happens, this requires options to be discreet things with discreet benefits and drawbacks. Discreet options contain a list of things and then the statement of work ends up simply being a list of those things and off we go into the blind leading the blind in a process that in software engineering we know is not going to work well. We do not live in a simple world

A plan is not a fact, a plan is a tree of possible decisions reaching towards an objective (that will move)

A plan, for me, is the same thing as reading ahead in a game. There is a process to follow to know we have exhausted possibilities. It goes something like this - this is a depth first search because I want to get to a goal fast. In a game with a timer this lets me have an answer and refine it into the best answer.

I learn’t this approach from playing go, it allows me to read 10-15 moves ahead in local situations. This was taught to me by Jiang Mingjiu 7p.

  1. We use intuition to determine a sequence of moves (lets say we go 3 deep). Hopefully it meets our needs - if it does not, nothing changes we are about to look for better moves anyway.
  2. We then take the final move in the sequence and evaluate it against all alternatives and look to refine
  3. We then systematically back out and repeat - often going to different depths depending on each sequences needs. This sometimes requires this kind of regression.
  4. We arrive back at the first move and now have the pattern of future moves mapped out - this forms a graph of possible outcomes that we have planned for and expect.
  5. If we like the outcome we often just play the move - but we can also look for alternatives, eg given the outcome here does this move still make sense globally? Did we notice something tactical that we could exploit better if something else happened first?

This is the process of coming up with a plan - it produces a graph of possible decisions and should be repeated after every move because as we go down the tree our finite brains can forget bits and go into more detail on other bits.

a tree of a plan

So, I think, an option analysis should really consist of these kinds of graphs, it isn’t a thing its a goal orientated decision tree that changes as more information arises.

You can do a,b,c which look largely independent … however we know later on these open up different options and those options also interact.

the way you take a graph of options down into one is by making a boat ton of assumptions in order to help them have a simple view. The problem is that most of those assumptions will depend on information not available to anybody yet - so making a decision about the paths now is well … brave.

But the people evaluating the options analysis Cannot have this level of detail because they simply do not (and should not) have the domain knowledge. They need things to have a start and a finish and a budget, a living being on the other hand requires carers, food and entertainment ongoing until someone puts it out of its misery.

So how exactly do we bring these 2 worlds together?

I don’t have the foggiest - today. But the answer will involve agreements and contracting not for things, but for desired behaviors and identification of the value of those potential behaviors over time vs some kind of investment rate. Beyond those numbers what do people financing really care about? Those things becomes constraints that then define a solution space instead of defining a specific solution.

Solution: Do what is right, not was is contracted

But if we do not figure out how to address this then we cannot hope to have sensible delivery processes in these kinds of engagements.

Everything else is a band aid over an ill conceived notion over what digital delivery is actually delivering - it is not the software thing, it is a cybernetic system, it is alive and it is reacting to what we are doing and changing as we are doing it.

The result is that my strategy on any engagement is very simple - and is shared by many with job roles that cannot be defined.

Essentially it can be summed up as: ‘Lets ignore the SOW for now, what is it that you really want? Because with the game of whispers we know what we have been asked to do is not what you asked for, lets work together to salvage what we can from this’

  1. build a high degree of trust by being my authentic self (which often unsettles people to start with until they understand that I really do want the same thing, i’m just not going to play a slow game, be direct with deep empathy)
  2. use this trust to get agreement to ignore the statement of work and do the right thing
  3. suffer because at some point the delivery people will come along and demand all the contractual obligations where met - despite the client being happy - because there is a legal obligation to meet the contract.

You might ask why does nobody change the contract? I have no idea, but I have never worked on a project where a delivery person who has done this.

Point 3 causes me a great deal of anxiety

It should be noted that this is a strategy I see all over, used by many people - including being a goal of departments in their engagements with external parties - because this sucks for them as much as it does for the external party.

It is evidence of what David Snowden describes as people exploiting the informal systems to work around overly (badly) constrained formal systems. This is where having a network of strong personal relationships matters. This is also why we are seeing pressure to go back into the office - our delivery and systems are broken and the only way to work around this (whilst not admitting and changing them) is to bring people together so they can break the rules and become effective.

However, any recognised notion of planning involves knowing that you want to do something, but have many options for what would follow - indeed creativity, expertise and risk management is essentially the ability to invent more on the fly

Final Conclusion

You are Not Done, you have only just started and because we are not done you are forcing us to go behind everyone’s backs to do the right thing.