Modern XP Should Not Be Focussed on Engineering

Who were the XP People

When XP came about it was from a position of highly skilled people with a wealth of experience attempting to describe a sane way of doing the whole thing that was known as engineering. This is back when everyone from delivery to production were considered engineers.

These people were generalists, they understood the customer, the business, they were entrepreneur minded people, they wrote books and created community in an era where most communication took place via a printing press and book shops. Everything was connected and deeply understood by all working together.

The communities and meetups and events we all goto and take for granted were in part started by these (and other) groups of people. They were pioneers who were pushing boundaries - and largely still are.

When you look at Kent Beck - for instance - he didn’t just write the book on TDD and then XP and stop, he has continued to push, communicate and try things. I reference him over others here because his story of meeting facebook, the challenges there and then the movement into 3x and what that means for product life cycles and WoW is fascinating stuff. Its the journey of someone who had a really good idea and has tried it in different places and come to some really interesting insights that we should all be listening to and bringing to life in our own work and universes. There are many other people I could of referenced here, a lot of them on a certain manifesto.

What are they trying to do?

Their goal is not to define and implement a set of complex best-practises, its a highly pragmatic desire to ship value to the customer as soon as humanly possible because that enables a relationship based upon trust and putting money where mouth is. It is about building trust, empathy and courage to do what is right from the outset. A conversation around a concrete thing and its next step is completely different to a theoretical debate about how to start.

These are not people defining static best-practice processes that become ensconced. They are temporary practices that only exist to serve principles and values whilst they are valuable, and so would change rapidly as the problem changes. That they stand the test of time is a testament to them being good, but also how little as an industry we have evolved.

In large teams - particularly public sector the pre-conditions are not as true. Half of the roles that were once engineering are now product, delivery, UX, UR, QA etc. The list is endless. What we are going to come back to is how this can be brought back around and be inclusive and it is simply a different perspective on what we are trying to do.

The role of defined practices here is really a kind of temporary support scaffolding to induce positive behaviors.

I don’t think that XP people care about TDD, I think they care about

1. A small piece of work that can fit on a card defined with the customer
2. Being able to identify examples for what that piece of work should do
3. Pick the set of examples that we are going to turn into production behavior based on how much we think we can do in a time period
4. Write tests to encode that desired behavior + examples because in 6 months we are going to need to change it and all documentation will be out of date, the team will be different and so we look after the code (which is the source of truth).
5. Implement the thing because we know if the other steps are in place we are not going to have any defects

On great teams this is almost a daily cycle.

The side effect of this is

• Direct customer involvement from the start
• We have story slicing of a small behavior that is valuable end to end into production
• Because the work is small we can get it into production to get real feedback on how it is used ideally in the same day
• We bring all the people together and get input from all representative user groups (because they are represented in the team - eg security, operational awareness)
• High levels of quality + no regression problems
• when we estimate 2 days, because there are no feedback loops later that estimate is realistic

But ultimately whilst most teams are discussing a sprint zero XP teams just shipped something to see how it works and get feedback. they know whilst this is possible everything else is wasteful middle management, the conversation changes from getting permission to start into ‘What is the next best small step?’.

XP today

Not XP

Most non-xp teams decided to not include observability, logging, DR etc in the original requirements, defined them as ‘fast follows’ and so now have to go back and remedy all the defective work that doesn’t meet production requirements. These things that are core to operability and defining how the system should change in the future end up being external systems with compromised data when the could of been quite simple design features.

These teams are trying, but are structured so badly by people who do not have the foggiest about the disastrous effect that their team structure, work structure and safe processes are having, that its just not possible for them. The reasons for this is a negative feedback loop caused by an inability to deliver faster than change creating the need to multiple coordination points that breaks the flow across multiple teams.

Therefor I would argue most teams are producing fraudulent reports of progress because the work they are saying is done is ‘done’ to some definition. But if the business cannot use it then its not done - more work is required to remediate the lack of doneness. It is a way of saying ‘we have spent all this money here, that is of value to you even though it delivers more value because later we will do something else that will let you get value!’. The inverse of this is, do you really want the delivery of a log format to unlock the entire value of 6 months of investment? Log formatting is so far from the intent of the product. I want to deliver some thing that relates to business objectives - a change that reduces the number of phone calls on a service desk. People will turn up for that demo!

I would rather deliver smaller complete things than lots of un-releasable value (its actually increasing liability/debt). This whole theatre / ceremony that has grown up around this increase of debt whilst trying to convince customers that spending money for nothing-yet is good is core to what is going wrong and the massive psychological difference.

XP mindset

People who talk about XP have a very different understanding of what done is because the values of communication, simplicity, feedback, courage and respect demand this.

The entire premise is that the way we work needs to accomplish these things by finishing and shipping things that deliver value now.

I regularly see work split into 20 parts because it needs to be distributed across multiple teams and none of that work delivers value in part. This causes everyone involved in this delivery to become one team and deliver in lockstep. Any failure in any team becomes a fail condition for the whole.

This is crazy from XP perspective

A XP mindset would be trying to establish how to define a piece of behavior end to end and then how to construct a group of people to deliver that as a single team so that it can be properly defined, build and delivered in totality.

It would be looking to define architecture and code to serve this structure and it would be constantly reflecting upon whether the structure we put in place previously is still valid and finding ways to keep the cost of changing this stuff constant. This is what the art of refactoring code is - if we distribute responsibilities across job roles and teams then it follows that refactoring also needs to cross these roles.

Rather than build a system and fight against it, it would seek to change the system to make the change easy (which is hard) and then make the easy change. That Kent Beck wrote that about code whilst talking about coupling and cohesion is fascinating to me because it also applies to all our delivery problems.

XP, TDD and the engineering understanding (along with practice, study, workshops and training we do under measurement) about how to model the flow of information through systems and deliver things of value quickly seems an awful lot like a highly systematic approach to understanding delivery - that I do not see delivery or product people practising.

The dreaded tech test is about ensuring engineers can actually engineer, use their tools and think in an iterative, communicated way in collaboration with others. If we don’t bring the repeatability to other places then engineering is never going to get better and will regress into ‘engineers’ being simply typists. I am not sure why the women left engineering in the first place (70’s/80’s), but I imagine it was something like deskilling and de-creativifying engineering like this.