The Appeal of a Pod

Inevitably, while writing this blog, I have a coffee in front of me. Over the last few years, in the course of researching and writing (then rewriting) two books and a host of accompanying blogs, I have consumed a lot of coffee. It just so happens that this particular coffee was provided courtesy of British Airways, as I’m writing this while making good use of some travel time in Europe, but most my writing is done at home. I therefore have good reason to be thankful to Nestlé and their introduction of Nespresso capsules – either they have saved me a lot of time over the last few years, or they have saved me from a lot of instant coffee. I’ll return to Nestlé later in this post as an example of how bravery is important during testing.

First, though, I’ll point you to a 2019 paper I co-authored about controlled field testing of a new directional drilling technology and how it was deliberately split into manageable blocks. A suitable test facility was secured and booked, and testing carried out to a defined cadence. Tests were conducted approximately once per month, with one week of testing followed by an approximately three-week period in which issues seen during the tests were diagnosed, and solutions identified and implemented. This strict cadence was used to keep the project on a rapid schedule and proved very effective in terms of identifying, rapidly deploying and testing improvements. It changed the nature of the conversation with the engineering team when we returned to base. The direction was not “Lick your wounds, find out what went wrong, create solutions, and let’s repeat the test when we are ready”. It was more like “Right, we are going back in three weeks – fix as much as you can but whatever happens you must be ready on the agreed date”. This introduced more urgency, more tension, more risk – and forced prioritisation. Anything that was “perfect” according to the Voltaire quote that heads this blog was rejected, and the team focused on “good enough”. It won’t surprise you to read that not every problem was solved in each of the three-week periods, but enough improvement was made to ensure that the next test was meaningful and drew out additional problems and failure modes. This strict cadence was used to force the pace and to keep the project to a rapid schedule and proved effective in terms of identifying, rapidly deploying, and testing improvements. When combined with a willingness and a desire to make things fail during early testing, such a cadence, or drum beat, with imposed “ready” dates for testing can lead to rapid learning and therefore rapid development and early commercialisation with much lower risk.

To stay on track, and to accelerate learning as much as possible, you need to think of your testing schedule as being fixed in time. If technical progress is delayed, do not succumb to the temptation to reschedule your testing. Instead, try to figure out what you can deliver on time to your test location, or to your testers, that will still give you useful results, even if not everything is tested at the first pass. I guarantee that this will provide you with the fastest route to market.

The most important thing while testing is to reveal as many problems as possible as quickly as possible and, as perverse as it sounds, you should aim for rapid failure and not success as you plan your testing – allowing you to learn and rapidly recover. Identify somebody who is accountable for delivering as much learning as possible from a given test and imbue them with the authority to make appropriate decisions as testing progresses without having to refer to bureaucratic approval processes. During testing, make the buck stop with your test coordinator or test manager. Make it clear to them that, as described by Capodagli and Jackson (2010) “it is easier to ask for forgiveness than permission”.

It’s important, though, that any failure must be productive. Gary Pisano (2019) pointed out that successful organisations understand the difference between productive failure and incompetence. Failure that results from genuine uncertainty about how new technologies or business models will work is acceptable and should be celebrated because it teaches you the way forward, and it can do so much more quickly than had you not failed. Pisano suggested that it’s really the learning we should be celebrating, not the failure – and he makes a good point. He went on to remind us that failure resulting from poor management or poor analysis (remember the Ford Edsel?) is nothing more than incompetence and should be dealt with ruthlessly. I agree with him.

And, of course, testing is not just about proving technical capability. Returning to my coffee, in The Business Model Navigator (2014), Oliver Gassmann, Karolin Frankenberger and Michaela Csik wrote about how Nespresso spent its first few years looking like a product failure for Nestlé. Then, because of testing a new route to market, the company hit on the right business model: selling capsules via the Internet or in its own dedicated Nespresso stores and appealing to domestic consumers instead of just aiming the product at businesses. The authors used this example to illustrate their recommendation to de-emphasise traditional, formalised business models, instead taking a rapid prototyping “design-prototype-test cycle” to try new business model ideas, test them quickly, and thereby accelerate learning. You’ll see that this is very similar to the approach I’ve suggested for testing new technologies.

Once your offering becomes commercial, you need to think a little differently about ongoing testing but there are still opportunities to learn rapidly and to fine-tune and improve what you are providing. Stefan Thomke, professor at Harvard Business School, wrote in 2020 about Booking.com and its approach to testing: anybody in the company can test anything by modifying the company’s customer-facing website without asking for permission. Think about this. Can you try out a new idea on one of your customers without permission? It sounds like a crazy idea, but it can be powerful. Of course, there are some controls. In the case of Booking.com, the modified user experience is only seen by a relatively small number of randomised customers in an A/B test – a randomised controlled experiment to compare two versions of the interface. The experiment is watched carefully and is quickly withdrawn if metrics are adversely affected. What Booking.com achieves by allowing this, and a driver for its success, is a rich and diverse seam of ideas that it might not otherwise be able to mine. To be clear, it benefits from these ideas because of the trust it places in its own employees to do the right thing when they are experimenting live with real customers. Perhaps it’s easy to do this with a website, where you can quickly introduce and withdraw experiments and where you have a large pool of customers to experiment on?

In my experience, I have found that you can also do this with hardware projects, and I’ve done so myself when developing and testing several different products. I have found that if you trust people, including people outside the direct control of the engineering function, to introduce changes that could improve the performance of a product, service or business model – and you give them the authority to do so – you will be more successful than if you don’t offer that trust and if you try to maintain some form of centralised control over what gets tried out.

Trust people, and allow them to fail, and your offerings will get through testing and to market much more quickly and much more effectively.