Software Delivery Assessment 
by Conflux

The success of a modern software delivery effort depends on a broad range of cultural, technical and procedural factors. Efforts to identify what these factors are started soon after computers were invented. The field is probably more than half way to discovering how to do this job well, but the answer is not getting any shorter. Additional factors continue to be discovered and refined in amazing detail.  

How can a leader ensure that all the factors needed are present in an organisation? 

Techniques such as Squad Healthcheck allow a team to autonomously determine the completeness of their own learning journey, and allow for managers to offer support where teams have issues in common. But while Squad Healthcheck has proven popular and scalable, it only offers only a broad picture of your teams' situation, with limited depth.

Developed by Team Topologies and DevOps innovator Matthew Skelton for Conflux, the Software Delivery Assessment is a set of six themes and 66 success factors. The assessment follows the same operating model as Squad Healthcheck but is expanded both in breadth and considerably in depth.

Software Delivery Assessment Card deck by Matthew Skelton
Software Delivery Assessment Card deck by Matthew Skelton
Software Delivery Assessment Card deck by Matthew Skelton

Software Delivery Assessment Card deck by Matthew Skelton

When to use the technique

The technique is easy to apply and results simple to analyse so it can be run as often as needed. Many teams value quarterly sessions covering all the themes.

We believe the technique is particularly useful at times of change, where there has been a turnover of staff, redundancies, where teams have been drastically reconfigured, or in startups or new departments where teams are new.

In short: at any time you have reason to believe that the best cultural, technical and procedural factors have been eroded.

Themes and Cards 

The first theme is a slightly extended version of the Squad Healthcheck cards, taking into account the insights of Google's Project Aristotle. The other five themes cover diverse areas of engineering practices in the same or greater detail.

1. Team Health
2. Deployment
3. Flow
4. Continuous Delivery
5. Operability
6. Testing and Testability

We have taken these themes and expressed them as a card deck with each suit representing a software delivery theme. Individual success factors within each theme are distilled onto the 66 cards in the form of a goal, and a description of healthy and unhealthy practices.

Our decks also include a sets of 5 point Emoji Cards to help teams give a clear answer quickly.

There are also assessment sheets for each set of criteria available that help you to quickly set up and tear down a session - keeping the important data. We've put together our version in excel to help easy calculation of scores for individual teams and analysis of scores across multiple teams. 

The worksheet has individual tabs for recording votes (not individual points) across each of the 6 themes of Team Health, Deployment, Flow, Continuous Delivery, Operability and Testability. This worksheet can be picked up by any team and allows the facilitator to record the 

each members' points during the assessment. The average score against each factor is automatically calculated with checks in place to ensure accuracy of the data captured.  

Once you input your teams' points, the detailed and summary views in separate tabs will be auto-populated giving you average scores across each factor and multiple teams instantly. 

Summary view for a single team

How to play

Setting up

  • Find a facilitator and gather your team. 
  • Keep a stack of index cards handy 
  • Plan ahead to ease a roll out. Ensure a shadow is present to witness the facilitation and learn to facilitate sessions on their own. 
  • Print the assessment sheets for each set of criteria or use one of our excel worksheets for easy recording and calculation of scores. 
  • Obtain the details of each of the assessment criteria in the Software Delivery Assessment card deck 

Playing the cards 

  • Take the suits one at a time. 
  • Read the text on the card or encourage players to take turns to read the text.  
  • The team members rate each of the criteria by holding up one of the 5 point cards BAD (1 OR 2) / MEH (3) / GOOD (4 OR 5) based on the Tired and Inspired guidelines. 
  • Write down the team scores and notes on the assessment sheets. 
  • Take photographs of your members on video conference calls holding up their scores against the criteria 
  • Keep the team on schedule by asking for some discussions to be held outside the session. Note these down on the Index Cards.  
  • Get feedback from the team on the VALUE and EXECUTION of the engineering assessment - Emoji Cards are sufficient 

Tip for remote sessions

With teams working remotely, one of the ways to capture scores is for the team members  to hold up their scores on a video call. The facilitator can take a screenshot of the players holding up their cards and paste them into a document.  Scores can then calculated later after the session.

This approach has a significant advantage of running the sessions faster and completing in the suggested 2 hours.  Counting everyone's scores during a session can be surprisingly time consuming, but with video conference calls, taking a screenshot becomes a clear record which can be processed later calculate overall scores. 

Where it came from

The Software Delivery Assessment represents thousands of pages of knowledge from several named authors such as Henrik Kniberg, Mirco Hering, Nicole Forsgren, Jez Humble, Gene Kim, Don Reinertsen, Dave Farley, Alex Moore, Rob Thatcher, Lisa Crispin, Janet Gregory, Steve Freeman, Nat Price, Michael Feathers, Ash Winter and Rob Meaney. Matthew used his years of consulting experience and insights to identify the key elements from all this reading and distill them into a brief card deck. The results have been tested successfully at enterprises around the world by Conflux Digital.