Here is the seventh post about Dan Vacanti’s book Actionable Agile Metrics for Predictability, An Introduction. In the previous installment we got to know Dan’s thought about Flow Debt and learned the following:
- If the Average Cycle Time is not constant over time then predictability is at risk.
- Watch out for when the Approximate Average Cycle time becomes greater than the actual average Cycle Time: you are incurring in Flow Debt.
- Flow Debt is what you will incur and have to pay off later whenever you decide to expedite work.
- Flow Debt happens whenever work-items age artificially because of some interference in their natural flow through the process.
- The degree to which interferences have a negative impact on predictability depend on their frequency of occurrence and on their handling policies.
- Even well intended polices might cause Flow Debt!
- Paying off Flow Debt will also damage predictability!
- The length of a horizontal line on a Cumulative Flow Diagram is not an exact time. It is not even an exact average time. It is an APPROXIMATE average time.
Now we will find out what Dan teaches about Cycle Time Scatterplots which he describes in chapters 10 and 11.
Chapter 10 Introduction to Cycle Time Scatterplots
Dan starts by explaining what a Cycle Time Scatterplot is, and how to construct one. Dan strongly criticizes those tools that draw standard deviation lines on a Scatterplot, and then qualify them as Upper/Lower Control Limits, with the intent of using the Scatterplots as if they were Control Charts. A Scatterplot is not a Control Chart.
The biggest error such tool vendors do, beside misappropriating the term “Control Chart,” is assuming that Cycle Time data is normally distributed. Cycle Time data is not normally distributed.
In place of using lines representing standard deviations, Dan gives reasons for standard Percentile Lines to the diagram (like 50th, 85th and 95th percentile lines). Percentile Lines can be drawn independently of the data’s distribution. They are easy to calculate and they are not skewed by outliers.
Dan also introduces Cycle Time Histograms and gives advice on how to draw them. They are closely related to Cycle Time Scatterplots, since they are just another view of the same data. Scatterplots give a temporal view and can uncover trends over time.
Histograms give a compact spatial view of frequency of occurrence. Histograms give an idea about the shape of the data’s distribution.
Chapter 11 Interpreting Cycle Time Scatterplots
Dan starts this chapter with words of wisdom about the deeper meaning of Scatterplots.
The shapes of Scatterplots are really a reflection of organizational policies. Any anomaly should trigger questions about what might be causing the data to look as it does.
Analyzing the shapes and patterns that form on a Scatterplot might suggest to change policies and then validate that the change was effective.
Some of the most common shapes are: Triangle, Clusters and Gaps. Such shapes might indicate, respectively: arrivals exceeding departures, or accumulation of Flow Debt; data is skewed, maybe because of overtime; and periods when work items are not finished (which correspond to flat lines on Cumulative Flow Diagrams and have the same causes: holidays, blockers, batches).
According to Dan: You cannot identify special/common causes simply by looking at a Scatterplot. He criticizes those that suggest so. The randomness of the plots reflects the variation in the process.
What matters is understanding the causes of variation, and pursue investigating them to improve predictability.
Dan then explains how there is always variability that is inherent to the process, and that some variability is necessary to protect flow. It is really important to figure out if any variability is self-imposed rather than being out of control.
Outliers and higher percentiles dots on the Scatterplot often indicate events outside of your control, but not always. The lower percentile dots mostly indicate events under your control, but not always.
You need to investigate and think; but at least now you have a tool for understanding where to focus your attention, and learn. It pays to identify Internal and External Variability.
TameFlow and Scatterplots
In TameFlow, Scatterplots have not been used extensively, mainly because of the lackluster support by tools — not because they are not useful. On the other hand, Cycle Time Histograms have been employed successfully. Most of the information found in Scatterplots can be read off Cumulative Flow Diagrams or Cycle Time Histograms. However, there are some instances where the Scatterplot visualization of a project’s execution can be advantageous. In particular, for quickly identifying single work-items that are at risk of exceeding the expected due date.
In TameFlow Kanban, which employs Minimum Marketable Releases (MMR) with Buffer Management, any schedule risk is detected by means of buffer signals. In most cases this is sufficient. Concern is about reliable delivery of the MMR as a whole, rather than of any single work-item. However, when MMRs are made smaller and smaller, the margins also become smaller. In these instances, being able to quickly detect schedule risk for any single work-item is valuable. Even more so, in a continuous flow situation, such as TameFlow Scrum, where MMRs are forgone entirely. Given this positive trait, Scatterplots will be added to the charts recommended by the TameFlow approach.
While Dan highlights that you cannot reliably identify special and common causes simply by looking at a Scatterplot, and that it is more beneficial to try classifying variation as internal or external, the TameFlow approach does both.
In fact, TameFlow goes to great lengths to distinguish special and common causes, mainly through Frequency Analysis on Reason Logs. A Reason Log keeps track of the reasons why there were negative buffer signals. Identifying common reasons is a way to recognize common causes.
TameFlow is more refined when it comes to categorizing internal vs external variability. In TameFlow there is a lot of attention given to what falls inside your Span of Control, and this corresponds to Dan’s Internal Variability. However, regarding the External Variability, in TameFlow there is the further distinction of what might fall within your Sphere of Influence and what is truly out of control. The Sphere of Influence is where you have no direct control over possible causes of variability, yet you might influence people who have. This is where the tools of Psychological Flow come in handy, and can support improving Operational Flow.
Dan’s teaching about how to use Scatterplots with Standard Percentile Lines is the most valuable lesson in this chapter. Like his treatment of Cumulative Flow Diagrams, these concepts should be considered as fundamental for any TameFlow practitioner.
Links:
- Dan Vacanti’s book is Actionable Agile Metrics for Predictability, An Introduction.
- For more information about TameFlow, read the book Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban; or
- Check the site https://tameflow.com.