| Pages: 1 2 |
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
![[*]](./images/grass/default_icon.gif) posted on 05 Sep 06 at 04:08 AM |
|
|
Measuring Agility
This is probably a topic for the forum here.
There is often a dispute about whether something is agile or not, whether someone is being agile or not, etc. So, how can we measure this
property?
I think there are so many dimensions to agility that no single measure will work. I envisage a set, probably open-ended, of 'Touchstones' for
agility.
A touchstone would ideally have the following properties:
1. It gives a definitive, objective answer for one specific measure.
2. If what is measured is agile, the touchstone would normally give a positive answer.
3. In attempting to meet the touchstone criterion, what is being measured will normally become more agile.
A potential Touchstone of Agility would be "Deliver value to the business at minimum every X weeks" where X could be 1, 2, 4 or 8 and the Touchstone
would give a larger positive value for the smaller number of weeks.
My question:
Should we start assembling and agreeing on a set of such Touchstones?
Paul Oldfield
Capgemini, Telford UK
|
|
|
Rachel
Member
Posts: 15
Registered: 05 Sep 06
Location: Rugby UK
Member Is Offline
Mood: calm
|
| posted on 05 Sep 06 at 05:32 AM |
|
|
I am not sure that agility is a property that can be measured. Assuming that by "whether something is agile or not" you are interested in gauging
whether team activity in developing software is agile or not. I would first look at whether what the team does is a reasonable match with principles
of Agile Manifetso.
I would also take into consideration the project context. I think it would be difficult to come up with a set of Touchstones that would be relevant
for all types of project. I would like to hear more examples of Touchstones and meanwhile will try and think of some myself.
|
|
|
adrianh
Member
Posts: 4
Registered: 05 Sep 06
Location: UK
Member Is Offline
|
| posted on 05 Sep 06 at 06:13 AM |
|
|
My definition of agile is more like a journey from where somebody is now.
We look at the current situation, look at the change we're thinking of making, and ask whether the change would make us more or less agile (according
to the values/principles of the agile manifesto - to pick one list.)
Saying something is/isn't agile seems to me to be an unhelpful way of approaching change.
|
|
|
jasonyip
Member
Posts: 4
Registered: 05 Sep 06
Location: Sydney, NSW, Australia
Member Is Offline
|
| posted on 05 Sep 06 at 07:07 AM |
|
|
The Lean manufacturing community has something called the Shingo prize for the "best" Lean companies.
From what I understand, one of the controversies is that Shingo prize winners are not actually doing that well financially, some going bankrupt.
Measuring the Agility of someone reminds me of this. Worrying about whether something is Agile or not or whether someone is Agile or not is precisely
the wrong behavior. I'd want to address that instead.
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 05 Sep 06 at 07:12 AM |
|
|
"Saying something is/isn't agile seems to me to be an unhelpful way of approaching change."
Perhaps there are contexts where being able to say something is/isn't agile would be useful. Maybe not. I am thinking of cases where advice is
offered, but the recipient isn't sure how good the advice is, or how good a grasp of 'agile' the advisor has.
If we had a checklist, then people could work at being able to check the list. That in itself isn't very good, but if in the process of putting
themselves in a position where they can check an item off the list they learn more about and become more agile, then this in itself might be a good
thing.
I do have several reservations:
- I can't yet see how to have measurable Touchstones without measuring practices, and I don't believe in any universal good practice.
- I don't think there's any yes/no answer to "Is it agile"... in some cases there's a definite "No" but usually it's a matter of degree.
- As you say, I'm not entirely sure it's useful to measure how agile something is, but I do know it's a recurring question so if anyone's going to
provide a way to answer I'd rather it was a way devised by people who are agile... though I'd settle for a definitive statement from the Alliance
that it cannot or should not be done.
Last time I went along this route I concluded it was more useful, and more tractable a problem, to try and assess how appropriate a particular
solution was in a given context.
Paul Oldfield
Capgemini, Telford UK
|
|
|
johnbr
Member
Posts: 6
Registered: 05 Sep 06
Member Is Offline
|
| posted on 05 Sep 06 at 08:34 AM |
|
|
Agility is a concept, not a value. As such, probably the only way to measure it is the same way you can measure "employee satisfaction" - employee
surveys.
A simple, and minimally controversial one would be to ask every software developer on a 1 to 5 scale how much they agree or disagree with the
following statements:
1) My company prefers to focus on Individuals and Interactions over processes and tools
2) My company prefers to focus on Working Software over comprehensive documentation
3) My company prefers to focus on Customer Collaboration over contract negotiation
4) My company prefers to focus on Responding to Change over following a plan
Like fantasy/science fiction? I invite you to read my book- Stone Magic - now
freely available online.
Feedback welcome and appreciated! |
|
|
PierG
Member
Posts: 7
Registered: 05 Sep 06
Member Is Offline
|
| posted on 05 Sep 06 at 09:17 AM |
|
|
I love this (http://alistair.cockburn.us/images/c/ca/Agiledevjoinsthewouldbecrowdcitj0102.pdf) article where Alistair Cockburn describe the
concept of 'would-be-agile' vs the 'being agile' picture.
PierG
|
|
|
ehendrickson
Elisabeth Hendrickson
Member
Posts: 22
Registered: 05 Sep 06
Member Is Offline
Mood: Pensive
|
| posted on 05 Sep 06 at 03:05 PM |
|
|
I think agility can only be measured in the results.
Imagine a team that uses a lightweight plan-as-you-go highly collaborative approach. They practice TDD, refactoring, CI, etc. The team is
co-located. They work well together. They're focused. They use Big Visible Charts. They work in two week iterations. The customer is in the
driver seat, directing the development effort on the features with the highest value to the business. And management supports the Agile Manifesto and
behaves accordingly, demonstrating in their actions that they value people and teams, working software, collaboration, and response to change.
Is this team agile? At first blush, yes. Sounds great.
Now imagine that the team, despite all this, finds that after a year they have nothing to show for their work. They've been unable to release new
features. Nothing has shipped. Management shifts uncomfortably in their ergonomic chairs. Finally someone mutters, "At least with a phased
methodology and big bang releases we got SOMETHING out the door. Let's go back to the way things were."
"But we did everything right!" the team protests. "We're Agile-with-a-capital-A!"
But they're not. Demonstrably. Despite good intentions and good practices, the team has not achieved agility.
We could ask what happened. Perhaps the customer failed to prioritize correctly. Perhaps the development practices weren't quite as good as
everyone imagined. Perhaps the team fell into the trap of mistaking chaos and turbulence for responding to change. Whatever the cause, the team is
not agile. They have not delivered value.
Rather than looking at intentions and practices, we have to find ways to look at results. And that means asking hard questions like:
- Do our stakeholders including executives, investors, customers, users, etc. agree that we're delivering value? How are they measuring us?
- Are we delivering faster, better, and more consistently now than before we claimed to be agile?
- What hard evidence (increased deployed features, faster delivery, higher revenue, lower costs, other) tells us we're more agile?
- What could we change that might increase our ability to deliver value quickly and consistently?
It may or may not be possible to attach meaningful quantitative measures to agility. But attempting to do so is likely to raise interesting
qualitative discussion points.
Though I'm not sure that these count as touchstones in the sense that Paul intended.
Elisabeth
|
|
|
RonJeffries
Ron Jeffries
Super Administrator
Posts: 126
Registered: 29 Aug 06
Location: Pinckney, MI
Member Is Offline
Mood: fascinated
|
| posted on 05 Sep 06 at 03:08 PM |
|
|
I'd agree on the results notion. The other stuff can be seen as means to an end. In particular, it seems to me that shipping real live deployable
code is a necessary result, if a team wants to get their Agile Merit Badge.
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 05 Sep 06 at 03:36 PM |
|
|
Okay, Ron, Elizabeth; the first Touchstone I suggested was in effect measuring results, though one could say delivery of business value within X weeks
(for a small X) is a practice.
What other 'results' might we hope for? Or are all other results of little importance?
Responsiveness to change is probably an important one to tick. It's all very well delivering business value regularly every 2 weeks or so when the
requirements are all known up front. An agile team would continue to do this while the requirements change.
Paul Oldfield
Capgemini, Telford UK
|
|
|
ehendrickson
Elisabeth Hendrickson
Member
Posts: 22
Registered: 05 Sep 06
Member Is Offline
Mood: Pensive
|
| posted on 05 Sep 06 at 05:46 PM |
|
|
What if we consider the actual responses given a changing requirement?
Responses that indicate the team isn't grokking agile:
- "uhhhh...you can't have that change"
- "you don't want that change even if you think you do"
- "I swore off that technology after a bad experience 5 years ago"
- "you'll give us 5 years and $50M, right?"
But "OK" isn't necessarily the most agile response either. "OK" sometimes means "OK we'll do it," and sometimes means, "we're going to say
whatever we have to say to make you go away."
Engagement is the key. Are the people implementing a change engaged? Do they engage the people requesting the change in a conversation to understand
the needs more fully? For example:
- "How will you use this change?"
- "What value does it have to you, either qualitative or quantitative?"
- "What, if anything, could we do instead that would have similar value?"
And, of course, we should measure the effort:
- time from change request to any actual effort to make it happen (perhaps indicated by a story in an iteration or a feature in a sprint)
- time from change request to related delivery
And we should measure the overall effectiveness too, perhaps in terms of resulting additional revenue, customer satisfaction, etc.
For that matter, we should also consider customer/stakeholder perceptions of responsiveness.
Is that what you had in mind?
Elisabeth
|
|
|
CarolWellington
New Member
Posts: 1
Registered: 05 Sep 06
Member Is Offline
|
| posted on 05 Sep 06 at 06:04 PM |
|
|
| Quote: | Originally posted by ehendrickson
I think agility can only be measured in the results.
|
I fully agree with this. However, there is one risk that we have to watch out for: just producing functionality isn't enough; we have to produce
functionality that the customer needs (is willing to pay for). Many agile planning mechanisms weigh the customer's needs (either implicitly or
explicitly), but thinking about how much "value" we have delivered (as opposed to how many "features" we delivered) helps prevent those projects
that fail because they solved the wrong problem.
One metric that I've found to be useful is "bells delivered/iteration" where "bells" is the customer's measure of how useful a feature is (from
the concept "bells & whistles"). This does focus the team on the value of the functionality being delivered, but it has some strange behaviors.
For example, bells/iteration tends to go down over the life of a project as the low hanging fruit are completed. This means that it's a good
short-term measure for weighing decisions (which option increases this iteration's bells delivered) and might give a clue when a project is no longer
worth pursing (when it gets low enough to point out that the project is no longer producing valuable features at a rate that makes it viable). In
other words, it is a metric that helps teams focus on things that should improve their ability to deliver functionality, but it really cannot be used
alone as an absolute measure of a teams "agility." (I agree with many of the other people that there may not be such an absolute measure)
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 06 Sep 06 at 04:42 AM |
|
|
This article certainly is very pertinent and recommended reading for anyone looking at the topic, Thanks PierG.
Talk of intentions reminds me of another concept pertinent to this topic: How we are measured affects how we behave.
Combining the two leads me to think that we should not consider how we are to measure agility until we have answered the question: "Why do we want to
be agile?"
Different answers to this question may lead us to choose to measure different aspects of what we do or what we produce.
Paul Oldfield
Capgemini, Telford UK
|
|
|
bradapp
Brad Appleton
New Member
Posts: 1
Registered: 10 Sep 06
Location: Northwest suburban Chicago, IL
Member Is Offline
|
| posted on 10 Sep 06 at 08:59 AM |
|
|
I like the idea of "touchstones." I think the things in Alistair's article are more along the lines of "essesntial elements" or "key
characteriistics". In the past, I've listed what I thought were the "key characteristics" of agility using AGILE as an acrostic:Adaptive – responsive to change in needs & requirements via continuous feedback and reflective
retrospection
Goal-driven – focus on producing executable-results (working functionality) in order of highest business value.
Iterative – short development cycles, frequent releases, regular feedback
Lean – simple design, streamlined processes, elimination of redundant information, minimal intermediate artifacts
Emergent behavior – highly collaborative self-organizing teams in close interaction with stakeholders
Now, whether or not you use the above, or the one's Alistair mentions, I do think its fair to say that if a "key characteristic" is missing, then
it's not "Agile". For example, I think if Iterative development is not used, then unless the releases are "iteration-sized", the process isnt
doing "agile".
Measuring "how agile am I?" would then correspond to "closeness to" or "distance away" from each key characteristic. This wouldnt tell me how
much value Im getting, it would simply tell me whether or not what was called "agile" was indeed agile. And it is very remiscent of the thread about
"agile auditing" on the agilemanagement Yahoo group.
I do think it could be useful to have some set of key indicators for things like:how frequent/infrequent various 'events'
happen at various levels of scale (e.g., changes-committed, integration builds, stand-up meetings, restrospecting, iterations, releases)
how collocated people are
how responsive (versus plan-driven) the planning & prioritizing is
how much collaboration (as opposed to phase-based 'handoffs')
I suppose it could be used (abused?) to try and give absolute numbers/comparisons rather than a distance from an (ideal) agile "center".
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 11 Sep 06 at 05:59 AM |
|
|
Here's a contribution from "Adail" copied from another list:
"It seems that the Cutter group has an Agile Assessment offer, where they gather and monitor five core metrics: quantity of function, productivity,
duration, effort, and reliability.
http://www.cutter.com/consulting-and-training/agile-assessment.html
"
Paul Oldfield
Capgemini, Telford UK
|
|
|
jasongorman
Jason Gorman
Member
Posts: 4
Registered: 13 Sep 06
Member Is Offline
|
| posted on 13 Sep 06 at 10:04 AM |
|
|
Measuring agility might be as simple as measuring how quickly the direction of a project can change. If the direction is defined by
the plan, then it would be a measure of how quickly the plan can change.
Or maybe it could be measured by how quickly teams respond to new requirements.
If team A takes an average of 6 weeks to satisfy new requirements (e.g., from the day the customer wrote the story card to the day it passed the
acceptance test), and team B takes an average of 6 months, who would say that team A is not more agile than team B?
Measuring Agility - now that's a different matter...
|
|
|
staciabroderick
Stacia Broderick
New Member
Posts: 1
Registered: 25 Sep 06
Member Is Offline
|
| posted on 25 Sep 06 at 12:35 AM |
|
|
Ron, thanks for a fresh new forum....
Wouldn't one set of 'agililty metrics' be a function of
1) how effective product owners/customers are at defining highest value stories
2) the extent to which a team is left to be self-managed and create innovative solutions to problems
to result in
3) the 'match' of customer's problem to team's solution
***
This would imply that the measurements (or touchstones) could be in the following areas:
a) how to define and assign value in backlogs
b) effective backlog management (stories are 'staged' in time for iteration, etc.)
c) the extent that the software is working (ahem, code passing all unit, functional and acceptance tests)
d) self-management metrics (there could be over 100 here)
e) visibility metrics (burndown made visible to stakeholders, project communicated regularly, etc). This is especially important for multi-project
teams in one solution space.
whew, it's late... I know there are more touchstones but I am out of time.
|
|
|
jeffo
Jeff Olfert
Member
Posts: 5
Registered: 28 Sep 06
Member Is Offline
|
| posted on 28 Sep 06 at 01:30 AM |
|
|
I would think that a team that is working well, will respond to change (changing business needs) quickly and continue to deliver business value.
Wouldn't this be a measure of agility?
It also seems that an agile measurement would need to be a function of {amount of time to respond, money spent, money made, change count, change
frequency, scope of change}.
But how do you make an mathmatical formula out of something as complicated as human/ group behavior. Maybe an anthropologist might be able to help
here.
Maybe measuring agility is not really possible, maybe it's like measuring style, or taste, or some other very subjective matter.
In any case what would be the purpose of measuring agility anyway?
Regards,
Jeff Olfert
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 28 Sep 06 at 06:58 AM |
|
|
| Quote: | Originally posted by jeffo
In any case what would be the purpose of measuring agility anyway? |
That's surely the key question.
One 'purpose' I imagined was the case where someone, not well acquainted with 'agile', wanted advice, but was unsure of the quality of advice
being given. How could this person know that the advice was good?
Paul Oldfield
Capgemini, Telford UK
|
|
|
debhart
Member
Posts: 72
Registered: 04 Sep 06
Location: Toronto, Ontario
Member Is Offline
|
| posted on 28 Sep 06 at 09:39 PM |
|
|
| Quote: | Originally posted by PaulOldfield1
What other 'results' might we hope for? Or are all other results of little importance? |
I think that the balancing measure for this one might be team morale. Delivering value without a deathmarch, for example.
|
|
|
craigatwork
craig davidson
Member
Posts: 17
Registered: 28 Sep 06
Location: Scotland
Member Is Offline
|
| posted on 29 Sep 06 at 04:10 AM |
|
|
| Quote: | Originally posted by PaulOldfield1
| Quote: | Originally posted by jeffo
In any case what would be the purpose of measuring agility anyway? |
That's surely the key question.
One 'purpose' I imagined was the case where someone, not well acquainted with 'agile', wanted advice, but was unsure of the quality of advice
being given. How could this person know that the advice was good? |
Hi Paul,
Do you see any conflicts here with the stated agile value of "people over process"?
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 29 Sep 06 at 08:29 AM |
|
|
| Quote: | Originally posted by debhart
I think that the balancing measure for this one might be team morale. Delivering value without a deathmarch, for example. |
Interesting - do you see these as being in conflict? Agreed, team morale is something we may want to measure; something we may want to work on
improving. Agreed, if we push to deliver too much value we go beyond a sustainable pace - but then the measures of 'value' would deteriorate.
Paul Oldfield
Capgemini, Telford UK
|
|
|
PaulOldfield1
Member
Posts: 62
Registered: 05 Sep 06
Location: Perthshire, Scotland
Member Is Offline
|
| posted on 29 Sep 06 at 08:43 AM |
|
|
| Quote: | Originally posted by craigatwork
Hi Paul,
Do you see any conflicts here with the stated agile value of "people over process"? |
Good to hear from you again, Craig.
I'm sure we could measure agility in ways that would conflict with agile values; I'm sure we could measure agility in ways that did not.
We have at least 3 distinct high level things we may measure;
How agile are our processes?
How agile are our people?
How good are our results?
Off the top of my head and without delving into detail, I'd suggest that one of these 3 would be more likely to bring us into conflict than the other
two.
In case you were specifically asking about my 'example reason:
| Quote: | | One 'purpose' I imagined was the case where someone, not well acquainted with 'agile', wanted advice, but was unsure of the
quality of advice being given. How could this person know that the advice was good? |
Well, if we accept that good agility is knowledge based (which we may choose not to accept) then some measure of a person's relevant knowledge may go
some way to answering this question. There may be other measurements we could make.
I don't see this as being in conflict with valuing people over process. If you disagree, help me on this.
Paul Oldfield
Capgemini, Telford UK
|
|
|
WayneMack
Wayne Mack
Member
Posts: 26
Registered: 18 Sep 06
Location: Fairfax, VA US
Member Is Offline
|
| posted on 29 Sep 06 at 09:12 AM |
|
|
Probably one of the leaders in measurements of agile is Laurie Williams of North Carolina State University. Her general website is
http://collaboration.csc.ncsu.edu/laurie/
and look under Publications for a list of downloadable papers. A couple of interest include:
http://collaboration.csc.ncsu.edu/laurie/Papers/ADC.pdf
http://collaboration.csc.ncsu.edu/laurie/Papers/QUTE-SWAP.pdf
She has two measurements sets. One compares the applicability of agile versus waterfall based on project qualities and the other covers adherence to
agile practices. I have also seen a paper measured software quality between agile and waterfall, but I can't seem to find it any more.
|
|
|
craigatwork
craig davidson
Member
Posts: 17
Registered: 28 Sep 06
Location: Scotland
Member Is Offline
|
| posted on 29 Sep 06 at 10:27 AM |
|
|
| Quote: | Well, if we accept that good agility is knowledge based (which we may choose not to accept) then some measure of a person's
relevant knowledge may go some way to answering this question. There may be other measurements we could make.
I don't see this as being in conflict with valuing people over process. If you disagree, help me on this. |
I am probably reading you wrongly but I guess I feel uncomfortable on a few points here:
- I believe that agility to be as much attitude based as it is knowledge based.
- When Agile comes down to four bullet points before diverging into "processes", who sets the measures?
Yes there are some techniques and practices that have often been found to contribute to the agility of software teams.
Adherance to these techniques could be measured: "What is your test coverage? How many hours were spent programming in pairs?"
However, encouraging blind conformance to these techniques in my view would discourage agility. I think we should always be striving to find a better
way than what we are doing now.
I think this is where I am feeling a conflict.
|
|
|
| Pages: 1 2 |
RSS Feed
