Scrum is an Agile development framework that Jeff Sutherland invented at Easel Corporation in 1993. Jeff worked with Ken Schwaber to formalize Scrum at Nokia Test: Where did it come from?
In 2005, Certified Scrum Trainer Bas Vodde was coaching teams at Nokia Networks in Finland and developed the first Nokia test focused on Agile practices. He had hundreds of teams and wanted a simple way to determine if each team was doing the basics.
The Nokia test is a similar to a maintenance check on your car. It looks at whether your tires have air, your tank has gas, all cylinders are firing, and makes sure there are no critical missing pieces to your car. You should perform it before you go out for a drive with your Scrum team.
It does not provide the secret sauce for hyperperforming teams. However, it is the first line of the recipe for high performance. We give this test to Scrum teams at OpenView Venture Partners and to their portfolio companies as the venture group does not expect good performance from Scrum teams without passing the Nokia test. They are also very interested in predictability of release dates which is impossible without passing grades on the test.
On 19 June 2006, announced a joint venture with Nokia Networks to form Nokia Siemens Networks. Full operations started on 1 Apr 2007 . Combined 2005 revenues were estimate at 15 billion Euro. Bas Vodde moved to China to train Nokia Siemens Networks staff on Scrum and updated the Nokia Test to include Scrum practices.
In 2007, Jeff Sutherland tuned the Nokia Test for Scrum Certification and in 2008 developed a Nokia Test scoring system. In 2009, a team question was added to the Nokia test. Click here for the lastest version.
Each person on the team takes a sheet of paper and prepares to score questions on a scale of 1-10. Teams average their score and team scores are averaged across a training class or a company to determine the Nokia test score.
Scrum training classes average a score of 4 to 5 during the first morning of the course. At the end of the course, they feel they can bring their teams up to a 6 by the end of their next Sprint using what they learned in the course.
Labels: nokia scrum test
posted by Jeff Sutherland @ 5:42 PM
15 Comments:
y 2007, Siemens had acquired Nokia Networks to form Nokia Siemens Networks with over 60,000 employees and 15 billion Euro in revenue.
What kind of statement is this ? Get your facts right Jeff
OK, the effort was billed as a joint venture. Announced in 2006, full operations started on 1 Apr 2007. The 15 billion Euro in revenue was estimated combined 2005 revenue. Are there other facts you are referring to?
Reading that Siemens had acquired Nokia networks was actually a bit too much to digest for me:-)
Actually its a 50-50 JV and the financial results of Nokia Siemens Networks are consolidated as part of Nokia's results. So I dont know how you got this idea about Siemens acquiring Nokia Networks.
I've written an paper about nokia test. I applied this test in one software company, but I didn't found anywhere how to calculate the score, could you provide me one material that explains how to calculate the score?
My e-mail is danilo.caetano1987@gmail.com
The latest version of the Nokia test has been posted at:
http://jeffsutherland.com/scrum/nokiatest.pdf
A number of our Scrum teams (30+) took this modified Nokia Test and there were a lot of common questions regarding scoring. Is there a forum where these questions can be posted and are likely to be answered?
Questions on the Nokia test can be posted here and I will answer them.
[Jeff, I thought I already posted a link to these questions at http://palfvin.blogspot.com/, but I don't see it here, so I'm not sure if there was a problem with it. Anywway, I've pasted in the content this time.]
The following are questions that arose in my department after taking the version of the Nokia Test which has ratings from 0 to 10.
Iterations - Our iterations are roughly one month in length, scheduled a year in advance to end on a Monday in the middle of the month (to avoid holidays and other standing meetings). That works out to 8 iterations of 4 weeks and 4 iterations of 5 weeks during a given year. Is that "variable length"? For those that claim "fixed length", what do they do if the iteration boundary occurs in the middle of a holiday week?
Testing within the Sprint - What does "dedicated QA" mean? Is that a reference to a person or role? If so, is having an individual a Scrum team who is dedicated to testing considered superior to having the work fully distributed? If it's a reference to QA work, what does "dedicated" mean? Also, in scoring this question, can you only came credit for a higher number if you've satisfied all the "good things" implied by lower level questions?
Agile Specification - Is interpolation permitted or encouraged? For example, what about "poor requirements"? Is there a reference for what is meant by "specifications" in this context (and what distinguishes it from "requirements")?
Product Owner - Again, are lower level "good things" required for higher level scores? For example, if development team and ScrumMaster are doing the lion's share of the work in preparing the Product Backlog, the Product Owner is only peripherally involved, but the backlog is clear and estimated before the Sprint Planning meeting, is that still a 5?
Product Backlog - Not really a scoring question, but if "story point" is based on size(effort) and not value, then how is cost per story point a measure of ROI?
Team Disruption - There are lots of potential sources of disruption (e.g. other Scrum teams seeking assistance, support personnel seeking help to restore service for a "down" customer). Is this question not intended to measure those?
Team - Same question about whether/how higher levels depend on lower levels (see below for generalization). What does "necessary competency" mean? Our teams are in a constant state of challenge/growth in terms of their skills relative to a large legacy code base.
Generalized Scoring Question - Each of the scoring elements is expressed as either a "bad thing in at least some respect" or a "good thing" relative to Scrum. My understanding of the intended scoring algorithm is as follows:
* You can't score higher than any "bad thing" which applies to you, even if you satisfy something with a higher number
* You can't score higher than any "good thing" you don't satisfy, even if you satisfy a good thing with a higher number
Is that correct?
Fixed length iterations - I'll do my best to clarify questions above, one at a time. The purpose of the fixed interactions is to establish a "heartbeat" of the team which leads to higher performance. It is also to give the Product Owner a predicable roadmap based on team velocity. In the iteration question above, the target was a monthly iteration with some four weeks and some five weeks to work around holidays. For the purposes of the Nokia test, I would consider that a fixed iteration.
Testing within a sprint - there must be testers on the team that complete testing at the feature level during the sprint, i.e. the acceptance tests defined by the Product Owner and enhanced by the testers are completed. Only stories with completed tests are DONE and count towards team velocity. On some teams developers test (but not the same developer that wrote the code). For other teams, it is essential to have professional testers when the consequences of failure are high - a production system for treating patients in a hospital, for example.
Hello Jeff,
how are you? I'm here again. This question is very simples, this scores are right about Nokia Test?
ScrumButt (Average 7,0 or less)
Pretty Good Scrum (Average 8,0 or more than 7,0)
Good Scrum (Average 9,0 or more than 8,0)
Great Scrum (Average 10 or more than 9,0)
Most teams who take the Nokia test score an average between 4 and 5. They are doing Scrum BUT they are missing some critical features that are essential to performance. Teams that raise their scores to between 6 and 7 typically run at 200% of their velocity when they scored between 4 and 5. Teams that raise the score further to between 8 and 9 often run at 300% of their initial velocity. Many companies I work with triple their velocity (or more). Keeping it there requires continual process improvement through constant removal of impediments. Failing to do this will cause teams to rapidly decay to mediocre velocity and quality.
Material on this site is free for reuse with attribution for non-commerical purposes under a Creative Commons license.
I appreciated your two posts answering the first two of my several questions on scoring.
Not to be pushy, but can I look forward to you answers to the rest?
Post a Comment
<< Home