Object Technology Jeff Sutherland

Beyond Java: Ruby on Rails with Scala backend

2009-04-10T08:06:00.003-04:00

Twitter on Scala
A Conversation with Steve Jenson, Alex Payne, and Robey Pointer
by Bill Venners
April 3, 2009

Summary
Three Twitter developers, Steve Jenson, Alex Payne, and Robey Pointer, talk with Bill Venners about their use of Scala in production at Twitter.

Twitter is a fast growing website that provides a micro-blogging service. It began its life as a Ruby on Rails application, and still uses Ruby on Rails to deliver most user-facing web pages. But about a year ago they started replacing some of the back-end Ruby services with applications running on the JVM and written Scala. In this interview, three developers from Twitter�Steve Jenson, system engineer; Alex Payne, API lead; and Robey Pointer, member of the service team�sit down with Bill Venners to discuss Twitter's real-world use of Scala. They describe the production issues that led them to consider Scala in the first place, what issues they ran into using Scala in production, and how Scala affected their programming style.

ATT charges over $20000 for three days of Gmail in Iceland!

2008-12-04T06:55:00.007-05:00

The Darth Vader of USB Modems. ATT USB 881. It will suck all your money out of your wallet!

Digg this post!

Evidently, deceptive business practices are the norm for ATT and the iPhone. William Gillis in San Diego is already suing Apple for false advertising.

Is it OK for vendors to blatantly lie about their products? It is very difficult to introduce Agile practices based on truth, transparency, and trust when vendors are constantly lying.

Evidently there are laws against this.

Prohibited Practices: Federal Communications Commission
Overcharging or engaging in other unjust or discriminatory practices by telephone or telegraph companies. 47 U.S.C. �202(a)

Unfair or Deceptive Acts and Practices: Department of Commerce
Engaging in unfair or deceptive acts or practices in or affecting interstate commerce.
9/15 U.S.C. �45(a) (1)

Consumer Products - Packaging and Labeling
Unfair or deceptive labeling of consumer commodities.
15 U.S.C �1452(a), 1453
Engaging in deceptive packaging, including misrepresentation of retail sale price.
14/15 U.S.C. 1452(a); 16 C.F.R. 502

------

Apple argues only a fool would believe its iPhone 3G ads
By Sam Oliver
Published: 04:10 PM EST

Apple isn't lying in television ads that tout the iPhone 3G as twice as fast as its predecessor, but customers would have to be fools to take those claims at face value, the company argues.

That's essentially Apple's legal response to a lawsuit filed by San Diego resident William Gillis back in September alleging that Apple and AT&T knowingly oversold the new iPhone alongside misleading ads that promised it would perform twice as fast as the original model.

------

When I recently purchased an iPhone, I told the sales guy I needed a USB modem for my Mac that worked globally as I spend 2-3 weeks a month in Europe. He said the ATT USBConnect 881 was an ideal choice as it worked in 177 countries, covered all of Europe and supported Global roaming. He didn't tell me if I used it in Iceland, I would have to buy an ATT corporate exec a Mercedes!

While I was in Iceland recently, I got a call from ATT telling me to dispute an outrageous charge. They were billing me over $39000 for three days of Gmail usage in Iceland. After filing a complaint with the Better Business Bureau, ATT lowered the charge to $20000.

So instead of buying an ATT executive a Mercedes for Christmas, they will settle for a Toyota Prius, all for using Gmail for three days. Maybe Google should get into this business!

It turns out that ATT cannot notify you of outrageously high fees until after you have already used the modem. Their system doesn't support this!

Needless to say, our corporate lawyers propose litigation if we cannot settle for a reasonable fee from ATT. I had a Vodafone USB modem in my bag that would have cost me less than $400 for the same usage in Iceland. Stupid me thought ATT would cost less. Duh!

American Express has already refunded all money paid to ATT for this fiasco. Evidently they think the charges are a little unreasonable.

Recent comments from others indicate there are many iPhone and ATT USB modem users out their who feel they have been mistreated. Is there anyone interested in joining a class action suit against ATT if they fail to resolve this issue in a reasonable way?

Arduino: Open Source Hardware

2008-10-26T15:04:00.003-04:00

Build It. Share It. Profit. Can Open Source Hardware Work?
By Clive Thompson Wired Magazine 10.20.08
A circuit board for the masses: the Arduino microcontroller.
Photo: James Day

Check this out," Massimo Banzi says. The burly, bearded engineer wanders over to inspect a chipmaking robot�a "pick and place" machine the size of a pizza oven. It hums with activity, grabbing teensy electronic parts and stabbing them into position on a circuit board like a hyperactive chicken pecking for seeds. We're standing in a one-room fabrication factory used by Arduino, the Italian firm that makes this circuit board, a hot commodity among DIY gadget-builders. The electronics factory is one of the most picturesque in existence, nestled in the medieval foothills of Milan, with birdsong floating in through the open doors and plenty of coffee breaks for the white-coated staff. But today Banzi is all business. He's showing off his operation to a group of potential customers from Arizona. Banzi scoops up one of the boards and points to the tiny map of Italy emblazoned on it. "See? Italian manufacturing quality!" he says, laughing. "That's why everyone likes us!" Indeed, 50,000 Arduino units have been sold worldwide since mass production began two years ago. Those are small numbers by Intel standards but large for a startup outfit in a highly specialized market. What's really remarkable, though, is Arduino's business model: The team has created a company based on giving everything away. On its Web site, it posts all its trade secrets for anyone to take�all the schematics, design files, and software for the Arduino board. Download them and you can manufacture an Arduino yourself; there are no patents. You can send the plans off to a Chinese factory, mass-produce the circuit boards, and sell them yourself � pocketing the profit without paying Banzi a penny in royalties. He won't sue you. Actually, he's sort of hoping you'll do it.

Roots of Scrum: Object Technology

2008-09-27T01:02:00.002-04:00

Scrum was originally designed to support emerging object technology environments, which have now become the dominant paradigm in software development. One of the goals was to get the organization of the team to reflect the potential flexibility of the software since software rigidity always reflects the organization that built it. This is Conway's Law. Fred Brooks cited Mel Conway's 1968 Datamation paper in The Mythical Man-Month and the name stuck. "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."

Object-oriented languages have a small set of interoperating principles. Break any one of them in creating a language and many of the benefits of object technology evaporate. A similar analogy could be made about Scrum. So here is a candidate set of principles.

Inheritance
It is easy to extend the Scrum pattern to fit the local environment without modifying the metaClass represented by the Scrum organizational pattern.

Polymorphism
The same message sent to a different Sprint can produce a result dependent on the environment.

Encapsulation (information hiding)
Work is packaged in increments. Scrum teams can scale by making a Scrum of Scrums look like a Scrum tream to another Scrum of Scrums. Using an object-oriented network to scale allows potentially unlimited scaling.

Emergent Design
A small change can have a ripple effect that causes refactoring and a new design emerges.

Messaging
OO is a messaging environment. Constant high bandwidth communication works best.

Self-Organization
There is typically no "control" object. Behavior emerges from the interaction between objects in languages and between people in Scrums.

Aggregation
Objects can be aggregated into components. Components aggregate into an architecture. Scrums can be aggregated into Scrums of Scrums orchestrated by a metaScrum. A metaClass creates the framework for all classes. The Scrum framework could be viewed as a metaClass laying out the minimal attributes of a Scrum and the relationship between components.

Reflection
Constant review and analysis at a meta-level leads to increased functionality and flexibility. This is an extremely powerful feature missing from some languages, which tends to cripple them. It leads to the Retrospective in Scrum.

All software implementations of objects use the metaClass of a specific language to create coherent structures. There were hundreds of people who created new objects languages and 99% of them failed in the market. These are like homegrown implementations of Agile. "We are doing Agile" has no meaning. It is like saying we are doing object technology. In the last decade we would look into the code of a C++ application and see nothing but old C procedures. There was often not a single object to be found. Today we see hierarchical organizations implementing hierarchical teams and calling it Agile. They can't help themselves. They have to change the organization first to get it right or Conway's Law assures that they will fail.

Certain languages are structured to work together like JRuby with Java. This is analogous to Scrum and XP. Others like Java and C++ inherit features from a common ancestor but do not interbreed well. Those that take pieces of Agile languages and cobble them together get what you would expect - a horse designed by a committee. It can't run but maybe it can store water.

Cool Tools on the Web - DropBox, Code Collaborator, and Poodwaddle

2008-09-16T12:14:00.004-04:00

There is no end of cool tools popping up on the web. Do you want to have file folders that automatically sync on every Windows, Mac, and Linux box that you own instantaneously and transparently - you need DropBox. Or maybe you are having problems with code reviews and want to make them fun, easy, and totally automated - check out Code Collaborator. Or maybe you just want to know the worldwide numbers in real time on the environment and health - see poodwaddle.com below:

iPhone, ATT, T-Mobile and how to avoid getting ripped off

2008-09-14T02:49:00.004-04:00

My phones - ATT iPhone 3G, HTC T-Mobile Touch, Sony/Ericsson Walkman

I spent over half my time for the last few years outside of the U.S. doing Scrum training and experimenting with mobile phones. There are lots of options and some of them can cut your costs by 98% compared to U.S. vendors. That's right, you can make local calls for 2% of the cost of a U.S. roaming cellphone. ATT and T-Mobile cash registers make a loud k-ching sound when you travel abroad and you may not know it but they sometimes think Canada is as far away as Europe.

I couldn't resist getting the iPhone 3G to test out on my travels. The iPhone is a very cute and very expensive toy. I love it. Too bad I can't use it as I travel around the world. I spent a week in Canada last month and got charged $100 extra on top of my $100 monthly charge for using Gmail and Google maps. Pretty expensive map.

ATT claims an award for "the best wireless network in the world." They must be drinking their own Koolaid. At Logan aiport in Boston the signal on the iPhone is so weak my wife can't hear me so I have to pull out my HTC T-Mobile phone which works fine. Same thing happened to me in Toronto. Also, my HTC T-Mobile has all the services of the ATT iPhone for $25.27 a month added to my wife's Blackberry bill as a second line. This includes no extra charge for international roaming data services.

So for the price of spending a week in Canada with my iPhone, I could have eight T-Mobile phones added to my wife's account. I'm keeping my T-Mobile phone so I don't have to use my iPhone, particularly when traveling abroad which is about 2/3 of my time. Also, I have the TOM TOM GPS system on the HTC. I need a real GPS, not a fake GPS like the iPhone for many things although I really like Google maps on the iPhone.

Now let's look at call charges. I'm in Denmark right now where phone charges are cheaper than most countries. Here is what a local call costs me:

ATT iPhone - $1.29/minute
T-Mobile - $0.99/minute
Lebara Mobile - 9 ore = $0.017/minute

Always get a local SIM for your phone. Buy a European phone if you need to. Otherwise you will be spending 100 times more for local calls. I might hack my iPhone to swap SIMS but I can already swap them in my T-Mobile phone so it is probably not worth it.

A markup of 10000% is a little steep so maybe ATT could qualify as the biggest ripoff of cellular networks in the world, although other U.S. vendors are not far behind.

But maybe they should be compared to major European vendors like Telenor, Orange, or Vodaphone and for their SIMs (I have several of them) I will get charged 20-30 cents a minute so maybe the ATT markup is only 500-600%. The iPhone is so cute it might be worth it as long as you don't travel more than a week a year.

Scrum and Fighter Aircraft

2008-07-06T08:35:00.000-04:00

In 1993 at Easel Corporation when we started the first Scrum, I thought the ScrumMaster needed to see how to land a Sprint on a date, just like landing a fighter aircraft at the end of the runway. If you are high on the glidepath in a fighter you can easy land in the middle of the runway and if it is wet or icy can slide off the end of the runway. At the end of every runway there are trees, a deep hole, or water. It is always bad.

So the Burndown Chart along with the Scrum Emergency Procedure was developed. What do you do when bad things start to happen in a Sprint?

More specifically, what do you do when an Agile system becomes unstable?

But I digress. Jim Coplien just sent me a great video of an unstable Agile system, the latest Euro fighter. It takes 70 computers just to keep it flying and if they fail it stops flying.

Here we see a race between a Bugatti and the fighter aircraft which reminds me of the time I flew F4 Phantoms in Europe in the air while I was driving a Porsche 911 on the ground. Here the Bugatti races the fighter to the end of the runway and back while the fighter takes off, flies straight up for a mile, then does a reverse loop in an attempt to beat the Bugatti back to the takeoff point.

Click here for an exciting ride!

HICSS 42 Call for Papers due 15 June 2008

2008-04-04T11:01:00.006-04:00

It's time for you to get your most scintillating Agile theories
together, write a kick-ass paper that could get published in the IEEE
library and spend a week in beautiful Hawaii next January. Sound good?
Then get writing!

HICSS-42 CALL FOR PAPERS - due 15 June 2008
http://www.hicss.hawaii.edu/hicss_42/minitracks/st-asd.htm
January 5-8, 2009
Hilton Waikoloa Village Resort
Waikoloa, Big Island, Hawaii

HICSS-42 offers a unique, highly interactive and professionally
challenging environment that attendees find "very helpful -- lots of
different perspectives and ideas as a result of discussion." HICSS
sessions are comprised primarily of refereed paper presentations; the
conference does not host vendor presentations. All papers are peer
reviewed and accepted papers are published in the IEEE Digital Library.

Software Technology Minitrack
Agile Software Development: Lean, Distributed, and Scalable
(Jeff Sutherland and Gabrielle Benefield)

Agile software development processes have been influenced by best
practices in Japanese industry, particularly by lean product
development principles implemented at companies like Honda and Toyota,
and knowledge management strategies developed by Takeuchi and Nonaka, now at the Hitotsubashi Business School in Japan, and Peter Senge at
MIT. This Minitrack will focus on advancing the state of the art or
presenting innovative ideas related to agile methods, individual
practices and tools.

Accepted papers will potentially enrich the body of knowledge and
influence the framework of thought in the field by investigating Agile
methods in a rigorous fashion.

We are open to research papers on multiple aspects of agile methods,
particularly those that bring best practices in knowledge management
and lean development to scalable, distributed, and outsourced Scrum,
eXtreme Programming (XP), and other agile practices. Topics include:

1. Research on existing or new methodologies and approaches: informal
modeling techniques and practices, adapting/trimming existing methods,
and new product/project planning techniques
2. Research on existing or new techniques or practices: pairing,
war-rooms, test-first design, paper-based prototyping, early
acceptance test driven development, exploratory testing, refactoring,
or others.
3. Research on special topics or tools: configuration and resource
management, testing, project steering, user involvement, design for
agility, virtual teams or others.
4. Research on integrating ideas from other fields, e.g. interaction
design, requirements engineering, cognitive science, organizational
psychology, usability testing, software security, into agile processes.
5. Research studies of development teams using ethnographic or social
6. Research on agile software engineering economics.
7. Quantitative and qualitative studies of agile methods, practices,
and tools.
8. Research on agile compliance and cost benefits within CMMI, ISO
9000, and FDA certified development projects.

Papers are particularly relevant when agile process implementations
are shown to produce quantitative and qualitative benefits on
distributed, outsourced, large, or standards compliant software
development projects which have been previously been viewed
(erroneously) as unsuited for agile development.

To submit papers and read more about the conference please go to:
http://www.hicss.hawaii.edu/hicss_42/minitracks/st-asd.htm

Jeff Sutherland
Scrum, Inc.
332 Congress St., 3rd Floor
Boston, MA 02210

Gabrielle Benefield
Scrum Training Institute
Mountain View,
CA 94040
Email: gabriellebenefield@yahoo.com

Agility and Organizational Patterns

2008-02-02T15:05:00.002-05:00

In the early days of what we now know as Agile processes, Mike Beedle was influenced by the online description of Scrum, implemented the process in his own company, and led the effort to drive Scrum through the Pattern Languages of Programming Design conferences. This made Scrum the first (and only) formal organizational pattern that describes a complete Agile process. One of the patterns books contains the Scrum pattern:

M. Beedle, M. Devos, Y. Sharon, K. Schwaber, and J. Sutherland, "Scrum: A Pattern Language for Hyperproductive Software Development," in Pattern Languages of Program Design. vol. 4, N. Harrison, Ed. Boston: Addison-Wesley, 1999, pp. 637-651.

Recent work by Jim Coplien shows that Scrum is deceptively simply while compressing a complex array of organizational patterns in his book, "Organizational Patterns in Agile Software Development." Jim was surprised when he found that Scrum compresses at least 33 patterns from his book into a concept that can be explained in 2 minutes. It takes over 60 pages of rather dense text to describe these patterns.

One of Scrum's design goals was to encapsulate best practices from 40 years of software development into a process that was simple enough for the average IT worker to use for development in less than 2 days of startup time. Jim's research shows that we did a good job of accomplishing that goal. You can download a copy of the complete Scrum pattern as it is part of a draft of "The Scrum Papers."

Click here for Jim's presentation on Scrum as a set of patterns.

Agile Software Development: Lean, Distributed, and Scalable

2008-01-08T14:55:00.000-05:00

HICSS 2008 Agile MiniTrack
Co-Chairs: Jeff Sutherland and Hubert Smits
HICSS 41, January 7-10, 2008 - Hilton Waikoloa Village Resort
Waikoloa, Big Island, Hawaii

If you are in Hawaii now, congratulations! If not, eat your heart out ...

ST 1 Thursday; Kona 3; 8:00 � 9:30

This session has three papers. Authors will have 15 minutes for presentation and 15 minutes for discussion. Papers will be presented in the following order.

Bridge Methods: Complementary Steps Integrating Agile Development Tools and Methods with Formal Process Methodologies
Stephen J. Cohen and William H. Money
Rolling Out Agile in a Large Enterprise
Gabrielle Benefield
Great Scrums Need Great Product Owners: Unbounded Collaboration and Collective Product Ownership (in competition for best paper)
Ken H. Judy and Ilio Krumins-Beens

ST 2 Thursday; Kona 3; 10:00 � 11:30'

This session has four papers. Authors will have 15 minutes for presentation and 10 minutes for discussion.

Effects of Agile Methods on Website Quality for Electronic Commerce
David F. Rico
A Case Study: Introducing eXtreme Programming in a US Government System Development Project
Ann Fruhling, Patrick McDonald, and Christopher Dunbar
Retrofitting Cyber Physical Systems for Survivability through External Coordination
Kun Xiao, Shangping Ren, and Kevin Kwiat
Scrum and CMMI Level 5: The Magic Potion for Code Warriors
Jeff Sutherland, Carsten Ruseng Jakobsen, and Kent Johnson

Douglas Crockford on The State of Ajax

2007-11-14T11:10:00.001-05:00

In a recent Yahoo YUI video, Yahoo's Douglas Crockford gives an overview of the state of Ajax application development on the Web, and points to ways in which the Web can improve. Advertisement

Douglas Crockford, inventor of the JSON data exchange format, and Yahoo's chief DHTML evangelist, presents an overview of the State of Ajax and client-side Web development. Among the topics he discusses are Java, the flaws and benefits of JavaScript, why CSS should be replaced, why mashups are significant, and what can be done about Web technologies, such as HTML and JavaScript, that saw their last major updates in 1999.

Kleiner Perkins gets real on global warming

2007-10-31T20:56:00.000-04:00

John Doerr gave a great talk on climate change at the recent TED conference. Check out "Seeking salvation and profit in greentech."

-----
John Doerr, a partner in famed VC firm Kleiner Perkins Caufield & Byers, made upwards of $1 billion picking dot-com stars like Amazon, Google, Compaq and Netscape. (He also picked some flops, like Go Corporation and the scandal-ridden MyCFO.com.) He was famously quoted saying, "The Internet is the greatest legal creation of wealth in history," right before the dot-com crash.

But now he's back, warning that carbon-dioxide-sputtering, gas-powered capitalism will destroy us all, and that going green may be the "biggest economic opportunity of the 21st century." So Kleiner Perkins has invested $200 million in so-called greentech, a combination of startups that are pioneering alternative energy, waste remediation and other schemes to prevent the coming environmental calamity. But Doerr is afraid that it might be too little, too late.

"[John Doerr] is, by all accounts, the most influential venture capitalist of his generation." Fast Company

Why Time Sheets are Lame ...

2007-10-07T14:29:00.000-04:00

Programmer time vs. Quality Software Score. Totally uncorrelated!

Actually time sheets are worse than lame:
* they demotivate developers
* 10-15% loss of productivity is the minimum
* developers have to fake the time to fill them out properly
* erroneous data is used for reporting and management makes bad decisions
* customers are deceived
* they have nothing to do with quality code production
* they focus the whole organization on phony data instead of production

Nevertheless, this is not enough for many managers to give up time sheets. Just like the waterfall process, there is a psychological dependency so strong, it is as if they are on drugs.

However, the situation is even worse. Most management has completely wrong information in their head and thus continually make bad decisions. One of my students recently said to me, "You mean everything my manager told me is wrong!"

Yes, Jose, everything your manager ever told you is wrong:
* there is no correlation between developer time and software production
* there is no correlation between time spent and quality of code
* there is no relationship between "quality people" and code production

The only correlation between developer time and quality production code is the quality story points measured as a deliverable for a specific team.

Research over many years at Yale University provides some of the best data on this topic - see Joel on Software
:
* for a single project worst/best coding times are 1/10
* across many projects worst/best coding times are 1/25
* the ratios above are the same for the worst and best Yale students
* the quality of the code produced is completely independent of time spent

For some reason, many development managers makes decisions without any data at all on this issue and their assumptions are completely out of reality.

Tom Poppendieck told me recently a competent manager actually did some research on his XP teams to see what number of hours per week produced the maximum amount of quality production code. After trying shorter weeks and overtime weeks, the best number of hours for teams to produce the most quality code was a 16 hour work week!

Trust me, you need to dump those lame time sheets and get focused on real software production before an Agile competitor puts you out of business!

Scrum and CMMI Level 5: A Magic Potion for Code Warriors

2007-09-05T14:18:00.000-04:00

An earlier blog item commented on the dramatic advantages of using Scrum with CMMI, particularly with a CMMI Level 5 company. See Scrum supports CMMI Level 5.

At the Agile 2007 Conference in Washington, D.C., an experience report was presented on the results of introducing Scrum into a CMMI Level 5 environment to replace waterfall projects for large defense and healthcare contracts. See Scrum and CMMI Level 5: A Magic Potion for Code Warriors. The paper was written by:

Jeff Sutherland, Ph.D. - Co-Creator of Scrum
Carsten Jakobsen - Systematic Software Engineering Process Leader
Kent Johnson - CMMI Level 5 Auditor

Systematic Software Engineering is a company which executes the waterfall process better than almost all companies in the world, with an ontime, on budget delivery rate of over 95% with estimates within 10% of actuals. The Scrum results were extraordinary, similar to introducing a team of Toyota consultants into a manufacturing plant. This was the result of driving the Scrum implemenation by lean principles which assured a disciplined and measured introduction of Agile practice.

- Productivity doubled in less than six months reducing total project costs by 50%.
- Defects were reduced by 40% in all Scrum projects (despite the fact this company already had one of the lowest defect rates in the world.)
- Planning costs were reduced by about 80%.
- User satisfaction and developer satifaction were much higher than comparable waterfall implementations.
- Projects were linearly scalable, something never seen before. The productivity of individual developers remains the same as the project increases in size.

The data in this study is some of the best in the industry and puts to rest the argument about whether the waterfall is preferable in some cases. The waterfall will always be less productive with higher defects on any project compared to a well executed Scrum.

Systematic Software engineering has revised its standard processes to use Scrum everywhere. See Scrum and CMMI Level 5: A Magic Potion for Code Warriors.

Agile 2007 Video Contest 1st Place: Developer Abuse

2007-08-20T19:31:00.000-04:00

At the Google party at Agile 2007 we all voted (screamed) for the best Agile video advertisement. Developer Abuse won easily.

Camp Scrum: A Powerful New Concept to Accelerate your Agile Program

2007-08-10T01:15:00.000-04:00

Looking for a powerful way forward into the world of Agile development? Whether you're just getting started or looking for ideas that can improve your current Agile implementation, you don't want to miss this opportunity to learn from the world's top experts in Scrum and Agile software development.

Imagine having a whole week together with four of the most authoritative personalities in Agile software development today... Not only a week of seminars, but a week of living with the experts, and of informal discussions of everyday practical problems of project management... Imagine gaining deep insights into the fundamentals of Scrum and Agile that could make all the difference in the world on your current project... Imagine being able to discuss these issues with others like you who bring their experiences to this event... Then imagine it all taking place in a beautiful rural Swedish setting, surrounded by nature...

If you can imagine this, you can imagine Camp Scrum.

Hawaii International Conference on Soft ware Systems 2008: Call for Papers - due 15 June 2007

2007-06-09T02:34:00.000-04:00

Get your papers in by 15 June 2007 - don't miss Hawaii in January 2008!

Agile Software Development: Lean, Distributed, and Scalable
(Jeff Sutherland and Hubert Smits)

Agile software development processes have been influenced by best practices in Japanese industry, particularly by lean product development principles implemented at companies like Honda and Toyota, and knowledge management strategies developed by Takeuchi and Nonaka, now at the Hitotsubashi Business School in Japan, and Peter Senge at MIT. This Minitrack will focus on advancing the state of the art or presenting innovative ideas related to agile methods, individual practices and tools. Accepted papers will potentially enrich the body of knowledge and influence the framework of thought in the field by investigating Agile methods in a rigorous fashion.

We are open to research papers on multiple aspects of agile methods, particularly those that bring best practices in knowledge management and lean development to scalable, distributed, and outsourced Scrum, eXtreme Programming (XP), and other agile practices. Topics include:

Research on existing or new methodologies and approaches: informal modeling techniques and practices, adapting/trimming existing methods, and new product/project planning techniques

Research on existing or new techniques or practices: pairing, war-rooms, test-first design, paper-based prototyping, early acceptance test driven development, exploratory testing, refactoring, or others.

Research on special topics or tools: configuration and resource management, testing, project steering, user involvement, design for agility, virtual teams or others.

Research on integrating ideas from other fields, e.g. interaction design, requirements engineering, cognitive science, organizational psychology, usability testing, software security, into agile processes.

Research studies of development teams using ethnographic or social research techniques.

Research on agile software engineering economics.

Quantitative and qualitative studies of agile methods, practices, and tools.

Research on agile compliance and cost benefits within CMMI, ISO 9000, and FDA certified development projects.

Papers are particularly relevant when agile process implementations are shown to produce quantitative and qualitative benefits on distributed, outsourced, large, or standards compliant software development projects which have been previously been viewed (erroneously) as unsuited for agile development.

Jeff Sutherland (primary contact)
Chief Technology Officer and Worldwide Scrum Consulting Practice Manager
PatientKeeper Inc.
275 Washington St., 2nd Floor
Newton MA 02458
Tel: (617) 987-0394
Email: jeff.sutherland@computer.org

Hubert Smits
Services Dept
Rally Software Development
3333 Walnut Street
Boulder, CO 80301
Email: Hubert.Smits@rallydev.com

Graduation Time: MIT PhD Thesis uses cheap supercomputer

2007-06-05T09:38:00.001-04:00

All at Once Computing
.........|....................|..................|..................|..................|

.........|....................|..................|

Dr. Andrew Sutherland will be awarded his Ph.D. degree at MIT on 7 June 2007. He needed a supercomputer to complete thesis thesis and his approach was to link eight HP a6030m dual processor AMD systems together to parallel process new mathematical techniques that will be useful for encryption. Breaking through a previous unsolved problem required developing algorithms that ran thousands of times faster than previous approachs. A calculation that required 15 days on a Sun Sparcstation4 now takes less than 3 seconds on a PC.

I bought one of these inexpensive HP systems to test it out and is extremely fast, almost totally silent, and produces almost no heat compared to the Dell dual processor Intel system sitting next to it. I can see why Dell is "feeling the heat" from the competition. Suffice to say, this HP is blazingly fast compared to my watercooled IBM Thinkpad T42 which I have relegated to the dustbin of computing history. I'm thinking about buying a couple more of the HP systems. For the orginal price of my Dell workstation, I could have a eight system supercomputer lab.

For those of you who are math geeks, here is the thesis abstract:

Order Computations in Generic Groups
by Andrew V. Sutherland
Submitted to the Department of Mathematics on May 16, 2007, in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY
For PDF of thesis click here.

Abstract

We consider the problem of computing the order of an element in a generic group. This is known to require exponential time. The two standard algorithms, Pollard�s rho method and Shanks� baby-steps giant-steps technique, both use Θ (N^1/2) group operations to compute |?| = N, and a lower bound of Ω (N^1/2) has been conjectured. We disprove this conjecture, presenting a generic algorithm with complexity o N^1/2) in any group. The worst case occurs when G is a cyclic group of (unknown) prime order, but for approximately half the integers N ε [0,M], the complexity is O (N^1/3).

We prove that a generic algorithm can perform any number of order computations in near linear time, plus the cost of a single order computation with N = #955;(G), where λ(G) is the group exponent (without knowledge of λ(G)). We show that in abelian groups, λ(G) can be computed for the same cost, and also consider some non-abelian cases.

We then use λ(G) to compute the structure of an abelian group. Given an O (N^2-ε) bound on |G| = N, we can compute group structure in o (N^1/2) group operations for all but a small set of pathological cases. The complexity is essentially that of an order computation on a single random element. A lower bound of Ω N^1/2) had been assumed, based on a similar bound for the discrete logarithm problem.

We apply these results to compute the ideal class groups of imaginary quadratic number fields, a standard test case for generic algorithms. The record class group computation by a generic algorithm, for discriminant ?4(1030 + 1), involved some 240 million group operations over the course of 15 days on a Sun SparcStation4. We accomplish the same task using 1/1000th the group operations, taking less than 3 seconds on a PC. Comparisons with non-generic algorithms for class group computation are also favorable in many cases. We successfully computed several class groups with discriminants containing more than 100 digits. These are believed to be the largest class groups ever computed.

Scrum World Tour T-Shirt

2007-05-23T10:02:00.000-04:00

Thanks to �rjan Hillbom CPG we now have a Scrum World Tour T-Shirt produced by Trifork in Denmark. �rjan is a rock group/motorcycle gang video expert who attended a Certified ScrumMaster training by me and Jens Ostergaard in Copenhagen recently. He said Scrum really "rocks" and it is definitely "faster, better, cooler" and must have a world class T-shirt which he designed for free.

A Type C Scrum Danish company produces really nice T-shirts so I asked them to produce this one. It costs about $30 plus a few dollars for shipping. Get your Scrum World Tour T-Shirt at Trifork.

Regenerative Build Tools

2007-05-16T11:34:00.000-04:00

Regenerative Build Tools
by Michael Feathers, May 15, 2007

Summary
Dust off an old computer and put it to work.

I think that when people look back at the way we program today, one of the things that they'll be amazed at is the fact that we have so much computing power but we don't do much with it.

Seriously, how many computers do you have? If you're like me you have a least a couple of stray ones sitting around the office, and they aren't doing anything. What a waste. And, it's only going to get worse as multicore processors become more prevalent.

When I first heard of continuous integration, way back in the day, I misunderstood it. I thought that some computer someplace was going to build my project continuously. That is, regardless of whether I or anyone else had checked in. Silly? Perhaps. But, wasteful? I don't think so, the computer is running anyway. It might as well work.

A while back, I visited a C++ team that had found a wonderful use for a spare computer. They set it up to build continuously. Then they did something fascinating. They created a script that went into their code and commented out a single #include. Then it built their project. If the build and the tests passed, the script deleted the include. If it didn't, the script uncommented the include and went on to the next one. Yes, it took quite a while to march through the code, but with that build running continuously in the background, they were able to eliminate a large number of superfluous dependencies. Their build got better.

That script that they wrote is an example of something I call a regenerative build tool. There are others. Jester is a regenerative build tool which uses mutation testing to find problems. It changes your code in subtle ways and runs your tests to see whether they still pass. If they do, you either don't have the tests you need to have, or you could have code which is really doesn't contribute to your results at all -- zombie code. Guantanamo is another regenerative build tool, a rather severe one. It deletes all code that isn't covered by a set of tests.

Deep Agile: A Dialogue Between Scrum and XP

2007-04-13T14:01:00.000-04:00

Photo from cover of Crosstalk, Apr 2007

Spend two days with two signatories of the Agile Manifesto - Jeff Sutherland, Co-Creator of Scrum, and Ron Jeffries, XP author and consultant.

You wouldn't want to miss a two day deep dive into Agile: Scrum, Extreme Programming(XP) and Lean steps to software development success. Not to mention how to get Scrum and XP working together ...

When: April 28 and 29th, 2007
Where: MIT Tang Center/E51-345, Cambridge, MA

Register here. All proceeds go to the Boston chapter of the ACM.

Scrum Montage

2007-03-16T10:39:00.000-04:00

Montage-a-Google

Montage-a-google is a simple web-based app that uses Google's image search to generate a large gridded montage of images based on keywords (search terms) entered by the user. Not only an interesting way of browsing the net, it can also be used to create desktop pictures or even posters.

The Scrum montage was created by setting Scrum as the keyword for a Montage-a-Google search.

Get an Experienced Developer on the Test Team

2007-03-04T14:59:00.001-05:00

In 1993, at Easel Corporation, we went to the local university and hired their best Master of Computer Science graduate to join the test team and focus 100% on automation. This was one of the decisions that put the first Scrum into a hyperproductive state where we delivered software so fast, the management, sales, and marketing people asked us to slow down.

Alberto Savoia proposes the same thing in his recent blog posting and his proposed job description is below. Click here to read the full post that makes the case for an experienced developer on the test team.

Agitating Thoughts & Ideas
Developer Testing Masters and Brain Surgeons
by Alberto Savoia
February 26, 2007

Developer Testing Master

This is a high-visibility, high-impact, senior software development position, reporting to the VP of Software Engineering at Company ABC. The successful candidate will help our software engineering organization bridge the gap between development and QA by providing developers with the infrastructure, tools, training, and support necessary to be successful at developer/unit testing.

Job Requirements:

[Should start with the organization�s standard requirements for non-junior software developers. Remember, this should not be an entry-level or junior position. This person has to be at least as good as other developers � ideally better than most of them since the DTM will often have to understand, clean-up, and make testable some of the un-testable code the other developers have created.]

Must have demonstrated passion for, and experience in, developer testing. Should be extremely familiar with continuous integration and testing tools and frameworks (CruiseControl, xUnit, code-coverage tools, etc.)

Must have leadership ability to evangelize, motivate, and train developers in the art and science of unit testing.

Job Responsibilities

Help to set-up a software development environment that enables continuous integration and testing. Environment must support automated hourly and nightly builds. The former must run smoke tests and the latter the full battery of developer tests. Might have to implement distributed testing framework to allow parallel test execution.

Analyze the existing code base and recommend and/or implement re-designs and refactorings to make the code base testable.

Extend and customize xUnit framework to standardize and simplify unit test writing for the other developers. Extend the �TestCase� class by creating �CompanyABCTestCase� and maintain the latter. Implement initial set of mocks, test fixtures, and other set-up methods to enable other developers to write true unit tests that don�t depend on external system resources.

Create and deliver basic unit testing training material to educate all developers in the art and science of unit testing.

Work with the team to decide on developer testing metrics and objectives. Create, maintain, and publish a developer-testing dashboard to track those objectives.

Ruby Implementations Shootout

2007-02-26T15:36:00.000-05:00

Antonio Cangiano reported on Ruby benchmarks recently:

Many brilliant developers are working on improving the current implementation of Ruby and on creating alternatives. I was curious about their current respective speeds, so I installed and ran some benchmarks for the most popular implementations. In this article, I�m sharing the results for the community to see...

Benchmark Environment

My tests were conducted on an AMD Athlon� 64 3500+ processor, with 1 GB of RAM.

The tested Ruby implementations were:

* Ruby 1.8.5-p12 stable on Linux;
* Ruby 1.8.5-p12 stable on Windows Vista;
* Ruby 1.9 on Linux;
* JRuby on Linux;
* Rubinius on Linux;
* Cardinal on Linux;
* Gardens Point Ruby .NET Beta 0.6 on Windows Vista;

The operating system that was used for all � but Ruby.NET � is Ubuntu 6.10 (for x86). Ruby.NET currently runs on Microsoft Windows only, therefore I�ve used Vista with the .NET Framework 2.0 and have also run Ruby 1.8.5-p12 on Windows as a means of having a more direct comparison with Ruby.NET.

Bjarne Stroustrup, inventor of C++, on what's wrong with most software ...

2006-11-30T15:58:00.000-05:00

The Problem with Programming
By Jason Pontin, Tuesday, November 28, 2006

In the 1980s and 90s, Bjarne Stroustrup designed and implemented the C++ programming language, which popularized object-oriented programming and influenced numerous other programming languages, including Java.

C++ remains the archetypal "high level" computer language (that is, one that preserves the features of natural, human language), and it is still used by millions of programmers. Many of the systems and applications of the PC and Internet eras were written in C++. For all that, the language remains controversial, largely because it is notoriously difficult to learn and use, and also because Stroustrup's design allows developers to make serious programming mistakes in the interest of preserving their freedom.

Stroustrup, for many years a researcher at AT&T Bell Labs, is now a professor of computer science in the Department of Engineering, at Texas A&M University, near Houston.

Technology Review: Why is most software so bad?

Bjarne Stroustrup: Some software is actually pretty good by any standards. Think of the Mars Rovers, Google, and the Human Genome Project. That's quality software! Fifteen years ago, most people, and especially most experts, would have said each of those examples was impossible. Our technological civilization depends on software, so if software had been as bad as its worst reputation, most of us would have been dead by now.

On the other hand, looking at "average" pieces of code can make me cry. The structure is appalling, and the programmers clearly didn't think deeply about correctness, algorithms, data structures, or maintainability. Most people don't actually read code; they just see Internet Explorer "freeze."

I think the real problem is that "we" (that is, we software developers) are in a permanent state of emergency, grasping at straws to get our work done. We perform many minor miracles through trial and error, excessive use of brute force, and lots and lots of testing, but--so often--it's not enough.

Software developers have become adept at the difficult art of building reasonably reliable systems out of unreliable parts. The snag is that often we do not know exactly how we did it: a system just "sort of evolved" into something minimally acceptable. Personally, I prefer to know when a system will work, and why it will.

TR: How can we fix the mess we are in? ... Click here to find out!