Wednesday, January 22, 2003

SCRUM: Eliminating bottletnecks is a core systems design principle

How do you optimize throughput in any systems process?
Goldratt's book is extremely useful to any techncial expert, whether in software development, or a surgeon in the Operating Room of the Future. It is written as a novel, easy and fun to read, and based on sophisticated mathematical principles.

The book shows that any industrial (or software, or clinical) process is a complex adaptive system. It is not possible to directly optimize the whole system because side effects overwhelm any analysis. The key to optimization is to look for bottlenecks. What is getting in the way? Remove the obvious bottleneck and throughput will increase. Then the next bottleneck with appear. By adopting a strategy of eliminating bottlenecks one by one, the system will evolve into radically improved throughput. This is why the third key question in a SCRUM every day is, "What is blocking progress?" The primary responsibility of management is not managing a SCRUM. It is removing bottlenecks identified by the SCRUM.

  • Goldratt. Eliyahu M. The Goal : A Process of Ongoing Improvement, 2nd Revised Edition. North River Press, 1992

  • Certainly the best novel on project management ever written and probably one of the most sophisticated project management books. Based on constraint theory, it's a business school favorite.

  • You can make many process improvements, but just as in tuning a computing system, one local bottleneck is always controlling throughput. If you miss that bottleneck you are hosed.

  • Maximizing net profit will maintaining an excellent return on investment and adequate cash flow is the goal of a business, but this doesn't translate well to the programmer in the cube or the union worker on the factory floor. In a typical hospital, there is clearly no one responsible for maximizing throughput as bottlenecks exist everywhere.

  • Maximizing throughput is easily understood, but focusing on efficiency does not lead to optimizing the business. In fact, you can go out of business by focussing on efficiency, which the story of the book.

  • Focussing on maximizing (Throughput minus Inventory minus Operating Expense) does optimize the business. It seems to me that Throughput is software SOLD, Inventory is software built but UNSOLD, and Operating Expense is the usual expense budget.

  • I'm interested in any analysis or experience you may have with this book and the application of its concepts to software development.

  • Goldratt's home page
  • Sunday, January 19, 2003

    Architecture: A Blueprint for Introducing Disruptive Technology Into the Internet


    Innovative technology leaders are always preoccupied about the next big thing. To get a startup funded for building the next big thing means it must be at least 10 times better, or 10 times cheaper, and have a huge potential market. Better yet, it must disrupt the established way of doing business by distributing power and control in a new way that unseats established players and allows revenue to flow to the innovators.

    Java was a disruptive technology which Microsoft tried to crush because it threatened to unseat Windows. Napster was another major disruptive technology on the internet. The music industry crushed it, only to create a multi-headed monster worse than Napster. What is the next great disruptive technology on the Internet? Larry Peterson of Princeton and his colleagues at the University of Washington and Intel think it is enabling a distributed development environment for peer to peer applications that becomes a platform for moving these applications seamlessly into production. Most applications don't have the viral explosion of a Napster, so how do you build, sell, and deploy bread and butter applications that service a market niche? How do you set up the Internet so that typical applications are developed with an infrastructure that solves the performance, security, and usability problems surfaced by applications that use distributed resources on the internet?

    Peterson, Larry et al. A Blueprint for Introducing Disruptive Technology Into the Internet. Proceedings of the First ACM Workshop on Hot Topics in Networks (HotNets-I), Princeton, NJ, October 2002.

    It's not easy reading, but thinking a new thought is never easy. There are no established concepts and vocabulary to describe it.

    Background reading: I cited the classic text on disruptive techology on 22 April 1999 after the CEO of Harvard Business School Press faxed me a copy, saying it was a must read for technical leaders. There is a new edition, hot off the press this month:

    Christensen, Clayton M. The Innovator's Dilemma : The Revolutionary National Bestseller That Changed the Way We Do Business. HarperBusiness, 2003.

    The compelling point about this book is that it shows how the best managed companies will fail, in spite of doing the right thing every step of the way, in the face of a disruptive technology. The market dynamics are similar to the martial arts (Aikido or JuJitsu) where the more powerful opponent winds up flat on the mat because the weaker opponent uses the attackers energy to neutralize him. It makes Bill Gates and Andy Grove paranoid.

    Friday, January 17, 2003

    Java Wars: Sun vs. Microsoft


    Sun Microsystems won an injunction against Microsoft on 12 Dec 2002 and the court's opinion provides some interesting insight into Microsoft competitive tactics. See SUN MICROSYSTEMS, INC. v. MICROSOFT CORPORATION Civil No. JFM-02-2739. It is important for technical leaders to understand the details of competitive strategies in the marketplace because it directly impacts the rational selection of architectures and technologies.

    "Because of the competitive threat Java presented, Microsoft devised and implemented a strategy to “wrest control of Java away from Sun” and to “turn Java into just the latest, best way to write Windows applications.” (Pl.’s Ex. 21, 4/14/97 Slivka email to Gates.) In order to preserve Java’s cross-platform functionality, Sun required in all of its license and distribution agreements (including the one with Microsoft) that a licensee’s implementation of the Java platform meet a test suite demonstrating that the implementation was compatible with the core Java platform. Though pretending to embrace the goal of compatibility, Microsoft intentionally took various steps to defeat that goal.

    "First, Microsoft made unauthorized modifications to the core Java class libraries. Sun Microsystems, Inc. v. Microsoft Corp., 999 F. Supp. 1301, 1309-10 (N.D. Cal. 1998).

    "Second,its implementation Microsoft failed to include support for what is known as Java Native Interface, a technology designed to permit programs written in Java to draw upon the native code of an underlying operating system. (FOF 388-390, 84 F. Supp. 2d at 105-06; Sun Microsystems, Inc. v. Microsoft Corp., 21 F. Supp. 2d 1109, 1119-22 (N.D. Cal. 1998), vacated, 188 F.3d 1115 (9th Cir. 1999).

    "Third, Microsoft altered its developer tools and virtual machine to recognize Microsoft-specific keywords and compiler directives that would run only on Microsoft’s implementations. FOF 394, 84 F. Supp. 2d at 106-07; Sun Microsystems, 21 F. Supp. 2d at 1122-25. This had the effect (as Sun correctly characterizes it) of changing the Java language to create a dialect only Microsoft products could understand.

    "Fourth, Microsoft prevented developers from having ready access to a set of class libraries, useful for creating distributed computing applications, known as RMI (remote method 5 invocation). FOF 391-393, 84 F. Supp. 2d at 106. Instead, as found by Judge Jackson, “it buried the link in an obscure location and neglected to include an entry for it in the site’s index. Referring to RMI and any Java developers who might access Microsoft’s site looking for it, a Microsoft employee wrote to his approving manager, ‘They’ll have to stumble across it to know it’s there. . . . I’d say it’s pretty buried.’” FOF 392, 84 F. Supp. 2d at 106.

    "Fifth, Microsoft intentionally deceived developers into believing the software products they were developing with Microsoft tools were cross-platform. United States v. Microsoft Corp., 253 F.3d 34, 76-77 (D.C. Cir. 2001); see also FOF 395-403, 84 F. Supp. 2d at 107-09 (explaining how Microsoft induced developers to use the Microsoft implementation rather than the Sun-compliant implementation).

    "The purpose of the strategy devised and implemented by Microsoft was described in its internal documents. A November 1996 email stated: “[W]e should just quietly grow j++ [Microsoft’s incompatible developer tools] share and assume that people will take more advantage of our classes without ever realizing they are building win32-only java apps.” FOF 394, 84 F. Supp. 2d at 107. Another document was even more succinct: “Kill cross-platform Java by grow[ing] the polluted Java market.” Microsoft, 253 F.3d at 76-77 (quoting Government Ex. 259).

    "While thus deliberately fragmenting the Java platform to make it less attractive for developers and users, Microsoft also successfully embarked upon a campaign to destroy Sun’s channels of distribution for Java. In May 1995, Netscape Corporation “agreed . . . to include a copy of Sun’s Java runtime environment with every copy of Navigator [Netscape’s web browser], and Navigator quickly became the principal vehicle by which Sun placed copies of its Java runtime environment on the PC systems of Windows users.” FOF 76, 84 F. Supp. 2d at 30. In order to curtail this line of distribution, Microsoft “undertook a number of anticompetitive actions that seriously impeded distribution of Navigator.” Microsoft, 253 F.3d at 75. Microsoft also entered into “First Wave Agreements” with dozens of independent software vendors that “conditioned receipt of Windows technical information upon the ISVs’ agreement to promote Microsoft’s JVM [Java Virtual Machine] exclusively.” Id. Finally, Microsoft successfully stopped Intel Corporation from cooperating with Sun and Netscape in the development of a cross-platform Java runtime environment by threatening not to distribute Intel technologies bundled with Windows and to support one of Intel’s competitors in connection with one of its products unless Intel ceased its support for Java."

    It's too bad Microsoft could not live up to the vision expressed in my 1996 interview with Microsoft executives:

    Sutherland, J. Microsoft and the Internet Wars: Freedom Fighters. Homepage Journal, Mar 1996.

    Friday, January 10, 2003

    Network Meltdown: Your Company May be Next!


    When IT fails: Q&A with John Halamka, CIO, CareGroup Healthcare System
    by Caroline Broder, iHealthBeat editor, January 9, 2003

    When a network outage at Beth Israel Deaconess Medical Center brought the hospital’s network to a grinding halt for four days in November, workers who once relied on computers to access e-mail, lab results, prescription orders, electronic medical records and billing information were forced to return to the days of paper.

    A researcher who had accidentally dumped a large volume of data into the network caused it to flood the system and crash. But John Halamka, CIO Harvard Medical School and Boston-based CareGroup Healthcare System, said the real problem was Beth Israel’s network. Beth Israel, which is one of CareGroup’s hospitals, has a network that was designed piecemeal and had been patched together when Beth Israel and Deaconess Hospitals merged in 1996. Halamka, who has posted a Web site that explains the technical details of the network’s failure, said the system also relied on old data protocols that couldn’t handle the medical center’s data needs...

    iHB: What has been the reaction from other hospitals after this incident? I would guess something like this might have happened somewhere before, but wasn’t reported.

    Halamka: It’s happened hundreds of times before. Twenty of the Fortune 100 companies have called me and told me they had the same thing happen. I’ve been called by a vast number of government agencies, and the same thing has happened to them. No one has talked about this before. They see it as a short coming. Before this happened, most CIOs were not aware that their network was at risk.

    Check out Halamka's web site for technical details. You can also listen to NPR coverage of the event.

    Technology Adoption: Tablet PCs


    Technical leaders are wondering what to do about the Tablet PC. My recommendation has been to make web pages work on them but no more. Massive adoption is not in the cards. This means that they will remain an expensive niche device. Scott Weiss has written a nice article that summarizes the issues. For his list of the Microsofts Top 10 Falsehoods, see:

    Weiss, Scott. Tablet PCs: The Latest Flop. handheldusability.com, Jan 2003.

    Key Arguments
    1. The new tablets do not recognize handwriting well. Microsoft has completely backpedaled from handwriting recognition by providing offline, background handwriting processing. Not a great way to launch a pen-based platform.

    2. The new tablets are expensive. The Tablet PC retails at nearly $3,000, about $1,000 more than a comparably equipped notebook computer running Windows XP. A well-equipped Pocket PC unit retails at $500, so the combination of the two products yields more convenience for less money.

    3. The new tablets are heavy, unwieldy, and fragile. Tablet PC units weigh at least three pounds. Pocket PC units weigh in at about one-half pound. Tablet PC units cannot be dropped without breaking them, but there are hardened PDAs that can be dropped without damage. Symbol makes many such units.

    4. Software on the Windows XP platform is not optimized for handheld use. If you have used Windows XP, you know how much configuration and pain is necessary. Handheld devices have zero boot time, launch applications flawlessly, do not require direct “Save” actions, and have uncomplicated file systems. They are also optimized for use with a stylus or directional buttons. Tablet PC is simply Windows XP without the mouse and the keyboard, but with a stylus instead. XP software is less responsive and requires more direct interaction than software designed for a PDA, and so the user spends a lot of time writing and tapping rather than getting his or her work done.

    Wednesday, January 08, 2003

    SCRUM: Saving the Product While Downsizing by 96%


    Mike Cohn has written a very cool article on how he used SCRUM to assure product survival and a company sale with 12 people after laying off 88. SCRUM works best under pressure!

    Cohn, Mike. From the Front Line: The Upside of Downsizing. STQE Magazine 5:1:18-21, Jan/Feb 2003.

    Thursday, January 02, 2003

    Grid Computing: New Book on the Web


    Bob Marcus sent out a New Year's message on his new:

    GREAT GLOBAL GRID BOOK

    Happy New Year! This message is being sent out to a mailing list on Emerging Internet Technology (EMIT). The list was inactive in 2002 while I took a sabbatical to write a book on enterprise applications of the next generation Internet. The book is entitled "Great Global Grid: Emerging Technology Strategies". A draft version, contents, references and description are available at http://www.great-global-grid.com . The book is an outgrowth of the 2001 Software Services Grid Workshop whose proceedings are available at http://swradio.omg.org/workshop/

    The technologies covered in the book include portals, EAI and B2B messaging, Web Services, peer-to-peer, wireless, virtual clusters and Grid middleware. These technologies are analyzed from a pragmatic enterprise viewpoint including their future impact.

    The next step for the Great Global Grid will be beyond the enterprise. My current work as a senior research engineer at SRI (http://www.sri.com) is creating a middleware architecture for mobile ad hoc networks. This infrastructure includes agent-mediated peer-to-peer communication with links to multiple resource-sharing grids.

    I plan to send out periodic mailings on technical issues and events involving all aspects of the Great Global Grid during 2003. Anyone who is interested in subscribing can send me a message with subject "Subscribe GGG" at bobmarcus1@attbi.com

    SCRUM: Agile database development


    Fowler, Martin and Sadalage, Pramod. Evolutionary Database Design

    Abstract:
    Over the last few years we've developed a number of techniques that allow a database design to evolve as an application develops. This is a very important capability for agile methodologies. The techniques rely on applying continuous integration and automated refactoring to database development, together with a close collaboration between DBAs and application developers. The techniques work in both pre-production and released systems.

    Wednesday, January 01, 2003

    Project Management: What is the right ratio of testers to developers?


    Managing the Proportion of Testers to (Other) Developers
    By Cem Kaner/Elisabeth Hendrickson/Jennifer Smith-Brock

    Summary: One of the common test management questions is what is the right ratio of testers to other developers. Perhaps a credible benchmark number can provide convenience and bargaining power to the test manager working with an executive who has uninformed ideas about testing or whose objective is to spend the minimum necessary to conform to an industry standard. The authors focused on staffing ratios and related issues for two days at the Fall 2000 meeting of the Software Test Managers Roundtable. This paper is a report of their thinking. (Editors note: Click here to read another paper on this topic, by Kathy Iberle and Sue Bartlett.)