This post resumes coverage from my notes on the Java Posse Roundup, 2008 conference.
Agile methodologies on large projects
The conventional wisdom is that agile development is a game for small teams. This session examined that issue, informed by the experience of attendees who have worked on large development projects.
Getting started on large projects
The discussion began with some opinions and suggestions which apply to getting started with agile methodologies:
- Agile works best (only?) if it is driven from the bottom up.
- Refactoring requires permission and inclination:
- Individual developers must be allowed to make pervasive changes to the codebase.
- Individual developers must be willing to follow the consequences of a change through to the end.
- Lead by example.
- Start with a pilot project with a small number of developers.
- Use the success of that project as a “poster child” to the rest of the organization.
- Modularity is a key for success.
A couple of the participants had been involved in a large, distributed project. The team comprised 500 developers (in a company which had a total of 1200 developers) located in India, Israel, the Bay area, Houston, and Europe. They were able implement continuous integration across that size and geographic distribution.
Agile is stereotypically described in terms of small teams. What is “large”?
- Small: one team
- Medium: 2–3 teams
- Large: more than 3 teams
One effort discussed had a limit of 6 people per team.
How does a project achieve modularity and team structure? One choice is to spend an iteration on defining the modular structure. Some agile methodologies emphasize delivering working, business-value code in each iteration. But kick-starting a multi-team effort may require using an iteration to get structure (of the modules and the teams!) in place.
However, there can be built-in tension with a layered effort that separates “framework” from “application”. The framework team(s) are naturally going to have a bias in favor of flexibility, to make the framework as general as possible. The application team(s) are naturally going to have a bias in favor of stability, to minimize the rework of application code due to framework evolution.
I think we can generalize this to any agile multi-team scenario where there’s a common interface or (especially) a producer-consumer relationship. It’s crucial to maximize communication and minimize “us-versus-them” thinking. Otherwise it will be natural to see “my team’s changes” as natural or necessary agile evolution, versus “your team’s changes” as “churn”.
There was some discussion of the roles of “gatekeeper” and “contributor”, and the separation of those roles as a way to help achieve stability. However, needs to be balanced by strong advocacy for the view that “anybody can fix the code” if the overall effort is to remain agile.
Engaging the business owner
The discussion turned to the non-developer aspects of an agile project. Getting a product owner may be much harder than getting buy-in from the technical side. There may be a greater need for education, as developers may be more likely to be familiar with the concepts of agile projects. However, the pain is still less than that of being stuck in waterfall mode. The consensus was to get the business side involved as early and thoroughly as possible.
There’s another contrast here. Because business value is achieved by integrating the total effort, the business owner for a large project is likely to fit most naturally at a cross-group level. However, agility demands level-of-effort to be understood at a fairly fine-grained level. Mike Cohn’s books on agile were highly recommended. I’ve put links to a couple of his books at the end.
Estimating and scoping
Estimates are critical to the communication from developers to the product owner and to planning (at all levels). How does the team come up with estimates level of effort?
Most of the participants leaned toward estimating in “points” rather than hours or other calendar units. One common scheme is to assess points along a Fibonacci scale: 1, 2, 3, 5, 8, 13, 21, etc. What’s the difference between a 5 and an 8? It’s simply the developers’ best-effort assessment of relative effort based on complexity. Product backlog is measured in points, and the sprint backlog is extracted from that. The planning conversation addresses the questions:
- How long is a sprint?
- What is our velocity (points per sprint)?
- How many sprints will it take for this feature?
The focus on continuous learning, communication, and adjustment makes this a very different conversation from the stereotypical “Gannt chart mentality” that just wants to put milestones on a calendar, set in concrete. Priorities can adjust, based on experience earned during the project.
Barry Hawkins pointed out that an important concept to keep in front of the product owner is: “You can have anything you want, but you can’t have everything you want!” It’s also critical to communicate the concept of “technical debt” to the product owner. (There are some good on-line discussions of technical debt which provide a comprehensive discussion, a quick, business-oriented metaphor, some nice visual aids, and reminders of the development process aspects.) Participants also referenced Mary Poppendiek’s work on lean software.
Barry also offered the view that agile methodologies don’t address some issues that become important on larger projects, including the cross-feature dependencies and how to partition functionality. He suggested domain-driven design as a good fit with agile development practices as a way to address these issues.
As the session was wrapping up, Barry offered a summary that seems applicable to any introduction of agile practice. He has observed that politics only diminished after the entire team (product owner, development, etc.) had a backlog of experience that showed the productivity benefits of the agile approach.
- User Stories Applied: For Agile Software Development (The Addison-Wesley Signature Series)
- Agile Estimating and Planning (Robert C. Martin Series)
- Lean Software Development: An Agile Toolkit (The Agile Software Development Series)
- Implementing Lean Software Development: From Concept to Cash (The Addison-Wesley Signature Series)