Fear and Testing

I’m encouraged and excited by what I see coming from user-led conferences. I’ve had great first-hand experiences at the Java Posse Roundup (rumor is February 21-25 this coming year), No Fluff Just Stuff, and StrangeLoop (2010); and I’ve really benefitted from materials published on-line by other conferences, such as Devoxx and—a recent discovery—JRubyConf.

I was particularly impressed and inspired by Must. Try. Harder. (also linked from here and here), a presentation by Keavy McMinn, who described her experiences training for Ironman. Part of Keavy’s first blog post on Ironman Cuzumel resonated strongly with something I once saw in some code, hence this article.

By way of background (with vagueness required by confidentiality considerations), I remembered looking at test code wrapped around a fairly subtle bit of business logic that had passed through multiple hands. I believed that I understood the intentions of the original designer, and wanted to confirm or correct that belief. All of which sounds like a perfect fit for test cases, right?

Hmmmmm.

The majority of the test cases dealt with various ways that something could go wrong in configuring the targeted business component. Additional tests dealt with incorrect inputs to a correctly-configured instance, and only a small fraction actually dealt with the behavior of a correctly-configured instance. In fact, there were no tests that answered the particular questions that I had in mind.

How can this be? (And what does this have to do with Keavy’s Ironman experience?)

About half-way down in her Ironman Cozumel blog entry, there’s a section entitled “Fear”. I thought about quoting a key sentence or two, but didn’t want to interfere with her well-crafted flow. So just take a minute and read it (at least—for now—the section under her “Fear” heading, between the two photos).

(Wow! You read quickly! 😉 )

So, as I looked at the test cases, a pattern began to emerge. The largest portion of the tests seemed to orbit around a particular aspect of the configuration that I took for granted, giving that configuration concept far more than its share of attention, IMHO. Which got me to thinking about similar patterns in other test cases. Which prepared me to have an “<em.Aha!</em.” moment when I encountered Keavy’s description of fear of the ocean.

Fear

Unbalanced tests that over-emphasize some issues and under-emphasize others may be a subtle hint that fear is influencing the work. I’ve observed two common responses to fear in myself and others: obsession and avoidance. On reflection, I believe that those attitudes show up in tests that I’ve looked at over the years.

Obsession

We may joke about it, but I’ll admit to going back a few times to make sure the coffee pot is off, jiggling the knob one more time to verify that I really did lock the door, or repeatedly checking my pocket to reassure myself that my airline tickets didn’t jump out and hide under the couch. (OK, forget the “hide under the couch” part.)

In this light, I see the set of test cases mentioned above as being obsessed with a particular configuration issue. Those tests were me, patting my pocket again, trying the door again, pushing the switch again (even though I can clearly see that the light is <em.not</em. glowing…) If I find that my tests appear to be obsessing over a particular aspect of the task at hand, that might be a symptom of fear—a need for comfort and reassurance in the face of of an unfamiliar domain concept, a language feature or API with which I’m uncomfortable, or a requirement that I don’t really understand. Or of moving onto the next use case, which brings me to…

Avoidance

Inadequate attention to testing an aspect of the design may indicate a reluctance to engage fully with the issue. In that connection, Keavy’s description of initial hesitation at the water’s edge brought back a vivid memory for me.

My family loves camping in the Great Smoky Mountains National Park. A popular spot along the Little River Road called “The Sinks” features a waterfall almost directly under a bridge, followed by a long, deep swimming hole with steep rocky banks on both sides. Visitors can climb to a ledge which offers a clear jump into a deep part of the pool. Although that ledge is only about 12 feet above the water level, it looks much higher to a first-time jumper. I don’t laugh at young (or not-so-young) first-timers who hesitate on that ledge, because I still remember my first leap.

Even though my head knew it must be safe—I had seen many people take the plunge before me that day—my stomach and legs hadn’t yet gotten the message. I “paused” for several long moments before stepping into the air.

(I’m not talking about truly high-risk activity here, I should point out. A friend of mine has a truly terrifying story about jumping from height into unfamiliar—and unsafe—water and impaling his foot. So let’s keep it in the pool and well-known swimming holes, kids. And don’t forget your buddy.)

Of course, we developers <em.never</em. procrastinate in the face of a fear-inspiring task! Then again, there’s that bit of legacy code that nobody wants to maintain, much less rewrite. And the lingering suspicion that “starting with the low-hanging fruit” can turn into an excuse to put off the parts we’re secretly fearing.

So, how do I fight back, when an imbalance in my tests shows evidence of fear, either positively (obsession) or negatively (avoidance)?

Suggested antidotes…

First, I should consider whether the test really are over- or under-emphasizing something. Different aspects of the code have different levels of risk and/or consequences, so I don’t expect absolutely even distribution of attention. If there is a true imbalance—even better, if I’m about to create one—I need to recognize that and apply an appropriate antidote.

…to obsession

When I start bogging down, these questions can be useful:

Is this test about the product or about me?

Let me tip my hat to Ruby Koans, which gives beautiful evidence of the power of testing as a way to explore a language or framework. If a test can confirm (or correct) my understanding of something I’m using in this project, I certainly should write it! But I’m equally certain that I should keep it outside my project’s code base.

What does/will this test teach me?

If it conveys useful new information, or affects the code in a material way, fine. But if it’s just a trivial variation on a theme, perhaps I should move on. If I’m wrong, I can always come back and add a test later.

How likely is this?

If this test is exercising a scenario that would almost certainly never occur in normal use, maybe I should take care of the more-likely cases first.

How bad can it be?

Do I really need this last little bit of yak hair?

…to avoidance

Keavy wrote about overcoming by focusing on technique, process, and details under our control. My high-school marching band director kept taking us back to the basics, with scales on our instruments and eight-to-five marching drills until our feet could hit the chalk line precisely, with no last-minute stretches or stutters. In fact, without even looking. For this side of my fears, here are some questions…

What do I not know?

If I can express it as a test, I can get moving.

How would I explain this to ___?

I have gotten past mental blocks by explaining my impasse to a friendly non-programmer. Being clear and non-technical often has amazing benefits! 😉 Fill in the blank with a name of your choice. Sometimes only imagining the conversation is enough. It also helps to imagine it out loud (but not where I could frighten or distract my colleagues!)

Am I trying to swallow an elephant?

The last time I got stalled out, I let circumstances outside the task at hand stampede me. I blurted out a train of thought into code and quickly ended up, as I once heard someone say, “crocheting in logic space“. Admitting that fact to myself allowed me to back out of my self-imposed impasse.

…and a couple of challenges:

Don’t just sit there, DO SOMETHING!

Sometimes even a stumbling start is better than stalling, because it gets me moving. As with explaining my problem to a non-programmer, thinking out loud in code may help me get a better perspective (even if I end up throwing away that first bit).

Don’t just do something, SIT THERE!

In my best ersatz-mystico-philosophical style, sometimes I need to do just the opposite. Stop fidgeting and pacing back and forth. Sit down. Close my eyes. Be still. Take a breath. Think a moment. In that jam there’s probably one log that I can shift a little, and it’s probably not the one I’ve been tugging on unsuccessfully for way too long. Now open my eyes. Look around not in the same place. Oh. Duh. There it is. 

Let me get back to you. I’ve got a test to write.

The value of persistence

After beginning this post, I learned that Keavy finished in Cozumel! Congratulations!

---

Code koans:

• in Java-Script;
• in Scala;
• I heard from Dianne Marsh that CodeMash featured work on koans last year; they may do so again this year.

I’m sure there are many more; please let me know.

---

Design by proof

The idea of proving programs correct has been around (and hotly debated) for roughly forty years. The irony is that some proof advocates anticipated by more than thirty years what the agile advocates are now saying:

Designing with verification in mind improves the quality of the resulting design.

To be even more specific, compare these statements:

Design the proof, then write the code so that it meets the obligations of the proof.		Write the test first, then write the code so that it passes the test.
—Edsger Dijkstra   (paraphrased)		—Kent Beck   (paraphrased)

I’ve experienced the benefits of following both pieces of advice. Let me apply this approach to a little bit of design which I’ll then use in the Project Euler series. There’s a small amount of algebra involved, but if you don’t want to bother with that, let me request that you at least look at the last section for the punchline.

Sum thing

Consider the following sums, where i in each case ranges from 1 to n:

Power	Sum	Expanded form	Closed form	Factored form
0	∑ i0	1 + 1 + 1 + … + 1	n1	n
1	∑ i1	1 + 2 + 3 + … + n	1/2 n2 + 1/2 n1	n(n+1)/2
2	∑ i2	12 + 22 + 32 + … + n2	?
3	∑ i3	13 + 23 + 33 + … + n3	?

Let’s assume that we need the solutions for the last two rows (and Google is down at the moment 😉 ). Using the “designing with verification in mind” approach, we can work them out for ourselves.

Sum of squares

Noticing that ∑ i⁰ is a polynomial of degree 1, and ∑ i¹ is a polynomial of degree 2, we might suspect that the sum of i² is a polynomial of degree 3. If so, the problem amounts to finding the coefficients for ∑ i² = an³ + bn² + cn + d. Let’s assume that as the structure of the solution, and apply test cases to that assumption.

Test case 0: The sum must be zero when n is zero; a0³ + b0² + c0 + d = d, therefore d must be zero. That streamlines the polynomial to ∑ i² = an³ + bn² + cn.

Test case 1: Adding (n+1)² to the sum for n must be equal to the result of substituting (n+1) for n in the polynomial.

	an3	+ bn2	+ cn	+ (n+1)2
=	an3	+ (b+1)n2	+ (c+2)n	+ 1
and
	a(n+1)3	+ b(n+1)2	+ c(n+1)
=	an3	+ (3a+b)n2	+ (3a+2b+c)n	+(a+b+c)

These two reduced expressions can only be equal if we can satisfy the equations implied by the coefficients for n², n¹, and n⁰:

n?		Coefficients				Satisfied if
n2		b+1	=	3a+b		a	=	1/3
n1		c+2	=	3a+2b+c		b	=	1/2
n0		1	=	a+b+c		c	=	1/6

So we have a solution of ¹/₃ n³ + ¹/₂ n² + ¹/₆ n, which we can factor to n(n+1)(2n+1)/6.

Sum of cubes

Based on that success, we’ll pursue the coefficients for ∑ i³ = an⁴ + bn³ + cn² + dn + e.

Test case 0: The sum must be zero when n is zero; a0⁴ + b0³ + c0² + d0 + e = e, therefore e must be zero, therefore ∑ i³ = an⁴ + bn³ + cn² + dn.

Test case 1: Adding (n+1)² to the sum for n must be equal to the result of substituting (n+1) for n in the polynomial.

	an4	+ bn3	+ cn2	+ dn	+ (n+1)3
=	an4	+ (b+1)n3	+ (c+3)n2	+ (d+3)n	+ 1
and
	a(n+1)4	+ b(n+1)3	+ c(n+1)2	+ d(n+1)
=	an4	+ (4a+b)n3	+ (6a+3b+c)n2	+ (4a+3b+2c+d)n	+ (a+b+c+d)

Again, we look at the coefficients for descending powers of n:

n?		Coefficients				Satisfied if
n3		b+1	=	4a+b		a	=	1/4
n2		c+3	=	6a+3b+c		b	=	1/2
n1		d+3	=	4a+3b+2c+d		c	=	1/4
n0		1	=	a+b+c+d		d	=	0

So we have a solution of ¹/₄ n⁴ + ¹/₂ n³ + ¹/₄ n², which we can factor to (n(n+1)/2)².

The punchline

“That’s no exponential, that’s my polynomial!” But seriously, folks…

The mathematicians among us–those who haven’t died of boredom–will recognize what we’ve done as an informal (hand-waving) inductive proof.

But I hope that the programmers among us will recognize what we’ve done as test-driven development and (re)factoring. Allowing verification to drive design offers a number of benefits, including:

• The result itself is verifiable.
• Each step along the way is more obvious and involves less risk.
• The result is usually cleaner and more understandable.

Incidentally, I first encountered the term “factoring” applied to programs in Leo Brodie‘s book Thinking Forth (1984). Given my background in Mathematics, it immediately clicked with me. Brodie introduced the idea with a quotation which seems entirely contemporary, once we get past the dated terminology:

“If a module seems almost, but not quite, useful from a second place in the system, try to identify and isolate the useful subfunction. The remainder of the module might be incorporated in its original caller.”

(The quotation is from “Structured Design“, by W.P. Stevens, G.J. Myers, and L.L. Constantine, IBM Systems Journal, vol. 13, no. 2, 1974, © 1974 by International Business Machines Corporation)

As a working developer, I’m excited by the potential of these techniques and the underlying ideas. But as a card-carrying member of the Sid Dabster fan club, I’m constantly amazed at the number of ideas that were actively in play in our field thirty or forty years ago that are now regarded as new, innovative, or even radical.

---

Recommended reading:

• Test Driven Development: By Example (The Addison-Wesley Signature Series)
• Test Driven Development: By Example (The Addison-Wesley Signature Series)
• Selected Writings on Computing: A Personal Perspective (Texts and Monographs in Computer Science)
• On the Shape of Mathematical Arguments (Lecture Notes in Computer Science)
• Thinking Forth

JPR 2008: Wednesday morning, part 1

This post resumes coverage from my notes on the Java Posse Roundup, 2008 conference.

Agile methodologies on large projects

The conventional wisdom is that agile development is a game for small teams. This session examined that issue, informed by the experience of attendees who have worked on large development projects.

Getting started on large projects

The discussion began with some opinions and suggestions which apply to getting started with agile methodologies:

• Agile works best (only?) if it is driven from the bottom up.
• Refactoring requires permission and inclination:
  • Individual developers must be allowed to make pervasive changes to the codebase.
  • Individual developers must be willing to follow the consequences of a change through to the end.
• Lead by example.
  • Start with a pilot project with a small number of developers.
  • Use the success of that project as a “poster child” to the rest of the organization.
• Modularity is a key for success.

A couple of the participants had been involved in a large, distributed project. The team comprised 500 developers (in a company which had a total of 1200 developers) located in India, Israel, the Bay area, Houston, and Europe. They were able implement continuous integration across that size and geographic distribution.

Agile is stereotypically described in terms of small teams. What is “large”?

• Small: one team
• Medium: 2–3 teams
• Large: more than 3 teams

One effort discussed had a limit of 6 people per team.

How does a project achieve modularity and team structure? One choice is to spend an iteration on defining the modular structure. Some agile methodologies emphasize delivering working, business-value code in each iteration. But kick-starting a multi-team effort may require using an iteration to get structure (of the modules and the teams!) in place.

However, there can be built-in tension with a layered effort that separates “framework” from “application”. The framework team(s) are naturally going to have a bias in favor of flexibility, to make the framework as general as possible. The application team(s) are naturally going to have a bias in favor of stability, to minimize the rework of application code due to framework evolution.

I think we can generalize this to any agile multi-team scenario where there’s a common interface or (especially) a producer-consumer relationship. It’s crucial to maximize communication and minimize “us-versus-them” thinking. Otherwise it will be natural to see “my team’s changes” as natural or necessary agile evolution, versus “your team’s changes” as “churn”.

There was some discussion of the roles of “gatekeeper” and “contributor”, and the separation of those roles as a way to help achieve stability. However, needs to be balanced by strong advocacy for the view that “anybody can fix the code” if the overall effort is to remain agile.

Engaging the business owner

The discussion turned to the non-developer aspects of an agile project. Getting a product owner may be much harder than getting buy-in from the technical side. There may be a greater need for education, as developers may be more likely to be familiar with the concepts of agile projects. However, the pain is still less than that of being stuck in waterfall mode. The consensus was to get the business side involved as early and thoroughly as possible.

There’s another contrast here. Because business value is achieved by integrating the total effort, the business owner for a large project is likely to fit most naturally at a cross-group level. However, agility demands level-of-effort to be understood at a fairly fine-grained level. Mike Cohn’s books on agile were highly recommended. I’ve put links to a couple of his books at the end.

Estimating and scoping

Estimates are critical to the communication from developers to the product owner and to planning (at all levels). How does the team come up with estimates level of effort?

Most of the participants leaned toward estimating in “points” rather than hours or other calendar units. One common scheme is to assess points along a Fibonacci scale: 1, 2, 3, 5, 8, 13, 21, etc. What’s the difference between a 5 and an 8? It’s simply the developers’ best-effort assessment of relative effort based on complexity. Product backlog is measured in points, and the sprint backlog is extracted from that. The planning conversation addresses the questions:

• How long is a sprint?
• What is our velocity (points per sprint)?
• How many sprints will it take for this feature?

The focus on continuous learning, communication, and adjustment makes this a very different conversation from the stereotypical “Gannt chart mentality” that just wants to put milestones on a calendar, set in concrete. Priorities can adjust, based on experience earned during the project.

Barry Hawkins pointed out that an important concept to keep in front of the product owner is: “You can have anything you want, but you can’t have everything you want!” It’s also critical to communicate the concept of “technical debt” to the product owner. (There are some good on-line discussions of technical debt which provide a comprehensive discussion, a quick, business-oriented metaphor, some nice visual aids, and reminders of the development process aspects.) Participants also referenced Mary Poppendiek’s work on lean software.

Barry also offered the view that agile methodologies don’t address some issues that become important on larger projects, including the cross-feature dependencies and how to partition functionality. He suggested domain-driven design as a good fit with agile development practices as a way to address these issues.

As the session was wrapping up, Barry offered a summary that seems applicable to any introduction of agile practice. He has observed that politics only diminished after the entire team (product owner, development, etc.) had a backlog of experience that showed the productivity benefits of the agile approach.

---

Recommended reading

• User Stories Applied: For Agile Software Development (The Addison-Wesley Signature Series)
• Agile Estimating and Planning (Robert C. Martin Series)

• Lean Software Development: An Agile Toolkit (The Agile Software Development Series)
• Implementing Lean Software Development: From Concept to Cash (The Addison-Wesley Signature Series)

• Domain-Driven Design: Tackling Complexity in the Heart of Software