Why would I simply summarise someone else’s article on this site? Isn’t that dishonest? Not in this case: Miško wrote essentially the same thing that I say over and over again in my own articles and in my training classes. I consider it unfortunate that he and I have never met nor worked together. We really should. You should read his stuff.

So… 10 ways to kill your design.

1. Instantiate Services wherever the hell you want.
2. Grab what you want when you want it.
3. Do real work in your constructors.
4. Publicise state.
5. Embrace singletons.
6. Make methods static. (Admittedly, this is more of a problem in Java/C# than Ruby/Python.)
7. Inherit implementation.
8. Reimplement polymorphism.
9. Mix Services and Values.
10. Keep conjunctions in your variable, method and class names.

Read more about the problems that these “techniques” cause.

When multiple assertions check very tightly related things, I don’t mind them, but when they check relatively loosely related things, they act as integrated tests for multiple behaviors that we should consider separating. This is even subtler than the simpler idea of “one action per test”.

If you’d like a Novice algorithm to follow:

1. Look for any test with multiple assertions.
2. Move those assertions to the bottom of the test. (If they aren’t already at the bottom, then you might have more than one action per test; this refactoring will help you discover that.)
3. Extract all the assertions together into a single method.

Now look at the new method. How many different objects does the method use?

If it’s more than one, then you almost certainly have unrelated assertions in the same place, so consider splitting the unrelated assertions into separate methods, then split the test into two so that each test invokes one of the two new separated assertion methods.

If your new assertion method uses only one object, then it might not be so clear whether those assertions are related. You can try this simple test: put all the values you’re checking in your assertions into a single object. Can you think of a good name for it? If yes, then perhaps you’ve just identified a missing abstraction in your system; and if not, then perhaps the assertions have too little to do with each other, in which case, try the trick in the preceding paragraph.

I apologise for not having a good example of this right now. If you point me to one, I’ll analyse it in this space.

• Where did that yak come from?
• When you try to learn a new library at the same time as explore the behavior and design of your application, you slow down more than you think.
• When you can’t figure out how to make the new library work for this thing you want to build, you might spend hours fighting, debugging, swearing.

Stop. Write a Learning Test.

1. Start a new test suite, test class, spec file, whatever you want to call it.
2. Write a test that checks the things you tried to check earlier with debug statements.
3. Write a test that has nothing to do with your application and its domain.
4. Remove unnecessary details from your test.

When this test passes, then you understand what that part of the library does. If it behaves strangely, then you have the perfect test to send to the maintainers of the library.¹

The Details

I just did this on a project using the context-free grammar parser treetop. Of course, I hadn’t used treetop before, so I had to learn it at the same time as design the grammar for the language I wanted to parse. I reached the point where I couldn’t write a grammar rule correctly, and spent probably an hour trying to figure out get it to work.² Fortunately, at that moment, my laptop ran out of power, so I left the coffee shop³ and did the usual thing: I explained the problem to my wife so that I could hear myself doing that. After about 15 minutes away from the problem, I decided to write some Learning Tests.

Summary of what I did

1. I wrote a Learning Test for a simple case that I thought I already understood well.
2. I wrote a Learning Test similar to the problem I had to deal with, to make sure I understood that well.
3. I wrote a Learning Test for the exact case that behaved unexpectedly.

The whole thing took an hour, and I understood the problem well enough to explain it to my wife. She understood it and agreed that it sounded like a mistake in the library.⁴ I used this Learning Test to open an issue at github. Now I can proceed without pulling my own hair out.

Do you want to see the Learning Tests?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

require 'treetop'

describe "Grammar with a simple rule" do
  let(:subject) { Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRule
  rule word
    [A-Za-z]+
  end
end
GRAMMAR
  )}

  let (:parser) { subject.new }

  it "doesn't match empty string" do
    parser.parse("").should be_false
  end

  context "matching single letter, the match result" do
    let(:result) { parser.parse("a") }

    it { result.should be_true }
    it { result.text_value.should == "a" }
    it { result.to_s.should_not == "a" }
    it { result.should_not respond_to(:word) }
  end

  context "matching many letters, the match result" do
    let(:result) { parser.parse("aBcDeF") }

    it { result.should be_true }
    it { result.text_value.should == "aBcDeF" }
    it { result.to_s.should_not == "aBcDeF" }
    it { result.should_not respond_to(:word) }
  end
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

require 'treetop'

describe "Grammar with a simple rule that uses a label" do
  context "Labeled subexpression followed by another expression" do
    let(:subject) { Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRuleWithLabel
  rule word
    letters:[A-Za-z]+ [A-Za-z]*
  end
end
GRAMMAR
    )}

    let (:parser) { subject.new }

    context "matching many letters, the match result" do
      let(:result) { parser.parse("aBcDeF") }

      it { result.should respond_to(:letters) }
      it { result.letters.text_value.should == "aBcDeF" }
    end
  end

  context "Labeled subexpression without another expression" do
    it "does not represent a valid grammar, even though I think it should" do
      lambda {
        Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRuleWithLabel
  rule word
    letters:[A-Za-z]+
  end
end
GRAMMAR
      )}.should raise_error(RuntimeError, /Expected \#/)
    end

    it "really should let me refer to the expression as #letters" do
      pending "https://github.com/nathansobo/treetop/issues/21"
    end
  end
end

• Invert the dependency on a Service, moving the new statement up one level in the call stack.¹
• Repeat for all dependencies on Services until the corresponding new statements arrive at your application’s entry point. The entry point now creates a large object graph of all your Services in its initialise function.
• Remove duplication in EntryPoint.initialise():
  • Instantiate common objects only once, passing them into the necessary constructors, replacing any Singletons with plain objects.
  • Extract the choice of implementation for each Service interface into a lookup table mapping interface type to implementation type.
  • Externalise the lookup table to a file, if you like.

Now you have a customised Dependency Injection Container for your application. To go a little farther:

• Remove duplication in EntryPoint.initialise() among three applications.

Now you have a generic Dependency Injection Container that probably provides 80% or more of the features you’ll ever need.

I recommend trying this incrementally. Think of the new statements flowing up the call stack, into the entry point, then changing from code into data. Nice, no?

The Details

I don’t have much to add.

I hope this helps to demystify dependency injection containers. To read about the technique of injecting dependencies, I refer you to one of my articles and then a trusty web search.

This technique applies the Dependency Inversion Principle repeatedly to move the choice of implementation for an interface up the call stack. This way, concrete things depend on abstract things.

Removing duplication in the entry point respects the principle Abstractions in Code, Details in Data, but it does rely on reflection, which can cause some problems. All the better not to scatter this reflection throughout the code causing a serious cohesion problem. Using reflection like this, all in one place, helps balance using a powerful technique with a design that everyone can understand.

Now you know what about 70% of what a dependency injection container does. You can build one. Even if you don’t go that far, the Dependency Inversion Principle will help reducing coupling and highlight cohesion problems in your design. It provides another set of mechanics to practise on the way to becoming an accomplished designer.

No doubt you already know about the Composed Method design pattern, which we commonly use to help understand code as we read it. Most commonly you encounter a block of code with a comment that describes what that code does. You then extract that block of code into a method whose name matches the comment. I’ve used this technique for well on a decade to raise the level of abstraction in code, help code document itself, and eliminate misleading comments. While introducing these methods helps me read the code, it sometimes hides the tangle of dependencies that makes separating responsibilities so difficult. For this reason, I recommend trying to extract pure functions instead.

To introduce a pure function, start with a block of code and extract a method for it. Now look at all the fields and global variables that the method reads and introduce each one as a parameter to the method. Now look at all the fields and global variables that the method writes to and turn these into return values. Where the old code invokes the new function, assign each new return value to the corresponding field or global variable. You know you’ve done this correctly if, of course, the overall behhavior of the program hasn’t changed and you can mark the new function as static or whatever your language calls a class-level function.

In some languages, like Java, you’ll have to introduce a new little class to allow you to return multiple values from the new function. If you don’t want to do that right away, then return a Map of return values. Once you see that you need to return similar Maps from different functions, consider replacing those Maps with a new class. Perhaps that new class will attract some code!

When I’ve used this technique, two key things have happened: either I’ve noticed duplication in the parameter lists of the new functions or introducing a parameter has changed the behavior of the system. In the first case, I introduce Parameter Objects for the duplicated parameters, which then probably attract code and become useful domain objects. In the second case, I’ve detected temporal coupling, which requires me to separate the function into two smaller ones so that some output from the first becomes input to the second. This helps me uncover cohesion problems, usually of the type of different things written too close together.

I realise that an example would help right about now, but I would rather create some screencasts than write out examples in code, but I don’t know when exactly I intend to do that. I wanted to share this idea with you without waiting for the energy to put together a suitable screencast.

I invite you to try introducing pure functions into some legacy code and practising the technique as a kata. Get used to the various maneuvers, like introducing Parameter Objects or Return Value Objects or solving temporal coupling by splitting the function in two. It sounds crazy, but I’d like to try a Legacy Code Retreat where we practise only this technique all day. I don’t know whether anyone else would find it valuable enough to try it together for an entire day, over and over and over.

Would you? Add your comments below.

I write Learning Tests to discover how a third-party library works. I isolate myself from the third-party library through a layer of interfaces and adapter classes that evolve from the common ways I use the third-party library. I call this the “Pattern of Usage API”, as it represents the way my application uses that third-party library. Now my application uses the third-party library through a layer of interfaces, which means that I can introduce Contract Tests on those interfaces. These Contract Tests effectively describe the subset of the third-party library’s behavior on which I depend.

Now when I upgrade the third-party library, I run the Contract Tests against my adapters to that library. Test failures usually indicate a backwards incompatible change in the third-party library. (Sometimes they indicate a trivial difference in the API which requires a trivial fix, such as an API call having been renamed or something.)

Of course, this only helps me detect behavior changes related to computing answers, and not related to responsiveness, reliability, scalability, and so on. For that, I’ll always need system tests.

The Contract Tests are almost always state-based integration tests. I simply limit these to the implementation of Pattern of Usage API and don’t let it leak farther up the call stack. At some point you have to integrated with the Outside World. I simply teach people to look to make that integration thinner.

When I can’t decide whether to remove a certain bit of duplication I’ve found, I fall back on two rules:

1. Remove duplication only after you see three copies.
2. If you don’t know how to remove this particular kind of duplication, then write more tests. Either you need more examples to see the pattern, or more examples will show a different, better pattern.

I also remember two guidelines:

1. Don’t be afraid to remove duplication by introducing a method or class or interface with a stupid name. Remember that you can always improve the name later.
2. If you sense duplication, but don’t really see it, or can’t explain it to others, then make the surrounding names more precise; then maybe you will see the duplication.

If you click here you’ll see an almost textbook example of a contract test: that is, a test class that can run the same set of tests for two different implementations of the same interface. I would change only one thing: I’d extract the tests into an abstract superclass—something I otherwise hate to do—and pull the declaration of the method MakeTestSubject() up there, leaving two subclasses, one for the real file system and one for the simulated one. “YAGNI,” you say, and I agree, but I prefer the symmetry of the abstract superclass design to the asymmetry of having one class inherit from the other. I find it easier to grok quickly.

Either way, I feel good seeing contract tests out in the wild. I’m not so crazy after all.

• Adding behavior confidently involves having fewer parts to change (low duplication), knowing which ones to change (high cohesion), ease of changing just the part you want to change (low coupling), and understanding how you’ve changed it (strong tests).
• Adding behavior requires breaking an existing assumption.
• In a well-factored design, we can easily find the one place we have made that assumption. (Otherwise, why bother refactoring?)
• First, make room for the new code, then add it.
• To make room for the new code, extract the existing code into a method whose name describes the generalisation we want to make, or the idea we want to introduce.
• By making room for the new code, we make that code easier to reuse by reducing its dependence on its surrounding context.

Want to learn more?

• Visit the links in this article.
• Attend Agile Design: Beyond the Basics.
• Pair with me.

The Details

We all want to add behavior to a system confidently, and I have observed that my confidence in adding behavior depends on two factors:

• I know where to add code.
• I understand the behavior of the code I am adding.

I use test-driven development as the main technique for handling the second of these two factors, but what about the first? I have uncovered a technique that I both use and teach, and I’d like to share that with you. I call this “making room for the new code”, naming it for a phrase I vaguely remember reading in one of the Grand Old XP books. (Did Kent Beck or Ron Jeffries write it? I can’t remember.) This technique helps me quickly find a reasonable first-draft place in the code base to put new code. After I have put the new code in place, and I feel confident that it does what I expect, I then use the Four Elements of Simple Design to guide me in refactoring to improve the design.

A premise

I start with the premise that adding behavior means breaking an assumption. By this I mean that whenever we add code to a system in order to extend its behavior, we have to falsify at least one assumption we’ve previously made. For example:

• In a payroll system, in order to support a second cheque printer, we likely have to break the assumption that there is only one cheque format.
• In a point of sale system, in order to support separate cash and card payment reports, we likely have to break the assumption that all “we made a sale” events look the same.
• In a mobile phone monitoring system, in order to support billing by the second, we likely have to break the assumption that we only have to count the number of minutes a call lasted.

Some of these seem obvious, and others less so, and it bears emphasising that the specific assumption or assumptions we break depends heavily on what we’ve built so far and the way we articulate the soon-to-be-added behavior. Even so, I conjecture that for every behavior we want to add to a system, we can identify a non-empty list of assumptions that we need to break.

The technique

1. Identify an assumption that the new behavior needs to break.
2. Find the code that implements that assumption.
3. Extract that code into a method whose name represents the generalisation you’re about to make.
4. Enhance the extracted method to include the generalisation.

The less duplication you have in the system, the better this works, because duplicate code makes it difficult to find all the code that implements the assumption in question. Similarly, the more appropriate the names in your system, the better this works, because unsuitable names make it difficult to know which code implements the assumption in question, as opposed to something unrelated. You’ll notice that these points relate both to the Four Elements of Simple Design and to the core concepts of Coupling and Cohesion.

Yes, yes…

An example

We are building a point of sale system, and we’ve just decided to implement sales tax. I live in PEI, Canada, where not only do we exclude sales tax from the price, we have two sales taxes, and we charge the second one on top of the first one. For example:

A $125 item that attracts both GST (the “Goods and Services” tax) and PST (provincial sales tax) costs a total of $144.38. GST at 5% costs $6.25, then PST at 10% of $131.25 (= $125 + $6.25) costs $13.13. The total is $125 + $6.25 + $13.13 = $144.38.

Notice that this example implies that GST or PST might or might not apply to a given product, so even before we identify the old assumption to break, we need to note the new assumption we’ll make: we assume that all products attract both GST and PST. Our customers won’t like it, but the tax authorities will love it, and only they have the power to treat us guilty until proven innocent.

Our code has a class like this in it.

What do we assume now that we can’t allow ourselves to assume any longer? The sale total should increment by the (net) price of the item we sell. By net price here, I refer to the pre-tax price, because of course, until now, our system has no notion of “tax”. Fortunately, because we’ve ruthlessly removed duplication so far, computing the running total of the sale requires only this line of code. We can pretty safely apply the technique of this article right here. To do this, we extract the assumption into a new method whose name represents the generalisation we’re about to make. In this case, we don’t want to increase the sale total by the product’s price, but rather by its cost, which includes any additional charges beyond net price. So, we introduce a method for accumulating the scanned product’s cost.

This creates space for the new code. We test-drive the new code, and end up with this delightful monstrosity.

It looks ugly, but it works. In addition to looking ugly, this method has Feature Envy. Specifically, the calculation part of #accumulate_cost only talks to product, and so it can move onto the class Product, leaving only the accumulating left behind. You could also say that this method had two responsibilities, so I separated them, then notice the feature envy in one of them, then moved it. I can almost always take smaller steps.

Notice the context independence we’ve achieved with the method Product#cost. We can now refine the notion of “cost” freely without worrying about how someone will use that information. We have an easy-to-find, easy-to-extend part of the system where we can add behavior for determining which products attract which taxes, supporting multiple tax calculation policies, and even including shipping, handling and restocking fees. Now we can really add future behavior with confidence.

A model for improving the names of variables, fields, interfaces, classes and namespaces in a system. Practise this and more in my course, Agile Design: Beyond the Basics.

You can probably see a straightforward way to start removing this duplication. To illustrate it, look at the following code snippet.

So we have two classes with a common field, a common method signature, and similar method bodies. Extract Superclass seems logical, even though I can’t think of a decent name for this class so far. No worry: it will come to me.

Next, we have to extract the parts that differ from the parts the match each other. I did that by introducing new methods.

Next, we make processMessage() identical in both classes, so as to pull it up into FooMessageListener. And here, we find our first warning sign about this refactoring. The signatures of handleBiddingStateEvent() don’t match:

• BidsForSniperMessageListener.handleBiddingStateEvent(Chat, BiddingState)
• UpdatesMainWindowMessageListener.handleBiddingStateEvent(BiddingState)

In order to find out whether this will become a problem, I have to add Chat to the parameter list for UpdatesMainWindowMessageListener’s implementation of the method, at least for now. I imagine I could do something more clever, but I’m not sure that introducing a closure over a Chat object would simplify matters. I can put that on my “to try” list for now. In the meantime, I add the Chat parameter. See the diff.

Now to turn similar processMessage() implementations into identical ones, I introduced an empty method. This is another warning sign, since I can’t envision a “default” UI update. Still, I reserve judgment until I have more evidence, and introduce the necessary changes. See the diff.

Now that processMessage() looks identical in both subclasses, we can pull it up into FooMessageListener, for which we still don’t have a good name. Either see the diff or look at the final result below.

Now that I look at this last version of Main, I see the value in having attempted this refactoring. It clarifies quite well the specific reasons why I won’t let inheritance play a long-term role in this part of the design. Specifically, look at the little problems with this design.

• I still can’t think of a good name for FooMessageListener, although maybe you’re screaming one at me as you read. (Sorry; I can’t hear you.)
• BidsForSniperMessageListener.handleAllOtherEvents() doesn’t need to do anything, which might or might not be a problem, but often points to the Refused Bequest smell.
• UpdatesMainWindowMessageListener.handleBiddingStateEvent() doesn’t need its chat parameter.
• Both subclasses need a reference back to Main, but the superclass FooMessageListener doesn’t need it. Of course, that says more about Main in general than the FooMessageListener hierarchy: Main violates the Single Responsibility Principle in a variety of delightful ways. That’s why we’re here.

So let me talk this through: we have subclasses of FooMessageListener that respond to the same logical stimuli, but different subclasses need different data and sometimes a subclasses doesn’t need to respond to the stimuli at all. That sounds an awful lot like an event mechanism to me.

And if you look ahead in Steve and Nat’s book, that’s not surprising.

So how do we get from here to there? I plan to attack it this way:

1. Make FooMessageListener an event source for this new type of event, which I’ll tentatively call an AuctionEvent.
2. Turn the subclasses into AuctionEventListeners.
3. Inline stuff back into place.

You can follow the detailed changes on this branch starting with this commit.

By the end of this session, I ended up with a new Auction Event concept, which has started separating the “chat message” concept from the “auction changed” event. It does, however, lead to other questions, which I’ve highlighted with SMELL and REFACTOR comments, so look for them. For now, I feel good that, while introducing a hierarchy had its problems, it helped me see how specifically to eliminate it. I trusted the mechanics, and it helped me see where to go next. I know I need to introduce an “auction closed” event and perhaps need to introduce an AuctionEvent object to eliminate the need for type checking in AuctionEventSourceMessageListener. We’ll go there next time, but in the meantime, peruse the current state of the code.

If you prefer, look at the changes in moving from MessageListeners to AuctionEventListeners.

I hope I’ve shown how a code base can reject an attempt to reuse code by inheritance, prefering instead, in this case, composition implemented as an event chain.

</p>

The guidelines

I have formulated these as Novice or Advanced Beginner rules, and so I have worded them more strongly than I tend to word my advice.

• When in doubt, inject each dependency directly into the method that requires it.
• Only when you inject the same dependency into multiple methods of the same class, move the parameter into the constructor.
• Do not use so-called “setter injection” except as an intermediate step in a refactoring aimed at injecting the dependency into a method or the constructor.
• Stop using the Service Locator pattern, and instead inject the service into the client.
• Stop instantiating collaborating services (in the Domain-Driven Design sense), and instead inject those services into the client.
• When a constructor parameter list becomes uncomfortably long, split the class so that the new classes’ constructor parameter lists don’t overlap.
• If you notice a class using the same dependency through two different object graph routes, split the class so that the new classes receive the dependency directly in their constructors.
• If you notice a class using groups of dependencies at different times, split the class so that the new classes only use each cohesive group of dependencies.

All these guidelines have their roots in “remove duplication” and “fix bad names”, two of the four elements of simple design.

So why inject dependencies at all?

Some people inject dependencies in order to stub, mock or spy on a method for testing. I used to use that as a primary motivation, but over the years that motivation has evolved into something more widely useful. I inject dependencies for one reason: to move the things that change in the direction of the client and the things that don’t in the direction of the supplier. Or, if you prefer, “abstractions in code, details in data”. Or, if you prefer, to avoid abstractions depending on details. Any of those will do.

You don’t buy it?

No problem. I’ll write more articles in this space showing examples of each of the guidelines, and you’ll have plenty of opportunity to share your opinions about them.

</p>

Integrated tests are a scam—a self-replicating virus that threatens to infect your code base, your project, and your team with endless pain and suffering.

Wait… what?

I mean it. I hate integrated tests. I hate them, and with a passion. Of course, I should clarify what I mean by integrated tests, because, like any term in software, we probably don’t agree on a meaning for it.

I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior.

I, too, would prefer a more rigorous definition, but this one works well for most code bases most of the time. I have a simple point: I generally don’t want to rely on tests that might fail for a variety of reasons. Those tests create more problems than they solve.

You write integrated tests because you can’t write perfect unit tests. You know this problem: all your unit tests pass, but someone finds a defect anyway. Sometimes you can explain this by finding an obvious unit test you simply missed, but sometimes you can’t. In those cases, you decide you need to write an integrated test to make sure that all the production implementations you use in the broken code path now work correctly together.

So far, no big deal, but you’ll meet the monster as soon as you think this:

If we can find defects even when our tests pass 100%, and if I can only plug the hole with an integrated tests, then we’d better write integrated tests everywhere.

Bad idea. Really bad.

Why so bad? A little bit of simple arithmetic should help explain.

You have a medium-sized web application with around 20 pages, maybe 10 of which have forms. Each form has an average of 5 fields and the average field needs 3 tests to verify thoroughly. Your architecture has about 10 layers, including web presentation widgets, web presentation pages, abstract presentation, an HTTP bridge to your service API, controllers, transaction scripts, abstract data repositories, data repository implementations, SQL statement mapping, SQL execution, and application configuration. A typical request/response cycle creates a stack trace 30 frames deep, some of which you wrote, and some of which you’ve taken off the shelf from a wide variety of open source and commercial packages. How many tests do you need to test this application thoroughly?

At least 10,000. Maybe a million. One million.

Wie ist es möglich?! Consider 10 layers with 3 potential branch points at each layer. Number of code paths: 3¹⁰ > 59,000. How about 4 branch points per layer? 4¹⁰ > 1,000,000. How about 3 branch and 12 layers? 3¹² > 530,000.

Even if one of your 12 layers has a single code path, 3¹¹ > 177,000.

Even if your 10-layer application has only an average of 3.5 code paths per layer, 3.5¹⁰ > 275,000¹.

To simplify the arithmetic, suppose you need only 100,000 integrated tests to cover your application. Integrated tests typically touch the file system or a network connection, meaning that they run on average at a rate of no more than 50 tests per second. Your 100,000-test integrated test suite executes in 2000 seconds or 34 minutes. That means that you execute your entire test suite only when you feel ready to check in. Some teams let their continuous build execute those tests, and hope for the best, wasting valuable time when the build fails and they need to backtrack an hour.

How long do you need to write 100,000 tests? If it takes 10 minutes to write each test—that includes thinking time, time futzing around with the test to make it pass the first time, and time maintaining your test database, test web server, test application server, and so on—then you need 2,778 six-hour human-days (or pair-days if you program in pairs). That works out to 556 five-day human-weeks (or pair-weeks).

Even if I overestimate by a factor of five, you still need two full-time integrated test writers for a one-year project and a steady enough flow of work to keep them busy six hours per day and you can’t get any of it wrong, because you have no time to rewrite those tests.

No. You’ll have those integrated test writers writing production code by week eight.

Since you won’t write all those tests, you’ll write the tests you can. You’ll write the happy path tests and a few error cases. You won’t check all ten fields in a form. You won’t check what happens on February 29. You’ll jam in a database change rather than copy and paste the 70 tests you need to check it thoroughly. You’ll write around 50 tests per week, which translates to 2,500 tests in a one-year project. Not 100,000.

2.5% of the number you need to test your application thoroughly.

Even if you wrote the most important 2.5%, recognizing the nearly endless duplication in the full complement of tests, you’d cover somewhere between 10% and 80% of your code paths, and you’ll have no idea whether you got closer to 10% or 80% until your customers start pounding the first release.

Do you feel lucky? Well, do you?²

So you write your 2,500 integrated tests. Perhaps you even write 5,000 of them. When your customer finds a defect, how will you fix it? Yes: with another handful of integrated tests. The more integrated tests you write, the more of a false sense of security you feel. (Remember, you just increased your code path coverage from 5% to 5.01% with those ten integrated tests.) This false sense of security helps you feel good about releasing more undertested code to your customers, which means they find more defects, which you fix with yet more integrated tests. Over time your code path coverage decreases because the complexity of your code base grows more quickly than your capacity to write enough integrated tests to cover it.

…and you wonder why you spend 70% of your time with support calls?

Integrated tests are a scam. Unreliable, self-replicating time-wasters. They have to go.

¹ True: few code bases distribute their complexity to their layers uniformly. Suppose half your 12 layers have only two branch points—one normal path and one error path—while the others have 5 branch points. 2⁶·5⁶ = 1,000,000 and for 4 branch points 2⁶·4⁶ > 262,000. You can’t win this game.

² Aslak Hellesøy points to a way to take luck mostly out of the equation. His technique for choosing high-value tests will certainly help, but it stops short of testing your code thoroughly. I believe you can achieve truly thorough focused tests with similar cost to writing and maintaining integrated tests even using the pairwise test selection technique. (Thanks, Aslak, for your comment on April 12, 2009.)

Read more in this series

“In the time between writing this essay and sending the book to be printed, a new dynamic proxy-based mock objects package has appeared on the scene, called jMock (www.jmock.org). It picks up where EasyMock left off, as the EasyMock project went through a temporary lull in activity, between October 2003 and May 2004. Being so new, we do not have any experience using it, and so we cannot say much about it, but it does look promising and bears a look. If you have used EasyMock, then it is worth experimenting with jMock to see the difference. You may find you prefer jMock’s approach to that of EasyMock.”

Since 2004 I have used JMock almost exclusively to drive my designs in Java. I even like the strange-looking JMock 2 syntax which, I know, puts me in the minority. In the last two years, other test double libraries have gained mindshare, among which Mockito has become quite prominent. While I can’t give you a feature-by-feature comparison, I can tell you this:

When I want to rescue legacy code, I reach for Mockito. When I want to design for new features, I reach for JMock.

Different central assumptions of JMock and Mockito make each one better at its respective task. By default, JMock assumes that a test double (a “mock”) expects clients not to invoke anything at any time. If you want to relax that assumption, then you have to add a stub. On the other hand, Mockito assumes that a test double (sadly, also a “mock”) allows clients to invoke anything at any time. If you want to strengthen that assumption, then you have to verify a method invocation. This makes all the difference.

When I work with legacy code, I mostly write learning tests to discover how different parts of that legacy code behaves. Usually legacy code has obscene and overwhelming levels of interdependency, and Mockito helps me manage that, by allowing me to worry about one crazy dependency at a time.

When I design for new features, I mostly write design tests that describe the new behavior I want to implement. With the nice green field of a new interface, I need JMock to encourage me to clarify the interaction I need. Whenever my production code attempts to use a collaborator, JMock effectively reminds me to ensure that I want that interaction. Most importantly, JMock stops me from introducing dependencies that I don’t need.

I really like Fred Brooks’ use of the terms essential complexity and accidental complexity. Briefly, a code base’s essential complexity reflects the complexity of the problem. Automating tax audits will result in high essential complexity. A code base’s accidental complexity reflects the complexity we programmers add because we don’t design simply. In short, if it isn’t essential complexity, then it’s accidental complexity. Our job as designers includes minimising accidental complexity.

Mockito helps me tolerate high accidental complexity while I work to reduce it.

JMock tries its best to stop me from introducing accidental complexity.

That explains why I use JMock when designing for new features and why I’ll recommend using Mockito for rescuing legacy code.

Hi everyone,

I’d like some advice/opinions on how to test some existing code. It’s a web application using Spring and struts.

I have a class called the ProcessedFilesManager which contains a number of methods used by Struts Action classes. This manager communicates with five different DAOs to get the information that some of the Struts actions are interested in. Now, I want to test this manager class (ProcessedFilesManager). The way I’ve started doing it is stubbing up each of the five DAOs, however, this is proving to be quite painful. I didn’t want to use a mocking approach, nor did I want to use a DB solution like Hypersonic, but now I’m open to suggestions.

Seeing as there a number of approaches I could use, what do you think would be best for this situation?

It feels wrong to stub the DAOs because what if I’m introducing behaviour in there that differs from the actual DAOs? My tests will not be accurate.

Any advice/comments would be much appreciated.

Read the thread

I used to have this fear, and I do something now that has eliminated that fear.

When I stub a DAO method, I make an assumption about what that DAO method does. I used to be worried about making the wrong assumption, but now I have a contract test for the DAO interface that tests for the assumption I’m making in my Service test. The contract test gives me confidence that any implementation of the DAO method passes the same tests, so every implementation of that DAO method behaves the way I assume it does. Once I have this, I feel comfortable stubbing that DAO method that way in a Service test.

A contract test is a test for an interface. I describe contract tests in some detail in JUnit Recipes, recipe 2.6, although back then I called them “abstract test cases” because I hadn’t yet discovered the better name “contract test”. If you prefer, I’ve provided a diagram showing some contract tests for a typical DAO class.

Since classes inherit methods from their superclasses, the Hibernate Customer DAO Test will inherit the contract tests from its superclass, as will the JDBC Customer DAO Test. This means that each implementation has to pass not only its own tests (like testClosesSession() or testClosesResultSet()) but also the tests inherited from Customer DAO Contract Test Template. (I call it a “template” because it plays the role of template in the Template Method design pattern.) When you test-drive a new implementation of Customer DAO, simply make the new test extend the contract test template and you’ll automatically inherit its contract tests. This way, I have confidence that any implementation of Customer DAO behaves the way I’d expect any Customer DAO to behave.

Returning to our example, these contract tests give me confidence to stub the DAO when I test-drive the Service, and that confidence brings with it a happy side effect. I am confident that findAllWithPendingOrders() only returns customers with pending orders, so I don’t have to worry about that issue at all when I design the Service that reports all customers with pending orders. Now that I notice it, Report All Customers With Pending Orders Service is really just a Report on Customers Service that needs a Customer Filter, which could be a Pending Orders Customer Filter. I don’t think I would have felt comfortable with this level of generalization if I weren’t so confident in the way I’ve separated the responsibilities.

The next time you want to avoid stubbing a method because you’re worried you’ll make a wrong assumption about what the method does, try writing enough contract tests to give you the confidence you need. I think you’ll like the results.

Ruby follows the Principle of Least Surprise, so I tried this:

That doesn’t work, as I found no #with_attribute nor anything like it. After I dug a little, I found out that I should write this:

I don’t like this, because it checks two things at once: it looks for an “RSS tag” and checks the href attribute. That HTML implemented the “RSS tag” as attributes on <link> creates the confusion. The extremist in me calls this an integrated test, but I prefer not to go there today. Suffice it to say that when this check failed, I didn’t immediately know why, and I insist on immediately knowing why a check fails. When I’ve done this in the path with XPath assertions in XHTMLUnit or XMLUnit, I’ve resorted to writing duplicate checks that build on one another, so I tried that here:

Here I sacrificed duplication to improve the clarity of the assertions. I just now noticed that that contradicts my usual rule that removing duplication matters more than improving clarity. I dno’t know what to say about that just yet. Even if I remove the duplicate code, I still have the duplicate check, so extracting to a method won’t really do much here. I want #with_attribute!

Not deterred, I tried introducing a custom RSpec matcher, since I don’t know what benefit that would give me, but it would benefit me somehow. When I tried to do that, RSpec told me that it couldn’t understand response.should have_tag because it couldn’t find a has_tag? on the response. I didn’t like the looks of the stack trace; I felt I would have to delve deeply into RSpec or assert_select, and I didn’t find myself in the mood to do either, so I switched to Nokogiri.

Here I got to write the spec the way I wanted to: assume you will find “RSS feed tag”, then check that its href attribute matches the right URL. If the response has no “RSS feed tag” at all, then complain violently, because of the higher severity of the mistake.

Of course, if you have another suggestion, I’d like to see it. Add your comment below.

Special thanks to Frivolog for helping me get the original have_tag code working.