Re. “primitive obsession,” I say this poker-scoring program wouldn’t be bettered avoiding integers for rank and suit. jsoftware.com/jwiki/TracyHar…

— Tracy Harms (@kaleidic) March 3, 2013

I can’t read J, so I can’t decide much about the quality of the code Tracy wrote, but I do notice one thing:

There aren’t many functions that operate on the primitive data (card rank, card suit, hand classification), they’re close by each other, and they’re all quite short.

Regarding this one example, then, Primitive Obsession doesn’t pose much of a problem yet. Using primitives to represent values with special constraints can lead to low cohesion and high duplication. The low cohesion consists of scattering related ideas throughout the code base, where programmers have a hard time finding them. Such scattering can lead to duplicating special processing logic, and worse, duplicating it inaccurately. That leads to mistakes. This makes me ask a question:

When does Primitive Obsession not smell?

I suppose it doesn’t smell when we use other means to keep cohesion high and duplication low. We can use primitives safely when we keep the special processing logic for those primitive values close together and use willpower (effectively) to avoid duplicating it.

I still can’t account for Tracy’s observation that Primitive Obsession doesn’t figure prominently in array programming, where it ought to run rampant and create serious problems. Some guesses:

• Teams of programmers working in these languages don’t create large, low-cohesion systems, like they do in Java/C#/C++.
• Teams of programmers working in these languages name things better in general, and so create fewer opportunities for misunderstanding.
• Teams of programmers working in these languages work together more closely in general, and so smooth over their misunderstandings more easily and more promptly.

Any ideas?

References

Martin Fowler, Refactoring: Improving the Design of Existing Code. This book introduced me to Primitive Obsession as a code smell.

Ward Cunningham, “The CHECKS Pattern Language of Information Integrity”. This article includes a section on Whole Value, which counters the effects of Primitive Obsession. Whole Values should become attractive code when you introduce them into a system.

James Shore, “Primitive Obsession”. I recommend this article to my students to learn more about Primitive Obsession.

Corey Haines and J. B. Rainsberger, “Primitive Obsession”. Corey and I discussed Primitive Obsession while in Bucharest in early 2010. This article includes a 14-minute video of the two of us chatting about the topic.

Code Retreat. We very commonly use the “no primitives” constraint at Code Retreat to encourage programmers to practise introducing Whole Values very early. I don’t always design this way, but I believe that programmers should understand more deeply the differences between design with and without promitives, so I encourage them to practise.

Modularity. Details. Pick one.

I routinely work with programmers who feel uncomfortable with tiny methods and tiny classes, in spite of all the benefits of tiny methods and tiny classes:

• easier to test
• easier to read
• easier to understand
• easier to build correctly
• easier to compose into working systems

The programmers who don’t like them cite these problems most often:

• “I can’t find stuff!”
• “I have to click too much to see what’s happening!”

This article by Kevin Rutherford reminded me of this issue. While Kevin extols the virtues of tiny methods, he doesn’t address this phenomenon, although he admits to running into it himself. I wanted to share what I do with him, but it doesn’t fit well into a comment. When I hear programmers complain about these things, I offer the following two responses to them.

You appear to be navigating. Would you like help with that?

I have paired, albeit mostly briefly, with hundreds of programmers by now. Most of them, to my surprise, navigate their code bases to find things. By this, I mean that they open up folders in a navigator view in their IDE, or navigate the file system, in order to locate a file containing the code that they want to read or change. Not wanting to make pair programming suck, I hold my tongue until I’ve built up more trust with the person. If I think they trust me, then I stop them and suggest:

Don’t navigate when you can search.

I just don’t care where to find things any more. Just knowing that “they’re somewhere in this room”¹ suffices. Only when searching fails do I resort to navigating.

The classic book The Pragmatic Programmer teaches us to learn one editor well, which I try to do. As part of classroom training and mentoring hundreds of programmers, I find myself working in unfamiliar environments quite often. When I work in a new one, I spend some time learning how to search for things. In Eclipse, I use “Open Type” and “Open Resource” extensively. Before I knew anything useful about vim, I knew to press / to find text in a file. Before I knew anything useful about Unix, I could clumsily use grep, and more recently I’ve learned about ack and even ag. I don’t use Emacs any more, but I love its incremental search feature. I see this as an extension of the maxim that humans ought to delegate mechanical tasks to the computer, such as searching text for a pattern.

I still see programmers scan web pages with their eyes to find text, rather than use the “search” feature of their browser. I just don’t understand why they do this. I prefer to work in environments where I don’t even know that it stores my code in files!²

This brings me to my other response.

Modularity. Details. Pick One.

If you ask programmers whether they want modularity, they usually claim that they do. It sounds like something they ought to want, books on design tell them to want it, and so I imagine that some programmers claim to want it in order to look good among their peers. Unfortunately, many such programmers don’t appear to understand that modularity comes from abstraction, and that abstraction literally means hiding details, and that doesn’t mesh with their clinging to details. To them, I suggest repeating a little mantra all day today:

I don’t care. I don’t need to know. I don’t want to know.

Giving up details won’t lead you directly to writing modular code, but clinging to details will keep you from it. The more your code knows about its neighbors, the less you can change or replace those neighbors. Changes ripple out to the rest of the system. When you do this, you’re writing more legacy code.

Modularity needs abstraction. Abstraction means hiding details. If you cling to details, then you won’t create abstractions, and you won’t achieve modularity. Modularity begins with your refusal to let others burden you with unnecessary detail.

I noticed that my designs improved once I began stubbornly refusing to know any more than I needed to achieve my current goal. Try starting there. “I don’t care. I don’t need to know. I don’t want to know.”

Look at the SOLID principles and how they relate to not wanting to know details.

• Single Responsibility: I don’t want to know about any other behavior in this system right now.
• Open/Closed: I don’t want to know how things work in order to extend them.
• Liskov Substitution: I don’t want to know which implementation of each interface this code uses.
• Interface Segregation: I don’t want to know anything more than the minimum I need to do my job.
• Dependency Inversion: I don’t want to know how the rest of the system uses me.

I don’t think I can make it any plainer than that.

Of course, I’ve assumed in this article that we want modular systems. You might not. I don’t want to argue the merits of modularity here. Instead, I want to make one thing clear:

Modularity and obsession with details exclude one another. If you want one, then you can’t have the other, so pick one.

Of the SOLID principles, the Dependency Inversion Principle remains somewhat misunderstood. I’d like to help with that. I hope you’ll ask questions and challenge these ideas, so that I can improve this little article.

I learned the Dependency Inversion Principle from Bob Martin’s article, the salient part of which states:

High level modules should not depend upon low level modules. Both should depend upon abstractions.

or

Abstractions should not depend upon details. Details should depend upon abstractions.

I’ve noticed that I spend the vast majority of my time introducing abstractions and inverting dependencies (abstractions that use concrete things) when rescuing legacy code or adding behavior to legacy systems. Sometimes I feel like I don’t know any other useful trick than invert this dependency. One could make a drinking game out of it.

Even so, not enough people understand and apply DIP. Perhaps if I share a few examples or equivalent formulations, they will.

Move Specificity Towards the Tests

Moving details from production code towards tests clarifies the tests. In particular, I find it easier to relate the inputs to the expected outputs. It eliminates duplication between tests and production code. It highlights opportunities for abstraction and reuse. Corey Haines interviewed me about this in summer 2009.

JB Rainsberger - Move Specificity Towards the Tests from Corey Haines on Vimeo.

(If you can’t watch the embedded video, then click here.)

Move Implementation Choices Up the Call Stack

Dependency injection confuses and annoys people, partly because some people miss the point. Injecting a dependency makes testing easier, by letting us substitute implementations in our tests that make those tests easier to write and understand. Don’t stop there. When in doubt, push new up the call stack. In particular, don’t instantiate a service if you can get your caller to do that. Keep pushing, even more than you think you should. Let all the choices of implementation of each service interface bubble up the call stack until they collect in your application’s or component’s entry point. This shines a huge floodlight on otherwise obscure duplication. It also paves the way for sane use of a dependency injection container, even though you don’t need a container to inject dependencies well. This movement of new up the call stack leaves in its wake a loosely-coupled network of abstractions talking to each other over well-defined interfaces with well-understood contracts. Using this technique helps me remove duplication, which decreases errors, particularly of the “I forgot to change that one” type. Using this technique also encourages me to write code that depends on narrower interfaces, which slowly become more generic interfaces, which makes the code depend less on its context and more suitable to use in other contexts.

Abstraction in Code; Details in Metadata

I first learned this principle from the book The Pragmatic Programmer. You’ll find it as “tip 38”. I encourage you to buy the book and the read the tip yourself. The authors state among the benefits:

• a more robust program
• a more flexible program
• opportunity to reuse the “engine” with different metadata
• opportunity to express metadata in the language of the domain
• opportunity to customise code without rebuilding it

Who could argue with all that?

Quick Summary

If you agree with the specific principles I’ve cited here, then you also agree with the Dependency Inversion Principle. Said differently:

Push details up towards the client and push generic patterns down towards the server.

This principle, applied everywhere, makes testing easier, promotes reuse, both of which reduce the incidence of errors and the cost of development.

So tell me:

• If you already felt you understood the DIP well, how faithfully did I represent it here?
• If you didn’t quite understand the DIP well before, how much has this article helped or hindered your understanding?
• If you explain the DIP to others, how do you do it?
• If you still think the DIP is a load of nonsense, how do you justify that? I’d like the chance to help you as well as fix holes in my own understanding and reasoning.

One More Thing

If you hate Spring’s dependency injection container, then I know why: you have code that depends on the container.

Don’t do that.

Dependency injection is related to the Inversion of Control principle, also known as the Hollywood Principle:

Don’t call us; we’ll call you.

Spring’s dependency injection container calls you; you never call it. Never.

Fine: you call it only from a smoke test that verifies that you’ve wired up your dependencies correctly, but you never call your dependency injection container from production code.

Of the SOLID principles, the most pompous-sounding is the one named for a person: Barbara Liskov. I don’t label her pompous – I never knew her – but of those five principles, only LSP has a common formulation that looks like the kind of mathematics so many programmers like to avoid, while others flock to.

Let q(x) be a property provable about objects x of type T. Then q(y) should be provable for objects y of type S where S is a subtype of T.

Or, if you prefer:

Subtypes must not change any supertype’s significant behavior. Here, “significant behavior” means behavior upon which clients of those objects expressly depend.

If you like the language of contracts, then you might prefer this formulation:

Subtypes must respect the contracts of their supertypes.

If, like me, you like to check contract compliance with contract tests, then you might prefer this formulation:

Subtypes must pass the same set of contract tests that their supertypes pass.

Remember that subtype here means any implementation of an interface or subclass of a class.

So, what does it look like when we violate the Liskov Substitution Principle? Usually like this:

1

// This should never happen

You can’t trace all such problems to a violation of LSP. Sometimes you have a simpler disagreement between a client and its collaborator. Even so, subtyping without careful attention to the type’s contract opens the door wide to breaking the contract, which creates disagreement between a client and a collaborator whose type we sometimes only know at runtime. You’ve experienced this if you’ve ever ended a 2-hour-long debugging session by yelling “You idiot! How did you expect that ever to work?!” You’ve discovered a fundamental contract disagreement.

While the LSP doesn’t receive the same attention as the Single Responsibility Principle, breaking the LSP leads to painful, costly mistakes. Understanding LSP, clarifying and respecting contracts helps you avoid this frustration.

I hope this helps.

I read about Design by Contract first, didn’t understand it, then read about Test-Driven Development and began to think of TDD as a kind of DbC by example. Accordingly, I have respect for both practices and have even combined them on projects. Moreover, “Integrated Tests are a Scam” amounts to little more than Isolating Layers by Contract.

When I saw the Vigil programming language today, I fell in love. To quote its README:

When a Vigil program is executed, Vigil itself will monitor all oaths (implorations and swears) that have been made. If an oath is broken, the offending function (the caller in the case of implore and the callee in the case of swear) will be duly punished.

I know for sure I’ll try Vigil at the next Code Retreat.

A month ago I announced a new book. The book is now nearing completion.

In the past month I have published over 240 pages, filled with code, demonstrating how to test-drive the first feature of an Android app.

If you want to see how to bring the benefits of evolutionary design to Android, then this book is absolutely for you. In it, you’ll see example after example of designing with tests, refactoring, and running commentary on what goes in to making the design decisions I make. It’s my opportunity to show you how all the principles, practices, techniques and ideas I have about design work together.

• Get out of the simulator as quickly as possible
• Integrated tests are still a scam!
• Move details down the call stack and abstractions up the call stack
• How fast tests lead to great design

Android development can make Java programming fun again. In this book, I show you how I do it, so buy your copy today!

I really like Octopress, which is built on Jekyll. You can read a lot of articles about the advantages and disadvantages of pre-baked blogs, so I won’t bore you with those details. Instead, I’ll bore you on the subject of the one feature I miss with pre-baked blogs: scheduling posts for later publication.

When I switched to a pre-baked blogging tool like Octopress, I gave up the ability to upload a post, but publish it later. Well, I didn’t really give it up entirely, but pre-baking my blog means thinking of my blog as a static web site, rather than a dynamic one, which implies having to re-deploy every time I wanted to change its content. This makes scheduling posts in the future a bit more tricky.

Using Heroku and its read-only file system make it even trickier, since I can’t regenerate the blog in production.

I just wanted to share my solution with you, in case it interests you.

First, I added to Octopress the ability to generate the blog without future-dated posts. This looked difficult, especially with Octopress’s existing “preview mode” feature, but I discovered that Jekyll supports omitting future-dated posts directly with jekyll --no-future, so I augmented Octopress to use this feature. If you’d like to explore the details, click here.¹

Next, I needed a way to republish my Octopress site regularly in order to include new posts as their publication instant arrives. I asked my tweeps for ideas, and we agreed that we couldn’t get around having a machine Out There Somewhere that’s always on that would poll a git repository for changes, then regenerate and publish to Heroku. I settled on the following architecture.

I followed these basic steps:

1. Added an SSH public key from “Monitor/Staging” to “Production”, so that the former could push to the latter.
2. Installed a script into “Monitor/Staging” to, well, monitor, stage, then push.
3. Cloned my local git repository to “Integration” so that it will always be available to “Monitor/Staging”.

Complicated, sure, but–and you’ll tell me if you disagree–just barely sufficiently complicated, and no worse. I can handle that.

My workflow hasn’t changed much. I compose posts, preview, and push to “Integration” when I finish. “Monitor/Staging” runs the following script every 10 minutes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#! /bin/bash
pushd $HOME/Workspaces/blog.thecodewhisperer.com
# I have to do this by hand, apparently.
. .rvmrc

# I can't assume that I'm on the right branch
git checkout -q master
git pull -q origin master
git checkout -q publish
git merge -q master

# Check the diff /before/ regenerating, to avoid spurious
# timestamp differences
DIFF="$(git diff --stat publish heroku/master)"
NOW=`date`
if [ -n "$DIFF" ]; then
  echo "Publishing $NOW..."
  bundle exec rake generate
  git add -A && git commit -m "Published $NOW" && git push heroku publish:master
else
  echo "No need to publish at $NOW.";
fi
popd

I write posts to the branch master, and “Monitor/Staging” merges them to the branch publish, then deploys them to “Production”’s branch master.

Of course, I will monitor this closely over the next few days, looking for anomalies. In particular, I worry a little about what happens when “Monitor/Staging” can’t merge master to publish automatically. I suppose in that case I’ll have something to worry about, and in the worst case, no changes will publish to production, which could cause annoyance, but sounds like safe default behavior.

If you see any problems with this setup, please don’t keep them to yourself!

In Part 1, Dancing with the Android SDK, I explore a sensible architecture for the basic Android app, emphasising context independence to avoid letting the Android framework swallow my app whole. I use tactics that have worked to great effect in working with older frameworks such as Servlets and EJB.

I toyed with the idea of writing an Android app in Ruby using Rhodes, but decided to return to my roots and Java, so if you’re writing Android apps in Java, and have run into maintenance or extensibility problems, then I’m writing this book for you.

I say “I’m writing”, because I’ve decided to use LeanPub to compose and publish the book. This means that you can read the book as I write it, and your feedback and questions could help direct its contents. As of this writing, I’ve published three versions of the book, and plan to publish a new version approximately twice per week until I’ve written enough.

The sooner you purchase the book, the less you pay, and once you purchase the book, you receive all future updates free of charge. I have already raised the minimum price of the book once, as I have published more content.

If you’d like to take a look, click here to download a sample chapter, then click here to buy the book.

Please add your comments at gist.github.com.

(If you can’t see the code snippet inline, then click here.)

Why would I simply summarise someone else’s article on this site? Isn’t that dishonest? Not in this case: Miško wrote essentially the same thing that I say over and over again in my own articles and in my training classes. I consider it unfortunate that he and I have never met nor worked together. We really should. You should read his stuff.

So… 10 ways to kill your design.

1. Instantiate Services wherever the hell you want.
2. Grab what you want when you want it.
3. Do real work in your constructors.
4. Publicise state.
5. Embrace singletons.
6. Make methods static. (Admittedly, this is more of a problem in Java/C# than Ruby/Python.)
7. Inherit implementation.
8. Reimplement polymorphism.
9. Mix Services and Values.
10. Keep conjunctions in your variable, method and class names.

Read more about the problems that these “techniques” cause.

When multiple assertions check very tightly related things, I don’t mind them, but when they check relatively loosely related things, they act as integrated tests for multiple behaviors that we should consider separating. This is even subtler than the simpler idea of “one action per test”.

If you’d like a Novice algorithm to follow:

1. Look for any test with multiple assertions.
2. Move those assertions to the bottom of the test. (If they aren’t already at the bottom, then you might have more than one action per test; this refactoring will help you discover that.)
3. Extract all the assertions together into a single method.

Now look at the new method. How many different objects does the method use?

If it’s more than one, then you almost certainly have unrelated assertions in the same place, so consider splitting the unrelated assertions into separate methods, then split the test into two so that each test invokes one of the two new separated assertion methods.

If your new assertion method uses only one object, then it might not be so clear whether those assertions are related. You can try this simple test: put all the values you’re checking in your assertions into a single object. Can you think of a good name for it? If yes, then perhaps you’ve just identified a missing abstraction in your system; and if not, then perhaps the assertions have too little to do with each other, in which case, try the trick in the preceding paragraph.

I apologise for not having a good example of this right now. If you point me to one, I’ll analyse it in this space.

• Where did that yak come from?
• When you try to learn a new library at the same time as explore the behavior and design of your application, you slow down more than you think.
• When you can’t figure out how to make the new library work for this thing you want to build, you might spend hours fighting, debugging, swearing.

Stop. Write a Learning Test.

1. Start a new test suite, test class, spec file, whatever you want to call it.
2. Write a test that checks the things you tried to check earlier with debug statements.
3. Write a test that has nothing to do with your application and its domain.
4. Remove unnecessary details from your test.

When this test passes, then you understand what that part of the library does. If it behaves strangely, then you have the perfect test to send to the maintainers of the library.¹

The Details

I just did this on a project using the context-free grammar parser treetop. Of course, I hadn’t used treetop before, so I had to learn it at the same time as design the grammar for the language I wanted to parse. I reached the point where I couldn’t write a grammar rule correctly, and spent probably an hour trying to figure out get it to work.² Fortunately, at that moment, my laptop ran out of power, so I left the coffee shop³ and did the usual thing: I explained the problem to my wife so that I could hear myself doing that. After about 15 minutes away from the problem, I decided to write some Learning Tests.

Summary of what I did

1. I wrote a Learning Test for a simple case that I thought I already understood well.
2. I wrote a Learning Test similar to the problem I had to deal with, to make sure I understood that well.
3. I wrote a Learning Test for the exact case that behaved unexpectedly.

The whole thing took an hour, and I understood the problem well enough to explain it to my wife. She understood it and agreed that it sounded like a mistake in the library.⁴ I used this Learning Test to open an issue at github. Now I can proceed without pulling my own hair out.

Do you want to see the Learning Tests?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

require 'treetop'

describe "Grammar with a simple rule" do
  let(:subject) { Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRule
  rule word
    [A-Za-z]+
  end
end
GRAMMAR
  )}

  let (:parser) { subject.new }

  it "doesn't match empty string" do
    parser.parse("").should be_false
  end

  context "matching single letter, the match result" do
    let(:result) { parser.parse("a") }

    it { result.should be_true }
    it { result.text_value.should == "a" }
    it { result.to_s.should_not == "a" }
    it { result.should_not respond_to(:word) }
  end

  context "matching many letters, the match result" do
    let(:result) { parser.parse("aBcDeF") }

    it { result.should be_true }
    it { result.text_value.should == "aBcDeF" }
    it { result.to_s.should_not == "aBcDeF" }
    it { result.should_not respond_to(:word) }
  end
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

require 'treetop'

describe "Grammar with a simple rule that uses a label" do
  context "Labeled subexpression followed by another expression" do
    let(:subject) { Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRuleWithLabel
  rule word
    letters:[A-Za-z]+ [A-Za-z]*
  end
end
GRAMMAR
    )}

    let (:parser) { subject.new }

    context "matching many letters, the match result" do
      let(:result) { parser.parse("aBcDeF") }

      it { result.should respond_to(:letters) }
      it { result.letters.text_value.should == "aBcDeF" }
    end
  end

  context "Labeled subexpression without another expression" do
    it "does not represent a valid grammar, even though I think it should" do
      lambda {
        Treetop.load_from_string(
<<GRAMMAR
grammar SimpleRuleWithLabel
  rule word
    letters:[A-Za-z]+
  end
end
GRAMMAR
      )}.should raise_error(RuntimeError, /Expected \#/)
    end

    it "really should let me refer to the expression as #letters" do
      pending "https://github.com/nathansobo/treetop/issues/21"
    end
  end
end

• Invert the dependency on a Service, moving the new statement up one level in the call stack.¹
• Repeat for all dependencies on Services until the corresponding new statements arrive at your application’s entry point. The entry point now creates a large object graph of all your Services in its initialise function.
• Remove duplication in EntryPoint.initialise():
  • Instantiate common objects only once, passing them into the necessary constructors, replacing any Singletons with plain objects.
  • Extract the choice of implementation for each Service interface into a lookup table mapping interface type to implementation type.
  • Externalise the lookup table to a file, if you like.

Now you have a customised Dependency Injection Container for your application. To go a little farther:

• Remove duplication in EntryPoint.initialise() among three applications.

Now you have a generic Dependency Injection Container that probably provides 80% or more of the features you’ll ever need.

I recommend trying this incrementally. Think of the new statements flowing up the call stack, into the entry point, then changing from code into data. Nice, no?

The Details

I don’t have much to add.

I hope this helps to demystify dependency injection containers. To read about the technique of injecting dependencies, I refer you to one of my articles and then a trusty web search.

This technique applies the Dependency Inversion Principle repeatedly to move the choice of implementation for an interface up the call stack. This way, concrete things depend on abstract things.

Removing duplication in the entry point respects the principle Abstractions in Code, Details in Data, but it does rely on reflection, which can cause some problems. All the better not to scatter this reflection throughout the code causing a serious cohesion problem. Using reflection like this, all in one place, helps balance using a powerful technique with a design that everyone can understand.

Now you know what about 70% of what a dependency injection container does. You can build one. Even if you don’t go that far, the Dependency Inversion Principle will help reducing coupling and highlight cohesion problems in your design. It provides another set of mechanics to practise on the way to becoming an accomplished designer.

No doubt you already know about the Composed Method design pattern, which we commonly use to help understand code as we read it. Most commonly you encounter a block of code with a comment that describes what that code does. You then extract that block of code into a method whose name matches the comment. I’ve used this technique for well on a decade to raise the level of abstraction in code, help code document itself, and eliminate misleading comments. While introducing these methods helps me read the code, it sometimes hides the tangle of dependencies that makes separating responsibilities so difficult. For this reason, I recommend trying to extract pure functions instead.

To introduce a pure function, start with a block of code and extract a method for it. Now look at all the fields and global variables that the method reads and introduce each one as a parameter to the method. Now look at all the fields and global variables that the method writes to and turn these into return values. Where the old code invokes the new function, assign each new return value to the corresponding field or global variable. You know you’ve done this correctly if, of course, the overall behhavior of the program hasn’t changed and you can mark the new function as static or whatever your language calls a class-level function.

In some languages, like Java, you’ll have to introduce a new little class to allow you to return multiple values from the new function. If you don’t want to do that right away, then return a Map of return values. Once you see that you need to return similar Maps from different functions, consider replacing those Maps with a new class. Perhaps that new class will attract some code!

When I’ve used this technique, two key things have happened: either I’ve noticed duplication in the parameter lists of the new functions or introducing a parameter has changed the behavior of the system. In the first case, I introduce Parameter Objects for the duplicated parameters, which then probably attract code and become useful domain objects. In the second case, I’ve detected temporal coupling, which requires me to separate the function into two smaller ones so that some output from the first becomes input to the second. This helps me uncover cohesion problems, usually of the type of different things written too close together.

I realise that an example would help right about now, but I would rather create some screencasts than write out examples in code, but I don’t know when exactly I intend to do that. I wanted to share this idea with you without waiting for the energy to put together a suitable screencast.

I invite you to try introducing pure functions into some legacy code and practising the technique as a kata. Get used to the various maneuvers, like introducing Parameter Objects or Return Value Objects or solving temporal coupling by splitting the function in two. It sounds crazy, but I’d like to try a Legacy Code Retreat where we practise only this technique all day. I don’t know whether anyone else would find it valuable enough to try it together for an entire day, over and over and over.

Would you? Add your comments below.

I write Learning Tests to discover how a third-party library works. I isolate myself from the third-party library through a layer of interfaces and adapter classes that evolve from the common ways I use the third-party library. I call this the “Pattern of Usage API”, as it represents the way my application uses that third-party library. Now my application uses the third-party library through a layer of interfaces, which means that I can introduce Contract Tests on those interfaces. These Contract Tests effectively describe the subset of the third-party library’s behavior on which I depend.

Now when I upgrade the third-party library, I run the Contract Tests against my adapters to that library. Test failures usually indicate a backwards incompatible change in the third-party library. (Sometimes they indicate a trivial difference in the API which requires a trivial fix, such as an API call having been renamed or something.)

Of course, this only helps me detect behavior changes related to computing answers, and not related to responsiveness, reliability, scalability, and so on. For that, I’ll always need system tests.

The Contract Tests are almost always state-based integration tests. I simply limit these to the implementation of Pattern of Usage API and don’t let it leak farther up the call stack. At some point you have to integrated with the Outside World. I simply teach people to look to make that integration thinner.

When I can’t decide whether to remove a certain bit of duplication I’ve found, I fall back on two rules:

1. Remove duplication only after you see three copies.
2. If you don’t know how to remove this particular kind of duplication, then write more tests. Either you need more examples to see the pattern, or more examples will show a different, better pattern.

I also remember two guidelines:

1. Don’t be afraid to remove duplication by introducing a method or class or interface with a stupid name. Remember that you can always improve the name later.
2. If you sense duplication, but don’t really see it, or can’t explain it to others, then make the surrounding names more precise; then maybe you will see the duplication.

If you click here you’ll see an almost textbook example of a contract test: that is, a test class that can run the same set of tests for two different implementations of the same interface. I would change only one thing: I’d extract the tests into an abstract superclass—something I otherwise hate to do—and pull the declaration of the method MakeTestSubject() up there, leaving two subclasses, one for the real file system and one for the simulated one. “YAGNI,” you say, and I agree, but I prefer the symmetry of the abstract superclass design to the asymmetry of having one class inherit from the other. I find it easier to grok quickly.

Either way, I feel good seeing contract tests out in the wild. I’m not so crazy after all.

• Adding behavior confidently involves having fewer parts to change (low duplication), knowing which ones to change (high cohesion), ease of changing just the part you want to change (low coupling), and understanding how you’ve changed it (strong tests).
• Adding behavior requires breaking an existing assumption.
• In a well-factored design, we can easily find the one place we have made that assumption. (Otherwise, why bother refactoring?)
• First, make room for the new code, then add it.
• To make room for the new code, extract the existing code into a method whose name describes the generalisation we want to make, or the idea we want to introduce.
• By making room for the new code, we make that code easier to reuse by reducing its dependence on its surrounding context.

Want to learn more?

• Visit the links in this article.
• Attend Agile Design: Beyond the Basics.
• Pair with me.

The Details

We all want to add behavior to a system confidently, and I have observed that my confidence in adding behavior depends on two factors:

• I know where to add code.
• I understand the behavior of the code I am adding.

I use test-driven development as the main technique for handling the second of these two factors, but what about the first? I have uncovered a technique that I both use and teach, and I’d like to share that with you. I call this “making room for the new code”, naming it for a phrase I vaguely remember reading in one of the Grand Old XP books. (Did Kent Beck or Ron Jeffries write it? I can’t remember.) This technique helps me quickly find a reasonable first-draft place in the code base to put new code. After I have put the new code in place, and I feel confident that it does what I expect, I then use the Four Elements of Simple Design to guide me in refactoring to improve the design.

A premise

I start with the premise that adding behavior means breaking an assumption. By this I mean that whenever we add code to a system in order to extend its behavior, we have to falsify at least one assumption we’ve previously made. For example:

• In a payroll system, in order to support a second cheque printer, we likely have to break the assumption that there is only one cheque format.
• In a point of sale system, in order to support separate cash and card payment reports, we likely have to break the assumption that all “we made a sale” events look the same.
• In a mobile phone monitoring system, in order to support billing by the second, we likely have to break the assumption that we only have to count the number of minutes a call lasted.

Some of these seem obvious, and others less so, and it bears emphasising that the specific assumption or assumptions we break depends heavily on what we’ve built so far and the way we articulate the soon-to-be-added behavior. Even so, I conjecture that for every behavior we want to add to a system, we can identify a non-empty list of assumptions that we need to break.

The technique

1. Identify an assumption that the new behavior needs to break.
2. Find the code that implements that assumption.
3. Extract that code into a method whose name represents the generalisation you’re about to make.
4. Enhance the extracted method to include the generalisation.

The less duplication you have in the system, the better this works, because duplicate code makes it difficult to find all the code that implements the assumption in question. Similarly, the more appropriate the names in your system, the better this works, because unsuitable names make it difficult to know which code implements the assumption in question, as opposed to something unrelated. You’ll notice that these points relate both to the Four Elements of Simple Design and to the core concepts of Coupling and Cohesion.

Yes, yes…

An example

We are building a point of sale system, and we’ve just decided to implement sales tax. I live in PEI, Canada, where not only do we exclude sales tax from the price, we have two sales taxes, and we charge the second one on top of the first one. For example:

A $125 item that attracts both GST (the “Goods and Services” tax) and PST (provincial sales tax) costs a total of $144.38. GST at 5% costs $6.25, then PST at 10% of $131.25 (= $125 + $6.25) costs $13.13. The total is $125 + $6.25 + $13.13 = $144.38.

Notice that this example implies that GST or PST might or might not apply to a given product, so even before we identify the old assumption to break, we need to note the new assumption we’ll make: we assume that all products attract both GST and PST. Our customers won’t like it, but the tax authorities will love it, and only they have the power to treat us guilty until proven innocent.

Our code has a class like this in it.

What do we assume now that we can’t allow ourselves to assume any longer? The sale total should increment by the (net) price of the item we sell. By net price here, I refer to the pre-tax price, because of course, until now, our system has no notion of “tax”. Fortunately, because we’ve ruthlessly removed duplication so far, computing the running total of the sale requires only this line of code. We can pretty safely apply the technique of this article right here. To do this, we extract the assumption into a new method whose name represents the generalisation we’re about to make. In this case, we don’t want to increase the sale total by the product’s price, but rather by its cost, which includes any additional charges beyond net price. So, we introduce a method for accumulating the scanned product’s cost.

This creates space for the new code. We test-drive the new code, and end up with this delightful monstrosity.

It looks ugly, but it works. In addition to looking ugly, this method has Feature Envy. Specifically, the calculation part of #accumulate_cost only talks to product, and so it can move onto the class Product, leaving only the accumulating left behind. You could also say that this method had two responsibilities, so I separated them, then notice the feature envy in one of them, then moved it. I can almost always take smaller steps.

Notice the context independence we’ve achieved with the method Product#cost. We can now refine the notion of “cost” freely without worrying about how someone will use that information. We have an easy-to-find, easy-to-extend part of the system where we can add behavior for determining which products attract which taxes, supporting multiple tax calculation policies, and even including shipping, handling and restocking fees. Now we can really add future behavior with confidence.

A model for improving the names of variables, fields, interfaces, classes and namespaces in a system. Practise this and more in my course, Agile Design: Beyond the Basics.