def fibo(

  (map first (iterate (fn[a b] [b (+ a b)]) [0 1]))

These learning tests are in Shubox - check it out.

A few colleagues and I recently discussed whether or not traditional (aka object-oriented) TDD techniques could get you to the fibo implementation. Aaron Bedra from ThinkRelevance was there and brought up some interesting points about things we need to bear in mind as we approach TDD in a functional language:

1. Are we hanging on to our OO-specific ideas?
2. Do we really understand the data abstractions in functional programming (FP)?
3. Does TDD create the best feedback loop for FP?

Here are my observations:

1. Are we hanging on to our OO-specific ideas? This is the crux of moving from one paradigm to another. You can write fibo recursively the way you might in Java, but you’ll hit a wall and either perform horribly at best or blow the stack at worst [http://en.literateprograms.org/Fibonacci_numbers_(Java)]. Stu does a good job of exploring the consequences of writing this kind of bad Clojure in the book, so I won’t belabor the point here. Suffice it to say, beware of approaching the problem with the wrong paradigm.

2. Do we really understand the data abstractions in functional programming? TDD will not get you to lazy sequences if you don’t already know them. You may discover the need for lazy sequences when you write a test that blows the stack, but you’re going to need reference material to find the solution. This is where I think TDL can help: By coding up some learning tests with which you can drill, you can explore the nuances of functional idioms until you’ve mastered them. You can then employ these techniques intuitively when needed.

3. Does TDD create the best feedback loop for FP? There’s no substitute for the REPL in quickly experimenting, but it’s no replacement for building up a suite of learning tests. Whatever you’ve coded up in the REPL is gone when you close your session. Learning tests, just like unit tests, can be composed in suites that you can review and practice with later. The REPL makes a great companion to a learning test or TDD session, however, for experiments with one-liners.

Given the need for tests, how do you gradually get to the solution in an FP TDD session? Sam Newman recently blogged about struggling with test-driven Clojure:

“My first instinct is to start decomposing functions, passing in stubs to the functions under test. But this just feels like I’m trying to shoehorn IOC-type patterns into a functional program. But what am I left with – testing large combinations of functions? That feels wrong too.”

I think it depends on what you mean by “large”, Sam. If the functions are core library functions such as the one in fibo, I’d be comfortable having a test such as:

(deftest fibo-is-an-infinite-sequence

  (are [description x expected] (= x expected)

    "first seven members" (take 7 (fibo)) '(0 1 1 2 3 5 8 )

    "5th member of the sequence" (nth (fibo) 5) 5

    "last 5 numbers of the ten thousandth fib number" (rem (nth (fibo) 10000) 100000) 66875))

Jay Fields will remind me to use one assertion per test, but I usually only listen to Jay when he’s talking about food, prosecco, or scotch. If my base functions were bigger one’s that I’d written, I’d write separate unit tests for them. Those tests would also probably have more than one assertion, though. Sorry, Jay.

“Stuart Halloway said during his Clojure talk at Qcon SF that despite being a TDD fan he finds it hard to TDD in a new language, and I get exactly what he means. A big part of it is that you’re getting to grips with the idioms, capabilities, libraries and tools associated with your new language – and a lack of this knowledge is going to impact on your ability to write good tests, let alone worry about implementing them.”

I thought the same thing, so I packaged up a set of scripts in a gem called Shubox. Shubox provides the ability to generate a TDD sandbox with all the libs and scripts you need for an automated suite of regressions tests.  I have environments for Ruby, Java, and Clojure at the moment. The name comes from Shu Ha Ri - when you find yourself in a novice Shu state with all or part of a language, I hope that Shubox will provide a training ground for you to experiment, learn, and eventually move to the next level.

All the documentation you need to get started is with the code on Github at http://github.com/bobbyno/shubox. Shubox is built with RubiGen, the same generator framework in Rails. Adding support for a new environment really comes back to answering Sam’s question about finding an idiomatic approach for TDD in each language. I plan on adding many more environments and would like to at least keep up with Bruce Tate’s Seven Language in Seven Weeks. Patches for new language support or updates to the existing environments would be welcome.

Those of you who saw my presentation about Test-Driven Learning at Software Craftsmanship North America last summer know that I’m interested in using learning tests to better learn languages, patterns, idioms, etc. through repeated practice. I’ve found test-driven learning of this sort to be an excellent compliment to other forms of experimentation, such as katas or small applications.

I’ve been using TDL to experiment with functional programming as of late: Shubox contains an example in Clojure from Stu Halloway’s Programming Clojure. More on that soon.

Enter unit testing. Unit testing gives you a way to build yourself a library of examples that are verifiable, executable, and self-documenting. This technique isn’t new. 4 years ago, Mike Clark wrote the best material I’ve found on the subject:

•    Ruby Learning Test #1: Are You There, World?
•    Ruby Learning Test #2: Active Reading

I only found these posts about 6 months ago, however, and I suspect many people haven’t seen this technique. I’ve found it to be a great way to practice in a safe environment rather than subjecting production code to my experiments. It’s the code equivalent of learning to play the daf in a sound studio: You can record yourself, use reference materials, and do as many takes as you want. And you won’t disturb the neighbors.

Andy Hunt of the Pragmatic Programmers has a blog post on test-driven learning in which he talks about mixing the technique with concepts like the spacing effect and spaced presentation. “Spaced repetition is a learning technique that exploits this psychological principle by incorporating increasing intervals of time between subsequent review of previously learned material.” I’m interested in exploring how this system could be used in mastering software API’s. In the meantime, I’ll keep working on test driving my way through a few more API’s and will blog any interesting surprises I find along the way.

Application code is one part of the system. So are the database, the configuration, the network, the documentation, and the host machine, to name a few others. One by one, these systemic concerns are being pulled into the CI process.

As the technical roadblocks to automation are removed, we’re also solving the political challenges: Operational silos are being knocked down in favor of collaboration. Product managers are thinking about how to structure features such that they can add value incrementally. Rapid delivery of high quality, valuable software is what agile methods is all about, and bringing automated deployment into the CI process is a natural extension of any agile process.

Just as Scrum introduced an “expanded definition of done”, the definition of “integrated” is expanding. Deployment is traditionally considered something that happens after the CI build. “Build, test, deploy”. I think we’re all coming around to the realization that we really need a “build, deploy, test” CI process in which we deploy the application to a staging environment before we run our functional tests, for example. Since this deployment is part of the CI build, this has to be automated. If we’ve solved deployment to a staging environment, why not deploy to production?

SmartFrog’s Patterns of Deployment includes the CI server right in the definition of continuous deployment. This theme resonated at CITCON NA 2009, with the discussions frequently turning to configuration and deployment. I mentioned I’d post the outcome of the discussion I facilitated on continuous deployment we had there. Here’s the first part, a summary of the top ten indications that continuous deployment makes sense for your team:

* To more quickly add value; get the code into the hands of users; remove inventory - inventory is waste.

* To elicit faster user feedback about defects, quality of the application, and value. Bugs should be fixed. Quality issues should be addressed immediately. Features that don’t add value should be dropped.

* For rehearsal; practice shipping to production using the same automation used to ship to a staging environment. Remove bottlenecks through early identification.

* The organization values happiness. How do people want to spend their time? Fighting fires and fixing production bugs isn’t fun. Why not spend the time up front to automate, rehearse, and avoid the “Last Mile Problem” that occurs when code piles up and before shipping? The organization must be willing and empowered to acknowledge and remove roadblocks.

* The cost of a failed deployment isn’t catastrophic - manual review gates in between automated steps might be deemed necessary as a safety valve in order to catch errors that can’t be automatically detected in the test scripts. Automation reduces risk, though, so even apps with a high cost of failure can benefit from automation as long as there is an element of control and auditing.

* Support costs of a new release are low. Companies don’t want to support hundreds of versions of a COTS product on store shelves, whereas a hosted web application effectively only has one live version at any given time.

*The licensing / revenue model is compatible with frequent releases. Subscription models, free upgrades within a minor revision, etc.

* The users cost of consumption is low, or users can opt to not upgrade. The previous point was about the cost of a deployment to the developing company. This point is about the cost, whether cognitive or financial, for consumers to make use of the new features as they roll out. An application that requires extensive training is obviously harder to continuously deploy than a chat avatar, for example.

* There aren’t complex legal issues with deployment. A new feature might have regulatory implications. Perhaps SOX compliance must be proven before a feature can be released. Situations like this aren’t the norm, but they do happen.

* The team is capable of excellence. One unreliable release will cost trust, user loyalty, and team morale. Too many unreliable releases will cost the product line and possibly the company. This is true regardless of whether or not automation used, obviously, but when we’re talking about continuous deployment with releases that can happen many times a day, we’re hopefully talking about feature releases, not just bug fixes. A solid development process that includes CI is a pre-requisite; I wouldn’t recommend continuous deployment to amateurs who haven’t yet written their first unit test.

With the justifications for deployment being a core part of continuous integration, I’m up for moving on to how it’s done. Now that distributed enterprise CI servers are becoming the norm, we can have a deployment agent on the machines in our QA and production environments that can grab our tested artifacts and run the same deployment scripts used in testing, for example. This is only one approach, though, and I’ll save a more in-depth discussion for another post.

The downtime and confusion that would ensue could hardly be considered agile.

Of course, this isn’t what’s intended by the terms ‘continuous integration’ or ‘continuous deployment’. Continual means ‘frequent, repeating at intervals’ and continuous means ‘going on without pause or interruption’. So shouldn’t we be talking about ‘continual integration’?

“Continuous Integration increases your opportunities for feedback. Through it, you learn the state of the project several times a day. CI can be used to reduce the time between when a defect is introduced and when it is fixed, thus improving overall software quality.” - Continuous Integration: Improving Software Quality and Reducing Risk

As agile developers, we choose to build and deploy as frequently as we can in order to maximize feedback and the opportunities for learning about the state of a system that is otherwise obfuscated. An interval as often as every check in seems to be a reasonable compromise between frequent feedback and absurdity.

The point of the term continuous isn’t to be linguistically precise: It’s to act as a challenge to improve our process by automating as many of our release management tasks as is practical. It’s about a commitment you make with your team to keep the build green and to fix broken windows as soon as they appear. Is “post check-in integration” going to change your team? Is “pretty frequent deployment” going to revolutionize your organization? Somehow those terms don’t quite have the same call to action to them, do they?

The benefits of continuous integration have been well-documented. Applying the same automation challenge to deployment is relatively new, but it makes sense. As continuous integration breaks out of it’s agile home ground and goes mainstream, more people are realizing that they’re already producing their release candidates as part of the build, so why not actually deploy the same way, using nothing more complicated than scripts and a CI server?

The why’s and why not’s of continuous deployment took up an hour when I facilitated a discussion with about 50 participants at CITCON NA 2009 this weekend; I’ll be posting the results of that discussion soon.

I’m happy to see this happening: It validates a lot of the theory we built into Cruise. The Cruise team and others at ThoughtWorks are as interested in deployment as they are in keeping the builds green. Getting code to production after it passed through successive stages of automated and manual testing fits with a build pipeline metaphor, so the pipeline became the core abstraction in Cruise.

The Cruise team uses Cruise as a CI and deployment server: There’s a UAT environment used by the Cruise development team, a pre-production environment used by about 100 ThoughtWorks developers, and the public release that’s sold to the world. Although the deployment was automated through Ant scripts, we left the gates between the deployment stages set to manual to make things a bit more predictable for the development team and, more importantly, our early users. A post on the XP mailing list I received at the beginning of my Cruise tenure presaged this outcome:

On November 27, 2007,  

The biggest barrier to continuous deployment
is gaining the trust that what you're going to
deploy isn't going to destabilize the environment.
This takes time and effort.

John Roth

Indeed. No developer wants the CI server to go down during the workday. We were using Cruise to build itself within a couple of iterations of starting the project, but of course the product took some time to stabilize. We rolled out our first production release two weeks after we started using Cruise in UAT. Once our user base increased, many felt that the risks of the production server being down due to a failed deployment outweighed the benefits of getting new features and bug fixes out as they became available: Users were perfectly content to wait until we’d fielded it in UAT, and even the dev team didn’t necessarily want to use every check in; it was enough to know that we could trigger a deployment on demand, as we frequently did throughout an iteration.

Should a CI server be concerned with deployment? I think so. Why do you use a CI server in the first place? Immmediate feedback. Visibility. Reporting. The peace of mind that follows. CI also implies automation, and that means executable documentation, repeatability, reduction in errors.

Don’t we also need all of these benefits in our deployment process? For many new to CI, the continuous integration process leaves off after testing, contributing to the “Last Mile” problem. I’ve seen some cases where the supposed CI build doesn’t have any tests! I hoped these were isolated incidents, but there are enough experience reports to the contrary that lead some to ask whether or not we need standards for CI. Assuming you’ve got a working CI build, why not go to the next level and deploy using the CI server?

Cruise, AnthillPro, and ControlTier have the concept of deployment baked right in. There are doubtless other servers offering these features. Either we’re all off on a tangent, or were on to something and Continuous Integration (CI) is becoming Continuous Integration & Deployment (CID). Maybe in a few years, leaving deployment out of your CI solution will seem gauche as leaving out tests.

There’s a lot more to say about how the challenge of continuous deployment affects the software lifecycle and how it fits with more established configuration management practices in CMMI and ITIL. Although I’ve since left ThoughtWorks, I’m working on a continuous deployment solution for my new employer, DRW Trading. I’ll continue to post more of my thoughts on CI and CD, and I’m looking forward to discussing more of these ideas with members of the community at CITCON North America in April.