Continuous Integration

Supports Pillars:
    Confidence, Collaboration

Dependencies on Other Skills:
   

    Collective Ownership

Definition:

    When a team of developers works on a project, changes that each one makes impact the work of the others. Conflicting changes can cause problems during integration. The more frequently we integrate the less painful it is when such conflicts occur, because the number and size of conflicts is small.

    To do so, we first integrate other people's changes from source control and make sure that the build passes locally. Then, we check in our own changes and run the build somewhere else. That verifies that everything works outside of our own environment (We could have local changes or configuration that we forgot to check in.)

CI is often seen as a two stage process, firstly the act of integrating our code with all other code on a main branch, ensuring that the build passes which includes all our unit tests. This first stage will normally happen hourly or many times an hour. This type of build should be very fast, typically taking minutes, as all unit tests are run in memory. This first stage will generally run on a CI server amd will normally not communicate with products outside of itself.

    Once the first stage passes all tests,  a second stage of CI should occur. This can occur at regular intervals, typically daily or a few times a day, which includes the running of automated acceptance or requirements based tests.


Resources:
    The Art of Agile Development, Shore/Warden, 183-192


Steps to Mastery:

    Can identify what kinds of code changes will cause problems for other developers, and alerts them in advance.

   Writes code iteratively, in small tested units, which can be released at least daily.

    Designs code so that the number of classes impacted by any feature enhancement is minimized, while avoiding huge classes that know everything.

    Performs integrations in a way that is non-disruptive, fails fast, and rolls back graciously


Organizational Support of this Skill:

    An automated build, including all the steps necessary to have a working, tested, production ready application.
    A continuous integration server, kept in a stable, healthy state.

Subpages (1): Using Version Control

Comments

D. André Dhondt - Dec 22, 2009 12:06 PM

On Physical Tokens, by Adam Sroka:

CI, as originally practiced by the C3 folks, involved actually running the build on another machine to verify it. Since this was a manual process it was useful to have the token to say, "It's our turn, wait until we see it pass or else we rollback."

Without the token we are in a reactive mode. I have to wait for CruiseControl to tell me I broke it several minutes ago and then reintegrate with my cohort's subsequent commits. This is usually not that hard, because most people don't commit as often as they could. Still, we aren't in a known state. There may be false negatives - my teammate checks in something and the build fails, but it's failing because of my change. There may also be masked failures - the build is not failing due to my checkin, because it is failing on someone else's mistake before it ever gets to mine. It's more like Continuous Churn than Continuous Integration.

Some modern CI servers (TeamCity, Quick Build, possibly others) support a two-phase commit so that broken builds are never submitted to the shared repository. I think that is an even better solution. (I have a version of this in a tool that I built to do distributed CI, but I don't know if it will ever see the light of day. Too many other things on my plate.) With such an approach the token is entirely unnecessary, but without it I think the token is a good idea.

Jeff Hoover - May 19, 2010 8:44 AM

I recently read a blog post entitled "I invite you to break my build". ( http://www.agiledeveloper.com/blog/PermaLink.aspx?guid=65f3a449-3f65-41d4-85dc-61de8421da32 ) in it, the author suggests that leaving a build broken is much worse than breaking the build in the first place. Breaking a build and fixing it quickly is an opportunity to learn where there are holes in a team or individual developer's process. Fear of punishment for a broken build makes developers cautious and robs them of the information that breaking a build provides.
In summary, teams may want to do their best not to break the build, but not from a fear place. Broken builds should be fixed quickly, and they should be used as information about how the development process can be improved so the build isn't broken the same way again next time.

Jeff Hoover - May 19, 2010 8:52 AM

A concern I have about an integration token is that if it is not returned quickly, it will block other people that want to integrate, slow the team down, and possibly cause larger merges as developers continue to code.

I'm less concerned with the Continuous Churn that Andre mentions. I don't often see the case where it's hard to whose checkin caused compile or link fails. And if a checkin causes tests to fail, good unit tests tell me precisely where the failure is and it will be easy to tell who caused it.

Richard Foster - May 21, 2010 5:28 AM

Like Jeff, Continuous Churn has not really been a problem. I agree that a multi-stage integration (like that offered by TeamCity) offers significant advantages, but we've been using single-stage without too-many problems. Our acceptance tests are still, sadly, manual - mostly because they require interaction with custom hardware that cannot sensibly be simulated (mocked).

I think CI should also have a dependency on Testing (probably Unit Testing in particular). If you have missing (or fragile) tests, you may get a false sense of security from CI. "It passed the build, therefore it's ready for the next stage" is only as good as the tests the integration is based on.