Continuous Integration Requires Integrating Continuously

LMAX Exchange

Sarah Goff-Dupont has a post on the Atlassian blog about how Atlassian is using feature branches and still doing continuous integration:

In
the not-so-recent past, continuous integration in the context of story
branching was considered so impractical as to be outright incompatible.
 Who has time to manually configure a CI scheme for dozens of
active branches that will only live a couple of days?  Nobody
that’s who.  And story branch-loving teams would frequently
encounter *ahem* “surprises” when merging work onto the main code line
”master”, in Git-speak.  But recent versions of Bamboo have
essentially removed that overhead and allowed these two frenenmies to
become genuinely harmonious partners in our development lifecycle.

Unfortunately,
what Sarah describes is not continuous integration. At best, it’s
regular integration but really it’s just automated testing. The key
principal of continuous integration is that all developers are merging
all changes, all the time. That way you see the messy combinatorial
effects of concurrent changes and can fix them before you waste a lot of
time going down incompatible paths. Anything that reduces the frequency
of doing that integration is a step away from continuous integration,
regardless of how often automated tests are run.

Let’s take a look at the basic process they’re using:

Next,
we go into development and testing cycles on the story branch.
 The developer is making local commits on her workstation.
 […]  When she reaches a point where she wants to run tests
against her work, the developer pushes those local commits up to the
repository still on the story branch, of course.  Bamboo sees that
there’s been a change, and starts building the corresponding Plan
branch/es.

This is not continuous integration. The
developer’s changes are being kept away from the main team and not
integrated while they’re being developed, while they sitting just in the
local git repository and while they’re in the story branch. Even though
Bamboo runs tests against the code, there has been no integration done
here. This is just automated testing, not continuous integration.

It seems that Atlassian have discovered this lack of integration and have developed ways of combating it:

To
help find conflict before merging to master, the developer can choose
to have Bamboo merge master into her story branch each time she pushes
changes to the repo, then run through the CI Plan against the merged
code. If the Plan passes, Bamboo will commit the merged code to her
branch.

Finally we get some integration happening,
but only one way. Since everyone’s off working on feature branches this
is really only integrating with completed stories. All the work going on
in parallel on other stories is still not integrated so we have no idea
if they will work when combined.

Some teams are
taking this workflow a step further and requiring developers to merge to
an integration branch during development, before finally merging the
completed story to master. This practice ferrets out integration issues
even faster. Developers get to mash their in-progress stories together
several times a day, rather than waiting until one has merged a
completed story to master (which may happen only every 2-3 days) and the
other has pulled code from master onto their story branch.

The
key lesson here is that if we integrate more often, it’s easier to
ferret out integration issues. It’s like continuous integration but with
the extra fun of managing branches…

Of course, Atlassian aren’t
alone in the use of feature branches it’s become all the rage since
DVCS supposedly made branching cheap and merging painless. Unfortunately
while DVCS has eliminated many of the basic mechanics of merging, they
haven’t done anything to reduce the risk of different developers push
the codebase in incompatible directions at a structural level. The only
way to avoid that is to integrate all changes as often as possible
that’s continuous integration.

The only way that feature branching
and continuous integration can be at all compatible is if the build
system automatically builds and tests every possible combination of
branches, all the time a massive waste of build system resources.
Anything less leaves potential combinations untested until they are
finally merged on to master and all the problems are discovered.

Or you know, we could just do continuous integration…

Any opinions, news, research, analyses, prices or other information ("information") contained on this Blog, constitutes marketing communication and it has not been prepared in accordance with legal requirements designed to promote the independence of investment research. Further, the information contained within this Blog does not contain (and should not be construed as containing) investment advice or an investment recommendation, or an offer of, or solicitation for, a transaction in any financial instrument. LMAX Group has not verified the accuracy or basis-in-fact of any claim or statement made by any third parties as comments for every Blog entry.

LMAX Group will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information. No representation or warranty is given as to the accuracy or completeness of the above information. While the produced information was obtained from sources deemed to be reliable, LMAX Group does not provide any guarantees about the reliability of such sources. Consequently any person acting on it does so entirely at his or her own risk. It is not a place to slander, use unacceptable language or to promote LMAX Group or any other FX and CFD provider and any such postings, excessive or unjust comments and attacks will not be allowed and will be removed from the site immediately.