Development - 'Extreme' Style!

This spring, we won a significant piece of business for our Small Business Marketing Product group. Game changing for Marchex, but one that presented the Engineering team with a clear and significant challenge; how could we take a relatively new (and growing) development group and deliver against a list of over fifty features (with fluid requirements) in a compressed time frame?

Time to adopt a new paradigm

What’s great about working at Marchex is the culture of innovation and the openness of the management team to try new ideas, and it was in this spirit that the idea of moving our development approach to ‘XP’ was born. Some of us in the group had prior experience working within an XP framework, and we were confident that not only could we deliver the required feature set, but that we could do so quickly, and improve our development quality. Armed with a set of 4 x 6 note cards and corkboards we set about to revolutionize how we translate product strategy into development excellence – bringing great products to market, quickly.

So what is XP?

At its core, XP (Extreme Programming) is a discipline of development that places communication and constant feedback at the center of the development cycle. Development is done by paired programmers who first write code to test the code they will write for the feature or story. This is TDD (test-driven development), which enables refactoring with the safety net of good test coverage to be sure nothing is broken by the changes.
In XP, the customer and the customer’s needs are always in the forefront of everything we do.

Pilgrim’s Progress

We are currently at 20 weeks and the process continues to gather momentum despite some bumps along the way. The team has delivered 5 major releases and 7 weekly maintenance releases during those 20 weeks with each release being easier and higher quality than any that came before.

‘Getting it right’ takes time and patience. As the team makes its way into uncharted territory the mantra of ‘crawl before you run’ has helped remind us that we are not ‘there’ yet and probably won’t be ‘there’ for some time to come.

Big Challenges

How do QA and UX Design fit into the XP process?

We were stumped at first on how to create an iterative process with incremental delivery that can incorporate QA and major UX work both of which need clear milestones and firmer timelines.

There is scant documentation or even chat from forums on how to roll QA and UX into a highly agile XP team so the answer to this question has been ongoing and challenging.

The specifics will be covered in a future post but the core answer is not surprising --- once the team consciously focused on better communication and expectation setting between all stakeholders, working across the disciplines became much easier and more productive. The key has been to understand how each stakeholder communicates and what their needs are.

When are we ‘done’?

As obvious as this might seem, we really didn’t have a clear idea of what ‘done’ meant.

For those of us using XP at a former company, we did not have a QA team and thus considered a task completed once the final code was written. As developer velocity (or ability to move through work) accelerated, QA became bogged down especially as new projects were started in development while older ones were still moving through the testing cycle.

Developers were focused on the new project and at times couldn’t even remember specifics about code they had written a month before.

There was great frustration on all sides as QA felt stuck and Development felt torn between the old and new priorities. Each group was context-switched between the old and new projects leading to even more churn and frustration.

The first part of the solution was incorporating a lesson from Kanban, another highly Agile discipline, which teaches “reduce Work In Process (WIP)”. Reducing the number of active projects helped the entire team to re-focus and work on the most important priorities first.

The second part of the solution centered on communication (as mentioned above) as well as holding the team to the stated tenet that no code is complete until it is successfully in production.

As a developer, part of writing the code is to support QA while they test it. As QA, part of testing is clearly asking for developer support if and when it is needed. Reducing the WIP meant developers would pitch in, however, it was productive to get the project complete (in production) so development on new features could begin.

Lessons Learned

• A core principle of XP was shown again to be vital: the Retrospective is the agent of change. The “Retro” with the entire team (Dev, QA, Ops, PgM) figures out what does and doesn’t work. The team owns the process and has the power and responsibility to make changes in it to improve constantly.
• Fail early and often – when a process/idea doesn’t work for the team have courage to discard it and try something different.
• Include QA and Ops from the beginning of a project.
• Start the development project with tasks to set the QA process up for success.
• Lack of TDD = Abundance of bugs.
• The most important part of any project is the communication between all of the stakeholders.
• Co-location matters - communication is easier and more effective when everyone is in immediate proximity.


On the Horizon

The ‘experiment’ phase is for the most part behind us now. As we go forward, we will be continuously improving and tinkering with process to make it better serve the team and our customers.

Areas on the radar:
• Integrate more tightly with IT/Ops using the model we have created for development/QA.
• Support the IT/Ops organization in using the XP model.
• Actively expand the XP process to other teams within Marchex (assuming it’s appropriate for their product and there is interest).


Final Thoughts

How specifically does XP benefit our customers ….. are they *really* seeing differences in the end products created using XP?

We believe so. Our weekly release rhythm for routine requests and ability to push out high-quality features on aggressive timelines are providing our customers the ‘proof in the pudding’ and we look forward to even more velocity as the team reaches the next level of experience with our systems.

• Quick turn-around on requests:
   o Even non-emergency requests can be coded/tested and released in two iterations (roughly 2 weeks) or less.
   o ‘Hot’ issues or requests can be turned around ASAP.

• Ability to respond to market changes in a timely fashion:
   o XP embraces change and can move quickly to keep our business on the cutting edge.

• Customer is King – NOT technology:
   o What the customer needs always comes ahead of ‘cool’ technology instead of the other way around.

We would love to hear about your experience (or questions) about XP! Email me!