Development - 'Extreme' Style!

This spring, we won a significant piece of business for our Small Business Marketing Product group. Game changing for Marchex, but one that presented the Engineering team with a clear and significant challenge; how could we take a relatively new (and growing) development group and deliver against a list of over fifty features (with fluid requirements) in a compressed time frame?

Time to adopt a new paradigm

What’s great about working at Marchex is the culture of innovation and the openness of the management team to try new ideas, and it was in this spirit that the idea of moving our development approach to ‘XP’ was born. Some of us in the group had prior experience working within an XP framework, and we were confident that not only could we deliver the required feature set, but that we could do so quickly, and improve our development quality. Armed with a set of 4 x 6 note cards and corkboards we set about to revolutionize how we translate product strategy into development excellence – bringing great products to market, quickly.

So what is XP?

At its core, XP (Extreme Programming) is a discipline of development that places communication and constant feedback at the center of the development cycle. Development is done by paired programmers who first write code to test the code they will write for the feature or story. This is TDD (test-driven development), which enables refactoring with the safety net of good test coverage to be sure nothing is broken by the changes.
In XP, the customer and the customer’s needs are always in the forefront of everything we do.

Pilgrim’s Progress

We are currently at 20 weeks and the process continues to gather momentum despite some bumps along the way. The team has delivered 5 major releases and 7 weekly maintenance releases during those 20 weeks with each release being easier and higher quality than any that came before.

‘Getting it right’ takes time and patience. As the team makes its way into uncharted territory the mantra of ‘crawl before you run’ has helped remind us that we are not ‘there’ yet and probably won’t be ‘there’ for some time to come.

Big Challenges

How do QA and UX Design fit into the XP process?

We were stumped at first on how to create an iterative process with incremental delivery that can incorporate QA and major UX work both of which need clear milestones and firmer timelines.

There is scant documentation or even chat from forums on how to roll QA and UX into a highly agile XP team so the answer to this question has been ongoing and challenging.

The specifics will be covered in a future post but the core answer is not surprising --- once the team consciously focused on better communication and expectation setting between all stakeholders, working across the disciplines became much easier and more productive. The key has been to understand how each stakeholder communicates and what their needs are.

When are we ‘done’?

As obvious as this might seem, we really didn’t have a clear idea of what ‘done’ meant.

For those of us using XP at a former company, we did not have a QA team and thus considered a task completed once the final code was written. As developer velocity (or ability to move through work) accelerated, QA became bogged down especially as new projects were started in development while older ones were still moving through the testing cycle.

Developers were focused on the new project and at times couldn’t even remember specifics about code they had written a month before.

There was great frustration on all sides as QA felt stuck and Development felt torn between the old and new priorities. Each group was context-switched between the old and new projects leading to even more churn and frustration.

The first part of the solution was incorporating a lesson from Kanban, another highly Agile discipline, which teaches “reduce Work In Process (WIP)”. Reducing the number of active projects helped the entire team to re-focus and work on the most important priorities first.

The second part of the solution centered on communication (as mentioned above) as well as holding the team to the stated tenet that no code is complete until it is successfully in production.

As a developer, part of writing the code is to support QA while they test it. As QA, part of testing is clearly asking for developer support if and when it is needed. Reducing the WIP meant developers would pitch in, however, it was productive to get the project complete (in production) so development on new features could begin.

Lessons Learned

• A core principle of XP was shown again to be vital: the Retrospective is the agent of change. The “Retro” with the entire team (Dev, QA, Ops, PgM) figures out what does and doesn’t work. The team owns the process and has the power and responsibility to make changes in it to improve constantly.
• Fail early and often – when a process/idea doesn’t work for the team have courage to discard it and try something different.
• Include QA and Ops from the beginning of a project.
• Start the development project with tasks to set the QA process up for success.
• Lack of TDD = Abundance of bugs.
• The most important part of any project is the communication between all of the stakeholders.
• Co-location matters - communication is easier and more effective when everyone is in immediate proximity.


On the Horizon

The ‘experiment’ phase is for the most part behind us now. As we go forward, we will be continuously improving and tinkering with process to make it better serve the team and our customers.

Areas on the radar:
• Integrate more tightly with IT/Ops using the model we have created for development/QA.
• Support the IT/Ops organization in using the XP model.
• Actively expand the XP process to other teams within Marchex (assuming it’s appropriate for their product and there is interest).


Final Thoughts

How specifically does XP benefit our customers ….. are they *really* seeing differences in the end products created using XP?

We believe so. Our weekly release rhythm for routine requests and ability to push out high-quality features on aggressive timelines are providing our customers the ‘proof in the pudding’ and we look forward to even more velocity as the team reaches the next level of experience with our systems.

• Quick turn-around on requests:
   o Even non-emergency requests can be coded/tested and released in two iterations (roughly 2 weeks) or less.
   o ‘Hot’ issues or requests can be turned around ASAP.

• Ability to respond to market changes in a timely fashion:
   o XP embraces change and can move quickly to keep our business on the cutting edge.

• Customer is King – NOT technology:
   o What the customer needs always comes ahead of ‘cool’ technology instead of the other way around.

We would love to hear about your experience (or questions) about XP! Email me!

Call Mining: Insights and Lessons Learned

Recently, my colleague Matthew Berk posted about the marketing benefits of being able to move beyond call transcription to a more meaningful level of customer insight provided by a deeper and scalable understanding of advertiser – consumer conversations.

Today we are announcing the availability of Call Mining, a technology-based speech analytics product designed to capture the most useful call data for advertisers that we are currently rolling out in beta to Marchex Call Analytics customers. I’ve been deeply involved with developing the product and on-boarding customers, and I wanted to share some trends and early insights that Call Mining is showing us.

The first conversation with an advertiser that leads them to become a customer of Marchex Call Analytics often starts like this: “I know that my search and display campaigns are generating phone calls. I just don’t know how many.” These conversations often beg deeper questions, such as “Where are my customers calling from?” or “How many of these phone calls are longer than a couple of minutes?” Marchex Call Analytics effectively answers these questions for millions of conversations each month.

Over the past year, though, questions from advertisers have increasingly been about the contents of the actual conversation between callers and agents (the person answering the phone). “Are callers booking appointments? What features are they asking about? Why aren’t they buying?”

When we set out to build Call Mining, these were the types of questions we wanted to answer, and the product allows advertisers to understand the types of conversations that their customers are having with their business. Coincident with our announcement, I wanted to share a few insights and surprises that came up as we started to trial this product with customers.

Our product data suggests that longer calls aren’t necessarily better calls.
One of the questions we had as we were developing Call Mining was whether or not there would be a strong correlation between the duration of a phone call and the likelihood of a purchase or conversion. In other words, is knowing the average duration of a call “good enough”?

In most cases, the answer is “no”. Generally, when a call is “very short” or “shortish” it indicates that a conversion probably didn’t happen and the call can be chalked up to a wrong number, unqualified lead or a simple question about directions or store hours. However, “longish” or “very long” calls do not indicate a likeliness to convert. Let’s say that you’re calling an insurance company. You provide information about yourself, your driving record and your insurance needs. The price is quoted at the end of the call - which may be the single biggest factor in whether or not you decide to buy!

For most customers, it seems that a longer call means a reservation or appointment or purchase may have happened, but the duration of a call is not a reliable enough indicator upon which an advertiser can base ROAS calculations.

Calls Are Not the Same Across Media Channels.
If a consumer sees your advertisement in the newspaper, on their PC, or on their mobile device it can be for very different reasons. Take this customer, who received the following number of calls from two different online marketing sources.

Source #1: 281 Calls
Source #2: 202 Calls

This data allowed us to calculate an effective cost per call for each source. However, these calls also generate additional useful data for this customer. Of particular importance to the customer are the calls where the customer specifically referenced one of several different problems or objections:

Source #1: 91 Calls (32%)
Source #2: 18 Calls (9%)

Our conclusion? For source #1, a much greater percentage of the calls were from existing customers reporting a problem and much less important to a marketer held accountable for new customer sign-ups.

The Magic Word for Payment?
My favorite part of Call Mining is building topics of conversation to understand about a set of calls. Call Mining provides suggestions on topics to build, based on the set of calls and what we know generally about conversations, and each time I use the feature I discover something new.

For example, many phrases can indicate the potential of a purchase over the phone. “Credit card” and “American Express” are two good examples. If you’re evaluating car rentals or reservations at your B&B, “confirmation number” is an important phrase to look for.

In most cases, though, there is one word that I’ve found that is a virtual lock to indicate a telephone purchase. The word is “expiration”. In almost every conversation where someone discusses a credit card purchase, the agent asks the caller for the expiration date on his or her card. There is almost no other term or expression that refers to expiration, and there is almost no other context where people frequently use the term expiration in a conversation.

Further Insights.
In the process of building our Call Mining technology, I’ve arrived at the conclusion that conversations between your business and your customer provide the richest and deepest clues into your business, and I expect to be following up with further insights as more customers begin to extract the qualitative conversion data they are looking for. If phone calls are an important part of your business, and you’d like to know more about how your customers are interacting on the phone, please reach out to me at johnb(at)marchex(dot)com.

Applying Design Thinking to Software Development (Part 1 of a Series)

I recently attended an executive education program at the Stanford d.school called “design thinking boot camp.”  If you’re not familiar with the term, design thinking is a continuous process rooted in deep customer empathy, interdisciplinary ideation, rapid prototyping, and frequent user testing.  While these things aren’t entirely new concepts in software (especially SAAS), there were some nuggets that caused me to think about how I’ve shipped product in the past.  

Before we dive in, here are three caveats.  (1) This stuff may not be appropriate for every software development effort.  (2) I have imparted my own interpretation of “design thinking” and do not purport to be an expert, just an informed observer.  (3) We’re just starting to implement design thinking at Marchex so I can’t say it will work for sure and probably won’t know for 6-12 months.  I can tell you design thinking is a big deal at places like Intuit, Jet Blue, Pulse, and Audi and it’s basically the underlying process for all of the projects at Ideo (arguably, the world’s leading design innovation firm).

Below are five design thinking principles that weren’t obvious to me:


1. The #1 job of customer research is not product validation (nor is it asking what products users want).
2. Space can be a game changer.
3. Ideation isn’t just smart people brainstorming.
4. Prototypes aren’t future products.
5. Research isn’t a waterfall process.

I’ll elaborate on each one over the next couple of months and start with customer research today.  

Customer Empathy! = Customer Research

This was probably the biggest aha! moment for me.  I tend to think Marchex is a customer centered place and my previous products at Google and Microsoft involved major investments in customer research.  Over the years, I’ve probably run, participated in, or consumed the results of tens of thousands of customer touch points through quant surveys, usability studies, customer visits, persona boards, conjoint studies, and analyst reports.  Design thinking argues all of that doesn’t matter without empathy. 

Aside from the dictionary definition, empathy is about getting non-obvious insights about your users well beyond their interactions with your product.  In fact, it’s essential to glean a ton of it before you even first envision your product or feature idea.  You can gain empathy through in-depth interviews and anthropological observation.  In-depth interviews are open ended discussions with customers where the researcher really internalizes experiences that matter to the customer.  Then they debrief and  unpack the customers’ feelings, motivations, values, priorities, biases, etc.  It’s sitting down with a real human, face to face in their place of business, home, or favorite café (not a research lab with a big mirror) and learning about what makes them tick.  Ultimately, empathy is discovering the unexpected things about customers you would never learn by reading a quant study, looking at demographic pie charts, or even using a traditional focus group. 

As an example, a recent design thinking project looked at parents of children going to college as a user group.  The obvious need is a way to stay in touch and keep track of their kids’ lives in college.  However, the much deeper, richer, and more interesting insight divined through a series of in-depth interviews found that the parents more primal and core need was to be invited into their kids’ new lives.  It wasn’t just hearing about what classes they were taking or who they were dating but an invitation to bring mom and dad into their lives in a way that presumably showed recognition, validated relevance, and love.  That’s some pretty rich stuff and could give rise to a whole new vein of product, feature, design, and positioning ideas that something more sterile like “help parents stay in touch with kids in college” would likely miss.

A half joking litmus test for an in-depth interview is whether your research participant ends up crying.  If the entire interview is dry, staccato, and doesn’t tap in to emotions , uncover tensions, or highlight rich stories about previous experiences, you’re not getting deep enough.  You want quotes like “I hate it when….because…”  “ Last year, this happened to me and I did this because…”   It frustrates me that…. because…..”  “I love …. because ….”  You should ask “why?” and “tell me about a time when…”   five to ten times over the course of the interview.  One great tool for gaining empathy is building a post-interview “empathy map.”  An empathy map is a way for your interview team to unpack the customer’s words, actions, thoughts, and feelings.  It’s super cool and you can learn more about that tool here http://bit.ly/azzWOR  .

Another point about customer empathy is the argument that six in-depth interviews may drive more insights and unlock more innovative opportunities than a series of quant surveys.  If you don’t really understand the personas and motivations of your users, you might not be asking the right questions in your quant surveys to begin with.  In hindsight, this makes a lot of sense since I’ve always found that the free text responses to open ended questions in online surveys tend to be much more interesting than the structured answers. 

Lastly, you’ll notice there isn’t a lot of talk about market size and industry structure or customer segmentation.  That’s because an underlying belief about design thinking is it’s better to build a product that extreme groups of users love and taps into a deep emotional pull versus a product that is targeted at a large group of middle-of-the-road users but doesn’t really nail it for any of them.  Obviously, there’s a balance to strike here but the focus on the user and not the market was kind of refreshing.  As an example, OXO kitchen tools were originally designed to meet the needs of arthritic users and that passion for those users’ needs led to a winning mainstream product.  It’s very possible those OXO handles might not have gotten there had the products’ designers focused on the home cooking market in general from the outset.

Happy empathizing.

Tom 

Customer Loyalty in the Age of Foursquare

Despite all the fragmentation and contrasting definitions, the one thing all businesses, big and small, clearly value at their core is the importance of loyal customers.  A common industry statistic is that it is five times more profitable to spend marketing dollars on keeping your best customers than acquiring new ones. Small businesses get this equation in spades. A Harvard Business Review study demonstrated that recovering only five percent of abandoning customers could increase profitability by 30 to 85 percent, and when Marchex asked several hundred small businesses in a recent survey what was most important, “keeping existing customers” was on top (46%) followed by “getting new customers” (26%).

The importance of customer loyalty isn’t a new concept for small businesses—but understanding what loyalty means in the digital age is imperative. What is new for businesses is how quickly consumers are shifting their conversations and other social actions to the online world (to Foursquare, Yelp, Citysearch, Twitter, Facebook, blogs, and the like).

 Primarily, the power has moved dramatically to the consumer, which means no longer does a loyal customer represent simply some repeat and referral business.  Customers now have to ability to broadcast sentiment to thousands instantly. This means loyal customers have become a small business’s de facto marketing department with the power to generate new business, craft a brand image, and inspire loyalty through tweeting, blogging, reviewing, following, and so on. Given a small business’ limited time and resources, this can be a wonderful thing if managed correctly.  So maintaining good relationships with customers has reached a whole new level of importance in the digital age.  A small business’s loyal customers will generate online word of mouth with positive reviews, mentions, and by broadcasting a visit on Foursquare -- but, with a little effort and appropriate digital tools, loyal customers can be mobilized to ignite referrals, generate positive air cover, shift opinions, and help soften the impact of bad reviews.

So what should a small business owner or operator do to manage this seemingly complex situation? He or she first take a deep breath, relax, and then start participating. Here are three suggestions to get the ball rolling:

1. Listen to the conversation: A business can’t truly understand how to engage customers—especially best customers—if they don’t know what they care about. Review sites, blogs, etc. are a goldmine of valuable information. Customers will tell a business exactly what to do to succeed—but they first need to hear and make sense out of all the chatter.  Effectively monitoring the Internet can be time consuming—but there are online products like Marchex Reputation Management that aggregate the conversation all in one place and provide simple yet invaluable analysis.

2. Get social—Once comfortable observing and getting a better understanding of what customers are saying small businesses should dip their toes in social media. Like it or not, small businesses need to be there, and they’ll find it’s not that scary once they jump in with a Twitter account, Facebook page, Foursquare and Groupon memberships, etc. (in fact they’ll likely find it’s a lot of fun).  

3 .Engage your customers--Lastly, it’s important to actively participate and communicate with customers (e.g., respond to a bad review, broadcast or thank a customer for a good one, ask for reviews, etc.). This is also a great opportunity to engage best customers who are active online by getting them to do more—such as reward them for referrals or just send them bits of interesting information they can broadcast (new menu item, upcoming sale, and so on).  However, communicating with customers can be challenging given the limited time and resources small businesses have. Many will inevitably find that effectively managing social media and the dialog with customers takes some help. That’s ok—there are affordable online products and services emerging --either self-serve or managed— that aggregate and simplify the engagement process for small businesses that will be an essential additions to a small business’s marketing toolkit.

The bottom line is this:  Like it or not, online conversations are happening, and are only likely to increase in volume. The good news is that this presents small businesses with a fantastic opportunity to listen to, learn from, and engage with, their customers on a scale never before possible. And help, in the form of the online products that are emerging to manage this opportunity, is ready and available.

Beyond Call Transcription

Call Analytics, which for Marchex represents a fundamental technical must- have for advertisers dependent on call-based leads, has the potential to go very deep: unlike clicks, which merely beget more clicks, a consumer connecting with an advertiser over the phone is engaged in an active, live, relationship-building conversation. And hearing the richness of these connections, as I now know very intimately, is an education in what really happens when marketing turns to sales and leads become customers....

To maximize the learning potential of call analytics, it's an incredibly valuable (albeit technically very challenging) half step to reach beyond the who-what-where-why-when's of tracking phone calls to start making sense of actual human conversation. This kind of insight can tell you things website and other kinds of marketing analytics can only guess at: how the call center and agents are behaving, the emotional state of the caller, what specific product or service attributes are critical to the customer, what their real pain points are, whether they are price sensitive, whether competitors are a threat, and much, much more.

So in a move that's ripped straight from the pages of the engineering/business disconnect handbook, I've lately heard a ton of noise about the value of transcription, long considered one of the choicest problems in software engineering. So, faced with tens of thousands of hours of otherwise opaque but insanely valuable audio, the marketer, who wants insight, correlation, and proof of ROI, has asked the engineer what the options are; the answer, not surprisingly, is to pick the most relevant engineering challenge, speech-to-text transcription.

But call transcription by itself merely shifts the locus of opacity from audio to text. Tens of thousands of hours of recorded audio are exchanged for millions of pages of text. Handing the marketer transcripts only works at tiny, tiny scale, and only if the marketer has the time and wherewithal to actually read the transcripts. That's why transcription is a necessary, but by itself insufficient, first step in the process of extracting fruit from a consumer/advertiser dialogue. Think of transcription alone as having a Web without a search engine....

Transcription by itself may provide a product, but it doesn't grasp the solution and its opportunity: making sense of live conversation to provide insight and leverage for the marketer.

At Marchex, we've been hard at work on the subsequent steps required to tap
into the opportunity we see in the storm of audio data. We call it Call Mining, and will lay out our vision for it in subsequent posts! Stay tuned....