The “Lean Startup” is killing growth experiments

Over the past few years, I’ve seen the “Lean Startup” grow to biblical proportions in Silicon Valley. It has introduced a lot of clever concepts that challenged the old way of doing business. Even Enterprises such as GE, Intuit and Samsung are adopting the “minimum viable product” and “pivoting” methodologies to operate like high-growth startups. However just like any dogma, the “lean startup” when followed with blind faith leads to a form of obscurantism that can wreck havoc.

Understanding “activation energy”

A few weeks ago, I was discussing implementing a growth experiment with Guillaume Cabane, Segment’s VP of Growth. He wanted to be able to pro-actively start a chat with Segment’s website visitors. We were discussing what the MVP for the scope of the experiment should be.

I like to think of growth experiments as chemical reactions, in particular when it comes to the activation energy. The activation energy is commonly used to describe the minimum energy required to start a chemical reaction.

The height of the “potential barrier”, is the minimum amount to get the reaction to its next stable state.

In Growth, the MVP should always be defined to ensure the reactants can hit their next state. This requires some planning which at this stage sounds like the exact opposite of the Lean Startup’s preaching: “ship it, fix it”.

The ol’ and the new way of doing

Before Eric Ries’s best seller, the decades-old formula was to write a business plan, pitch it to investors/stakeholders, allocate resources, build a product, and try as hard as humanly possible to have it work. His new methodology prioritized experimentation over elaborate planning, customer exposure/feedback over intuition, and iterations over traditional “big design up front” development. The benefits of the framework are obvious:
– products are not built in a vacuum but rather exposed to customer feedback early in the development cycle
– time to shipping is low and the business model canvas provides a quick way to summarize hypotheses to be tested

However the fallacy that runs rampant nowadays is that under the pretense of swiftly shipping MVPs, we reduce the scope of experiments to the point where they can no longer reach the “potential barrier”. Experiments fail and growth teams get slowly stripped of resources (this will be the subject for another post).

Segment’s pro-active chat experiment

Guillaume is blessed with working alongside partners who are willing to be the resources ensuring his growth experiments can surpass their potential barrier.

The setup for the pro-active chat is a perfect example of the amount of planning and thinking required before jumping into implementation. At the highest level, the idea was to:
1- enrich the visitor’s IP with firmographic data through Clearbit
2- score the visitor with MadKudu
3- based on the score decide if a pro-active sales chat should be prompted

Seems pretty straightforward, right? As the adage goes “the devil is in the details” and below are a few aspects of the setup that were required to ensure the experiment could be a success:

  • Identify existing customers: the user experience would be terrible is Sales was pro-actively engaging with customers on the website as if they were leads
  • Identify active opportunities: similarly, companies that are actively in touch with Sales should not be candidates for the chat
  • Personalize the chat and make the message relevant enough that responding is truly appealing. This requires some dynamic elements to be passed to the chat

Because of my scientific background I like being convinced rather than persuaded of the value of each piece of the stack. In that spirit, Guillaume and I decided to run a test for a day of shutting down the MadKudu scoring. During that time, any visitor that Clearbit could find information for would be contacted through Drift’s chat.

The result was an utter disaster. The Sales team ran away from the chat as quickly as possible. And for a good cause. About 90% of Segment’s traffic is not qualified for Sales, which means the team was submerged with unqualified chat messages…

This was particularly satisfying since it proved both assumptions that:
1- our scoring was a core component of the activation energy and that an MVP couldn’t fly without it
2- shipping too early – without all the components – would have killed the experiment

This experiment is now one of the top sources of qualified sales opportunities for Segment.

So what’s the alternative?

Moderation is the answer! Leverage the frameworks from the “Lean Startup” model with parsimony. Focus on predicting the activation energy required for your customers to get value from the experiment. Define your MVP based on that activation energy.

Going further, you can work on identifying “catalysts” that reduce the potential barrier for your experiment.

If you have any growth experiment you are thinking of running, please let us know. We’d love to help and share ideas!

Recommended resources:
https://hbr.org/2013/05/why-the-lean-start-up-changes-everything
https://hbr.org/2016/03/the-limits-of-the-lean-startup-method
https://venturebeat.com/2013/10/16/lean-startups-boo/
http://devguild.heavybit.com/demand-generation/?#personalization-at-scale

Images:
http://fakegrimlock.com/2014/04/secret-laws-of-startups-part-1-build-right-thing/
https://www.britannica.com/science/activation-energy
https://www.infoq.com/articles/lean-startup-killed
https://en.wikipedia.org/wiki/Activation_energy

Improve your behavioral lead scoring model with nuclear physics

According to various sources (SiriusDecision, SpearMarketing) about 66% of B2B marketers leverage behavioral lead scoring. Nowadays we rarely encounter a marketing platform that doesn’t offer at least point based scoring capabilities out of the box.

However, this report by Spear Marketing reveals that only 50% of those scores include an expiration scheme. A dire consequence is that once a lead has reached a certain engagement threshold, the score will not degrade. As put it in the report, “without some kind of score degradation method in place, lead scores can rise indefinitely, eventually rendering their value meaningless.” We’ve seen this at countless companies we’ve worked with. It is often a source of contention between Sales and Marketing.

So how do you go about improving your lead scores to ensure your MQLs get accepted and converted by Sales at a higher rate?

Phase 1: Standard Lead scoring

In the words of James Baldwin, “If you know whence you came, there are absolutely no limitations to where you can go”. So let’s take a quick look at how lead scoring has evolved over the past couple of years.

Almost a decade ago, Marketo revolutionized the marketing stack by giving marketers the option to build heuristical engagement models without writing a single line of code. Amazing! A marketer, no coding skills required, could configure and iterate over a function that scored an entire database of millions of leads based on specific events they performed.

Since the introduction of these scoring models, many execution platforms have risen. The scoring capability has long become a standard functionality according to Forester when shopping for marketing platforms.

This was certainly a good start. The scoring mechanism had however 2 major drawbacks over which much ink has been spilt:

  • The scores don’t automatically decrease over time
  • The scores are based on coefficients that were not determined statistically and thus cannot be considered predictive

Phase 2: Regression Modeling

The recent advent of the Enterprise Data Scientist, formerly known as the less hype Business Analyst, started a proliferation of lead scoring solutions. These products leverage machine learning techniques and AI to accommodate for the previous models inaccuracies. The general idea is to solve for:  

Y = ∑𝞫.X + 𝞮

Where:

Y is the representation of conversion
X are the occurrences of events
𝞫 are the predictive coefficients

 

So really the goal of lead scoring becomes finding the optimal 𝞫. There are many more or less sophisticated implementations of regression algorithms to solve for this, from linear regression to trees, to random forests to the infamous neural networks.

Mainstream marketing platforms like Hubspot are adding to their manual lead scoring some predictive capabilities.

The goal here has become helping marketers configure their scoring models programmatically. Don’t we all prefer to blame a predictive model rather than a human who hand-picked coefficients?!

While this approach is greatly superior, there are still a major challenge that need to be addressed:

  • Defining the impact of time on the scores

After how long does having “filled a form” become irrelevant for a lead? What is the “thermal inertia” of a lead, aka how quickly does a hot lead become cold?

Phase 3: Nuclear physics inspired time decay functions

I was on my way home some time ago, when it struck me that there was a valid analogy between Leads and Nuclear Physics. A subject in which my co-founder Paul holds a masters degree from Berkeley (true story). The analogy goes as follows:
Before the leads starts engaging (or being engaged by) the company, it is a stable atom. Each action performed by the lead (clicking on a CTA, filling a form, visiting a specific page) results in the lead gaining energy, thus furthering it from its stable point. The nucleus of an unstable atom will start emitting radiation to lose the gained energy. This process is called the nuclear decay and is quite well understood. The time taken to free the energy is defined through the half-life (λ) of the atom. We can now for each individual action compute the impact over time on leads and how long the effects last.

Putting all the pieces together we are now solving for:

Y = ∑𝞫.f(X).e(-t(X)/λ) + 𝞮

Where:

Y is still the representation of conversion
X are the events
f are the features functions extracted from X
t(X) is the number of days since the last occurrence of X
𝞫 are the predictive coefficients
λ are the “half-lives” of the events in days

 

This approach yields better results (~15% increase in recall) and accounts very well for leads being reactivated or going cold over time.

top graph: linear features, bottom graph: feature with exponential decay

 

Next time we’ll discuss how unlike Schrödinger’s cat, leads can’t be simultaneously good and bad…

 

Credits:
xkcd Relativistic Baseball: https://what-if.xkcd.com/1/
Marketo behavioral lead score: http://www.needtagger.com
Amplitude correlation analysis: http://tecnologia.mediosdemexico.com
HubSpot behavioral lead score: http://www.hubspot.com
MadKudu: lead score training sample results

What we can learn from Ants to improve SaaS conversion rates

SaaS onboarding is the beating heart of your business. In our era of freemium, trials and other piloting processes, ramping up prospects who signed up for your product can make or break your forecasts. Increasing free-to-paid conversion rates is therefore a daunting task. You may feel overwhelmed by the incredible amount of factors you can tamper with. The myriad of solutions out there while doing a great job at solving specific problems rarely help identify the main levers of improvement for SaaS conversion rates.
Today, we’ll discuss an approach to identifying these levers and how to execute against them.

Ant colony optimization

At this point you might be wondering what’s this business about Ant Colonies helping improve SaaS conversion rates.
In the real world, ants have developed a rather intriguing heuristic to optimize their path to food patches. They initially wander in random directions away from the colony, laying a pheromone trail on their path. As they find food and return, they increase the amount of pheromone on the path to the food. The other ants from the group are attracted to the strongest trail which will be the closest to a food source. As the pheromones evaporate, the shortest paths become increasingly more attractive until the optimal path is found.This optimization algorithm is called the ant colony algorithm. Its goal is to mimic this behavior with “simulated ants” walking around the graph representing the problem to solve.

At MadKudu, we’ve built such an algorithm and its goal is to mimic this behavior with “simulated ants” (trial users) walking around the graph (performing sequences of events) representing the problem to solve.

Identify milestone events

You’ve probably heard about Facebook’s famous “7 friends in 10 days“. The key drivers of conversion, or “key conversion activities” are user activities that are most associated with conversion. Identifying those key activities allows to focus your engagement efforts on things that truly move the dial. For example, you can write content that most effectively helps users get value from the product, and convert them.

At MadKudu, we use a standard decomposition of onboarding events into 3 groups. Using advanced analytics, we identify and distinguish between those 3 types of activities:

Activation Activities

These are activities that users absolutely need to do to convert, even though doing them does not indicate they will convert. In other words, they are required but not sufficient.
These activities are typically things like “setting up an account” or “finishing the onboarding steps” or “turning on a key integration”.

Engagement Activities

These are the core activities of your product. This is where users get recurring value from your product. Users who perform these activities often will convert. Those who don’t will most likely not.
The key is to find which activities truly matter and how many occurrences are necessary until the point of diminishing returns is reached.

Delight Activities

These are activities that are done by few users, your most advanced users. Users who don’t do those activities are not less likely to convert. But those who do are very likely to convert.
Make sure to identify what these activities are and promote them to advanced users when the time is right.

DIY

In order to map out your onboarding events, you can calculate for each event:
– the conversion rate of users who performed the event: P(X)
– the conversion rate of those who did not: P(¬X)

You can then determine the impact of performing the event (average conversion – P(X)) and the impact of not performing the event (average conversion – P(¬X)).

Finally you can graphically represent your onboarding events as such:

Anything on the left is a requirement to have a chance to convert. Anything at the top is strongly correlated to converting.

Or you can contact us ;-)

From analytics to results

There are many ways to make this actionable, here are just a few:

  • Create a behavioral onboarding drip (case study)
  • Close more delighted users by promoting your premium features
  • Close more delighted users by sending them winback campaigns after their trial (50% of SaaS conversions happen after the end of the trial)
  • Adapt your sales messaging to properly align with the user’s stage in the lifecycle and truly be helpful

If you’d like to dive deeper into your onboarding funnel or discuss implementing some of the tactics above, you can signup for MadKudu or reach out to us.

Photo: www.cusuitemusings.com/
Image: Multiobjective Optimization of an Operational Amplifier by the Ant Colony Optimisation Algorithm (http://article.sapub.org/)
Plot: MadKudu “Happy Path” Analysis Demo Sample

Achieving personalization at scale in B2B sales

I was trying to write a title as pompous and with as many buzz words as possible and I do believe I’m close. Who knows we might even get featured on TechCrunch with these ramblings on how “big data” is enabling the ultimate phase of the B2B sales & marketing revolution…

Over the past few weeks at MadKudu, we’ve run a thorough retrospective on 2016 to flesh out what we’ve learnt, which hypotheses were validated, which were proven wrong.
The exciting learning is that we’re onto something big, something HUGE!
We’ve validated the fact that lead prioritization enablement was commonly sought. But more importantly we’ve realized that lead scoring solutions as they exist today are only duct-tape on a broken process. Since companies aren’t able to handle personalized onboarding at scale, they reduce the scale by focusing on a subset of leads to manually personalize the experience for. Welcome to the world of the inbound SDR. MadKudu is set to change this and bring us one step closer to completing the marketing & sales revolution by operationalizing personalization (channel, message…) at scale.
In essence the main actionable learning is that operationalization is 10x more valuable than enablement. It’s actually a completely different sport.

The Sales & Marketing Revolution

The term revolution is mainly used to describe an overthrow of an order in favor of a new one. But the root of the words tie back to the concept of going full circle. So when we talk about the sales & marketing revolution we mean we’re getting back to a previous state. While we’ll dedicate a specific post to this topic, a high-level history of marketing would go as such:
– Before the industrial revolution, people bought from local stores and suppliers. This was the era of one-to-one personalization of the product to the customer’s needs.
– The industrial revolution changed everything, the product was now king. Our newly discovered ability to mass produce meant we needed to find ways to ship these products. This started the era of the marketing mix’s 4P (product, price, promotion, placement) in marketing.
– In more recent days, the rise of the internet 2.0 marked the rise of the SDR. With online products being available for billions of people and marketing strategies still focusing on bringing in as many prospects as possible, there was a new need to qualify potential customers.
– The “big data” revolution. Data science has started powering personalization and relevance at scale in eCom marketing for a few years now. Amazon led the charge with its recommendation engine and many companies have since then applied data science to make the B2C sales experience more relevant (at AgilOne, we did a lot of this). The shift from the 4Ps towards the 5Cs is another illustration of this trend of putting back the customer at the center of marketing activities.

What “big data” brings to Sales

There is a common misconception that big data equates huge quantities of data and thus is more appropriate for marketing than sales and for B2C rather than B2B. But there are really 3 aspects to big data:
– massive data sets (high volume)
This is what companies like facebook, google deal with. We’re talking trillions of records of data to process. The main challenge here is scalability and is only seen in B2B2C or B2C companies.
– fast data (high velocity)
This is what real time analytics systems deal with. Recommender systems, trading algorithms are great examples of systems dealing with high velocity data.
– complex data sets (high variety)
Here’s the least sexy and known aspect of the lot. B2B companies generate big data with customer records coming from sales data, product usage, customer records, support tickets… While real-time analytics and scalability are challenges the hard nut to crack is the identity layer or combination of all the information in a comprehensible data set. Machine Learning algorithms will only ever be as good as the input they are fed.

Why is B2B Sales broken

The final aspect has been ankylosing the B2B space and has thus become a great source of innovation. Companies are spending billions of dollars to get their data together (getBirdly, Jitterbit), stitching it together (leanData, AgilOne). The hardest part though remains in rendering the data actionable. This is where Big data can help reach the holy grails of sales and marketing: “personalization to foster relevance, at scale”.
Lead scoring tools so far have been built with this in mind. They leverage the multitude of data points available to automate -to some extent- the qualification historically run by SDRs.

BANT Qualification process:
B => mainly firmographic data to determine if the account would have budget for your top tier pricing
A => mainly demographic to determine how close is this person to having a budget line item for your product
N => mainly firmographic to determine if the account likely to be a successful user of your product or at least have a need for it
T => mainly behavioral to determine if the account’s aggregated behavior is indicative of a strong likelihood to purchase your product in the near future.

And so this is where big data has been helping so far. Lead scoring solutions have been doing a great job at getting SDRs to focus on a small subset of leads that they can then write personal emails to through bulk email solutions like Yesware or Salesloft…

Where this approach falls short is that sending emails manually don’t make them personal, let alone relevant. We all receive tens of emails like this every day:
right_person_email

From cartography to self-driving cars

A couple weeks ago, Guillaume Cabane, VP Growth at Segment, made a striking analogy between cartography and B2B sales. Cartography is the representation of the overall landscape of your leads. It is used to determines the routes you need to follow to reach your destination. This is your initial ideal customer profile analysis. The GPS is an automated way of telling how to get to your destination. This is lead scoring as we know it today. The self-driving car is build upon a GPS and executes the commands reliably and automatically. This is the future of B2B sales, the idea of a “software SDR”.
In essence, the great opportunity to seize in 2017 lies in realizing the era of the GPS as a stand alone tool is over. We are now heading into a world of self-driving cars.
Not only are we convinced about this, the early tests we’ve been running so far are encouraging. Our software SDR has consistently outperformed by at least 66% regular SDRs on the amount of qualified demos booked. Not only were we generating more meeting, we also free-ed up time for the sales team so they could focus on what they do best: adding value to prospects whom we’ve engaged with them.

Here’s to 2017, year of the true sales automation!

Image credit : A future lost in time