How MadKudu makes Salesforce Einstein better

…Or why Salesforce Einstein won’t be the next IBM Watson.

Is the AI hype starting to wither? I believe so, yes.
Reality of the operational world is slowly but steadily catching up with the idyllic marketing fantasy. The report Jeffries put together challenging IBM Watson proves alarm bells are ringing. The debacle of the Anderson implementation goes to show how marketing promises can be unrealistic and the downfall will be dreadful.

With that said, not all is lost as we keep learning from our past mistakes. Being part of the Salesforce Einstein focused incubator, we are witnessing first-hand how the CRM giant is looking to succeed where Watson and others are struggling. Hopefully these insights can help others rethink their go-to-market strategy, in an era of unkept commitments.

Salesforce, a quick refresher

A few weeks ago, I was being interviewed for an internal Salesforce video. The question was “how has the Salesforce eco-system helped your startup?”. To contextualize my thoughts it’s important to know that while Salesforce is one of our main integrations, we consider it as an execution platform among others (Segment, Marketo, Intercom, Eloqua…). I’ve always admired Salesforce for its “platform” business model. Being part of the Salesforce ecosystem facilitated our GTM. It gave MadKudu access to a large pool of educated prospects.

However I believe the major value add for startups is the focus induced by working with Salesforce customers. Since Salesforce is a great execution platform there are a plethora of applications available addressing specific needs. This means, as a startup, you can focus on a clearly defined and well delimited value proposition. You can rely on other solutions to solve for peripheral needs. As David Cohen reminded us during our first week at Techstars, “startups don’t starve, they drown”. Salesforce has helped us stay afloat and navigate its large customer base.

What is Salesforce Einstein?

I’m personally very excited about Salesforce Einstein. For the past 5 years, I’ve seen Machine Learning be further commoditized by products such as Microsoft Azure, Prediction.io… We’ve had many investors ask us what our moat was given this rapid democratization of ML capabilities and our answer has been the same all along. In B2B Sales/Marketing software, pure Machine Learning should not be considered a competitive advantage mainly because there are too few data sets available that require non-generic algorithms. The true moat doesn’t reside in the algorithms but rather in all the aspects surrounding them: feature generation, technical operationalization, prediction serving, latency optimization, business operationalization… The last one being the hardest yet the most valuable (hence the one we are tackling at MadKudu…).
Salesforce Einstein is the incarnation that innovation will be in those areas since anyone can now run ML models with their CRM.

We’ve been here before

Just a reminder, this is not a new thing. We’ve been through this not so long ago.
Remember the days when “Big Data” was still making most of the headlines on Techcrunch? Oh how those were simpler times…


Big Data vs Artificial Intellligence search trends over the past 5 years

There were some major misconceptions as to what truly defined Big Data especially within the context of the Enterprise. The media primarily focused on our favorite behemoths: google, facebook, twitter and their scalling troubles. Big data became synonymous for Petabytes and unfathomly large volumes of data more generally. However scholars defined a classification that qualified data as “big” for 3 reasons:
– volume: massive amounts of data that required distributed systems from storage to processing
– velocity: quickly changing data sets such as product browsing. This meant offline/batch processing needed an alternative
– variety: data originating from disparate sources meant complex ERDs had to be maintained

In the Enterprise, volume was rarely the primary struggle. Velocity posed a few issues to large retailers and companies like RichRelevance nailed the execution of their solution. But the main and most challenging data issue faced was with the variety of data.

What will make Salesforce Einstein succeed

Einstein will enable startups to provide value to the Enterprise by focusing on the challenges of:
– feeding the right data to the platform
– defining a business playbook of ways to generate $$ out of model predictions

We’ll keep the second point for a later blog post but to illustrate the first point with DATA, I put together an experiment. I took one of our customers’ dataset of leads and opportunities.
The goal was to evaluate different ways of building a lead scoring model. The objective was to identify patterns within the leads that indicated a high likelihood of it converting to an opportunity. This is a B2B SaaS company selling to other B2B companies with a $30k ACV.
I ran an out-of-the-box logistic regression on top of the usual suspects: company size, industry, geography and alexa rank. For good measure we had a fancy tech count feature which looked at the amount of technologies that could be found on the lead’s website. With about 500 opportunities to work on, there was a clear worry about overfitting with more features. This is especially true since we had to dummy the categorical variables.
Here’s how the regression performed on the training data (70% of the dataset) vs the test dataset (30% that were not used for training and ensuring if a company is part of the training it is not part of testing – see we did not fool around with this test)

Regression model using firmo and technographic featuresmodel performance on test dataset using available data points

Not bad right?! There is a clear overfitting issue but the performance is not dreadful apart for a blob in the center

Now we ran the same logistic regression against 2 feature: predicted number of tracked users (which we know to be highly tied to the value of the product) and predicted revenue. These features are the result of predictive models that we run against a much larger data set and take into account firmographics (Alexa rank, business model, company size, market segment, industry…) along with technographics (types of technologies used, number of enterprise technologies…) and custom data points. Here’s how the regression performed:

Regression with MadKudu featuresmodel performance on test dataset using 2 MadKudu features

Quite impressive to see how much better the model performs with fewer features. At the same time, we are less running the risk of overfitting as you can see.

The TL;DR is that no amount of algorithmic brute force applied to these B2B data sets will ever make up for appropriate data preparation.

In essence, Salesforce is outsourcing the data science part of building AI driven sales models to startups who will specialize in verticals and/or use-cases. MadKudu is a perfect illustration of this trend. The expert knowledge we’ve accumulated by working with hundreds of B2B SaaS companies is what has enabled us to define these smart features that make lead scoring implementations successful.

So there you have it, MadKudu needs Salesforce to focus on its core value and Salesforce needs MadKudu to make its customers and therefore Einstein successful. That’s the beauty of a platform business model.
I also strongly believe that in the near future there will be a strong need for a “Training dataset” marketplace. As more of the platforms make ML/AI functionalities available, being able to train them out-of-the-box will become an important problem to solve. These “training datasets” will contain a lot of expert knowledge and be the results of heavy data lifting.

Feel free to reach out to learn more

Images:
www.salesforce.com
Google trends
MadKudu demo Jam 3

PS: To be perfectly clear, we are not dissing on IBM’s technology which is state of the art. We are arguing that out-of-the-box AI have been overhyped in the Enterprise and that project implementation costs have been underestimated due to a lack of transparence on the complexity of configuring such platforms.

Are Automation and AI BS?

A couple weeks ago, I ended up taking Steli’s click bait and read his thoughts on sales automation and AI. There isn’t much novelty in the comments nor objections presented. However I felt compelled to write a answer. Part of the reason why, is that MadKudu is currently being incubated by Salesforce as part of the Einstein batch. Needless to say the word AI is uttered every day to a point of exhaustion.

The mythical AI (aka what AI is not today)

The main concern I have around AI is that people are being confused by all the PR and marketing thrown around major projects like Salesforce’s Einstein, IBM’s Watson and others – think Infosys Nia, Tata Ignio, Maana.io the list goes on.

Two months ago, at the start of the incubator, we were given a truly inspiring demo of Salesforce’s new platform. The use-case presented was to help a solar panel vendor identify the right B2C leads to reach out to. A fairly vanilla lead scoring exercise. We watched in awe how the CRM was fed google street view images of houses based on the leads’ addresses before being processed through a “sophisticated” neural network to determine if the roof was slanted or not. Knowing if the roof was slanted was a key predictor of the amount of energy the panels could deliver. #DeepLearning

This reminded me of a use-case we discussed with Segment’s Guillaume Cabane. The growth-hack was to send addresses of VIP customers through Amazon’s mechanical turk to determine which houses had a pool in order to send a targeted catalogue about pool furniture. Brilliant! And now this can all be orchestrated within the comfort of our CRM. Holy Moly! as my cofounder Sam would say.

To infinity and beyond, right?

Well not really, the cold truth is this could have also been implemented in excel. Jonathan Serfaty, a former colleague of mine, for example wrote a play-by-play NFL prediction algorithm entirely in VBA. The hard part is not running a supervised model, it’s the numerous iterations to explore the unknowns of the problem to determine which data set to present the model.

The pragmatic AI (aka how to get value from AI)

Aside from the complexity of knowing how to configure your supervised model, there is a more fundamental question to always answer when considering AI. This foundational question is the purpose of the endeavor. What are you trying to accomplish with AI and/or automation? Amongst all of the imperfections in your business processes which one is the best candidate to address?

Looking through history to find patterns, it appears that the obvious candidates for automation/AI are high cost, low leverage tasks. This is a point Steli and I are in agreement on: “AI should not be used to increase efficiency”. Much ink has been spilled over the search for efficiency. Henry Ward’s eShares 101 is an overall amazing read and highly relevant. One of the topics that strongly resonated with me was the illustrated difference between optimizing for efficiency vs leverage.

With that in mind, here are some examples of tasks that are perfect fits for AI in Sales:

  • Researching and qualifying
  • Email response classification (interested, not interested, not now…)
  • Email sentiment classification
  • Email follow up (to an email that had some valuable content in the first place)
  • Intent prediction
  • Forecasting
  • Demo customization to the prospect
  • Sales call reviews

So Steli is right: No, a bot will not close a deal for you but it can tell you who to reach out to, how, why and when. This way you can use your time on tasks where you have the highest leverage: interacting with valuable prospects and helping them throughout the purchase cycle. While the recent advent of sales automation has led to an outcry against the weak/gimmicky personalization I strongly believe we are witnessing the early signs of AI being used to bring back the human aspect of selling.

Closing thoughts

AI, Big Data, Data Science, Machine Learning… have become ubiquitous in B2B. It is therefore our duty as professionals to educate ourself as to what is really going on. These domains are nascent and highly technical but we need to maintain an uncompromising focus on the business value any implementation could yield.

Want to learn more or discuss how AI can actually help your business? Feel free to contact us

3 reasons why B2B SaaS companies should segment trial users

99% of the B2B SaaS companies I talk to don’t segment their free trial users.

This is a shame because we all know our trial users can be very different from one another.

For example, have you heard of accidental users? Those users signed up thinking your products was doing something else and leave soon after realizing their mistake (much more common than what you might think!).

Or what about tire-kickers? Yes, a surprisingly large number of people like to try products with no intention of buying ever (more about it in this great post from Matt Pope).

There are also self-service users. They are actively evaluating your product but don’t want to talk to a human being, especially a sales person.

The enterprise buyer is an interesting profile. She will likely buy an expensive plan and will appreciate to get help from an account executive.

 

“Sure thing… why should I care now?”

Fair question. Here is what happens when little is done to identify the different types of trials.

1. The overall conversion funnel has little meaning

A SaaS company we work with was worried because their trial-to-paid conversion rate had decreased 30%. Is this because of the new product feature they just released? Or maybe there is an issue with the email drip campaign? The explanation was simpler: A large number of tire-kickers coming from ProductHunt suddenly signed up. Their very low conversion rate crashed the overall conversion rate.

Looking at the trial-to-paid funnels by customer segment is the best way to understand how your product and sales activities affect conversions, regardless of variations in customer signups.

2. You are selling and building the wrong product features

Understanding how your product is used is essential to effectively sell and improve your product.

But looking at overall product usage metrics is misleading. The accidental users and tire-kickers usually make up a large chunk of your customers. Looking at overall usage metrics means that you may well be designing your sales and product strategy to fit your worst customer segments!

When looking at product usage, make sure to focus on your core user segment. The features they care about are the features to sell and improve.

3. You are spending your time and money on the wrong trial users

There are lots of ways in which a lack of segmentation hurts your sales and customer success efforts:

  • Tire-kickers take away precious time from sales and customer success. This time could be spent on selling and helping core users.
  • Customers with high potential value don’t get extra love. Many sales teams spend huge amounts of time on tiny customers while underserving larger customers.
  • Trying to get buyers to use your product and trying to get users to buy is a waste of everybody’s time. In B2B, the buyer is often not a heavy user. For example, a CTO will pull the credit card and pay for an app monitoring software, but he or she will use the software only occasionally. Educating the CTO on the nuances of the alert analysis feature doesn’t help anyone!
  • Sales trying to engage self-service users hurts conversions. Some users appreciate having an account representative help them evaluate a product while others want to do their evaluation on their own. Knowing who’s who is critical for both customers and sales teams.

 

How to get started?

One way, of course, is to use MadKudu (passionate, self-interested plug). Otherwise the key is to start simple. Talk to your best customers to get a qualitative feel of who they are, and look at your customer data to find out what similar characteristics are shared by your best customers. Then put together a simple heuristic to segment your customers and implement this logic in your CRM and analytics solution.

This effort will go a long way to increase your trial-to-paid conversion rates.

Now back to you. Do you have different segments for your trial users? If no, why not? If yes, what are those segments? Who is using them? Continue the conversation on twitter (@madkudu) or email us hello@madkudu.com!

3 Things Demand Gen Teams Can Learn from Sales

Guest blog post from our friends at Clearbit

A demand generation manager must meld several operational roles (marketing, automation, analytics) into one core competency—increasing conversions, and ultimately, revenue. They’re the “growth hackers” of B2B marketing, working across disciplines to solve problems.

Just as growth hackers seek the viral loops that will skyrocket their growth, demand generation teams look for opportunities to automate and systematize the customer life cycle.

In practice, that’s meant that demand generation teams have been much more similar to marketing teams than sales teams. But from tools that automate formerly clunky processes to prospecting methods, there’s quite a bit that demand generation teams could take away from the techniques that sales teams are using to great effect today.

1. Prospect like Socrates

Asking questions is the most fundamental skill that salespeople need to be great. Steli Efti at Close.io has more than a half-dozen points he asks himself when talking to a new potential buyer — if he doesn’t know the answers, he figures it out by questioning the prospect:

  • How much value can I add to this person or organization?
  • How will the value I add be quantified?
  • How important will supporting this prospect be?
  • What are their overall wants and needs for my product?
  • What size is this deal and what is their budget?
  • How much additional usage will our product see as a result?

If you don’t know all of these angles on the deal you’re trying to close, then “there are all kinds of ways” that “quoting them a price too early can backfire,” according to Steli.

Extend this logic a bit further back in the customer life cycle. Imagine if your sales reps didn’t have to think through these kinds of questions at all.

Your job in demand generation is not limited to bringing in more traffic to your website. You have to make sure you make that traffic count. To do that, you want to have a clear sense of who exactly you’re generating this demand in.

There are many ways to understand your target customer base, but the old-fashioned way is still the best and most reliable. Pick up the phone, ask questions, listen, and learn. Don’t treat the Socratic method of prospecting as a one-off process that your sales reps have to do to ink deals. Understand your potential customers, their needs, and the amount of value you can bring their organization before the sales call, and the leads you take to your sales team will blow their minds.

2. Use Tools To Reach Peak Efficiency

Smart sales teams have fully embraced automation. Those doing it right are consistently reaping great prospects and warm leads while doing a fraction of the work they used to do.

Today, sales teams with developers are going all-in on Clearbit and MadKudu, who make a series of APIs designed to generate customized lists of leads, qualify them, score them, enrich them, and send them into CRMs.

However, if you don’t have development resources, the easiest way to get started is with Clearbit Connect, a Gmail extension that works like Rapportive used to. It lets you look up the contact information for virtually any employee at any company, segment by role, and see a large swath of company information at a glance.

Here’s an example of the kind of information you can get—name, address, company, role, title, site, social following, and more all from one request:

flowc

To use Connect, just install the Chrome extension and it’ll appear as an option inside your Gmail dashboard:

Demand generation isn’t just about bringing in more leads or helping your sales team succeed—it’s about improving results at all stages of the funnel and driving growth. And there’s no better way to do that than with data.

Find your best customers. Understand who they are, inside and out. Then clone them.

3. Automate & Customize

Cold emailing may seem like a foreign concept if your job is demand generation, but with a refreshed technique it can be a hugely powerful tool for testing new markets and validating hypotheses about who your customers are.

First, you need the target markets you’re trying to reach, the roles and titles of the people that can get value out of your product, and a list of emails.

With a tool like Customer.io, you can take that information and turn it into personalized emails built on liquid tags—little snippets of code that, like variables, let you input whatever information you want.

That means rather than force yourself to rewrite new emails all the time, trying desperately to make them sound fresh, you can create personalized, evergreen emails that actually work. Here’s an example of an email that Customer.io posted on their blog:

This automated-yet-customized approach to the welcome email is how top sales teams are heating up their cold leads and optimizing their pipelines. The power of this technique is even greater, however, when taken in the broader context of a demand generation team’s job.

You’re not just going to get good click-through rates—you’re going to bring in qualified leads who understand the value of what you’re offering.

Top, Middle, And Bottom Of Funnel

Demand generation teams are responsible for one of the B2B startup’s most important objectives—growing inbound awareness, conversions, revenue, and growth.

To reach peak performance, they need to look across all departments and disciplines for the latest and greatest methods. These are three that I think are super valuable—let me know what you think in the comments below. What demand generation processes do you use at your company? What processes do you wish you used?

Make the right “build versus buy” decision with 3 simple steps

A couple weeks ago I attended a Point Nine and Algolia happy hour in Dublin. The premise was on point with a recurring question we deal with on a daily basis when it comes to software: “Should you buy versus build internal solutions?”

Many at the event shared the story of an in-house solution turning into a big costly distraction for their team. The main culprit? The decision to build in house was taken lightly without the hypotheses behind this decision being written down and communicated.

 

I’d like to share here a simple framework I’ve used and that I’ve seen work in this form or another at some of the SaaS rising stars (Algolia, Intercom, Mention…).

This framework helps support data driven, thus dispassionate, decisions on the topic of building vs buying.

The high level structure is:

Step 1: Validate the business need
Step 2: Get a rough but realistic estimate of the cost for the “build” option
Step 3: Decide and review your hypotheses in a given timeframe.

Step 1: Validate the business need

Even Chris Doig in his analysis of the problem writes that everything starts with well-defined requirements. However, as most founders know only too well, every decision to even think about doing something starts with a hypothesis of much positive impact the company can get from a new set of functionality.

Make sure to always go through the exercise of determining how much value you will get from this feature/product you’re considering.

Let’s take the example of building a lead scoring mechanism to help the sales team know which leads they should de-prioritize. The assumption is that the sales team is wasting time on leads that are unlikely to purchase your product at a high-enough price point. Seems fair. But how much value can we expect from implementing such a solution? Keep it simple. Let’s assume you have 10 SDRs, each at a base salary of $50k. If 20% (1 out of 5) of the leads they are reaching out to are unqualified, you are essentially wasting ~$100k annually. And this is without considering the opportunity cost from not spending that time on higher value leads.

With this rough estimate in mind, let’s proceed with evaluating the cost.

Step 2: Define basic requirements and compute an estimate of the cost for the “build” option

Whenever we consider building a solution internally, I like to approach it as I would if I were to write a RFI. This is a great forcing function to decompose the problem and identify the different required functionalities along with their impact on the expected value (aka their criticality). The individual costs are always higher than you initially thought, and the estimates for each item add up quickly!

For example, using the example of lead scoring, decomposing the problem could bring us to the following set of critical features:

– Build a programmatic way to fetch information about new leads from LinkedIn
– Define a heuristic to score leads based on the data obtained
– Build a scoring mechanism
– Build a pipeline to feed this score back into your CRM
– Add the score in a workflow to route leads appropriately
– Set up reports to measure performance in order to make adjustments if necessary

Once you have those listed, get an estimate from the engineering team for building each feature. This will enable you to have an idea of the cost of the “build” option.

You can use a simple spreadsheet to estimate the annual cost of building and maintaining a solution based on your team’s size, current MRR…

Download this calculator here.

For an early young company (6 engineers, $100k MRR), the cost of such a solution over the course of a year would be about $80,000.

This may seem high and the truth and that we all have a hard time estimating opportunity cost, maintenance cost (we are typically twice that of initial development)…

In parallel, look around to see what SaaS solutions are available to solve your problem and how much they would cost. A lot of them offer free-trials and/or demos. I recommend going through at least a demo as you will be able to get some valuable information about others who have worked on solving the problem you’re addressing. On the pricing aspect, if no pricing is shown and the product requires a demo, you can be fairly certain the cost will be at least $999/month.

Step 3: Decide and review your hypotheses in a given timeframe

You are now armed with all the data points to take a data driven decision! If you’ve decided to build in house, set a “gate” with your team to revisit the hypotheses you’ve made. For example, decide with your team to have a quick check-in in 90 days to discuss the learnings from building the solution in house and decide whether or not to continue or re-evaluate.

 

Notes
I want to emphasize that no answer can be right without context. What is initially true might very well become wrong. Therefore we’ve built a lot of software to help us determine what are the critical components we would be looking for when shopping around. In these cases it was always essential to timebox the build phase and to constantly remind ourselves that the objective was to reduce uncertainty and unknowns.

Secondly, there is a hidden cost in buying that can come from the rigidity and inadequacy of the SaaS product you buy with your problem. This is why trials and POCs are so popular nowadays (which is why we offer one).

Lastly, the example picked seems like a no-brainer as the solution is for the “business” team. The level of rigour required to go through this exercise for tools used by dev teams is much greater. The main fallacy lying in the illusion that an internal tool will always be a better fit for all the company-specific requirements. This is not only a highly inaccurate; it also leads to ill-defined features. Going through step 1 can save hours of wasted time and headaches.

Segment for CMOs

SaaS marketers are not usually engaged in the day-to-day engineering work. I’m no exception.

A few days ago I asked Sam Levan, our CEO, “what exactly are we doing with Segment?” I had visited Segment’s landing page but didn’t really get the value – it reminded me of the “middleware” concept IT departments have been talking about for years. Sam I talked about Segment and I finally understand it.

Since other CMOs might have similar questions I decided to write up our conversation as an interview. Hope it helps you!

Me: Pretend I’m a CFO, CSO, CMO, or CEO and not involved in day-to-day product development. What value does Segment provide to our business?

Sam: Engineering is the scarcest resource we have. Segment makes our developers more efficient and allows us to deliver value faster. Saves us time and money.

Me: Ok. That’s simple enough. How?

Sam: Like every business we need a set of third-party tools like email marketing, sales, help desk, and analytics to name a few. Sending our data to each service individually takes a lot of time to setup and support. By sending our data to Segment we can re-distribute it to ANY of these services almost instantly.

Segment maps data between our database and these applications.

Me: Can you give me an example?

Sam: Sure. Suppose you want to test out a new tool like Mixpanel.

Without Segment you have to ask the engineering team to integrate with Mixpanel’s API. This starts with a scoping meeting … which leads to a priority discussion … which leads to acceptance testing … and updates when we change our product … or they change their API.

As the CEO I don’t want our developers to have to spend time on this type of work – we need to be improving our product and delivering value for customers, not messing around with APIs.

With Segment you just flip a switch and you can start testing Mixpanel. It usually isn’t quite that simple but much, much easier than the alternative.

Me: Ok, I get it. So what is the downside of using Segment?

Sam: Well obviously Segment isn’t free, so I guess if you only need 1 or 2 tools it may not be worthwhile.

But to be perfectly honest we would probably use Segment even if we only used 1 tool in their platform. All they do is APIs and data mapping and they do it better than anyone.

Me: I’ve used Zapier before to automate rules and share data between apps like Wufoo and MailChimp. How is this different from Segment?

Sam: We don’t use Zapier and I’m not as familiar with their product. But my understanding is that Zapier is more popular with small and non-tech businesses and makes it easy to map fields between products like email, forms, file-sharing, etc.

It seems to be more geared towards solving a specific data-sharing problem between specific applications. Using your example, sharing the data you have on Wufoo with your MailChimp account.

This is very different from exposing all of our internal business data to be shared across different applications.

Me: Ok. Switching gears a bit … MadKudu is also an Integration partner with Segment. What is the value for us of this partnership?

Sam: A no-brainer. Segment makes it easier for potential customers to try MadKudu and send us their data.

Me: I guess that seems obvious … but doesn’t this also mean that MadKudu’s customers could quickly switch to a competitor? Isn’t this how companies like Microsoft became dominant? By building proprietary interfaces and locking-in customers?

Sam: It doesn’t really work that way anymore. Every SaaS company like MadKudu needs to constantly re-sell its value every billing cycle – otherwise customers cancel. I don’t know a single SaaS founder who thinks she can lock-in customers.

Even if that were the case we also benefit from lower support costs by being on Segment since it reduces the costs of supporting our own API.

Me: Ok. Now just to add to the confusion … Segment is also a MadKudu customer?

Sam: That’s right. We use data science to help Segment’s sales team identify their most promising leads and turn them into customers.

Me: Good grief now I know why I find our status meetings so confusing. Let me see if I get this straight:

MadKudu is a Segment customer.
MadKudu is also a Segment Integration partner.
Segment is a MadKudu customer.

How the heck do you guys keep track of all of this?

Sam: (laughs) Yeah, it makes for some confusing conversations. Usually it is obvious from context.

Me: Ok, final topic. You were a data scientist for years before starting MadKudu and have consulted for hundreds of companies. Can you explain the value of Segment’s new data Warehouses solution?

Sam: Data warehouses have been around for decades but have struggled to live up to their promise. One major reason is that getting all business data into one place is a huge PITA.

Moving the data and building APIs is part of the problem – basically the same challenge we discussed earlier about building and maintaining multiple APIs. An additional challenge is understanding what each field means and mapping it to the same logical entity in another application.

This is why a lot of “data science” projects have traditionally struggled to get off the ground – it just takes too much effort to get everything setup and organized in the data warehouse.

Me: Ok, I’ve been a part of projects like this. I can recall months of meetings with analysts and business owners building data dictionaries, mapping fields. It basically sucked.

Sam: Exactly. This is the power and promise of Segment’s data Warehouse. It allows an analyst to quickly run cross-application SQL queries to get answers to critical business questions.

Of course you still have to know what the fields and data means – it just helps overcome a main impediment from getting these type of projects started.

Me: Thanks Sam, I think I get it. Suppose another SaaS company CXO is thinking about using Segment – how can he or she contact you?

Sam: Just email me at sam@madkudu.com. I’ll be happy to jump on a call.

Closing mid-market deals faster is the key to SaaS sales velocity

Guest post from MadKudu.

Overview: what is the most effective way to increase SaaS MRR?

Regular readers of SaaScribe know increasing monthly recurring revenue (MRR) is the #1 challenge faced by SaaS founders. But what is the fastest way? Get more leads? Close bigger deals? Convert more trial users? Close deals faster?

That’s the question we tried to answer in this study. We analyzed the sales velocity of 45,000 qualified leads for 9 representative SaaS companies. Based on these results, the most immediate way for SaaS companies to increase MRR is by closing mid-market deals (deers) faster. We finish with some advice for creating a high-velocity sales closing workflow that targets mid-market leads.

Rabbits, deers, and elephants – oh my!

A few weeks ago we were chatting in MadKudu HQ about the sales practices at a few of our SaaS customers – how they identified the best leads, when they contacted them, etc.

We noticed that most sales reps focus almost exclusively on closing “elephants” (largest deals) and invest little time in “deers” (medium-sized deals). Traditionally this is how software sales has worked: since a rep can only manage a finite number of leads most sales teams will focus the largest deals.

But there are 2 drawbacks to closing elephants.

  1. They take longer to close.
  2. There are fewer of them.

Our hypothesis was that closing mid-market (deers) deals faster was the most actionable way to accelerate SaaS sales.

Sales velocity is our reference

We needed a way to make an apples-to-apples comparison between deers and elephants based on deal size, deal volume, and time to close. We decided to use Sales velocity since the equation considers all 3 variables.

 

 

Sales velocity measures how fast your team is making money. If normal velocity is “miles per hour” you can think of sales velocity as “money per month”. Thomasz Tunguz provides a detailed explanation of sales velocity in this post, but here are the basics.

The sales velocity variables

# The number of leads a sales team can work over a period of time. Deers have a higher inventory of available leads than elephants.

$ The average deal size. We expect elephants deals to be bigger than deer deals.

% Conversion rate, the percentage of leads that convert to paying customers. The rate of conversion for elephants could be bigger or smaller than deers depending on the amount of qualification.

T The average time for conversion, usually measured in days. We expect elephants to convert slower than deers since larger deals require more negotiation and touch points.

Thus the larger $ for elephants comes at a cost of larger T, smaller and possibly smaller %.

Methodology: how we calculated sales velocity for deers and elephants

We started by picking a representative sample of 9 SaaS companies. We then needed to categorize them into cohorts based on deal size.

Using number of employees as a proxy for deal size

Unfortunately we don’t often have the data we need to test our hypotheses. In this study we had no way of categorizing a lead based on deal size, so we used Clearbit to give us a best estimate.

We starting by identifying each company’s “good” leads based on domain, presence in Clearbit’s database, and behavior.

We broke each company’s good leads into 10 cohorts based on Clearbit’s employee_count data – this served as a basis for identifying elephants and deers. We ignored small deals – rabbits.

For each cohort we calculated the average time to convert, number of leads and conversions.

Example data for 1 company

Screen Shot 2016-04-12 at 8.12.14 AM

Results: what we learned from 45,000 qualified SaaS leads

Here are the results for the 9 companies we studied.

Screen Shot 2016-04-12 at 8.19.49 AM

Result 1: Deers are only closing 10% faster than elephants

We divided deer T by elephant T to see which is faster.

Surprisingly, deers are only closing 10% faster than elephants.

Screen Shot 2016-04-12 at 8.25.58 AM

Result 2: The conversion rate of deers is 3x more than elephants

It isn’t obvious why deer % should be so much higher than elephant %. Since sales teams invest more time selecting and engaging elephants you could argue that elephant conversions should be higher.

But deers convert at a much higher rate on average. There is also a large variance among these 9 sample companies.

Deer conversion rates are also high on an absolute basis. 7 out of 9 we studied are converting > 8% of their deers. (if this seems like b.s. remember we filtered these for “good” leads based on their presence in Clearbit’s database).

Screen Shot 2016-04-12 at 8.29.32 AM

Result 4: Deer deals can be 10x smaller and achieve the same Sales Velocity

Given these results for #, %, and T we can calculate the deer deal sizes needed to hit the same sales velocity. If you assume an equal SV for deers and elephants you can solve for relative $.

I’ll spare you the algebra – results are below.

Screen Shot 2016-04-12 at 8.35.48 AM

Thus on average elephant deal sizes need to be 10x bigger than deer deal sizes to achieve the same sales velocity.

Of course I’m assuming all deer leads are # – even given our filtering assumptions this is a stretch since reps can only work so many leads.

Analysis: your practical options for quickly increasing sales velocity

So how can you increase sales velocity? Conventional wisdom says “it depends” because these variables are codependent.

In the long run this is certainly true – you can adjust pricing or increase qualified leads. But you have fewer options in the short run because your team is already optimizing most of these variables.

You’re reading this because you’re not looking for a long-term theoretical plan – you’re looking for fast actionable, wins. Let’s consider your options in the context of this data.

SVblue

Increase $? Not easily.

Most SaaS companies have already tested pricing and are reasonably close to optimizing conversions and pricing. Unless your product is new there are probably no quick wins from price increases.

Increase elephant %? No way.

Your sales team is already calling every elephant – again … and again … and again. If there was an easy way to close more deals they would be doing it.

Increase deer %? Unlikely.

Based on this data the deers already have a high conversion rate. Increasing it dramatically is probably unrealistic.

Marketing automation and product are already doing a pretty good job at getting deers converted. The qualified deers who don’t convert are already pursued by sales after the trial ends.

Increase #? Yeah…right.

How about increasing your qualified leads? Maybe waive a magic wand so more wonderful customers suddenly show up?

Every SaaS company we know is already working hard to prospect for more leads. Any increases won’t come easily (or cheaply).

Decrease elephant T? Nope.

Elephants take multiple touch points to activate. They have customized workflows and often require purchase orders. Your sales team is already trying to close them yesterday.

Decrease deer T? Yes!!

The only remaining option is to close deers faster.

The 9 companies we studied have roughly the same average time to close for deers and elephants. The only logical conclusion is that these companies are not trying to close deers faster.

This is the key insight from this data.

Insight: The fastest way to increase your SaaS revenue is to close deers faster

Deers should close faster than elephants. Fewer people in the decision loop. Fewer meetings. Less negotiation. Deers also pay with credit cards – not purchase orders.

But according to this analysis deers are closing about as fast as elephants – too slow.

Why? Because SaaS companies are relying on the free trial conversion to close deers

These results are consistent with results from our previous study because most SaaS companies rely on trial expiration as the primary buying incentive.

From this previous study you can see how most SaaS trial conversions occur around the end of the trial period – 30 days in the graph below:

Screen Shot 2016-02-12 at 5.48.12 PM

IOW, deers sign up for your product, self-serve and you don’t try to get them to pay until the trial is about to expire.

Action: how you can close deers faster and accelerate sales velocity

You may be tempted to use marketing automation or product workflow to close deers faster – in our experience your inside sales team will be much more effective.

Here are few tips based on what we have implemented with our customers.

Don’t chase the whole herd – qualify your deers

1691205018_90bd181004_b

Flooding your sales reps with a pile of mid-market leads won’t work – you’ll probably decrease your sales velocity. If your sales reps start converting less than 15% of their leads they will become frustrated and less effective.

You need to identify the most qualified deers, engage them the moment they are ready to buy and developer a higher-velocity closing process.

Some tips for qualifying leads:

  1. Leads that are in Clearbit’s database are a good initial filter. You can also disqualify any free email accounts (gmail, hotmail, qq, etc.).
  2. Segment deers and elephants with simple rules – for instance, start with employee_count or plan.

Start with 1 dedicated “Deer Hunter” sales rep

We suggest starting with 1 dedicated sales rep to close deers. Let’s call her the “Deer Hunter”.

Start small and begin tracking the sales velocity of the Deer Hunter.

Build a higher-velocity engagement workflow

The Deer Hunter can’t simply manage list of deer leads and systematically work through them – this takes too much time. Instead, work with marketing to develop a sales automation workflow that gets a deer to take the first engagement step.

For example, an email campaign that asks a deer to reply to a question or schedule a call based on qualifying demographics or behavior.

Create a 1-touch deer closing script

The Deer Hunter needs a script that gets a deer to convert on a single call. The script should include any conversion incentives (i.e. discount, free feature) and minimize product education. Sales should be credit card only.

Target deers who are ready to buy now

Your customers can take specific steps that indicate a high likelihood of buying – invite a friend, add 5 projects, etc.

We call these “Acceleration” or “Delight” events and showed you how to find them in our series on Behavior-based Conversions.

These events are highly correlated with conversions and indicate a customer who is ready to buy now.

In this example hired_accountant is Acceleration event:

graph

Notify the Deer Hunter when a deer is ready to buy

Deers who complete Acceleration events are ready to buy now – these are the ones you want your Deer Hunter to target.

Send your sales teams notifications through Slack, email, or Salesforce when deers complete Acceleration events.

Include information about the customer, actions completed, and anything else the sale rep needs for the deal-closing script. Here is an example of what we send to our customers:

About MadKudu

 

MadKudu helps B2B SaaS apps accelerate revenue by qualifying leads based on demographics and in-app behavior.

Sign up for a free trial of MadKudu now .

Photo credits: the_boglin

Close deers faster: your quickest win for increasing SaaS MRR

Summary: We analyzed the sales velocity of 45,000 qualified leads for 9 representative SaaS companies. Based on these results the most immediate way for SaaS companies to increase MRR is by closing mid-market deals faster. We advise setting up a separate, high-velocity sales closing workflow that targets mid-market leads who are ready to buy now.

A few weeks ago we were chatting in MadKudu HQ about the sales practices at a few of our customers – how they identified the best leads, when they contacted them, etc.

We noticed that most sales reps focus almost exclusively on closing “elephants” (largest deals) and invest little time in “deers” (medium-sized deals). Traditionally this is how software sales has worked: since a rep can only manage a finite number of leads most sales teams will focus the largest deals.

But there are 2 drawbacks to closing elephants.

  1. They take longer to close.
  2. There are fewer of them.

Our hypothesis: the most immediate opportunity to grow SaaS MRR is by closing mid-market (deers) deals faster

So … we quit there. It seemed easier to hold on to an opinion rather than actually look at the data.

Of course I jest. We decided to research the sales data of a sample of our customers to see if there was data to support our hypothesis.

Sales velocity is our reference

We needed a way to make an apples-to-apples comparison between deers and elephants based on deal size, deal volume, and time to close We decided to use Sales Velocity since the equation considers all 3 variables.

 

 

Sales velocity measures how fast your team is making money. If normal velocity is “miles per hour” you can think of sales velocity as “money per month”.

The sales velocity variables

# The number of leads a sales team can work over a period of time. Deers have a higher inventory of available leads than elephants (although acting on them places more pressure on qualification).

$ The average deal size. We expect elephants deals to be bigger than deer deals.

% Conversion rate, the percentage of leads that convert to paying customers. The rate of conversion for elephants could be bigger or smaller than deers depending on how much qualification is done for the lead pool.

T The average time to for conversion usually measured in days. We expect elephants to convert slower than deers since larger deals require more negotiation and touch points.

Thus the larger $ for elephants comes at a cost of larger T, smaller and possibly smaller %.

Methodology: how we calculated sales velocity for deers and elephants

We started by picking a representative sample of 9 SaaS companies. We then identified each company’s “good” leads based on domain, presence in Clearbit’s database, and behavior.

We broke each company’s “good” leads into 10 cohorts based on the employee count attribute in Clearbit. We decided to use employee_count as a basis for identifying elephants and deers. We ignored small deals – rabbits.

For each cohort we calculated the average time to convert, number of leads and conversions.

Example data for 1 company

Screen Shot 2016-04-12 at 8.12.14 AM

Results: what we learned from 45,000 qualified SaaS leads

Here are the results for the 9 companies we studied.

Screen Shot 2016-04-12 at 8.19.49 AM

Result 1: Deers are only closing 10% faster than elephants

We divided deers T by elephant T to see which is faster.

Surprisingly, deers are only closing 10% faster than elephants.

Screen Shot 2016-04-12 at 8.25.58 AM

Result 2: The conversion rate of deers is 3x more than elephants

It isn’t obvious why deer % should be so much higher than elephant %. Since sales teams invest more time selecting and engaging elephants you could argue that elephant conversions should be higher.

But deers convert at a much higher rate on average. There is also a large variance among these 9 sample companies.

Deer conversion rates are also high on an absolute basis. 7 out of 9 we studied are converting > 8% of their deers. (if this seems like b.s. remember we filtered these for “good” leads).

Screen Shot 2016-04-12 at 8.29.32 AM

Result 4: Deer deals can be 10x smaller and achieve the same Sales Velocity

Given these results for #, %, and T we can calculate the deer deal sizes needed to hit the same sales velocity. If you assume an equal SV for deers and elephants you can solve for relative $.

I’ll spare you the Algebra – results are below.

Screen Shot 2016-04-12 at 8.35.48 AM

Thus on average elephant deal sizes need to be 10x bigger than deer deal sizes to achieve the same sales velocity.

Of course I’m making an assumption that all deer leads are # – even given our filtering assumptions above this is probably a stretch since sales reps can only work so many leads.

Analysis: your practical options for quickly increasing sales velocity

So how can you increase sales velocity? Conventional wisdom says “it depends” because these variables are codependent.

In the long run this is certainly true – you can adjust pricing or increase qualified leads. But you have fewer options in the short run because your team is already optimizing most of these variables.

You’re reading this because you’re not looking for a long-term theoretical plan – you’re looking for fast actionable, wins. Let’s consider your options in the context of this data.

SVblue

Increase $? Not easily.

Most SaaS companies have already tested pricing and are reasonably close to optimizing conversions and pricing. Unless your product is new there are probably no quick wins from price increases.

Increase elephant %? No way.

Your sales team is already calling every elephant – again .. and again … and again. If there was an easy way to close more deals they would be doing it.

Increase deer %? Unlikely.

Based on this data the deers already have a high conversion rate. Increasing it dramatically is probably unrealistic.

Marketing automation and product are already doing a pretty good job at getting deers converted. The qualified deers who don’t convert are already pursued by sales after the trial ends.

Increase #? Yeah…right.

How about increasing your qualified leads? Maybe waive a magic wand so more wonderful customers suddenly show up?

Every SaaS company we know is already working hard to prospect for more leads. Any increases won’t come easily (or cheaply).

Decrease elephant T? Nope.

Elephants take multiple touch points to activate. They have customized workflows and often require purchase orders. Your sales team is already trying to close them yesterday.

Decrease deer T? Yes!!

The only remaining option is to close deers faster.

The 9 companies we studied have roughly the same average time to close for deers and elephants. The only logical conclusion is that these companies are not trying to close deers faster.

This is the key insight from this data.

Insight: The fastest way to increase your SaaS revenue is to close deers faster

Deers should close faster than elephants. Fewer people in the decision loop. Fewer meetings. Less negotiation. Deers also pay with credit cards – not purchase orders.

But according to this analysis deers are closing about as fast as elephants – too slow.

Why? Because SaaS companies are relying on the free trial conversion to close deers

These results are consistent with results from our previous study because most SaaS companies rely on trial expiration as the primary buying incentive.

From this previous study you can see how most SaaS trial conversions occur around the end of the trial period – 30 days in the graph below:

Screen Shot 2016-02-12 at 5.48.12 PM

IOW, deers sign up for your product, self-serve and you don’t try to get them to pay until the trial is about to expire.

Action: how you can close deers faster and accelerate sales velocity

You may be tempted to use marketing automation or product workflow to kill deers faster – in our experience your inside sales team will be much more effective.

Here are few tips based on what we have implemented with our customers.

Don’t chase the whole herd

1691205018_90bd181004_b

Flooding your sales reps with a pile of mid-market leads won’t work – you’ll probably decrease your sales velocity. If your sales reps start converting less than 15% of their leads they will become frustrated and less effective.

You need to identify the most qualified deers, engage them the moment they are ready to buy and developer a higher-velocity closing process.

Start with 1 dedicated “Deer Hunter” sales rep

We suggest starting with 1 dedicated sales rep to close deers. Let’s call her the “Deer Hunter”.

Start small and begin tracking the sales velocity of the Deer Hunter.

Build a higher-velocity engagement workflow

The Deer Hunter can’t simply manage list of deer leads and systematically work through them – this takes too much time. Instead, work with marketing to develop a sales automation workflow that gets a deer to take the first engagement step.

For example, an email campaign that asks a deer to reply to a question or schedule a call based on qualifying demographics or behavior.

Create a 1-touch deer closing script

The Deer Hunter needs a script that gets a deer to convert on a single call. The script should include any conversion incentives (i.e. discount, free feature) and minimize product education. Sales should be credit card only.

Target deers who are ready to buy now

Your customers can take specific steps that indicate a high likelihood of buying – invite a friend, add 5 projects, etc.

We call these “Acceleration” or “Delight” events and showed you how to find them in our series on Behavior-based Conversions.

These events are highly correlated with conversions and indicate a customer who is ready to buy now.

In this example hired_accountant is Acceleration event:

graph

Notify the Deer Hunter when a deer is ready to buy

Deers who complete Acceleration events are ready to buy now – these are the ones you want your Deer Hunter to target.

Send your sales teams notifications through Slack, email, or Salesforce when deers complete Acceleration events.

Include information about the customer, actions completed, and anything else the sale rep needs for the deal-closing script. Here is an example of what we send to our customers:

Let us help you close deer faster

I can’t help you hunt deers – I’m a vegetarian and wouldn’t shoot poor little Bambi.

But MadKudu can help you increase your sales velocity by identifying the most qualified deers and telling you when they are ready to buy.

Sign up for a free trial of MadKudu now – you have nothing to lose.

P.S. Could you have written this post?

If so you should contact us – we’re hiring.

 

Photo credits: the_boglin, Kansas Tourism

Why are your job descriptions so boring?

We’re not that picky about who we hire – we just need a qualified warm body to fill the chair.
— Said by no startup CEO … ever

Things are going great here at MadKudu. We hit our growth milestone and our recent pivot allowed us to sign up awesome customers like Segment, Contactually, and Codeship.

But … alas … there is little time for celebrating in a fast-growing startup because success just creates new problems. We now have the #1 challenge facing every startup with traction: hiring initial team members.

We decided to get serious about building our team and invest time attracting the right people. We created a “careers” page and and started writing job descriptions.

I’m tasked with hiring our first full-time marketer. Before writing the job description I did – what you do – read a few dozen open job recs for ideas.

I’ve got > 2901 competitors for the right candidate

Shocker! We’re not the only startup trying to fill a marketing role – actually there are open marketing positions at 2,901 other startups on Angel List alone.

My challenge is getting the attention of the most talented people – those who could compete for jobs at companies like Medium or Slack.

At first I found this prospect intimidating – that is, until I started reading the job descriptions.

Most startup job descriptions are horrible

Generic. Repetitive. Uninspiring. BOOOORING.

Most look like they were written by Dilbert & edited by Catbert.

Don’t believe me? Spend a few minutes browsing jobs on Angel List. Go ahead, I’ll wait.

See what I mean?

“need … excellent communications skills … a team player … “

If “hiring” is a startup CEO’s biggest challenge why do so many put no effort into writing job descriptions?

I attempted to do better

I stopped looking to job descriptions on Angel List for inspiration. Instead I imagined myself talking to an incredible person – someone who has lot of choices – and trying to help her decide if the job is for her.

After a few hours I finished Want to be our Head of Marketing? It could be a lot better but it was the best I could do in the time I had.

How to write a job description that attracts the candidates you want

Sell. The. Job.

The people you want to hire have options. They are probably not even looking for a job. You need to sell the opportunity to work at your amazing company.

Write TO your ideal candidate. Tell her about your company and why you need her help. Make the opportunity sound so compelling that a friend would forward the job description to her.

Suppose your company keeps growing and she does amazingly well. What would be her opportunities? Sell a prospect on your vision for her awesome future.

Skip the superfluous bs

Everyone thinks they are …

… good at communication,
… a team player,
… a self-starter,
… and on … and on.

So don’t include these words in your job description – they just fill up space and make you sound like everyone else. In the history of job searching nobody has ever read a job description and thought …

“hmmmmm… this job sounds good but they don’t mention teamwork … guess I’ll pass since I won’t get to improve my teamwork skills”

good communication skills? Well that’s not me – I’m boring and ramble a lot. Better not apply for this job.”

Say what the job ISN’T – and who will HATE it

One of the easiest ways to attract the right candidates is to say what a job ISN’T. Talk about the pros and cons of your company.

For instance, Silicon Valley is full of legendary companies that “started in a garage.” Sounds cool AFTER the company is successful – but what is it like actually working in a garage? No windows … or air conditioning … plenty of dead crickets … not exactly an inspiring workplace for most people.

Except WE’RE NOT most people – and we want to attract people who share our values. We’re currently working in a shack we affectionally call “le donjon”. (yes, that’s French for “dungeon”).

donjon - 1

It has a whiteboard, a few desks, and lots of coffee stains. In the center is a little bell we ring whenever we close a new deal. I fell in love with the donjon the first time I saw it.

The right candidate will feel the same way.

Give examples. Tell stories.

Why are you hiring? What problem are you trying to solve? Can you describe a recent problem you need the ideal candidate to solve? Can you give a real example?

The right candidate will see the problem and think “I can do that!”

Yes, this takes more time – up front

Hiring can feel like a big distraction when you’re trying to do “real” work. It is much quicker to cut-and-paste someone else’s job description and hope it works out.

Invest the time selling your opportunity to the right candidate – you’ll get better results and save money in the long run.

Want to work in our beloved donjon?

Sweet – you’re one of us. I hope you’ll check out our open positions and apply.

 

photo credit: star5112

Startup vs bigco: the best career option in data science

Our fearless leader and CEO Sam Levan recently spoke at Galvanize in San Francisco about data science careers. A common student question is, “is a job at a startup or a big company better for data scientists”?

At MadKudu we’re in a good position to answer it – the 5 of us have more than 25 years experience in data science. We’ve built everything from the world’s largest fraud detection system to quick hacks in Google sheets.

So what is the best career option for an aspiring data scientist? Google or HotNewStartupWithCuteBlueDoggyMascot.com?

Before answering I’m going to share a secret about being a data scientist. Whether you work at a 2-person startup or CapitalOne, there is one attribute which best predicts your probability of success.

The super-duper-secret to being a great data scientist is …

… wait for it …
…… wait for it ……
……… WAIT FOR IT!!! ………

Data science is a SOCIAL skill

That’s it. That’s the big secret. Your career as a data scientist will be defined by how well you can communicate, write, listen, organize, lead, and empathize.

Do you need hard skills? Of course. You can’t do the job if you don’t know how to use R, Python, or MatLab. You have to know how to measure the statistical significance of your results.

But unless you fancy yourself the next Will Hunting, being a brilliant Python coder won’t make you any more effective than a good one if you can’t work well with others.

Data scientists don’t work alone

Data scientists try to solve business problems with data – an iterative activity which requires working with cross-functional teams.

Suppose you’re a knowledge engineer working at a bank. On any given day you will need to:

  • Talk to regulators about the riskiest type of criminal activity.
  • Help analysts understand customer data and what behavior the bank can track.
  • Ask (beg?) the operations to pull new data sources for you.
  • Testify in court.

The most effective data scientists are team players who make everyone else more effective.

If you want to be a lone hero data science isn’t for you.

Communication beats code

Every morning I read Nate Silver’s analysis on FiveThirtyEight. Is Nate the world’s greatest statistician?

Of course not. Nate Silver’s brilliance is his ability to help us understand how data answers important questions in politics, sports and life.

We try to deliver the same value in our work.

This is – by far – our most popular blog post. Why was it featured on Growth Hackers, Hacker News, and Growth Hacking digest? It wasn’t a great study – just 9 sample companies. We didn’t build any amazing models – everything was done in Google Sheets.

As an example of data science it is … meh. Our customers loved it because we helped them understand what the data means and how they can use it to solve business problems.

Any idiot can have an opinion. Lots of smart people can compile numbers.

Few people can help others understand why data matters and what they should do – be one of them and you’ve got the world at your feet.

Data science careers: Startups vs Bigco

Back to the students’ questions:

“What is the difference between being a data scientist at a startup vs a Bigco?”

Specialist vs generalist

The biggest difference between being a data scientist at bigco vs a startup is your degree of specialization. At a bigco you have the opportunity to work for months and years on the same problem.

Are you excited about spending 2 years creating the world’s greatest recommendation engine for Facebook? Do you like doing primary research? Becoming an expert in building models to solve 1 problem? Becoming a master in R, Python, or MatLab?

That’s life in a bigco. I know Knowledge Engineers who spend a career detecting violations of stock market wash sale rules.

Startups? Ha ha ha!

You won’t know what you’re working on next week, much less next year. You’ll be spending your time helping marketing, sales, and product teams answer basic questions. Since you’re constrained by data and time you’ll do much of your work in spreadsheets or SQL.

It isn’t uncommon for a data scientist at a startup to be juggling 5 different problems at the same time. Your expertise will be your ability to quickly acquire and apply new skills – fortunately this is a great skill to have.

Support system

Unless you work at MadKudu, you may be the only data scientist at your startup. Your colleagues may not understand what you do or how you can help them. You may have to define your own objectives. On your first day you might be told to “go help the sales team find the best leads”. Does this terrify or excite you?

At bigco you will have a support system. Your boss will tell you which project you’re working on. More experienced data scientists can answer your questions. Have a problem? Ask your boss – that’s what she’s for.

Getting dirty

Data science textbook examples are fairy tales. In 20 years I’ve never encountered such simple problems. In the real world:

  • Simply getting the data is HARD.
  • People don’t agree on what columns actually mean.
  • Everything changes while you’re doing analysis.

Bigcos have teams of people to help solve these problems: server-side developers to populate the data warehouse and business analysts who write data dictionaries.

At startups … well … it is probably up to you. The developers are all too busy finishing the next release and supporting customers to run SQL queries. You have look in the code to see how the product generates the account_activated event in Mixpanel.

Risk

At bigco you’ll have a nice salary, 401(k) plan, and benefits. You’ll work a little harder the 3 months before bonus time so you can get that new car. It feels safe – but is it?

Life at a startup is the opposite. Part of your compensation will potential, unknown upside from stock options. Will you have a job next quarter? It depends on whether the CEO can close the B round. It feels risky – but is it?

I’ve worked for the world’s biggest, most stable employer and at any-day-we’re-dead-lets-start-stealing-office-supplies startups. I’ve had friends lose $500K starting a company and others struggle for years to find a job after being laid off. Here is how I think about risk.

Working for startups is very risky in the short run but incredibly stable in the long run.

The stability of bigcos comes at a price – you develop fewer skills, build fewer relationships, and don’t get regular experience marketing yourself.

This risk is particularly true for a data scientist who can get stuck working on the same problem … with the same tools… and the same people … for years. A major industry downturn can be economically devastating when all companies in a sector are laying off employees.

Both bigco and startup careers have risks – you just need to understand the risks you’re taking and be smart about managing them.

What’s best for you – bigco or a startup?

After reading this post you’re probably more confused than ever – because there is no one answer.

My #1 piece of advice is to go out an interview with big and large companies. Meet the teams and ask lots of questions.

What is the #1 problem you would be solving? Why? Who else is on your team? What do they say about the problem? What tools would you be using?

It’s the only way you’re going to see what is best for you.

Best of all it will give you an opportunity to work on those social and communication skills – the most critical ones for your career in data science.

Photo credit: goingfar.org