This post was contributed by Saadat Qadri, Analytics Practice Lead at Outshine. This post discusses how and why data warehousing is part of the remedy to a siloed marketing stack, and what the most common misconceptions are that B2B SaaS companies have relative to data warehousing.
Behind every B2B SaaS organization is their technology stack. However, each department has their own source of truth which makes reporting on cross-functional performance difficult. This is where data warehousing comes in—a helpful tool in dealing with a siloed stack. At a certain point, a data warehouse is an inevitable investment for high-growth B2B SaaS companies. While building a data warehouse is a very important step, it is also complex, painstaking, and often organizations don’t have the internal expertise to execute on a successful implementation.
In this post, we discuss common misconceptions that executives have around data warehousing projects and offer solutions.
Data Warehousing for Revenue Intelligence
The type of data warehouse we refer to in this post is one that is built to inform marketing and sales decisions to drive revenue (which is very different from the type of data warehouse you build to store product data). Both are important investments, depending on your organization, but are vastly different in the purpose they serve, and how and who you need to build them. Before we dive into some of the pitfalls to avoid and best practices for success, let’s look at how to determine if it’s time to build a data warehouse.
Leading indicators that your organization needs a data warehouse
How do you know you’re ready to invest in a data warehouse? Here are the two most common signs your organization is ready to invest:
- As a senior marketer or a senior executive of the company, you’re asking a lot of questions about the performance of cross-channel marketing and you can’t get answers fast enough. This is the number one sign. It should take your analysts hours to answer performance questions—not days or weeks.
- Your analysts struggle to understand and identify basic trends in marketing and sales performance. This is very important for companies in a growth phase. Most companies rely on a superficial understanding of business performance. In order to be able to diagnose problems and/or opportunities, you have to be able to go deeper.
- For example, you understand your cost per lead by lead source, but need to report on which leads close (won) fastest and for the most money in Salesforce. Answering this question involves connecting marketing data (most likely Google Analytics) to data from your CRM. A data warehouse aggregates data across these channels (and others) to provide answers on cross-channel performance.
A data warehouse, like a sophisticated CRM, isn’t a justifiable investment for most early-stage companies, however, at a certain point, the opportunity cost of lost revenue is greater than the cost of a data warehouse. That’s when it’s time to invest.
Top Five Misconceptions:
1) A Data Scientist will Solve all my Data Problems
There has been a lot of hype around data science and machine learning lately, however, we believe that a data scientist is not the best hire to build or maintain a data warehouse. Rather, a more effective first hire to address your data problems is a Data Engineer; specifically, someone who:
- has experience cleaning and structuring data for reporting and analysis.
- understands scripting, SQL, distributed systems, and knows how to build for scale.
- understands the context and purpose of data at your organization.
- has experience implementing data warehousing technologies such as Amazon Redshift or Google BigQuery
A good Data Engineer will provide clean data schemas that can be reused for reporting and leveraged to help push business objectives forward.
Too often, organizations think a Data Scientist will solve both these problems. It’s untrue and unfair to expect those skills from one person. In fact, what we observe time and time again is, data scientists that do not have access to clean, structured data cannot provide actionable insights. Building a data warehouse and using the information produced by a data warehouse are two very different tasks that require two different dedicated resources.
To manage expectations, depending on the size or complexity of your data – it takes at least six months to a year for a Data Engineer to lay the proper foundations and get your data in the right shape. Once your data is in good shape, then you may want to hire a Data Scientist to build predictive modeling, forecasting, and other data-driven projections.
2) The Engineering Team Can Build a Data Warehouse as a Side Project
Don’t make the mistake of tasking your product or engineering team with building a data warehouse as a side project. These teams have a lot of responsibilities already and can’t execute on this in addition to their regular responsibilities. Even if someone(s) on the engineering team is able to build it, consider if they’ll also be able to maintain and upgrade the data warehouse over time. For most organizations, a successful data warehouse implementation is largely dependent on having a dedicated resource.
Also, it is worth noting that engineering teams don’t typically use or derive benefit from a data warehouse, nor are they familiar with the business case(s) for a data warehouse. Marketing and Sales departments most often directly benefit from a data warehouse and best understand its purpose; therefore, they should play a role in its creation. Given that your sales/marketing organizations typically lack the technical ability to execute on the development of a data warehouse, the best practice is to find a dedicated engineering resource within your company to help build and maintain the data warehouse.
3) Something is Better Than Nothing
A mistake we often see fast-growing organizations make is they only partially invest in building a data warehouse. In some cases, it gets deprioritized or the resources get reallocated to save time or money and they justify it by saying, “something is good enough.” In reality, the opposite is true. Incomplete or inaccurate data leads to making poor decisions and restarting or rework will cost the organization more in the long run.
4) Data Warehousing is a Sexy Project
Building a data warehouse may sound interesting but it isn’t what the bulk of technical hires will be passionate about. Most people are passionate about building a product that will be put out into the world. It can be difficult to find team members who think building a data warehouse is interesting and want to work on it. In most organizations, it’s a responsibility that’s cast upon someone rather than volunteered for.
Another prevalent misconception is that the person in charge of building a data warehouse should be technical. As mentioned, the person you’re looking for is a Data Engineer; ideally, someone that has both technical skills and enough business acumen to understand the use cases of deploying a data warehouse. This is very important—the most effective data warehouses allow people to self-serve. In some cases, when an overly technical hire builds a data warehouse, it’s built in a way that only answers a rigid set of questions.
“You want somebody who lives in both worlds, who understands the business use case and can execute on the technical requirements.”
5) It’s Fast and Easy
Another common mistake is underestimating how long building a data warehouse will take. The process, done correctly, can be difficult. Especially if the company has a lot of technical debt, like legacy code or faulty work-arounds, which most companies do. In our experience, it takes at least six months to a year to build a strong, reliable foundation and many months of tweaking the reporting requirements for key stakeholders in the organization.
Wrap Up
Reporting tells us what’s happened in the past. Predictive modeling focuses on forecasting an answer to: where do we want to go? This is a question every organization should be aiming to answer. That can’t happen in the absence of having a solid foundation. Predictive modeling and machine learning can’t be leveraged without the output of a data warehouse. Building a data warehouse sets your organization up for success in the long term.
If you’re a CMO or CRO, your first priority should be to get the foundation right. Once you recognize the signals don’t shy away from making an investment; it’s a slow, painstaking process but it will benefit your organization.
Your second priority should be to assess whether or not you have the resources in-house to successfully implement a data warehouse. If not, a good Data Engineer costs upwards of $200K (in San Francisco—this varies by region). In many cases, it’s more cost effective to hire a third party to execute on the implementation. At a minimum, you should have an outside party conduct an unbiased assessment of your organization’s technical debt before you decide which route is best for your organization.
