How to keep your data clean with proper alerting from your data system

No data is perfect, and data quality practices and processes are absolutely critical to put in place before any analysis can be performed.

How to keep your data clean with proper alerting from your data system
Photo by Luís Perdigão / Unsplash

Every business cares about making the most out of their data, but not everyone in the business cares about ensuring data quality. If there are any data professionals within that business, then they probably know that no data is perfect, and data quality practices and processes are absolutely critical to put in place before any analysis can be performed.

In cases where you’re working with highly customizable tools such as Salesforce, an abundance of manually-input fields often make documenting “best practices” impossible and cause project hand-offs to become transition nightmares.

As you can imagine, the issue only gets exponentially worse with multiple data management systems in play, as is the case in many businesses.

The benefit of data quality alerts

When you can’t automate those manual inputs, your next best solution is setting up data quality alerts. These are warnings automatically triggered by anomalies in your system’s records that arise due to human error.

Benefits of setting up event-triggered alerts:

  • What gets brought to the attention of stakeholders is relevant
  • When that information is brought up is also relevant
  • Setting up alerts defined by logic that’s limited to specific and high-importance scenarios is a lot more resource efficient than tracking everything all the time
  • Businesses can create a system that provides feedback on where resources should be allocated and what efforts should be prioritized

Narrator’s alerts story

Prior to setting up alerts for our own teams at Narrator, missing data entry errors and tedious manual fixes were the reality of working within a highly customizable CRM system like Salesforce.

Utilizing internal alerts enabled our teams to not only track relevant and complex sales, operations and customer support scenarios, but also perform analyses using data that we were confident had been vetted before.

As an example of when our alerts really came in handy, our calculations of Narrator’s Annual Recurring Revenue (ARR) and Average Customer Value (ACV) were underestimated due to missing contract emails, which in turn led to missing contracts on closed-won opportunities in Salesforce.

Before:

After:

Let’s take a closer look at what these alerts look like in Narrator.

Narrator’s solution

Narrator’s data quality alerts have allowed us and our customers to create custom validations for transformations beyond the standard validations Narrator runs when the transformation is first created. Leveraging custom alerts is a great way to ensure that the assumptions used in transformation queries still hold over time.

Alerts in Narrator are defined using SQL queries that can be scheduled to run regularly and alert specified users via email if it returns no rows or any rows.

The below example is the alert created for missing emails on closed-won opportunities. This would raise alarms for our Operations team since missing this field can lead to plenty of downstream reporting errors, beyond just ARR and ACV underestimates, that can become very time consuming to track down the longer they are left unfixed.

The below is what the emailed alert looks like when that alert’s SQL query returns any rows:

Conclusion:

Setting up data quality alerts is a necessary practice to ensure the quality of the data that comes out of systems requiring manual inputs like Salesforce. Narrator offers an easy way for you to set up those alerts so that your business can always rely on the data before using it to power analyses.

To learn more about how to create alerts in Narrator, check out our docs.


Check us out on the Data Engineering Podcast

Find it on the podcast page or stream it below