Just Launched: (No Code) Validation Monitors

The best data quality monitoring coverage is both broad and deep. It also is simple to deploy, without requiring any code.

Data teams need broad monitoring across their entire surface area to detect changing trends in their data behavior. For example, if a table normally gets data every hour and it’s been four hours with no update in sight, you should know.

These anomalies indicate systemic issues that need to be identified and resolved. AI monitors are the best tool for this job as they are easier to scale and can catch issues that could never be anticipated otherwise. (To learn more about broad monitoring you can check out our post on our Data Products and AI Metric Monitoring capabilities).

But this coverage needs to be supplemented with deep monitoring that sets hard thresholds on specific, important fields. For example, if Social Security Numbers are critical to an application involved in processing bank loans, then it is important to surface all the records where the data doesn’t conform to the standard format.

Many data teams depend on manually crafting SQL statements to validate individual records. This process can also involve subject matter experts providing requirements to more technical individuals who can then express them in SQL. Either way, it’s tedious and time intensive.

To solve this problem and help enterprise organizations make data quality more accessible,  Monte Carlo is excited to announce the release of  Validation Monitors. Let’s dive in to see how Monte Carlo empowers all members of the data team–regardless of their SQL proficiency–to set deep data monitors.

Deep monitoring with Validation Monitors

Validation Monitors makes it easy to define when Monte Carlo should alert you to bad data with only a few clicks. Once you’ve selected the table, field, and operator, relevant metrics appear in the drop down menu. 

For example, if the selected column is a timestamp, then there are options to specify the time-range (e.g. last 365 days) or check whether the entry is in the future. 

Numeric columns have the options to validate conditions such as NULL, zero, not a number, negative, or manually entered ranges. Strings can be monitored for specific completeness metrics, formats, and manually entered values. Check out the full list of metrics here.

You can even build complex validations involving multiple variables and conditions. Here, I’ve defined a validation to alert me if a column zip code contains “92630” AND state does not contain “California.”

As powerful and as extensive as Validation Monitors are, there will still be cases when what you want to monitor may not fit with the dozens of validations we built into Monte Carlo. For example, maybe your organization is a logistics company and your delivery times for a certain product follow a very reliable cadence of 18 days. 

The good news is anything that can be expressed in SQL can be monitored in Monte Carlo. Just write the condition in SQL and we’ll take care of the rest.

And you can also go a step further using SQL Rules to write any check you desire. If you don’t know or don’t want to write SQL, leverage our GenAI as is shown below:

Democratizing data quality across the enterprise

We are excited for how the extensibility and power of Validation Monitors will enable data teams to easily deploy deep data monitors. 

The key takeaway is that Monte Carlo’s data observability platform allows data teams to democratize data quality by allowing you to deploy monitors however you like to work whether that is automated across a data product, via code during the CI/CD process, using natural language with a chatbot, or with a simple no-code UI.

Looking forward, on the roadmap you can anticipate we will continue to develop ways for teams to make easy, strategic monitoring decisions including automatically recommending monitors based on data and metadata profiling.

Our promise: we will show you the product.