A.I. Confidential | The Unvarnished Truth From 5 Anonymous Data Leaders 

You can’t turn around in the data space without running into hot takes about GenAI. Will it make our jobs obsolete? Will robots take over the world? Will organizations figure out how to unlock unprecedented levels of value for their customers?

But how many of those hot takes are genuine and how many are for the clicks? What do data leaders say when the cameras aren’t rolling and the board decks aren’t public?

To answer that question, we asked data pros from a variety of industries, geographies, and company sizes who are building the infrastructure for these tools to share their unvarnished “hand on heart” opinions about the on-the-ground reality of GenAI…and they didn’t disappoint.

From fears of unrealized productivity gains to words of caution, we heard from all sides of the GenAI hype debate.

Curious what they had to say?

Well, open up those manilla folders labeled “confidential,” and let’s get started. 

(Names and titles have been redacted to protect the anonymity of our informants.)

How helpful would GenAI be for your organization today?

Informant 1
“Today we have a GenAI that’s integrated into our Data Catalog / Data Discovery tooling. Mostly, it’s helping us summarize conversations and post them for visibility. It’s helping us fill in definitions for business terms, measures, and metrics. It’s helping us answer more complex questions based on a variety of available metadata factors.

Does it do things perfectly? No. Is it killing jobs? No. Is it altering our lives? No. Does it seem to help in little ways today? Yes.

We are taking baby steps towards truly building solutions from scratch with GenAI. But today it is really just an assistant or helper.” 

Informant 5
“Bar a few use cases (chatbots, coding assistant), I don’t believe it will [fundamentally impact our organization, team, or the solution]. Happy to be proven wrong though!”

Informant 2
“My feeling about generative AI isn’t as specific as any one company’s platform. I think in its current state Generative AI is far better suited to support the “soft” skills of business. By that I mean feeding your GenAI engine the entirety of your confluence, Google, docs, policies, FAQ’s etc. 

That source and flow doesn’t really need grooming but the Chatbot style interface to results would increase productivity in ways that looking at your core data flow via Generative AI can’t.

Application databases and data sources are designed to be specific and targeted answer stores. Generative AI excels in the unformatted loose forming of data text and context. A lot of what applications store are numbers, dates and state. This is better formatted for classical ML applications.”

What’s the biggest opportunity for GenAI?

Informant 3
“Assuming you could build business cases that had value, the impact would be tremendous. I am currently leveraging ChatGPT to assist in code writing and I would expect that most cases would be similar productivity enhancements. I think anything you implement would fundamentally alter the data practice. Everything from architecture to security would be impacted. Especially if you truly build in a learning function to the system. Those touchpoints would add additional points of failure and require additional monitoring and functional testing to roll out.”

Informant 1
“We have implemented MS Copilot, MS 365 Copilot, and MS Sales Copilot (Dynamics). It is successfully helping us improve note taking, meeting summarization, translating MS Word files into PowerPoints, experimenting with creating MS Excel files out of conversations and translating MS Excel discovery analysis into Power BI reports.

The more deeply we integrate into our data stack and entire technology stack, the more answers it will be able to provide. But we need to spend greater time curating information to make sure the answers are as good as possible.”

What’s the biggest challenge to implementing GenAI at your organization?

When considering “how far away are you really from enabling a GenAI use case,” data quality, security concerns, and trust in the technology are some of the biggest hurdles data leaders face to realizing the potential of AI at an enterprise level.

Informant 3
“Most use cases are limited by the quantity and amount of data available. There is also a plethora of security issues that would have to be reviewed. I would think that most of the use cases I could see would have to work with some type of existing platform tool since the expertise to build and deploy one would take forever. 

The use case for a chatbot might be interesting, provided there is a business case that would validate the work. I would assume we could build an internal chatbot to test the validity.”

Informant 1
“Well, we would only gain a base level of use cases from GenAI if we don’t start training our people on prompt engineering or allow more of our stack to integrate with GenAI. Our team needs to be willing to hand over the things that GenAI does well, so they can focus on the more complex tasks that they cannot yet trust it with. 

We are leaving productivity on the table if we don’t allow GenAI to take over some things and if we don’t train our people to take advantage. But without curation and guidance, it’s not going to stay high quality, we run the risk of creating low quality data objects (even documentation) with GenAI and then the low quality data objects will be fed into the model and produce a garbage in, garbage out cycle that will lead the business in a bad direction. A larger focus on auditing information and curating it will matter more and more than creating it.”

Informant 4
“We could probably get away with a basic customer support chatbot.  For anything beyond that, we’d need a solid data catalog/data lineage in place to avoid just spewing nonsense.”

Informant 5
“I don’t think our infrastructure is the thing holding us back here. We’re treading quite cautiously here –  with the landscape moving so fast, and the risk of reputational damage of a ‘rogue’ chatbot, we’re holding fire and waiting for the hype to die down a bit!”

Is GenAI the solution to all our problems?

Informant 2
“Generative AI for the sake of Generative AI won’t solve any problems. Just like any engineering puzzle, you have to know the problem you are trying to solve before asking what tools can help us get there.

Generative AI is a tool in the tool belt. It might be a Swiss army knife but it’s still a tool and should be considered as an option to solve a problem, not a whole solution.”

So… what’s the verdict?

Data pros agree that the GenAI of today does help streamline small, “softer” tasks. But, in these early days of GenAI products, the most impressive use cases are still powered by real organizational/customer data. And that means, if it wasn’t there already, data quality is back at the top of every data team’s priority list.

Before diving head first into GenAI, data teams need to take a step back and double down on implementing strategies, processes, and technologies that are laser focused on delivering reliable data. Only with the right data strategy in place will organizations be primed and ready to drive value in the next era of GenAI use cases.

Is GenAI on your 2024 roadmap? Check out our tips to get GenAI-ready with RAG.

Do you want to be an anonymous informant about your GenAI efforts? Submit your hot takes here!

Curious how data observability can elevate your enterprise data for GenAI? Contact the Monte Carlo AI reliability team today!

Our promise: we will show you the product.