AI Hallucinations: Amusing at home, unwelcome at work

How we’ve built accuracy and confidence into Coda Brain.

Glenn Jaume

Product Manager at Coda

AI Hallucinations: Amusing at home, unwelcome at work

By Glenn Jaume

Share this article

Share on TwitterShare on LinkedInShare on Facebook
AI · 6 min read
When consumer AIs such as ChatGPT and Bing AI first entered the mainstream, the internet was awash with amazing examples of things they could do. However, there have also been a fair share of examples that aren’t quite so flattering. Many of these are instances of AI hallucinating, i.e., sharing false information not grounded in reality. These hallucinations can happen for a variety of reasons, including biases or lack of diversity in training data, “overfitting,” or an AI model’s inherent limitations in understanding context. Sometimes, these hallucinations are amusing or entertaining—think Google telling people to eat rocks or GPT-3 sharing that the Golden Gate Bridge was transported across Egypt in 2016. But while they can be easily dismissed as quirks of AI in the consumer world, this isn’t the case when it comes to enterprise AI.

Why hallucinations don’t cut it in enterprise AI.

 When using AI at work, the tolerance for inaccuracy is extremely low; it’s critical to have full confidence in the responses and to be able to verify their truthfulness. Without that trust, the benefits and efficiencies of AI are severely undermined. Imagine you’re putting together a presentation for your senior leadership, and you need to collect sales numbers for the past quarter. You ask whatever AI you’re using to gather this data for you. Right before the presentation—or, worse, during it—you realize that the AI has hallucinated a few sales deals that don’t actually exist. Or, even worse still, you already made decisions based on that false data. Now, you’ve lost all trust in the AI. Next time, you’ll hesitate to use it. Or, you’ll waste tons of time fact-checking its answers, which rather defeats the point of using AI in the first place. Clearly, this isn’t acceptable, and that’s why the bar for accuracy in enterprise AI is so high.

How we’ve built confidence into Coda Brain.

 Right from the start of building Coda Brain, our turnkey enterprise AI platform, we knew that trust and accuracy were a non-negotiable. There are four features we’ve built into Coda Brain to ensure the responses it gives are accurate and verifiable:
  1. Increasing relevancy with RAG.
  2. Showing our working with citations.
  3. Keeping the human-in-the-loop.
  4. Protecting security with permissions-awareness.
Let’s get into a little more detail about how each of these works to deliver an enterprise AI experience you can trust.

1. Increasing relevancy with RAG.

 RAG (Retrieval Augmented Generation) is an AI technique that’s rapidly gaining in popularity, especially in enterprise AI. RAG directs the AI to retrieve specific content or data first, then use those sources to answer the user’s question. Coda Brain uses RAG to pull relevant information from your docs and any tools you’ve connected to Coda before generating its own insights, responses, or actions based on these. For example, when you ask a question like “Should we build a desktop app?”, Coda Brain will collect relevant context like meeting notes, writeups, Jira tickets, Salesforce data, and more. Then, it will synthesize that information to generate a concise response to your question.
This not only increases accuracy and relevancy but also reduces the risk of hallucinations because the response is grounded in real, verifiable data. It also means Coda Brain’s responses can be much more personalized and context-aware—which is critical when it comes to the complexity and nuance of using AI in a work context.

2. Showing our working with citations.

 Speaking of real, verifiable data, the second approach we’ve taken with Coda Brain is citations. We’ve made Coda Brain’s responses as transparent as possible, so you know exactly where it has come from and can adjust if needed. Behind the scenes, when we ask for a response from the AI, we not only retrieve the right set of sources, but we also direct the AI to annotate where each part of the response comes from. These are then shown to you as citations. If you hover over the citation, it will show what was retrieved and from where. This means you can verify the results yourself and it helps prevent hallucinations.
In addition to showing these citations, Coda Brain delivers actual live data, too. For example, if you ask “Show me sales opportunities over $10k,” it can bring back a live table of Salesforce data rather than just a written list. This again makes it much easier to trust and verify that the data you’re seeing is real and accurate.

3. Keeping the human-in-the-loop.

 Further to our goal of transparency, we’ve built Coda Brain to automatically show you the steps it’s taking as it is working. For example, in our “$10k opportunities” scenario, you might see that it identified Salesforce as the source, then which table it chose, which filters it applied, and so on. You can then adjust this if needed—say if you want to refine further to just North American opportunities, or if you want to see data from a different tool instead. This approach—called “human-in-the-loop”—makes it easy to fine-tune the AI output to your specific needs. It also means you can be confident that what you’re seeing isn’t a hallucination, as you know exactly where the information came from.

4. Protecting security with permissions-awareness.

 When we first started exploring AI features, we were very focused on security outside of your company. But as we’ve gotten further into building enterprise AI, we’ve realized that the main challenges are actually inside your company. When your AI has access to your internal docs and data, how do you give users access to the information they need without revealing data they aren’t supposed to see? We’ve solved for this by making Coda Brain permissions-aware; it knows what docs and data you already have access to and only pulls from those sources. That way, it doesn’t accidentally reveal confidential information to an unintended audience. Think about a question like “Who is being promoted this cycle?” Likely, the answer to this lives in a doc that only some people have access to. Being permissions-aware means Coda Brain won’t share that information with anyone else, even if they ask the same question. This permissions-awareness is critical for enterprise AI because it means you can feel confident giving AI access to your docs and data in order to deliver you relevant answers.

Is your AI up to the task?

 In conclusion, while hallucinations might be excellent fodder for amusing social posts, they can have serious implications in the world of work. That’s why it’s crucial to use a platform specifically designed for business when using AI at work. Unlike consumer AIs, enterprise AIs like Coda Brain can access your context and data, not just the internet. This allows them to provide more relevant and personalized answers, reducing the risk of hallucinations, and enabling you to verify accuracy. You can read more about how enterprise AI differs from consumer AI here. Or, if you’re interested in experiencing the power of Coda Brain first-hand, sign up for our private preview.
Join the private preview

Related posts

Explore more stories about AI.
What is RAG?

How RAG works and why it’s key to enterprise AI.

Enterprise AI can do more

With access to your team’s data, enterprise AI becomes a powerful tool.

Enterprise AI vs consumer AI

Consumer AI tools don’t have context for your tasks at work.