Building your own AI bot at work? It might be harder than you think

Building an AI bot fit for work isn’t as simple as it might seem. Here are 4 reasons why.

Jeremy Olson

Product Manager at Coda

Building your own AI bot at work? It might be harder than you think

By Jeremy Olson

Share this article

Share on TwitterShare on LinkedInShare on Facebook
AI · 6 min read
Is your team considering building its own internal AI bot? As demand for enterprise AI rises, many companies are weighing their options and deciding whether to “build vs. buy.” The desire to build internally usually comes from cost or control reasons (i.e., not having to rely on a vendor) or wanting to customize the AI to their specific needs. Since we started building our AI products, Coda AI and Coda Brain, we’ve spoken to many teams about their needs for enterprise AI. We see time and time again that custom building an internal AI bot is more complex than it might at first seem. In addition to the usual challenges of building software internally—like resources, speed of development, and maintenance overhead—building an AI chatbot comes with additional complexities.

1. So. many. data. sources.

 One of the biggest challenges is data. Enterprise AI is only effective if it can access your content and data (not just the general internet), and the more of it the better. That means it needs to be connected to the many different tools being used across your company—docs, wikis, task trackers, messaging apps, design tools, and so on—which, in turn, means having to build many integrations. Anyone who has ever worked on integrations will tell you that this is a significant undertaking. Every tool’s API is slightly different, the quality of documentation varies, and keeping up with API changes across dozens of integrations takes significant time.

2. With data, comes permissions.

 Connecting all your data sources also comes with another challenge: permissions. How do you make sure the AI doesn’t reveal data or content to users who don’t have permission to view it? In order to avoid this, it’s critical that your AI is permissions-aware. That means it knows what information and tools you already have permission to access and only uses those sources when formulating its response. But when you’re dealing with data from many different tools, permissions get complex, fast. Firstly, every product does permissions slightly differently in terms of what you can and can’t lock down. It can be as simple as at a project or doc level, down to as granular as rows and columns (looking at you, Salesforce!). Secondly, every product has differences in what access to permissions they support in their APIs. And lastly, there is a speed element. If you accidentally share a doc with someone and then un-share, how quickly does the AI update its permissions? All of this means there is no one-size-fits-all solution and ensuring the AI understands these complexities can be a huge amount of work.

3. Text alone won’t do.

 With consumer AI, we’re accustomed to receiving a text-based response to our questions. But in a work context, the questions we’re asking AI often require more structured responses to be truly valuable. For example, you might ask for a specific set of data, like a list of your highest-spend customers, or a view of your Jira issues. Or you might ask for a calculation, like analyzing sales performance. Enterprise AI will return this data as a structured table or cards rather than just a written list, so that you can view the associated data, verify its accuracy, and use it elsewhere (in dashboards, for example).
Designing an AI chatbot that can manage both structured and unstructured responses is critical, but each requires a different technical approach and a different way of storing data. When dealing with unstructured responses, the AI is often looking through information to find the most relevant answer, usually using RAG (retrieval augmented generation). When dealing with structured data, things get a little more complex. When you ask for a set of data from Coda Brain, for example, it analyzes which tool the data is in, which table is relevant, which filters to apply, and so on. Building these separate approaches is, of course, possible but is more technically challenging than a simple Q&A bot.

4. Going beyond answers to action.

 When using AI at work, the answer to a question is usually just the start. That’s why enterprise AI is most valuable when it goes beyond answering questions and actually takes action for you, saving you time and manual effort. If you build a standalone AI bot, it won’t be integrated into the place where your actual work happens, limiting its ability to take actions for you. For example, say you want to create a project brief for a new feature. You can ask Coda Brain to find relevant customer feedback, feature requests, and win/loss data, and then use those response as the basis for a project brief doc. Or you could ask for accounts over $10k at risk of churning and include that in your weekly sales meeting agenda. These types of actions wouldn’t be possible with a standalone bot without copy-pasting responses between tools.
A natively integrated AI is helpful in other ways, too. I use Coda AI every day to give me feedback on my docs so I can improve them. Because Coda AI is native to the Coda product, I can ask it to review the specific page I’m writing or even a certain section. And it can give feedback as targeted comments on the actual content—I don’t have to copy-paste it into another tool or worry about switching back and forth to review suggestions. These are benefits that you simply don’t get if your bot isn’t integrated into the tool where most of your work happens.

We built an enterprise AI so you don’t have to.

 Clearly, building and maintaining an enterprise AI bot isn’t a simple task. It requires building multiple integrations, managing the complexity of permissions, handling structured and unstructured responses, and more. But, luckily, you don’t need to! With Coda Brain, our turnkey AI platform, all these challenges are taken care of. Coda Brain understands your company data and lets anyone take action on it. It integrates with 500+ tools to provide reliable, real-time insights. Plus, it’s permissions-aware and can take action for you right where your work happens. Sound interesting? To see it for yourself, sign up for our private preview. Or, read more about Coda Brain.
Explore Coda Brain

Related posts

Explore more stories about AI.
Buy or build your next tool?

Consider these six critical questions when deciding to buy vs build internally.

What is RAG?

How RAG works and why it’s key to enterprise AI.

Enterprise AI vs consumer AI

Consumer AI tools don’t have context for your tasks at work.