ChatKit Azure OpenAI: A developer’s guide to building custom AI chat

If you’ve been keeping an eye on the AI space, you know the conversation has moved beyond simple Q&A bots. The real excitement is around AI agents that can actually do things. OpenAI recently dropped some tools for developers, like AgentKit and ChatKit, that promise to make building these sophisticated agents much easier. But what are they, really? And what does it actually take to use them?

This guide is here to cut through the hype around ChatKit Azure OpenAI. We’ll break down what it is, how it works, and what you need to consider before diving in. We’ll also clear up some confusion between OpenAI's developer toolkit and other tools with similar names, so you can make a smart "build vs. buy" decision for your company.

What is ChatKit Azure OpenAI?

First things first, let's clear up what we're actually talking about. This isn't a single, off-the-shelf product. It’s two different technologies that you bring together: a toolkit for the front-end chat window and an AI service for the back-end brainpower.

Understanding OpenAI's ChatKit

OpenAI's ChatKit is a software development kit (SDK) for developers who want to build their own chat user interfaces. It’s important to know that this is not a ready-to-use app like the similarly named "chatkit.app".

Think of it as a box of Lego bricks for building the part of the chat app your users see and interact with. It gives you pre-built UI widgets and a framework to connect to a server, which helps handle things like streaming responses and showing interactive buttons. It provides the foundation to create that slick, ChatGPT-like experience right inside your own product.

Understanding Azure OpenAI

Azure OpenAI is Microsoft's cloud platform that gives you access to OpenAI's powerful large language models, like the GPT-4 series. The big deal here is that it comes with all the enterprise-level perks you'd expect from Microsoft: strong security, data privacy, and reliability, all neatly tucked into the Azure ecosystem.

When you build an app with ChatKit, Azure OpenAI is usually the engine running in the background. It’s the "brain" that takes a user's question, thinks about it, and generates the smart response that your ChatKit UI displays.

How to set up a basic ChatKit Azure OpenAI integration

Alright, let's get into the nitty-gritty. Connecting ChatKit with Azure OpenAI is definitely a job for your development team. While these tools make some parts of the process easier, it still involves writing code, setting up servers, and managing infrastructure. Here’s a high-level look at what’s involved.

Key requirements

To get a basic version up and running, a developer is going to need a few things:

• An Azure account: You'll need an active subscription with access to the Azure OpenAI service.

• A deployed Azure OpenAI model: This involves picking a model, like "gpt-4-turbo", and deploying it. This gives you an API endpoint and a key so your application can talk to the AI.

• A custom backend server: ChatKit doesn't talk directly to Azure OpenAI. It needs a server in the middle to manage the conversation. The official documentation has examples using Python with FastAPI, but you can use any web framework your team is comfortable with.

• A way to store data: Want to remember past conversations? You'll have to set up and connect your own database to save threads and messages. This doesn't come included.

While building a custom UI with ChatKit gives you ultimate control, it’s a big commitment in terms of developer hours. For teams that want a powerful AI agent without the engineering headache, no-code platforms like eesel AI offer a much faster path. You can connect it directly to your helpdesk and knowledge bases, and go live in minutes without writing a single line of code.

The core workflow

So, what does the conversation flow look like once you have everything set up? It's a neat, multi-step dance managed by that custom server you built.

1. User sends a message: Someone types a question into the custom chat window you built with ChatKit.

2. Request hits your server: That message travels to your backend server.

3. Server calls Azure OpenAI: Your server then securely calls the Azure OpenAI API, passing along the user's message and any relevant conversation history.

4. Azure OpenAI processes the request: The AI model does its thing and generates a response.

5. Response is streamed back: The answer is sent back to your server, which then streams it to the ChatKit UI. This is what makes the text appear word-by-word, just like in ChatGPT.

This back-and-forth is what creates that smooth, interactive feeling. But remember, your team is responsible for building and maintaining every single step of that process, from keeping API keys safe to managing the conversation history in a database.

Features and limitations

Going the custom route with ChatKit gives you a ton of freedom, but it's not all sunshine and rainbows. It’s worth weighing the pros and cons before committing your team’s time and resources.

Key features and benefits

• Total Customization: You have complete control over the look, feel, and functionality. You can design custom UI elements and perfectly match the entire experience to your brand.

• Real-time Streaming: ChatKit is built for streaming. It lets you send responses back token-by-token, which makes the conversation feel much more dynamic and alive.

• Framework Agnostic: Your developers aren't locked into one specific technology. The server can be built with whatever backend framework they prefer, whether it's FastAPI, Express, or Ruby on Rails.

• Integration with Agents SDK: It's designed to play nicely with OpenAI's other tools, like the Agents SDK, which is handy if you plan on building more complex agents that can use tools.

Practical limitations and challenges

• High Development Overhead: This is the big one. This isn't a plug-and-play solution. You need skilled developers to build the server, database, attachment storage, and front-end. And then you need to maintain it all.

• No Out-of-the-Box Features: Basic features you might take for granted, like conversation history, user login, and analytics, aren't included. You have to build them all from scratch.

• Complex Knowledge Integration: To make your bot truly useful, it needs to know about your business. Connecting it to internal knowledge sources like Confluence pages, Google Docs, or old support tickets means writing custom code for every single source.

This is where a lot of projects get bogged down. Building custom connectors for every app your company uses is a huge time sink. In contrast, solutions like eesel AI are designed to solve this exact problem. With over 100 one-click integrations for platforms like Confluence, Google Docs, and Zendesk, it can learn from all your scattered information instantly, providing more accurate answers from day one.

This infographic shows how eesel AI simplifies knowledge integration, a challenge with custom ChatKit Azure OpenAI setups.

Understanding the pricing

The ChatKit toolkit itself is free, but the AI brain it connects to, Azure OpenAI, definitely isn't. Getting a handle on its consumption-based pricing is key to avoiding any nasty surprises on your monthly bill.

How pricing works

Azure OpenAI's pricing is mostly based on "tokens," which are just small chunks of text. You get charged for the tokens you send to the model (the input) and for the tokens the model sends back (the output).

There are two main ways to pay:

• Standard (Pay-as-you-go): You pay a set rate per 1,000 or 1 million tokens. This is great if your usage goes up and down.

• Provisioned Throughput Units (PTUs): You pay a fixed monthly or annual fee to reserve a certain amount of processing power. This is a better fit for businesses with high, steady traffic.

The model you choose also makes a big difference. The newer, more powerful GPT-4 models cost more per token than the smaller, faster ones.

Example pricing for popular models

To give you a rough idea, here's a quick look at the standard pay-as-you-go pricing for a few popular models on Azure OpenAI. Just remember, these prices are only for the AI service. They don't include the costs of paying your developers or hosting and maintaining your custom app.

Model	Input Price (per 1M tokens)	Output Price (per 1M tokens)
GPT-4o	$5.00	$15.00
GPT-4o mini	$0.15	$0.60
GPT-4.1-mini	$0.40	$1.60

Pricing based on Azure OpenAI "Global" deployment as of late 2024. Always check the official Azure OpenAI pricing page for the most current information.

Trying to budget with unpredictable, token-based costs can be a real headache. This is why many teams prefer the transparent and predictable pricing of platforms like eesel AI. With plans based on a set number of interactions and no hidden fees, you can scale your support automation without sweating the end-of-month invoice.

This screenshot shows the predictable pricing model of eesel AI, an alternative to the complex token-based pricing of a custom ChatKit Azure OpenAI solution.

Is ChatKit Azure OpenAI right for you?

So, what's the verdict on ChatKit Azure OpenAI? It's a powerful combination for teams that want to build a completely custom AI chat experience from scratch. If you have the engineering resources and need total control over every pixel and every line of code, it’s a solid choice.

However, that "build" approach comes with a heavy price in development time, ongoing maintenance, and unpredictable costs. For most businesses, the goal is to use AI to solve a problem, not to become a full-time chat application development shop.

If you want the power of a custom-trained AI agent without the months of development work, a platform like eesel AI gets you there in minutes. You can unify all your company knowledge, automate support, and get helpful insights from a single, easy-to-use dashboard. Start a free trial today and see what's possible.