Copilot in Your Apps over Rectangular (SQL) Data

by Markus Egger | First published: June 21, 2023 - Last Updated: November 27, 2023

I recently showed a pretty cool demo in a State of .NET event showing how to use a custom Copilot in a custom application over data from SQL Server. It's a fun demo, because it shows a few things that are generally considered limitations of Large Language Models (LLMs). In particular, this demo includes a Copilot (which uses Azure OpenAI) which searches and reasons over data in dozens of tables.

Here is a video starting right at the point of the demo:

What's very cool about this demo is that a custom Copilot, through the use of a Large Language Model (LLM) based on Azure OpenAI (which in turn uses the OpenAI models, such as GTP 3.5 and GPT4), drives a custom application, reasoning and searching over data stored across dozens of SQL Server tables. This goes way beyond the usual examples you will see of having ChatGPT answer pre-trained questions, or fine-tuning models or indexing a bunch of PDF documents, to answer more specific questions. This incorporates a vast set of data, with all application security in play, in real-time. And here we're not talking about just a few tables or databases; we're talking about an AI navigating and reasoning over data from dozens of SQL Server tables—a feat once considered as a limitation for the LLMs.

This is also more than just a demo. It is something we used in our production application today. Oh, and it did it in the voice of John Cleese. Everything is better, when presented in the voice of John Cleese 😃