Tag · 33 posts
#openai
Durable Execution for AI workflows
If you follow me on LinkedIn, you know I have talked about agentic workflows versus agents. Although everybody talks about agents, workflows are often better suited to many tasks. A workflow is more deterministic, is easier to reason about and to troubleshoot. Anthropic also talked about this a while ago in Building Effective Agents. In…
aiDeploying a multi-agent solution with MCP and A2A to Azure Container Apps
In previous posts, we discussed multi-agent scenarios, how A2A servers work (here and here) and how to deploy the infrastructure to host a multi-agent application on Azure with Azure Container Apps and AI Foundry. In this post, we will take a look at deploying the different components of the solution as containers in Azure Container…
aiDeploying AI Foundry Agents and Azure Container Apps to support an Agent2Agent solution
In previous posts, I discussed multi-agent solutions and the potential use of Google’s Agent2Agent protocol (A2A). In this post, we will deploy the infrastructure for an end-to-end solution like follows: Here’s a short description of the components. Component Description Foundry Project Basic Foundry project with a private endpoint. The private endpoint ensures private communication between…
aiUsing tasks with streaming in Google Agent2Agent (A2A)
In a previous post we created a simple A2A agent that uses synchronous message exchange. An A2A client sends a message and the A2A server, via the Agent Executor, responds with a message. But what if you have a longer running task to perform and you want to inform the client that the task in…
aiBuilding multi-agent solutions: what are your options?
When we meet with customers, the topic of a “multi-agent solution” often comes up. This isn’t surprising. There’s a lot of excitement around their potential to transform business processes, strengthen customer relationships, and more. The first question you have to ask yourself though is this: “Do I really need a multi-agent solution?”. Often, we find…
aiBuilding Configurable AI Agents with the OpenAI Agents SDK
In this post, I will demonstrate how to build an AI agent system where agents can collaborate in different ways. The goal is to create agents that can either work independently with their own tools or collaborate with other agents through two distinct patterns: using agents as tools or handing off control to other agents.…
aiCreating an agent with Hugging Face smolagents and Azure OpenAI
Artificial Intelligence (AI) agents have garnered significant attention, with numerous posts discussing them on platforms such as LinkedIn and X/Twitter. In that sense, this post is not different. Instead of theory though, let’s look at building an agent that has a reasoning loop in a very simple way. Although you can build an agent from…
aiUsing WebRTC with the OpenAI Realtime API
In October 2024, OpenAI introduced the Realtime API. It enables developers to integrate low-latency, multimodal conversational experiences into their applications. It supports both text and audio inputs and outputs, facilitating natural speech-to-speech interactions without the need for multiple models. It addresses the following problems: Simplified Integration: Combines speech recognition, language processing, and speech synthesis into…
aiUsing the Azure AI Inference Service
If you are a generative AI developer that works with different LLMs, it can be cumbersome to make sure your code works with your LLM of choice. You might start with Azure OpenAI models and the OpenAI APIs but later decide you want to use a Phi-3 model. What do you do in that case?…
aiCreate a Copilot declarative agent that calls an API with authentication
In a previous post, we looked at creating a Copilot declarative agent. The agent had one custom action that called the JSONPlaceholder API. Check that post for an introduction to what these agents can do. Using a dummy, unauthenticated API is not much fun so let’s take a look at doing the same for a…
aiWriting an multi-service document extractor with the help of Diagrid’s Catalyst
Many enterprises have systems in place that take documents, possibly handwritten, that contain data that needs to be extracted. In this post, we will create an application that can extract data from documents that you upload. We will make use of an LLM, in this case gpt-4o. We will use model version 2024-08-06 and its…
aiLoad balancing OpenAI API calls with LiteLLM
If you have ever created an application that makes calls to Azure OpenAI models, you know there are limits to the amount of calls you can make per minute. Take a look at the settings of a GPT model below: Above, the tokens per minute (TPM) rate limit is set to 60 000 tokens. This…
aiUse Azure OpenAI on your data with Semantic Kernel
I have written before about Azure OpenAI on your data. For a refresher, see Microsoft Learn. In short, Azure OpenAI on your data tries to make it easy to create an Azure AI Search index that supports advanced search mechanisms like vector search, potentially enhanced with semantic reranking. On of the things you can do…
aiSuper fast bot creation with Copilot Studio and the Azure OpenAI Assistants API
In a previous post, I discussed the Microsoft Bot Framework SDK that provides a fast track to deploying intelligent bots with the help of the Assistants API. Yet, the journey doesn’t stop there. Copilot Studio, a low-code tool, introduces an even more efficient approach, eliminating the need for intricate bot coding. It empowers developers to…
aiFast chat bot creation with the OpenAI Assistants API and the Microsoft Bot Framework SDK
This post is part of a series of blog posts about the Azure OpenAI Assistants API. Here are the previous posts: Part 1: introduction Part 2: using tools Part 3: retrieval In all of those posts, we demonstrated the abilities of the Azure OpenAI Assistants API in a Python notebook. In this post, we will…
aiRetrieval with the Azure OpenAI Assistants API
In two previous blog posts, I wrote an introduction to the Azure OpenAI Assistants API and how to work with custom functions. In this post, we will take a look at an assistant that can answer questions about documents. We will create an HR Assistant that has access to an HR policy document. In addition,…
aiUsing tools with the Azure OpenAI Assistants API
Introduction In a previous blog post, I wrote an introduction about the Azure OpenAI Assistants API. As an example, I created an assistant that had access to the Code Interpreter tool. You can find the code here. In this post, we will provide the assistant with custom tools. These custom tools use the function calling…
aiA look at the Azure OpenAI Assistants API
Introduction A while ago, I looked at the OpenAI Assistants API. In February of 2024, Microsoft have released their Assistants API in public preview. It works in the same way as the OpenAI Assistants API while being able to use it with Azure OpenAI models, deployed to a region of your choice. The goal of…
aiWriting your first flow with Prompt Flow in Visual Studio Code
In this blog post, we will create a flow with Prompt Flow in Visual Studio Code. Prompt Flow is a suite of development tools to build LLM-based AI applications. It tries to cover the end-to-end development cycle, including prototyping, testing and deployment to production. In Prompt Flow, you create flows. Flows link LLMs (large language…
aiFinding images with text and image queries with the help of GPT-4 Vision
With the gpt-4-vision-preview model available at OpenAI, it was time to build something with it. I decided to use it as part of a quick solution that can search for images with text, or by providing a similar image. We will do the following: Describe a collection of images. To generate the description, GPT-4 Vision…
aiCreating a custom GPT to query any knowledge base with actions
A while ago, OpenAI introduced GPTs. A GPT is a custom version of ChatGPT that combine instructions, extra knowledge, and any combination of skills. In this tutorial, we are going to create a custom GPT that can answer questions about articles on this blog. In order to achieve that, we will do the following: create…
aiTrying the OpenAI Assistants API
If you have ever tried to build an AI assistant, you know that is not a simple task. In almost all cases, your assistant needs access to external knowledge such as documents or APIs. You might even want to provide your assistant a code sandbox to solve user queries with code. When your assistant is…
aiUsing Azure Database for PostgreSQL as a vector store
When we build LLM applications, there is always a recurring question: “What vector store will we use?”. In Azure, there are several native solutions. Some of them were discussed in previous posts. Azure Cognitive Search: supports vector search but also hybrid search with semantic reranking as discussed here Azure Redis Cache Enterprise: check my blog…
aiUse Azure OpenAI Add your data vector search from code
In the previous post, we looked at using Azure OpenAI Add your data from the Azure OpenAI Chat Playground. It is an easy-to-follow wizard to add documents to a storage account and start asking questions about them. From the playground, you can deploy a chat app to an Azure web app and you are good…
aiImprovements in Azure OpenAI Add your data
In a previous post, I talked about the Add your data feature in the Azure OpenAI Chat playground. Recently, there have been some updates to this feature, including vector search. Let’s take a look at the updated experience and focus on vector search. Starting point I have some PDF documents in a storage account container.…
aiUse the Azure Add your data feature from code
In a previous post, we looked at Azure Add your data (preview) that can be configured from the Azure OpenAI Chat playground. In a couple of steps, you can point to or upload files of a supported file type (PDF, Word, …) and create an Azure Cognitive Search index. That index can subsequently be queried…
aiBuilding a chatbot in Azure that works with your data
When I talk to customers about Azure OpenAI, I am often asked how to build a chatbot that uses your own data in the simplest way possible while still allowing for some customization. In Azure, there are a few solutions. We will look at one of the solutions in this post. Note: don’t feel like…
aiSemantic Kernel Planner 101
Introduction If you are a developer who wants to build AI-first apps with natural language processing and large language models, you might be interested in Semantic Kernel (SK), a lightweight and open-source SDK that aims to simplify the integration of AI with conventional programming languages. SK is part of the CoPilot Stack and Microsoft is…
aiEnhancing Semantic Search with a Streamlit UI
In a previous blog post, we discussed two Python programs, upload_vectors.py and search_vectors.py. These programs were used to create and search vectors, respectively. The upload_vectors.py script created vectors from chunks of a larger text and stored them in Pinecone, while the search_vectors.py script enabled semantic search on the text. In this blog post, we will…
aiEnhancing Blog Post Search with Chunk-based Embeddings and Pinecone
In this blog post, we’ll show you a different approach to searching through a large database of blog posts. The previous approach involved creating a single embedding for the entire article and storing it in a vector database. The new approach is much more effective, and in this post, we’ll explain why and how to…
aiStoring and querying for embeddings with Redis
In a previous post, we wrote about using vectorized search and cosine similarity to quickly query a database of blog posts and retrieve the most relevant content to a natural language query. This is achieved using OpenAI’s embeddings API, Pinecone (a vector database), and OpenAI ChatCompletions. For reference, here’s the rough architecture: The steps above…
aiPinecone and OpenAI magic: A guide to finding your long lost blog posts with vectorized search and ChatGPT
Searching through a large database of blog posts can be a daunting task, especially if there are thousands of articles. However, using vectorized search and cosine similarity, you can quickly query your blog posts and retrieve the most relevant content. In this blog post, we’ll show you how to query a list of blog posts…
aiStep-by-Step Guide: How to Build Your Own Chatbot with the ChatGPT API
In this blog post, we will be discussing how to build your own chat bot using the ChatGPT API. It’s worth mentioning that we will be using the OpenAI APIs directly and not the Azure OpenAI APIs, and the code will be written in Python. A crucial aspect of creating a chat bot is maintaining…
ai