Category · 41 posts
docker
Building an AI Agent Server with AG-UI and Microsoft Agent Framework
In this post, I want to talk about the Python backend I built for an AG-UI demo project. It is part of a larger project that also includes a frontend that uses CopilotKit: This post discusses the Python AG-UI server that is built with Microsoft Agent Framework. All code is on GitHub: https://github.com/gbaeke/agui. Most of…
aiLoad balancing OpenAI API calls with LiteLLM
If you have ever created an application that makes calls to Azure OpenAI models, you know there are limits to the amount of calls you can make per minute. Take a look at the settings of a GPT model below: Above, the tokens per minute (TPM) rate limit is set to 60 000 tokens. This…
aiGiving Microsoft’s Radius a spin
Microsoft recently announced Radius. As stated in their inaugural blog post, it is “a tool to describe, deploy, and manage your entire application”. With Radius, you describe your application in a bicep file. This can include containers, databases, the connections between those and much more. Radius is an open-source solution started from within Microsoft. The…
azureEnhancing Semantic Search with a Streamlit UI
In a previous blog post, we discussed two Python programs, upload_vectors.py and search_vectors.py. These programs were used to create and search vectors, respectively. The upload_vectors.py script created vectors from chunks of a larger text and stored them in Pinecone, while the search_vectors.py script enabled semantic search on the text. In this blog post, we will…
aiDraft 2 and Ingress with Web Application Routing
If you read the previous article on Draft 2, we went from source code to deployed application in a few steps: az aks draft create: creates a Dockerfile and Kubernetes manifests (deployment and service manifests) az aks draft setup-gh: setup GitHub OIDC az aks draft generate-workflow: create a GitHub workflow that builds and pushes the…
azureQuick Guide to Azure Container Apps
Now that Azure Container Apps (ACA) is generally available, it is time for a quick guide. These quick guides illustrate how to work with a service from the command line and illustrate the main features. Prerequisites All commands are run from bash in WSL 2 (Windows Subsystem for Linux 2 on Windows 11) Azure CLI…
azureQuick Guide to the Secret Store CSI driver for Azure Key Vault on AKS
Yesterday, I posted the Quick Guide to Kubernetes Workload Identity on AKS. This post contains a similar guide to enabling and using the Secret Store CSI driver for Azure Key Vault on AKS. All commands assume bash. You should have the Azure CLI installed and logged in to the subscription as the owner (because you…
azureQuick Guide to Kubernetes Workload Identity on AKS
IMPORTANT: the steps below are not relevant anymore; the steps in the quick guide have been updated; see https://github.com/gbaeke/quick-guides/blob/main/workload-identity/README.md for the correct steps. Some things that have changed: you can now use managed identities instead of app registrations; federated token configuration is at the managed identity level you do not need to install the webhook…
azureTaking Azure Container Apps for a spin
At Ignite November 2021, Microsoft released Azure Container Apps as a public preview. It allows you to run containerized applications on a serverless platform, in the sense that you do not have to worry about the underlying infrastructure. The underlying infrastructure is Kubernetes (AKS) as the control plane with additional software such as: Dapr: distributed…
azureKubernetes Blue-Green deployments with Argo Rollouts
In this post, we will take a look at 🟦/🟩 blue-green deployments in Kubernetes. With blue-green deployments, you deploy a new version of an application or service next to the live and stable version. After manual or automatic checks, you promote the new version to become the live version. Switching between versions is simply a…
cloudBuilding a GitHub Action with Docker
While I was investigating Kyverno, I wanted to check my Kubernetes deployments for compliance with Kyverno policies. The Kyverno CLI can be used to do that with the following command: kyverno apply ./policies –resource=./deploy/deployment.yaml To do this easily from a GitHub workflow, I created an action called gbaeke/kyverno-cli. The action uses a Docker container. It…
automationAKS Pod Identity with the Azure SDK for Go
In an earlier post, I wrote about the use of AKS Pod Identity (Preview) in combination with the Azure SDK for Python. Although that works fine, there are some issues with that solution: the container image is around 1GB, which is quite large (it is based on tiangolo/uvicorn-gunicorn-fastapi:python3.7) as expected, the image contains many vulnerabilities…
azureAzure Key Vault Provider for Secrets Store CSI Driver
In the previous post, I talked about akv2k8s. akv2k8s is a Kubernetes controller that synchronizes secrets and certificates from Key Vault. Besides synchronizing to a regular secret, it can also inject secrets into pods. Instead of akv2k8s, you can also use the secrets store CSI driver with the Azure Key Vault provider. As a CSI…
automationCertificates with Azure Key Vault and Nginx Ingress Controller
Let’s face it. If you deploy web applications and APIs of any sort, you need certificates. If you have been long enough in IT, there’s just no escape! In this article, we will take a look at getting a certificate from Azure Key Vault to Azure Kubernetes service. Next, we will use that certificate with…
containersDeploy and bootstrap your Kubernetes cluster with Azure DevOps and GitOps
A while ago, I published a post about deploying AKS with Azure DevOps with extras like Nginx Ingress, cert-manager and several others. An Azure Resource Manager (ARM) template is used to deploy Azure Kubernetes Service (AKS). The extras are installed with Helm charts and Helm installer tasks. I mainly use it for demo purposes but…
automationAn introduction to Flux v2
If you have read my blog and watched my Youtube channel, you know I have worked with Flux in the past. Flux, by weaveworks, is a GitOps Kubernetes Operator that ensures that your cluster state matches the desired state described in a git repository. There are other solutions as well, such as Argo CD. With…
automationDocker without Docker: a look at Podman
I have been working with Docker for quite some time. More and more however, I see people switching to tools like Podman and Buildah and decided to give that a go. I installed a virtual machine in Azure with the following Azure CLI command: Just replace RESOURCEGROUP, VMNAME and PASSWORD with the values you want…
azureFrom MQTT to InfluxDB with Dapr
In a previous post, we looked at using the Dapr InfluxDB component to write data to InfluxDB Cloud. In this post, we will take a look at reading data from an MQTT topic and storing it in InfluxDB. We will use Dapr 0.10, which includes both components. To get up to speed with Dapr, please…
cloudDapr Service Invocation between an HTTP Python client and a GRPC Go server
Recently, I published several videos about Dapr on my Youtube channel. The videos cover the basics of state management, PubSub and service invocation. The Getting Started with state management and service invocation: Let’s take a closer look at service invocation with HTTP, Python and Node. Service Invocation with HTTP The services you write (here Service…
azureGetting started with Consul on Kubernetes
Although I have heard a lot about Hashicorp’s Consul, I have not had the opportunity to work with it and get acquainted with the basics. In this post, I will share some of the basics I have learned, hopefully giving you a bit of a head start when you embark on this journey yourself. Want…
azureProgressive Delivery on Kubernetes: what are your options?
If you have ever deployed an application to Kubernetes, even a simple one, you are probably familiar with deployments. A deployment describes the pods to run, how many of them to run and how they should be upgraded. That last point is especially important because the strategy you select has an impact on the availability…
cloudWriting a Kubernetes operator with Kopf
In today’s post, we will write a simple operator with Kopf, which is a Python framework created by Zalando. A Kubernetes operator is a piece of software, running in Kubernetes, that does something application specific. To see some examples of what operators are used for, check out operatorhub.io. Our operator will do something simple in…
apiGiving Argo CD a spin
If you have followed my blog a little, you have seen a few posts about GitOps with Flux CD. This time, I am taking a look at Argo CD which, like Flux CD, is a GitOps tool to deploy applications from manifests in a git repository. Don’t want to read this whole thing? There are…
automationDeploy AKS with Nginx, External DNS, Helm Operator and Flux
A while ago, I blogged about an Azure YAML pipeline to deploy AKS together with Traefik. As a variation on that theme, this post talks about deploying AKS together with Nginx, External DNS, a Helm Operator and Flux CD. I blogged about Flux before if you want to know what it does. I added the…
azureGitOps with Weaveworks Flux – Installing and Updating Applications
In a previous post, we installed Weaveworks Flux. Flux synchronizes the contents of a git repository with your Kubernetes cluster. Flux can easily be installed via a Helm chart. As an example, we installed Traefik by adding the following yaml to the synced repository: apiVersion: helm.fluxcd.io/v1 kind: HelmRelease metadata: name: traefik namespace: default annotations: fluxcd.io/ignore:…
containersThe basics of meshing Traefik 2.0 with Linkerd
A while ago, I blogged about Linkerd 2.x. In that post, I used a simple calculator API, reachable via an Azure Load Balancer. When you look at that traffic in Linkerd, you see the following: Above, you do not see this is Azure Load Balancer traffic. The traffic reaches the meshed service via the Azure…
apiGiving linkerd a spin
A while ago, I gave linkerd a spin. Due to vacations and a busy schedule, I was not able to write about my experience. I will briefly discuss how to setup linkerd and then deploy a sample service to illustrate what it can do out of the box. Let’s go! Wait! What is linkerd? linkerd…
apiSecuring your API with Kong and CloudFlare
In the previous post, we looked at API Management with Kong and the Kong Ingress Controller. We did not care about security and exposed a sample toy API over a public HTTP endpoint that also required an API key. All in the clear, no firewall, no WAF, nothing… 👎👎👎 In this post, we will expose…
apiQuick overview of Traefik Ingress Controller Installation
This post is mainly a note to self 📝📝📝 that describes a quick way to deploy a Kubernetes Ingress Controller with Traefik. There is also a video version: We will install Traefik with Helm and I assume the cluster has rbac enabled. If you deploy clusters with AKS, that is the default although you can…
containersCloud Run on Google Kubernetes Engine
In this short post, we will take a look at Cloud Run on Google Kubernetes Engine (GKE). To get this to work, you will need to deploy a Kubernetes cluster. Make sure you use nodes with at least 2 vCPUs and 7.5 GB of memory. Take a look here for more details. You will notice…
containersCreating and containerizing a TensorFlow Go application
In an earlier post, I discussed using a TensorFlow model from a Go application. With the TensorFlow bindings for Go, you can load a model that was exported with TensorFlow’s SavedModelBuilder module. That module saves a “snapshot” of a trained model which can be used for inference. In this post, we will actually use the…
aiInfrastructure as Code: exploring Pulumi
Image: from the Pulumi website In my Twitter feed, I often come across Pulumi so I decided to try it out. Pulumi is an Infrastructure as Code solution that allows you to use familiar development languages such as JavaScript, Python and Go. The idea is that you define your infrastructure in the language that you…
automationResNet50v2 classification in Go with a local container
To quickly go to the code, go here. Otherwise, keep reading… In a previous blog post, I wrote about classifying images with the ResNet50v2 model from the ONNX Model Zoo. In that post, the container ran on a Kubernetes cluster with GPU nodes. The nodes had an NVIDIA v100 GPU. The actual classification was done…
aiRecognizing images with Azure Machine Learning and the ONNX ResNet50v2 model
Featured image from: https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852 In a previous post, I discussed the creation of a container image that uses the ResNet50v2 model for image classification. If you want to perform tasks such as localization or segmentation, there are other models that serve that purpose. The image was built with GPU support. Adding GPU support was pretty…
aiCreating a GPU container image for scoring with Azure Machine Learning
In a previous post, I discussed how you can add an existing Kubernetes cluster to an Azure Machine Learning workspace. Adding an existing cluster is necessary when the workspace does not support auto creation of a cluster. That is the case when you want to use the Standard_NC6s_v3 virtual machine image. I also used a…
aiAttaching Kubernetes clusters with NVIDIA V100 GPUs to Azure Machine Learning Service
Azure Machine Learning Service allows you to easily deploy compute for training and inference via a machine learning workspace. Although one of the compute types is Kubernetes, the workspace is a bit picky about the node VM sizes. I wanted to use two Standard_NC6s_v3 instances with NVIDIA Tesla V100 GPUs but that was not allowed.…
aiDeploying Azure Cognitive Services Containers with IoT Edge
Introduction Azure Cognitive Services is a collection of APIs that make your applications smarter. Some of those APIs are listed below: Vision: image classification, face detection (including emotions), OCR Language: text analytics (e.g. key phrase or sentiment analysis), language detection and translation To use one of the APIs you need to provision it in an…
analyticsDraft: a simpler way to deploy to Kubernetes during development
If you work with containers and work with Kubernetes, Draft makes it easier to deploy your code while you are in the earlier development stages. You use Draft while you are working on your code but before you commit it to version control. The idea is simple: You have some code written in something like…
apiCommunication between microservices in Kubernetes with Go Micro
In this post, we continue the story we started with two earlier posts: Getting started with Kubernetes on Azure Microservices on Kubernetes: a simple example in Go In the previous post, I described a very simple service written with the help of Go Micro. It exposes an RPC call Get that retrieves a device from a…
apiMicroservices on Kubernetes: a simple example in Go
In the previous post, Getting started with Kubernetes on Azure, we talked about creating a Kubernetes cluster and deploying a couple of services. There are basically two services: Data: a service that exposes an endpoint to pick up data for an IoT device; you call it with http://service_endpoint:8080/data/devicename Device: a service that can be used…
apiControl Sonos with an easy to use API
In an earlier post, Controlling Sonos from a Particle Photon, we created a small app to do just that. The app itself contained some C++ code to interact with a Sonos player on your network. Although the code works, it does not provide you with full control over your Sonos player and it’s tedious to…
api