# Detecting emotions with FER+

- URL: https://baeke.info/2019/01/06/detecting-emotions-with-fer/
- Author: Geert Baeke
- Published: January 6, 2019 (updated March 12, 2025)
- Topics: ai, analytics, api, azure, containers, ai, azure, azuremachinelearning, emotion, fer, golang, machinelearning, mi, onnx, webcam
- Outdated: Azure Notebooks and the Azure ML SDK v1 image API used here are retired.

In an [earlier post](https://baeke.info/2019/01/03/recognizing-images-with-azure-machine-learning-and-the-onnx-resnet50v2-model/), I discussed classifying images with the ResNet50v2 model. [Azure Machine Learning Service](https://azure.microsoft.com/en-us/services/machine-learning-service/) was used to create a container image that used the [ONNX ResNet50v2 model](https://github.com/onnx/models/tree/master/models/image_classification/resnet) and the [ONNX Runtime](https://github.com/Microsoft/onnxruntime) for scoring.

Continuing on that theme, I created a [container image](https://hub.docker.com/r/gbaeke/onnxferplus) that uses the [ONNX FER+](https://github.com/onnx/models/tree/master/emotion_ferplus) model that can detect emotions in an image. The container image also uses the ONNX Runtime for scoring.

You might wonder why you would want to detect emotions this way when there are many services available that can do this for you with a simple API call! You could use Microsoft’s [Face API](https://azure.microsoft.com/en-us/services/cognitive-services/face/) or Amazon’s [Rekognition](https://aws.amazon.com/rekognition/) for example. While those services are easy to use and provide additional features, they do come at a cost. If all you need is basic detection of emotions, using this FER+ container is sufficient and cost effective.

![](https://baeke.info/_astro/2019-01-image.Bw-h1HUa_2kf8a7.webp)

*Azure Face API (image from Microsoft website)*

A notebook to create the image and deploy a container to Azure Container Instances (ACI) can be found [here](https://notebooks.azure.com/geba/projects/gebaml/html/onnxferplus.ipynb). The notebook uses the [Azure Machine Learning SDK](https://docs.microsoft.com/en-us/python/api/overview/azure/ml/intro?view=azure-ml-py) to register the model to an Azure Machine Learning workspace, build a container image from that model and deploy the container to ACI. The scoring script score.py is shown below.

![](https://baeke.info/_astro/2019-01-scoreferplus.BgIP7Gim_1IXJoQ.webp)

*score.py*

The model expects an 64×64 gray scale image of a face in an array with the following dimensions: \[1\]\[1\]\[64\]\[64\]. The output is JSON with a results array that contains the probabilities for each emotion and a time field with the inference time.

The emotion probabilities are in this order:

```
0: "neutral", 1: "happy", 2: "surprise", 3: "sadness", 4: "anger", 5: "disgust", 6: "fear", 7: "contempt
```

To actually capture the emotions, I wrote a small demo program in Go that uses [OpenCV](https://opencv.org/) (via [GoCV](https://gocv.io/)). You can find it on GitHub: [https://github.com/gbaeke/emotion](https://github.com/gbaeke/emotion). You will need to install OpenCV and GoCV. Find the instructions here: [https://gocv.io/getting-started/linux/](https://gocv.io/getting-started/linux/). There are similar instructions for Mac and Windows but I have not tried those

The program is still a little rough around the edges but it does the trick. The scoring URI is hard coded to [http://localhost:5002/score](http://localhost:5002/score). With Docker installed, use the following command to install the scoring container:

```bash
 docker run -d -p 5002:5001 gbaeke/onnxferplus
```

Have fun with it!