Moderate comments on your website using Azure Content Moderator

21 Mar 2021 by Kolappan N

In this post we are going to see in detail about how Azure Content Moderator can be used to moderate comments on your website/blog. For those who prefer to view code instead of reading through blog post, here is the GitHub Link to my article.

First, what is Azure Content Moderator? According to Microsoft,

The Azure Content Moderator API is a cognitive service that checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable.

In simple terms, it is a cognitive service that can be used to validate and moderate various user generated contents. In this post we focus on moderating comments aka text.

Where is it useful?

Blocking harmful comments with just the help of a blocklisted words might allow some content to slip through it. This service allows you detect and prevent them and also do additional things like ensuring that Personal info is not posted in public forums or comments.

Setting up things on Azure

We will start by setting up things on Azure first and then proceed to the coding.

  1. It’s pretty obvious by now that you will need an Azure account.
  2. Login into your Azure Portal and create a Content Moderation service.
  3. Once the resource is created, go to the resource.
  4. Navigate to the Keys and Endpoint settings.
  5. You will see two key values here. Any one value can be used in place of another. These redundant keys are here to make key regeneration easier. Note down anyone key and the endpoint. We will need it later.
Screenshot of Key & Endpoint Settings
Key & Endpoint Settings for Azure Content Moderator
Setting up code

For the purposes of this demo, I will be using a C# console application. The application will read my input, analyze it and take corresponding action. You can easily modify this code for rest API or any other format that fits you best.

  1. My first step will be to create a Console application.
  2. Install the Nuget package Microsoft.Azure.CognitiveServices.ContentModerator which can be found here.
Authenticating with Azure service
  1. I am creating a new ModerationLib class to contain all the corresponding functions.
  2. Create two variable to hold the key and endpoint we noted down earlier.
    private static readonly string SubscriptionKey = "EXAMPLE_KEY";
    private static readonly string Endpoint = "https://example.cognitiveservices.azure.com/";
    
  3. Add the following references to the ModerationLib class.
    using Microsoft.Azure.CognitiveServices.ContentModerator;
    using Microsoft.Azure.CognitiveServices.ContentModerator.Models;
    
  4. Create a function that creates a client that can authenticate into the Azure service.
    private ContentModeratorClient Authenticate()
    {
    var client = new ContentModeratorClient(new ApiKeyServiceClientCredentials(SubscriptionKey))
    {
        Endpoint = Endpoint
    };
    return client;
    }
    
Calling the service to Analyze the text

The function we need to call to analyze the text via Azure is ContentModeratorClient.TextModeration.ScreenText. It requires the text to be passed as stream. We also need to use a ContentModeratorClient that has key and endpoint details i.e) the one we created in the previous step. Here is a simple wrapper function I use for Calling the Azure service,

private Screen CallAzureModerator(ContentModeratorClient client, string text)
{
    var textBytes = Encoding.UTF8.GetBytes(text);
    using var stream = new MemoryStream(textBytes);
    return client.TextModeration.ScreenText("text/plain", stream, language: "eng", autocorrect: false, pII: true, listId:null, true);
}

Notice the parameters I pass to the ScreenText function.

Processing the response

Here is a JSONized response I got for a sample text,

{
    "OriginalText": "Hello My email is 123@sample.com",
    "NormalizedText": "Hello  email  123@sample.com",
    "AutoCorrectedText": null,
    "Misrepresentation": null,
    "Classification": {
        "Category1": {
            "Score": 0.0016730600036680698
        },
        "Category2": {
            "Score": 0.20461790263652802
        },
        "Category3": {
            "Score": 0.0669788047671318
        },
        "ReviewRecommended": false
    },
    "Status": {
        "Code": 3000,
        "Description": "OK",
        "Exception": null
    },
    "PII": {
        "Email": [
            {
                "Detected": "123@sample.com",
                "SubType": "Regular",
                "Text": "123@sample.com",
                "Index": 18
            }
        ],
        "SSN": [],
        "IPA": [],
        "Phone": [],
        "Address": []
    },
    "Language": "eng",
    "Terms": null,
    "TrackingId": "578522d7-9e53-418b-abff-985c186e7377"
}

This provides us with a lot of information. For example, the index at which every PII starts is given to you. You can use it react the personal info and then process the comment. Here is how I intend process the info,

public void ModerateText(string text)
{
    var azClient = Authenticate();
    var azResult = CallAzureModerator(azClient, text);

    // TODO:
    // Check if the text contains a blocklisted word using List

    if (azResult.PII != null && (azResult.PII.Email.Count > 0 || azResult.PII.Phone.Count > 0))
    {
        // Don't post the comment and inform the user
        Console.WriteLine("Contains Personal Data");
    }
    else if (azResult.Classification.ReviewRecommended == true)
    {
        
        if (azResult.Classification.Category1.Score > 0.9)
        {
            // Automatically reject comments with greater confidence in inappropriate content.
            // Category 1 refers to potential presence of language that may be considered sexually explicit or adult in certain situations
            // There are a total of 3 categories.
            // Ref: https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/text-moderation-api#classification
            Console.WriteLine("Rejected the comment");
        }
        else
        {
            // Send the comment to your manual review system
            Console.WriteLine("Comment held for review");
        }
    }
    else
    {
        Console.WriteLine("Comment Published");
    }
}

You can even combine them with your existing custom workflows such as finding and removing links, etc…

What’s next

These blog posts are WIP. Links to these blog posts will be updated once they are published.

  1. Creating & Managing custom blocklists in the Azure Content Moderator service
  2. Using the review system within Azure for Manual reviews
Tags: Azure Azure-Content-Moderator Azure-Cognitive-Services .NET View Blog Archive