Out of the box, Open AI offers a free moderation API endpoint that lets developers check whether the content complies with OpenAI’s content policy. 

With DataGovs, teams get out-of-the-box detection and validation of data quality against custom policies for specific industry use cases tied to data standards. 

Use Cases

• Track reported user issues 
• End-Users IDS 
• Verify inputs and outputs 

To obtain a classification for a piece of text, make a request to the moderation endpoint as demonstrated in the following code snippets:

curl https://api.openai.com/v1/moderations \  -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{"input": "Sample text goes here"}'

Below is an example output of the endpoint. It returns the following fields:

• flagged: Set to true if the model classifies the content as violating OpenAI’s content policy, false otherwise.
• categories: Contains a dictionary of per-category binary content policy violation flags. For each category, the value is true if the model flags the corresponding category as violated, false otherwise.
• category_scores: Contains a dictionary of per-category raw scores output by the model, denoting the model’s confidence that the input violates the OpenAI’s policy for the category. The value is between 0 and 1, where higher values denote higher confidence. The scores should not be interpreted as probabilities.

{  "id": "modr-XXXXX",  "model": "text-moderation-001",  "results": [  { "categories": {  "hate": false,  "hate/threatening": false,  "self-harm": false,  "sexual": false,  "sexual/minors": false,  "violence": false,  "violence/graphic": false  }, "category_scores": {  "hate": 0.18805529177188873,  "hate/threatening": 0.0001250059431185946,  "self-harm": 0.0003706029092427343,  "sexual": 0.0008735615410842001,  "sexual/minors": 0.0007470346172340214,  "violence": 0.0041268812492489815,  "violence/graphic": 0.00023186142789199948 }, "flagged": false  } ] }