Toxicity Detection

This endpoint classifies text into one of several classes. It uses a few examples to create a classifier from a generative model. In the background, it constructs a few-shot classification prompt and uses it classify the input texts you pass to it.

The internet is dominated by user-generated content. While it provides an avenue for online platforms to grow, it is a bane for content moderators managing them. It is impossible for humans to manually moderate all the user content that is created. This is why an automated solution is needed, such as in flagging for toxic content.

Here we look at an example of classifying online user comments for toxicity by classifying them in Toxic or Not Toxic.