Moderation Glossary

Plain-language definitions of content moderation and AI terms

Content ModerationThe process of reviewing user-generated content and removing or acting on material that breaks a platform's rules or the law.Content Moderation APIA moderation API focused on text content like messages, comments, and posts, returning an instant safety decision for each one.DoxxingPublishing someone's private personal information without consent, usually to intimidate or enable harassment.False NegativeA case where a moderation system misses content that was actually harmful.False PositiveA case where a moderation system flags content as harmful when it is actually fine.HarassmentRepeated or targeted hostile behavior aimed at a specific person to intimidate, demean, or drive them away.Hate SpeechContent that attacks or demeans people based on protected characteristics such as race, religion, gender, or sexual orientation.Human in the LoopA design where people review or approve some automated decisions rather than letting the system act entirely on its own.Image Moderation APIA moderation API that analyzes pictures and returns an instant decision about whether an image contains unsafe content.Implicit ModerationCatching harmful intent that is implied rather than stated, such as coded language, insinuation, and indirect threats.Intent-Based ModerationJudging the meaning and purpose behind a message rather than matching keywords or blocklists.Large Language Model (LLM)An AI model trained on vast amounts of text that can understand and generate human language.Moderation APIA service that lets any app automatically check content for harmful material and get back an instant decision.Moderation LabelsThe categories a moderation system assigns to content, such as harassment, hate, or spam, describing what kind of harm was detected.Multimodal AIAI that can understand more than one type of content at once, such as text and images together.Natural Language Processing (NLP)The field of AI focused on enabling computers to understand and work with human language.Personally Identifiable Information (PII)Data that can identify a specific person, such as a name, address, phone number, or government ID.Precision and RecallTwo metrics that measure moderation accuracy: precision is how often flags are correct, recall is how much harm is caught.Prompt InjectionAn attack where crafted input tricks an AI system into ignoring its instructions or behaving unexpectedly.Rate LimitA cap on how many actions or requests are allowed in a period of time, used to prevent abuse and overload.Self-Harm ContentContent that expresses, encourages, or gives instructions for self-harm or suicide.SpamUnsolicited, repetitive, or irrelevant content posted in bulk, usually to advertise, mislead, or disrupt.ToxicityRude, disrespectful, or hostile language that makes people want to leave a conversation.

Stop Harmful Content Today

Join many communities already protected by Supervisor's AI moderation