What is the content moderation screen pattern?

The content moderation screen is a Jev request with Nouls for spam, harassment, sexual, self_harm, and malware. Each returns a probability your policy thresholds.

Does the content moderation screen replace OpenAI or Perspective APIs?

It is an alternative shape on TypeSafe Jev, not a drop-in for those vendors. Jev Patterns is not affiliated with TypeSafe AI or with those APIs.

How should self_harm be handled in a content moderation screen?

Escalate to a human playbook at a low Noul threshold. Do not generate supportive copy with Jev. Jev does not generate text.

Can one post fail multiple categories in the content moderation screen?

Yes. Independent Nouls can all be high. That is why they are not a single Choice.

Does the content moderation screen work on images?

Not directly. Jev accepts text today. OCR first, or use a vision stack for the binary, then send text as state.

ModerationNOUL

Content moderation screen Jev pattern

Run independent Nouls for spam, harassment, sexual content, self-harm, and malware.

Ink pads for parallel moderation labels on a cream desk — Five Nouls. No explanatory essay.

A single “is this bad?” score hides which policy fired. Five Nouls on the same state give you five numbers you can threshold independently — spam can be auto-dropped while self-harm always goes to a trained reviewer. Jev does not generate the content and does not explain it. It only answers the questions you asked.

This content moderation screen schema is a paste-ready TypeSafe Jev request for jev-latest. Copy the JSON, keep thresholds in your code, and calibrate on your labels. Jev Patterns is independent and not affiliated with TypeSafe AI.

Open in Studio

Use content moderation screen when

User-generated text hits your API before it is stored or shown.
Policies are independent and should not collapse into one label.
You want calibrated probabilities, not a single moderation enum.

Do not use content moderation screen when

The payload is an image or video — Jev is text-only today.
You need a user-facing explanation of the violation.
You are trying to rewrite the content to make it acceptable.

Confidence thresholds for content moderation screen

Keep these in your code. They are not part of the model call.

When	Action
self_harm.noul >= 0.40	Hold the post and route to a trained human queue — never auto-reply with advice
malware.noul >= 0.70 \|\| spam.noul >= 0.85	Reject without storing
harassment.noul >= 0.75 \|\| sexual.noul >= 0.75	Reject or quarantine per policy

Pass examples

Crack link

malware and spam high. The other Nouls should stay low.

{
  "text": "Free license key generator: http://bit.ly/not-a-real-store"
}

Clean product feedback

All five should sit near 0.

{
  "text": "The new export is missing tags. Otherwise it looks good."
}

Insult

harassment up, the rest down. Do not call it spam.

{
  "text": "You're an idiot and everyone knows it."
}

Ambiguous examples

Quoted abuse

Reporter vs perpetrator. Put role in state or you will over-flag.

{
  "text": "Reporting this message I received: 'you should disappear.'"
}

Security research

malware vs research. Surface and audience belong in state.

{
  "text": "PoC for CVE-2024-1234 attached for the security list."
}

Dark humor

Idiom, not a crisis. A 0.5 is unsure — that is not a medium severity.

{
  "text": "If this deploy fails I'm going to lose it."
}

The problem content moderation screen is for

Hosted moderation APIs return category scores with their own taxonomy. Fine-tuned classifiers need labels you may not have yet. Prompted LLMs return a JSON object of booleans that you still have to parse, and they will happily explain the sexual content they just classified — which you did not ask them to generate.

A content moderation screen on Jev is a set of Nouls: spam, harassment, sexual, self_harm, malware. Each is the probability a statement is true. They run in parallel on the same state. You threshold them in policy code. Nothing is written back to the user by this model.

This content moderation screen is a first-pass gate before storage, search indexing, or a writer model. It is not a counselor. self_harm at any non-trivial Noul should route to a human playbook you already own.

Why this content moderation screen schema uses Jev

Independent Nouls mean a post can be spam and harassment at once. A single Choice of “the worst category” hides that. Content moderation screen keeps the bits separate so your policy can AND/OR them.

There is no separate confidence field on a Noul. The number is the probability. 0.5 is unsure. Do not map it to a yellow “maybe harassment” badge that staff treat as a finding.

Jev currently accepts text, objects, and arrays of text — not images. Screenshots need OCR first. Say that in your UI if you moderate uploads.

What to put in state for content moderation screen

Send the body, the author age-band if you have it, recent strikes, and whether the destination is public. A content moderation screen without audience context will treat a private log as a public post.

Do not send a request to “be helpful.” Send the artifact. Extra instructions that beg for a narrative make people wire a chatbot by accident.

For self_harm, pair the Noul with a deterministic keyword/regex screen you already trust. Two cheap gates beat one clever gate.

How to wire content moderation screen in code

Example: block if malware.noul >= 0.70 or sexual.noul >= 0.85 on a teen surface. Queue for human if harassment.noul >= 0.60. Allow if all Nouls <= 0.20. The middle is review, not publish.

self_harm should use a much lower auto-escalate threshold because the cost of a miss is a person, not a bad ranking. You are routing to a human, not generating advice.

Never echo the flagged span back through a writer “to explain the decision.” Content moderation screen does not explain. Your policy page does.

Eval plan: use a labeled sample per category, including overlaps (spam+harassment). Check that the content moderation screen can raise two Nouls at once. Precision on malware can be higher-threshold than spam; recall on self_harm should be the opposite.

Failure modes

One Choice of category

A single winner cannot represent overlapping harms. Use parallel Nouls in the content moderation screen.

Treating 0.5 as a hit

Unsure is not a finding. Queue it or gather more state.

Image memes

Jev has no vision yet. OCR or skip. Do not pretend the alt text is the image.

Copy, run, calibrate

The JSON in the rail is the content moderation screen request for jev-latest. Copy it into your stack, or open Content moderation screen in Jev Studio and draw the fixture bars. Thresholds stay in your repository. Calibrate on your labels before you auto-apply. Official model docs live at docs.typesafe.ai. Jev Patterns is independent and not affiliated with TypeSafe AI.

Content moderation screen: FAQ

What is the content moderation screen pattern?: The content moderation screen is a Jev request with Nouls for spam, harassment, sexual, self_harm, and malware. Each returns a probability your policy thresholds.
Does the content moderation screen replace OpenAI or Perspective APIs?: It is an alternative shape on TypeSafe Jev, not a drop-in for those vendors. Jev Patterns is not affiliated with TypeSafe AI or with those APIs.
How should self_harm be handled in a content moderation screen?: Escalate to a human playbook at a low Noul threshold. Do not generate supportive copy with Jev. Jev does not generate text.
Can one post fail multiple categories in the content moderation screen?: Yes. Independent Nouls can all be high. That is why they are not a single Choice.
Does the content moderation screen work on images?: Not directly. Jev accepts text today. OCR first, or use a vision stack for the binary, then send text as state.

Last reviewed 21 September 2026. Independent of TypeSafe AI.