What is RAG chunk relevance in Jev?

RAG chunk relevance is a Score from 0 irrelevant to 3 exact for a retrieved chunk given the query. You drop weak chunks before calling a writer.

Is RAG chunk relevance the same as embedding similarity?

No. Similarity is nearest neighbors. RAG chunk relevance is a typed rubric about usefulness to the question.

How many chunks can I score at once?

Multiple Score questions in one System One call are evaluated in parallel. Stay inside TypeSafe’s token budget. Prefer one chunk per question.

What if all chunks score below 2?

Do not generate. Refuse or retrieve again. RAG chunk relevance is a gate, not a suggestion.

Does RAG chunk relevance cite sources?

No. It scores them. Citation is a writer behavior you should still validate with an LLM output guardrail.

GuardrailsSCORE

RAG chunk relevance Jev pattern

Score each retrieved chunk on a 0–3 rubric before it is stuffed into the writer prompt.

A query card beside four excerpts flagged 0 to 3 for relevance — 0 irrelevant · 1 weak · 2 usable · 3 exact.

Vector search returns near-neighbors, not answers. Score every chunk against the question, drop 0s and 1s, and only then call the writer. This is cheaper than generating a wrong paragraph and trying to catch it later.

This RAG chunk relevance schema is a paste-ready TypeSafe Jev request for jev-latest. Copy the JSON, keep thresholds in your code, and calibrate on your labels. Jev Patterns is independent and not affiliated with TypeSafe AI.

Open in Studio

Use RAG chunk relevance when

You retrieve k chunks and currently stuff all of them.
You want a graded keep/drop, not a boolean that hides weak matches.
The writer should see less, better context.

Do not use RAG chunk relevance when

You need the chunk rewritten or summarized.
You have no question — relevance is relative to a query.
The “chunk” is an image.

Confidence thresholds for RAG chunk relevance

Keep these in your code. They are not part of the model call.

When	Action
relevance.score >= 2.0	Keep the chunk in the writer context
relevance.score < 1.5	Drop it
no chunk scores >= 2.0	Refuse to answer rather than hallucinate

Pass examples

Exact policy hit

Exact. Keep.

{
  "question": "Refund window on Pro?",
  "chunk": {
    "text": "Pro subscriptions may be refunded within 14 days of first charge."
  }
}

Neighbor topic

Irrelevant or weak. Drop.

{
  "question": "Refund window on Pro?",
  "chunk": {
    "text": "Pro includes SSO and audit logs on all seats."
  }
}

Enterprise MSA aside

Usable as a contrast, not an answer. Around 2 if you keep it, 1 if you are strict.

{
  "question": "Refund window on Pro?",
  "chunk": {
    "text": "Enterprise refunds are negotiated in the MSA."
  }
}

Ambiguous examples

Old version

Topically exact, possibly stale. Freshness is metadata — pass a date if it matters.

{
  "question": "Refund window on Pro?",
  "chunk": {
    "text": "Pro refunds: 30 days (docs v2, 2023)."
  }
}

Partial overlap

Related billing rule, not the window. Weak vs usable is a product call.

{
  "question": "Refund window on Pro?",
  "chunk": {
    "text": "Refunds are not offered on usage overages."
  }
}

Question is broader than chunk

Usable for a sub-part. Do not expect Exact on a broad question.

{
  "question": "How does billing work?",
  "chunk": {
    "text": "Pro refunds within 14 days of first charge."
  }
}

The problem RAG chunk relevance is for

Vector search returns neighbors, not answers. Cosine 0.72 is not “this paragraph supports the claim.” Galileo’s chunk relevance, Bedrock’s relevance float, and cross-encoders exist because the retriever is optimistic. RAG chunk relevance is a System One Score on each candidate: 0 irrelevant, 1 weak, 2 usable, 3 exact.

People then stuff ten chunks into a writer and hope. A cheap Score per chunk, in parallel questions if you batch, lets you drop the junk before generation. Official Jev latency is 70–500ms; that is in the request path for a gate, not a nightly job.

This is not generation. RAG chunk relevance will not rewrite the chunk. It will not cite it. It will place it on a rubric you can threshold.

Why this RAG chunk relevance schema uses Jev

Score is ordered. Weak and usable share a border. A Choice of keep|drop cannot say “almost.” RAG chunk relevance can sit at 1.6 and you can still drop it if your bar is 2.0.

Ask one Score per chunk. Do not concatenate five chunks into one state and ask “are these good.” Isolation is the product. You want a number per id.

After you keep chunks at >= 2, send those to the writer, then optionally to the LLM output guardrail. Retrieval judge, write, groundedness judge. Three jobs.

What to put in state for RAG chunk relevance

Each call (or each question in a batch) needs the user query and one chunk: id, text, source title, date. RAG chunk relevance without the query is a quality score of the prose, which you do not want.

Pass the retrieval score if you have it. Jev may agree or disagree; disagreement is interesting in logs.

Stale docs: pass document_date. The rubric can mention freshness in criteria if your product cares. Do not hide that only in a prompt to the writer.

How to wire RAG chunk relevance in code

Drop chunks with score < 2.0. Keep 2.0–3.0. If none survive, refuse to generate — that is better than a fluent hallucination. Then run the LLM output guardrail on the draft.

Batch questions in one request: chunk_a, chunk_b, chunk_c as separate Score keys. They evaluate in parallel against whatever state you send. Prefer one chunk per state blob if texts are long; watch the 32k state+question budget in TypeSafe’s docs.

Eval: human keep/drop versus score >= 2. Tune the threshold, not the adjectives in criteria, once the rubric is stable.

Eval plan: sample queries where the writer cited a chunk a human would have dropped. Those are RAG chunk relevance misses. If exact and usable collapse, your 2 vs 3 criteria are synonyms — rewrite them with an example in the level text.

Failure modes

Scoring the writer’s answer

That is the LLM output guardrail. RAG chunk relevance scores retrieved text against the query, before generation.

One blob of all chunks

You will get one number. You cannot drop chunk #4. Split the questions.

Using cosine as the Score

Pass cosine in state if you want, but the Jev Score is a semantic rubric, not a copy of the embedding.

Copy, run, calibrate

The JSON in the rail is the RAG chunk relevance request for jev-latest. Copy it into your stack, or open RAG chunk relevance in Jev Studio and draw the fixture bars. Thresholds stay in your repository. Calibrate on your labels before you auto-apply. Official model docs live at docs.typesafe.ai. Jev Patterns is independent and not affiliated with TypeSafe AI.

RAG chunk relevance: FAQ

What is RAG chunk relevance in Jev?: RAG chunk relevance is a Score from 0 irrelevant to 3 exact for a retrieved chunk given the query. You drop weak chunks before calling a writer.
Is RAG chunk relevance the same as embedding similarity?: No. Similarity is nearest neighbors. RAG chunk relevance is a typed rubric about usefulness to the question.
How many chunks can I score at once?: Multiple Score questions in one System One call are evaluated in parallel. Stay inside TypeSafe’s token budget. Prefer one chunk per question.
What if all chunks score below 2?: Do not generate. Refuse or retrieve again. RAG chunk relevance is a gate, not a suggestion.
Does RAG chunk relevance cite sources?: No. It scores them. Citation is a writer behavior you should still validate with an LLM output guardrail.

Last reviewed 21 September 2026. Independent of TypeSafe AI.