All guides

Use cases and implementation

Community harassment reports

A worked process for limited evidence access, clear ownership, application action, and human appeal.

For
Community, social, dating, forum, and creator-platform safety teams
Last updated
29 August 2026

01

Starting point

A member reports repeated harassment across several conversations. The team needs enough message and relationship context to assess the pattern, while limiting who can view sensitive content and preventing two analysts from taking conflicting actions.

02

What must hold

  • Exposing sensitive conversation content to reviewers who do not need it.
  • Two analysts working the same case without seeing each other’s ownership.
  • Sending a generic notice that makes an affected user unable to understand or appeal the decision.

03

Step by step

  1. Publish the visual workflow with an authority mode, a passing fixture, an immutable version, and explicit failure paths. Pin that version to the active queue policy before new reports enter it.
  2. Collect structured allegations, referenced message IDs, reporter safety preferences, and optional supporting files through the report form.
  3. Route threats and imminent-risk signals directly to a protected human queue; route lower-priority conduct reports according to the configured policy.
  4. Fetch only the referenced conversation window and relevant relationship signals through signed, time-bounded enrichment webhooks.
  5. Show live analyst presence and a renewable claim so teammates know who is viewing, working, or waiting on the report.
  6. Apply the customer’s action, send separately scoped reporter and affected-user notices, and place every appeal in a human queue with the original record intact.

04

Operating boundary

The harassment queue is human-managed. AI may organize evidence and identify missing context, but it cannot decide threats, sexual harassment, protected-characteristic abuse, coordinated targeting, or any case involving a vulnerable user.

05

Data exchange

The process requests a conversation_window finding for the referenced message IDs. The customer service returns the excerpt authorized by policy, stable source links, and retention metadata. A failed retrieval stays visible and blocks the decision when that evidence is required.

Stable referenceSigned requestTyped result

06

Verification

The analyst sees who else is present, claims the report, reviews authorized evidence, consults the published policy, and explains the outcome. The audit timeline supports later quality review while the customer application remains the source of truth for its messages.

07

Implementation questions

Does analyst presence replace assignment?+

No. Presence is short-lived awareness; a claim or assignment is the durable indication of responsibility. Both are shown so teams can coordinate without accidentally locking abandoned work forever.

Should harassment ever be AI-managed?+

A customer may use AI for bounded preparation or low-risk triage, but safety-sensitive, uncertain, protected, high-impact, or contested cases must escalate to a human under the product’s operating boundaries.

Safest Resolve

Start with one report your team handles today.

Bring its source, required evidence, decision owner, application action, messages, and appeal path.