Trust & Safety
Hey Everyone! After months of development behind the scenes, we have introduced a new system which we are calling the Child Sexual Exploitation and Abuse (CSEA) Detection Tool. It is an automated detection that examines the pattern of a conversation for signs of child sexual exploitation and abuse is now live across our rooms. It routes what itn finds to trained human review, and it works alongside the reporting tools and the moderation team already in place in every room we run.
Why this matters
Safety is the product
We have been building this quietly for months, and it is running across our rooms today. This post is an account of what it does, published after the fact rather than before it — a safety system is worth announcing only once it is actually working, and the people it exists to catch read announcements too.
We have run free chat rooms since 1999. In that time the single thing that decides whether a community is worth belonging to is whether people are safe inside it — and nowhere is that more absolute than where young people are concerned.
We should also be straight about our own history. Our safety tooling has not always been what it is today. For a long stretch of this site’s life the options available to us were thin — there was a period when there was no way inside the chat for one user to report another at all. Most of the criticism that still attaches to our name dates from those years, and it describes a version of this site that no longer exists.
What changed it was not a single announcement. It has been several years of steady, largely invisible work: better tools for our moderators, reporting that actually goes somewhere, automated defences against the networks that mass-produce accounts, and now detection that reads conversations rather than scanning them for banned words. We are not claiming to have finished. We are claiming to be a long way from where we started, and still moving.
Keyword filters have never been enough. Anyone who has moderated a chat room knows the problem: the words that matter are rarely the words that get typed. Harm arrives spaced out, misspelled, wrapped in punctuation, or spread across a dozen innocuous-looking messages that only mean something when you read them together.
So we built something that reads the conversation.
New
Automated CSEA detection, with human judgement at the end
Our new system analyses messages as they are sent and looks for the recognised behavioral stages of grooming and exploitation rather than simply a list of forbidden words. Anything it surfaces is packaged with its evidence and sent to a dedicated review queue for a person to read and decide.
-
01
Every message is analysed
Room messages and private messages are examined in real time. The matching is built to see through the evasions people actually use — inserted spaces and punctuation, character substitution, look-alike letters from other alphabets, and deliberate misspelling.
-
02
Behavior is scored, not vocabulary
Grooming is a sequence: building trust, isolating someone from the people around them, introducing secrecy, offering incentives, asking for images, and moving the conversation to another platform. Any one of these is innocent on its own. Several of them, aimed by the same account, are not.
-
03
Context decides the priority
A flag is weighed against who is talking to whom — the declared ages on both sides of the conversation, and whether the behavior was concentrated on one person or scattered across many. An adult working through the stages with a single child is treated very differently from two people the same age talking.
-
04
The evidence travels with the flag
Every item in the queue carries the actual messages behind it, grouped and ordered by severity. A reviewer never has to act on a category name alone; they read what was written and judge it.
-
05
A person makes the decision
The detection layer records; it does not pass sentence. For anything requiring judgement, a trained reviewer decides whether to ban, escalate, or dismiss — and every action taken is logged and reversible.
Where there is no ambiguity, there is no delay
A narrow set of behaviors — chiefly the advertising, selling or soliciting of child sexual abuse material — is removed automatically on the very first message, with no warning and no second chance. These cases are not sent for review first. They are ended, and then recorded.
Reporting
Every user is a safety system
Automation finds patterns; people find meaning. Reporting is available directly in chat, on any user and any message, and it reaches the same moderation team that works the detection queue.
Reports do not disappear into a void. They accumulate against an account, so a pattern that no single reporter could see — four different people uneasy about the same user across three days — becomes visible as one picture. Moderator actions and the reasons written for them feed back into the same review surface, which means a member of staff acting on instinct in a room still contributes to the formal record.
If something is wrong, tell us. It is the fastest route we have.
At the door
Stopping the networks before they reach a room
The most damaging content rarely arrives from one account acting alone. It comes from networks that mass-produce accounts, burn through them, and make more.
Protection therefore starts before anyone types a word. Sign-up and guest entry are screened for the signatures of automated account creation — the structural tells that separate a script from a person, which are considerably harder for an operator to change than a username or an email address.
When we identify a network, we do not remove it one account at a time. We trace it across every site we run and shut the whole operation out at once, including the accounts it has created but not yet used.
How we hold ourselves to it
The rules we impose on our own systems
Detection records, humans decide
Outside the narrow automatic-removal categories, no account is actioned on a machine’s say-so. Every judgement call is made by a person who has read the messages.
Every action is reversible
Bans, dismissals and escalations are written to an audit trail with the means to undo them. A mistake should be correctable, not permanent.
Measured before it is trusted
Before any new rule is allowed to act, we test it against our own history and measure how often it would have been wrong about a real member. A rule that cannot clear that bar does not go live.
Separated from everything else
The child-safety review system is held apart from general site administration, with its own access control, so the people who need it have it and no one else does.
We do not publish the recipe
We describe what our systems do, never the exact signals they match on. Publishing that detail would hand a checklist to the people we are trying to catch.
It is never finished
Evasion techniques change weekly. Detection that is not actively maintained is decoration, so ours is reviewed and extended continuously.
Shared responsibility
No platform is the whole answer
Everything above is our responsibility and we take it seriously. But we would be misleading you if we suggested that any chat platform, ours included, can be a complete solution to online safety. It cannot, and a site that tells you otherwise is selling something.
Our reach ends at our own rooms
The most common move in online exploitation is getting a conversation off the platform where it started — onto a messaging app, a game, a social network where none of this applies. We look for that move and we act on it. What we cannot do is follow a conversation once it has left.
If you are a parent, that gap is the one that matters most. Knowing which apps your child uses, talking about it early rather than after something goes wrong, and making it safe for them to come to you will do more than any filter — ours or anyone else’s. A child who is certain they will not be punished for speaking up is protected in a way no software can reproduce. Supervision is not surveillance, and it is not distrust; it is the part of this that only you can do.
If you are using our rooms yourself, a few things are worth treating as warning signs no matter who they come from: someone pressing to move the conversation to another app, asking for photographs, asking you to keep the conversation between the two of you, or being unusually generous very early on. Those are among the patterns our detection is built to recognise. They are worth recognising yourself, because you will always see them before we do.
And tell us. Community reporting is not a formality we offer because we have to — it is genuinely one of the most effective safety mechanisms we have. We would far rather look at a hundred reports that turn out to be nothing than miss the one that was not.
If you see something
Use the report option on the user or the message. It goes straight to the moderation team, it is read, and it counts — even when you are not certain, and even when you would rather not make a fuss.
If you believe a child is in immediate danger, call your local police without waiting. In the United States, suspected child sexual abuse material and online child exploitation should be reported to the National Center for Missing & Exploited Children through the CyberTipline at report.cybertip.org or 1-800-843-5678, which operates around the clock. Anywhere else, report it to your national law enforcement agency. These organizations have powers and reach that no chat platform has, and going to them directly is always the right thing to do.
#1 Chat Avenue has provided free chat rooms since 1999. This page describes our child-safety approach in general terms; specific detection signals are deliberately withheld.