Panels recruit all kinds of participants, and some low-effort or fraudulent responses always slip through. To save you reading every transcript to find them, Tellet automatically rates the quality of each participant's answers once they finish the interview.
The ratings
Each completed transcript gets one of three ratings:
- Good: the participant properly answered what was asked.
- Medium: many answers are missing or don't really respond to the question, but the participant isn't faking it.
- Low: the answers are junk, nonsense or fraudulent.
Every rating comes with a one-sentence explanation of why it was given.
What the rating looks at
An AI reviewer reads all of the participant's answers and judges them against five criteria:
- Fraud signals: answers that have nothing to do with the question, oddly salesy language, or the same one or two words for every question.
- Specificity: real examples and situations rather than vague generalities.
- Comprehension: whether the participant answered the question that was actually asked.
- Consistency: whether the answers fit together, without unexplained contradictions.
- Engagement and depth: whether answers include reasons, detail and context.
The reviewer judges the interview as a whole, so one weak answer doesn't sink an otherwise good participant. The rating reflects patterns across the conversation.
The reviewer gives one overall rating per transcript, not a score per answer or per criterion. The explanation and the cited answers show what the decision was mainly based on.
Flags
On top of the rating, a transcript can get one or more flags:
- Fraud: for example, the same text pasted into unrelated questions, gibberish, copied text such as song lyrics, or "don't know" given to almost every question. A fraud flag always makes the rating Low.
- Offensive: hate speech, harassment, threats or explicit content. This also always makes the rating Low. Swearing used for emphasis ("this product is f***ing good") doesn't count.
- AI: the answers read as if a tool like ChatGPT wrote them. This is a warning only and doesn't change the rating. It's only raised when many signs show up together, and never on short answers.
What it won't penalise
- Short answers. A brief but genuine answer like "taste is not good" isn't treated as low quality on its own.
- Plain or non-native language. Simple wording is never taken as a sign of AI or fraud. The reviewer reads answers in the language of the interview.
- Speech-to-text mistakes. In voice and video interviews, a misheard word (for example "chad gpt" for "ChatGPT") is put down to the transcription, not the participant.
Answers to multiple-choice and scale questions aren't part of the rating, so a quick pick like "No" doesn't count as a lazy answer.
Where to find it
- Transcripts table: the
Quality column shows each transcript's rating and any flags. Hover over the rating to read the explanation.
- Transcript details: open a transcript to see the Answer quality card, with the full explanation, the verdict, an engagement score and the answers the rating was mainly based on.
- Quality flags filter: use it to list flagged transcripts that nobody has reviewed yet.
Reviewing flagged transcripts
- Filter the table to flagged transcripts.
- Open a transcript and read the explanation and the cited answers.
- Click Mark as reviewed once you've checked it. This records who reviewed it and when, and removes it from the flagged filter. It doesn't change the rating.
- If the response really is low quality, archive it to remove it from your analysis and exports.
The explanation is written so you can also use it as the reason when you reject a participant with your panel provider.
Good to know
- Ratings are a guide, not a final verdict. Always check a flagged transcript before rejecting a participant.
- When it runs: a transcript is rated once, when the participant completes the interview. For voice and video interviews, the rating waits until the recordings have been transcribed.
- Not rated: transcripts that were imported, and interviews where the participant gave no written or spoken answers.
