After a lesson, for students who agreed to be recorded, the diagnostic reads a transcript of their own microphone. It is an aid for the teacher and the student to see how speaking moves from one lesson to the next. It is not a grade and not a certificate.
What it measures
- Five aspects of speaking — range, fluency, accuracy, coherence and interaction — each on the steps A1, A2, B1, B2 and C1 (never C2).
- A level, read from 15 can-do descriptors (three for each step from A1 to C1) over the current lesson and up to the seven before it, shown with the working range around it. The five aspects are read alongside the level; they are not what it is computed from.
- Vocabulary, in five ways: how varied, how rich, which levels, how accurate and how fluent.
- Grammar: the structures used, and the mistakes that were found, with the excerpt to listen to again.
- Progress: how each of these moved, lesson by lesson.
How it is done
Part of it is counted: how many different words, how often a student pauses and for how long, which level list a word belongs to. Part of it is judged by language models that read the transcript: is this a mistake, is this a set phrase. A judgement is never taken alone — a second model checks the work of the first, and an item the two checks disagree on is left out, and said so.
Every result carries a fingerprint of the exact calculation: the versions of the models, the word lists and the thresholds. Two results with different fingerprints are not compared without saying so. The audio itself is not scored: pronunciation is not rated, the pronunciation review only points to passages worth listening to again.
Estimates
A firm level needs 8 lessons. Until the level rests on that many, or when there are fewer than 3 lessons or only a few minutes of speech, it is marked estimate with the number of lessons behind it. An estimate can move: a few more lessons will firm it up. An estimate is always written with the word and a dashed outline — never with colour alone.
Verdicts and milestones
A verdict is said only when at least 3 comparable lessons are behind it. Progress or a little lower also needs a change bigger than the normal variation of the measure; otherwise the verdict is steady. With fewer than 3 lessons the page says first lessons, and a trend drawn from only a few lessons is marked estimate. A milestone is a dated event — a structure acquired, a level that moved. The first lesson of a period is only a starting point, so it never counts as one.
Vocabulary
- Range — different words among the words said. MTLD and MATTR are two standard measures of that variety; MATTR looks through a moving window.
- Depth — set phrases, words used in a stretched meaning, phrasal verbs.
- Word levels — each word placed on a public level list, shown as A1, A2, B1, B2 and C1+ (C1 and C2 together). A word that is on no list is shown as “?”.
- Accuracy — of the words that were checked, how many were used correctly. The items the two checks disagreed on are not counted.
- Fluency — hesitations before a word of content, filler words, switches to another language.
Progress
The Progress page reads the lessons inside the chosen window — 30 days, 90 days, 180 days, or all. A mistake is recurring only when it comes back in more than one lesson. The sentence at the top of the page is written from measured facts, and every sentence is checked against them before it is shown.
Not calibrated yet
The diagnostic has not been compared to trained human raters yet, so its levels are not calibrated. Read a level as a working range, not a grade, and do not grade or place a student on it without a teacher's judgement.
The same goes for the thresholds behind verdicts and trends: the margin of normal variation uses cautious default values that have not been checked against human raters either. Some measures are only shown as a series, with no verdict: the lexical range (MTLD) is one, because a single word can move it too much.
Known limits
- Speech recognition makes mistakes — more with strong accents, background noise or a poor microphone — and a mistake in the transcript can become a mistake in the diagnostic. A word the system is unsure it heard is flagged or left out.
- With less than 15 seconds of speech there is no level, and with less than 30 seconds no fluency or vocabulary measure. The page says so instead of guessing.
- Two runs on the same lesson can give slightly different results, mostly on the judged items such as mistakes and set phrases.
- The level and the aspects read the current lesson and up to the seven before it, not the whole history. The aspects are an average over those lessons weighted by speaking time: a long lesson weighs more than a short one, and an older lesson weighs as much as a recent one of the same length.
- English only for now. The texts the diagnostic writes — summaries, advice, explanations — are in English.
- No pronunciation score.
- A measure that is missing is explained, never shown as zero.
Your data
What is recorded, who sees it and for how long is in the privacy policy.