Skip to content

Industry field guide

Language school speaking prompt and response data for AI

A spoken response can be clear but answer a different prompt than the one recorded in the rubric.

Sector
Education
2022 NAICS
611630
Updated

What this guide tests

Which prompt does each response address by content, and which claims cannot be established from these text artifacts?

See the specimen and fields · Work through the fictional records

Task and source context

A spoken response can be clear but answer a different prompt than the one recorded in the rubric. ACTFL publishes language performance descriptors. A school can use original prompts and scripted adult responses to test prompt matching and feedback attribution. ACTFL performance descriptors

Write two brief practice prompts and two approved responses. Give one an audio and transcript pair with a mismatched label. Ask what can be verified, which response belongs to each prompt and what needs a reviewer. Use independent criteria and keep pronunciation or proficiency claims specific to the exercise.

Use consenting adult voices or wholly synthetic text. Exclude learner identities, school recordings, actual proficiency ratings and protected curriculum. Clear audio, transcript and model-use rights separately.

Describe an original language practice sample

Industry research

The U.S. Census Bureau’s industry definitions establish which activities these codes cover. The workflow sources in this guide describe the task or record type. Fiund’s pilot below uses fictional material to test a specific decision within that scope.

611630 · Language Schools

This industry comprises establishments primarily engaged in offering foreign language instruction (including sign language). These establishments are designed to offer language instruction ranging from conversational skills for personal enrichment to intensive training courses for career or educational opportunities.

U.S. Census Bureau, 2022 NAICS Manual, p. 515 ↗

Build a pilot sample

Set up this pilot

Write two original speaking prompts and typed practice responses, then swap the folder labels. Grade the addressed prompt from words and leave pronunciation unassessed.

Prepare the test packets

  1. Start with the worked case. Use the fictional record types below as a guide to the source files you need. Give each packet a stable ID and save its source versions with it.
  2. Change one fact. Make one packet with enough evidence to answer the question and one where the evidence points to a different answer. Keep the other fields fixed so a reviewer can tell what changed.
  3. Remove the decisive evidence. Give the reviewer a third packet with a plausible near-match in its place. Have them write what can still be established and what evidence would resolve the rest.

Fields the reviewer needs

Prompt ID and exact question
Defines the content being answered.
Response words and source
Makes the proposed content association inspectable.
Container label conflict
Preserves the metadata error for correction.
Available modality
Limits conclusions to text evidence.

Use Build an AI evaluation sample from original teaching materials to turn the specimen into a brief with a defined task, source and review owner.

Fictional exercise

A clear transcript can still belong to the wrong speaking prompt

Two original prompts and scripted text responses test content association. The packet uses no learner audio, real proficiency rating or licensed curriculum.

Records supplied to the evaluator
Record IDSourceWhat it records
LS1Original prompt pairP1 asks for a fictional shop's opening day. P2 asks for its closing time.
LS2Response artifact AScript says The shop opens on Monday. Container label says P2; transcript and scripted source agree on the words.
LS3Response artifact BScript says It closes at six. Container label P1. No pronunciation recording or authorized proficiency score supplied.

Question for the model. Which prompt does each response address by content, and which claims cannot be established from these text artifacts?

Show the reference answer

LS1 and the scripted words in LS2 associate A with P1's opening-day question despite its P2 container label. LS3's closing-time response addresses P2 despite its P1 label. Flag both label mismatches for review rather than declaring the responses irrelevant from metadata alone. Text supports content matching here, but it supplies no evidence about pronunciation, audio quality or an individual's language proficiency.

Download the fictional records and answer key (JSON)

How to check the result

Ask for the answer, the record IDs that support it and any fact the evidence leaves unresolved. Have a reviewer compare the response against the reference before deciding whether the sample is ready to expand.

Errors to flag in this case

  • Following swapped container labels while ignoring the explicit prompt-response content.
  • Assigning a pronunciation or proficiency rating from text alone.

Change the inputs and run it again

Change the fictional shop to opening at six and closing on Monday. Test whether the model follows the revised prompts rather than memorizing day-versus-time associations from this example.

Score the decision and the evidence separately

Save the reviewer’s expected answer, supporting record IDs and unresolved fields in a separate answer key. Score the model on the decision, its citations and its handling of the deliberate gap. A correct answer citing a different case counts as an evidence error.

Before expanding collection, have a second reviewer run the same checks on the held-out packet. Keep related versions, views and exports of the same case together so they stay on one side of the training and evaluation split.

Check permission and scope

Use original text or separately consented adult recordings with explicit audio, transcript and model-use rights. Exclude school recordings, learner identities and actual proficiency assessments.

Exclude student records, names, private accommodations and licensed curriculum unless the specific use is authorized.

Describe an authorized archive through Fiund’s existing-data assessment, or describe a new capture capability through the collection brief. The contributor agreement covers the terms for participation.

Sources and further reading

Resources for this workflow

How the examples work

Company and program reviews

Use these reviews to compare sourcing, capture or evaluation approaches. Confirm the applicable program, accepted sources and terms with the provider.

All company reviews

Related industry examples

← Education