Skip to content

Industry field guide

Education agency reporting cohort and metadata for AI

Two education reports may use the same label while covering different school years, entities or populations.

2022 NAICS
923110
Updated

What this guide tests

What year-2 total is supported, and can the model call 100 to 80 a like-for-like decline or add school rows to the district row?

See the specimen and fields · Work through the fictional records

Task and source context

Two education reports may use the same label while covering different school years, entities or populations. The U.S. Department of Education’s Ed Data Express provides documentation for education program data, including data collected through EDFacts. An agency can test whether an assistant preserves those definitions when comparing aggregates. U.S. Department of Education Ed Data Express documentation

Create a fictional reporting workbook with district totals and a second export broken down by school. Change the eligible cohort in the second year. Ask which totals can be aggregated and why the year-over-year comparison needs an adjusted basis. A reviewer should verify the grain, year and cohort definition.

Use invented counts or cleared aggregates with release notes. Exclude student records, small groups that could identify a person, staff identities and internal program files.

Describe a synthetic education reporting example

Industry research

The U.S. Census Bureau’s industry definitions establish which activities these codes cover. The workflow sources in this guide describe the task or record type. Fiund’s pilot below uses fictional material to test a specific decision within that scope.

923110 · Administration of Education Programs

This industry comprises government establishments primarily engaged in the central coordination, planning, supervision, and administration of funds, policies, intergovernmental activities, statistical reports and data collection, and centralized programs for educational administration. Government scholarship programs are included in this industry.

U.S. Census Bureau, 2022 NAICS Manual, p. 594 ↗

Build a pilot sample

Set up this pilot

Make two synthetic cohort totals with a definition change and school rows nested under a district. Grade the current count and refuse a trend until both cohorts are rebased.

Prepare the test packets

  1. Start with the worked case. Use the fictional record types below as a guide to the source files you need. Give each packet a stable ID and save its source versions with it.
  2. Change one fact. Make one packet with enough evidence to answer the question and one where the evidence points to a different answer. Keep the other fields fixed so a reviewer can tell what changed.
  3. Remove the decisive evidence. Give the reviewer a third packet with a plausible near-match in its place. Have them write what can still be established and what evidence would resolve the rest.

Fields the reviewer needs

Entity grain and inclusion relationship
Prevents hierarchical double-counting.
School year and cohort definition
Identifies which populations are comparable.
Reprocessing availability
Shows whether a common-basis trend can be constructed.

Use Build a public-sector AI evaluation from versioned records to turn the specimen into a brief with a defined task, source and review owner.

Fictional exercise

School rows cannot be added to their district total as extra entities

A fictional education report changes both row grain and eligible cohort between years. It uses invented aggregates without students, small groups or internal program files.

Records supplied to the evaluator
Record IDSourceWhat it records
EA1Year 1 practice district reportDistrict D total 100 for cohort C1, all eligible organizations in its stated scope.
EA2Year 2 exportDistrict D total 80 for revised cohort C2; constituent school S1=30 and S2=50, both included in the district total.
EA3Metadata noteC2 excludes a category included in C1. No reprocessed year-1 C2 total or year-2 C1 total supplied.

Question for the model. What year-2 total is supported, and can the model call 100 to 80 a like-for-like decline or add school rows to the district row?

Show the reference answer

EA2 supports 80 at district grain, with its included school rows 30+50 reconciling to that total. Adding all three rows would double-count the same population. EA3 also makes C2 different from C1, so 100 to 80 is a reported difference across cohorts rather than a supported like-for-like decline. Preserve year, grain and definition, and request a common-cohort series for that interpretation.

Download the fictional records and answer key (JSON)

How to check the result

Ask for the answer, the record IDs that support it and any fact the evidence leaves unresolved. Have a reviewer compare the response against the reference before deciding whether the sample is ready to expand.

Errors to flag in this case

  • Summing district totals and their included school components as independent rows.
  • Attributing a cross-year difference to change in outcomes when the eligible cohort changed.

Change the inputs and run it again

Provide a reprocessed year-1 C2 total while retaining the old C1 report. Require the common-cohort comparison to cite the reprocessed source and preserve the original separately.

Score the decision and the evidence separately

Save the reviewer’s expected answer, supporting record IDs and unresolved fields in a separate answer key. Score the model on the decision, its citations and its handling of the deliberate gap. A correct answer citing a different case counts as an evidence error.

Before expanding collection, have a second reviewer run the same checks on the held-out packet. Keep related versions, views and exports of the same case together so they stay on one side of the training and evaluation split.

Check permission and scope

Use invented counts or appropriately cleared released aggregates. Exclude students, identifying small groups, staff identities and internal records.

Exclude beneficiary, student, veteran and patient information and individual eligibility decisions.

Describe an authorized archive through Fiund’s existing-data assessment, or describe a new capture capability through the collection brief. The contributor agreement covers the terms for participation.

Sources and further reading

Resources for this workflow

How the examples work

Company and program reviews

Use these reviews to compare sourcing, capture or evaluation approaches. Confirm the applicable program, accepted sources and terms with the provider.

All company reviews

Related industry examples

← Public administration