Industry field guide
Education agency reporting cohort and metadata for AI
Two education reports may use the same label while covering different school years, entities or populations.
- Sector
- Public administration
- 2022 NAICS
- 923110
- Practice pack
- Example and answer key (JSON)
- Updated
What this guide tests
What year-2 total is supported, and can the model call 100 to 80 a like-for-like decline or add school rows to the district row?
See the specimen and fields · Work through the fictional records
Task and source context
Two education reports may use the same label while covering different school years, entities or populations. The U.S. Department of Education’s Ed Data Express provides documentation for education program data, including data collected through EDFacts. An agency can test whether an assistant preserves those definitions when comparing aggregates. U.S. Department of Education Ed Data Express documentation
Create a fictional reporting workbook with district totals and a second export broken down by school. Change the eligible cohort in the second year. Ask which totals can be aggregated and why the year-over-year comparison needs an adjusted basis. A reviewer should verify the grain, year and cohort definition.
Use invented counts or cleared aggregates with release notes. Exclude student records, small groups that could identify a person, staff identities and internal program files.
Industry research
The U.S. Census Bureau’s industry definitions establish which activities these codes cover. The workflow sources in this guide describe the task or record type. Fiund’s pilot below uses fictional material to test a specific decision within that scope.
923110 · Administration of Education Programs
This industry comprises government establishments primarily engaged in the central coordination, planning, supervision, and administration of funds, policies, intergovernmental activities, statistical reports and data collection, and centralized programs for educational administration. Government scholarship programs are included in this industry.
Build a pilot sample
Set up this pilot
Make two synthetic cohort totals with a definition change and school rows nested under a district. Grade the current count and refuse a trend until both cohorts are rebased.
Prepare the test packets
- Start with the worked case. Use the fictional record types below as a guide to the source files you need. Give each packet a stable ID and save its source versions with it.
- Change one fact. Make one packet with enough evidence to answer the question and one where the evidence points to a different answer. Keep the other fields fixed so a reviewer can tell what changed.
- Remove the decisive evidence. Give the reviewer a third packet with a plausible near-match in its place. Have them write what can still be established and what evidence would resolve the rest.
Fields the reviewer needs
- Entity grain and inclusion relationship
- Prevents hierarchical double-counting.
- School year and cohort definition
- Identifies which populations are comparable.
- Reprocessing availability
- Shows whether a common-basis trend can be constructed.
Use Build a public-sector AI evaluation from versioned records to turn the specimen into a brief with a defined task, source and review owner.
Fictional exercise
School rows cannot be added to their district total as extra entities
A fictional education report changes both row grain and eligible cohort between years. It uses invented aggregates without students, small groups or internal program files.
| Record ID | Source | What it records |
|---|---|---|
| EA1 | Year 1 practice district report | District D total 100 for cohort C1, all eligible organizations in its stated scope. |
| EA2 | Year 2 export | District D total 80 for revised cohort C2; constituent school S1=30 and S2=50, both included in the district total. |
| EA3 | Metadata note | C2 excludes a category included in C1. No reprocessed year-1 C2 total or year-2 C1 total supplied. |
Question for the model. What year-2 total is supported, and can the model call 100 to 80 a like-for-like decline or add school rows to the district row?
Show the reference answer
EA2 supports 80 at district grain, with its included school rows 30+50 reconciling to that total. Adding all three rows would double-count the same population. EA3 also makes C2 different from C1, so 100 to 80 is a reported difference across cohorts rather than a supported like-for-like decline. Preserve year, grain and definition, and request a common-cohort series for that interpretation.
How to check the result
Ask for the answer, the record IDs that support it and any fact the evidence leaves unresolved. Have a reviewer compare the response against the reference before deciding whether the sample is ready to expand.
Errors to flag in this case
- Summing district totals and their included school components as independent rows.
- Attributing a cross-year difference to change in outcomes when the eligible cohort changed.
Change the inputs and run it again
Provide a reprocessed year-1 C2 total while retaining the old C1 report. Require the common-cohort comparison to cite the reprocessed source and preserve the original separately.
Score the decision and the evidence separately
Save the reviewer’s expected answer, supporting record IDs and unresolved fields in a separate answer key. Score the model on the decision, its citations and its handling of the deliberate gap. A correct answer citing a different case counts as an evidence error.
Before expanding collection, have a second reviewer run the same checks on the held-out packet. Keep related versions, views and exports of the same case together so they stay on one side of the training and evaluation split.
Check permission and scope
Use invented counts or appropriately cleared released aggregates. Exclude students, identifying small groups, staff identities and internal records.
Exclude beneficiary, student, veteran and patient information and individual eligibility decisions.
Describe an authorized archive through Fiund’s existing-data assessment, or describe a new capture capability through the collection brief. The contributor agreement covers the terms for participation.
Sources and further reading
Resources for this workflow
Includes worksheet
Build a public-sector AI evaluation from versioned records
Build a public or invented versioned record example and distinguish proposal, issue and outcome.
Includes worksheet
Build an AI evaluation from professional work
Write a rubric for the supported administrative decision and an abstention case.
Company and program reviews
Use these reviews to compare sourcing, capture or evaluation approaches. Confirm the applicable program, accepted sources and terms with the provider.
Compare the approach
Surge AI review for frontier data and agent evaluation
Compare how a grader and expert review should handle ambiguous document evidence.
Compare the approach
Sapien review: dataset quality review and expert checks
Read the quality review questions for source scope, reference judgments and unresolved cases.
Related industry examples
Industry guide · NAICS 923130
Human services program caseload definitions and reporting data for AI
A benefits dashboard can count participating families while another report counts individuals.
Industry guide · NAICS 923120
Public health surveillance revision and reporting basis data for AI
A published surveillance count can be revised as information arrives.
Industry guide · NAICS 923140
Veterans program form and instruction version data for AI
An archived instruction may reference a form revision that differs from the current listing.