Skip to content

INDUSTRY FIELD GUIDE

Restaurant prep videos for AI task and action recognition

A useful kitchen clip shows what the cook started with, what they changed, and the result.

By Fiund · Updated 2026-11-05

NAICS 722511 · Full-Service Restaurants

A useful kitchen clip shows what the cook started with, what they changed, and the result. Figure’s Index announcement identifies cooking and restaurant tasks among its capture examples. That is evidence that one robotics company is gathering human task video. An individual restaurant can use a scoped pilot to see whether its own footage meets a specific project brief. Figure Index

Start with a cold prep station

Record one assembly task using reusable training materials or an approved cold station. A cook receives a sample ticket, selects the correct vessel and utensils, portions three items and verifies the result. Capture one alternative order and one labeled correction after a wrong item is selected. Record the order version, ingredient identifiers, timestamps and final match against the mock ticket. Use a stable camera with the hands and work surface in view.

A buyer building a video understanding model might need time ranges for each action. A buyer testing a planning assistant might want the order, the expected sequence and the decision to correct a mistake. Confirm the target first, then decide whether video, event labels, or paired text is needed.

Separate the shoot from food service

Get permission from the operator and everyone you will record. Keep customers, live tickets, payment screens, proprietary recipes and staff schedules out of frame. Stage examples away from active hazards. Your kitchen’s safety and food-handling requirements still apply to a shoot. FDA Food Code

Start by describing one repeatable prep task. Include available camera views, timing, permission scope and the type of variation you can demonstrate. Describe collection capabilities

Original Fiund exercise

Check a practice assembly against the revised ticket

Reusable inert training pieces represent a cold-station task. The target is the correspondence between ticket, visible selection and result, with no real recipe or live cooking.

Fictional records supplied to the evaluator
RecordSourceWhat it records
ORDER-P1Original practice ticketUse bowl Blue and training pieces A, B and C.
ORDER-P2Confirmed revised ticketReplace C with D for the same practice order. Bowl Blue and pieces A and B remain unchanged.
TAKE-P3Observed demonstrationThe take shows Blue with A, B and C, followed by a correction in which C leaves view. The endpoint does not show D.
NOTE-P4Operator intentionA note says the operator intended to complete the revised order, without a final check.

Question for the model

Does the supplied evidence verify the result requested by P2? Describe the observed correction and its remaining gap.

Show the reference answer and checks

P2 requires Blue with A, B and D. P3 supports an initial old-version assembly and the visible removal of C, but it does not show D in the endpoint. P4 records intent and cannot establish the final state. The revised result remains unverified. Request a matching final view or independent check, rather than label the intended sequence as completed.

  • Treating the removal of the old piece as proof that the replacement was added.
  • Using the original ticket or an intention note to override missing final evidence.

Fields to preserve

Order revision
Identify the changed requirement before judging the take.
Action interval
Locate the initial assembly and correction separately.
Observed endpoint
Record the evidence for the final state and unresolved visibility.

Test another case

Use a different vessel and piece-label style with a clear revised endpoint. Test whether the model follows the new ticket rather than the original prop layout.

Use harmless training materials away from active hazards. Exclude customers, live tickets, payment screens, proprietary recipes and staff schedules.

Download this fictional example and answer key (JSON)

Discuss a contribution

Describe the sources you control, the task they could support and what has already been checked. Use the existing-data assessment for an archive or the collection brief for a new recording project. The contributor agreement defines Fiund’s licensing authority and compensation. A guide or fictional exercise does not establish an accepted collection or a sale.

Primary sources and further reading