Diligence & quality
How to evaluate a training-data vendor
A repeatable way to judge a vendor on provenance and rights auditability, not just samples and turnaround.
- Topic
- Diligence & quality
- For
- AI data buyers
- Reading time
- 7 min
- Updated
Evaluate a vendor’s evidence for your actual requirement, not just its best sample or overall catalog size. A small reviewed subset can be more useful than a large total whose sources, units and restrictions are unclear.
Use the same questions for Fiund and other suppliers. This guide is published by Fiund and is not independent certification or a ranking of vendors.
Request a traceable source package
Ask for source categories, stable asset IDs, counted units, collection method and known restrictions. Distinguish vendor statements from underlying records and actual tested files. Resolve who can authorize the proposed use.
A sample should connect to the described collection rather than be an unrelated showcase. Ask how it was selected and what difficult or incomplete material it omits. Do not assume a public demo represents the entire distribution.
Inspect technical and annotation evidence
Request original formats, timeline and sensor records where relevant. For labels, record the source, schema, software and human-review status. A model estimate and measured reference are different deliverables.
Use the synthetic package linked below only to understand file checks and manifest structure. It does not demonstrate a supplier’s real capture quality, privacy preparation or task coverage. A real pilot is still needed for those questions.
Run a reproducible pilot review
Agree the tested subset and criteria before checking. Record loading results, missing files, duplicates, relevant quality findings and permission holds. Tie decisions to package version and checksums.
Keep pass, partial pass, not tested and unresolved outcomes distinct. The buyer should know which fixes are required and who bears rework. A successful upload is not delivery acceptance or proof of model improvement.
Compare the complete proposal
List licence, curation, annotations, review, delivery, refresh and any minimum scope separately. Normalize accepted activity or assets rather than comparing incompatible headline units.
Review permitted use and source-specific restrictions. Fiund standard licences are non-exclusive and prohibit synthetic voice or likeness replication. Confirm actual availability and commercial terms after a brief instead of treating this checklist as an offer.
Frequently asked questions
Does this guide confirm a dataset is available?
No. Availability, permissions, included preparation and commercial terms require review of the actual proposed package.
Ready to scope a supplier evaluation?
Send an evaluation brief →Related resources
5 min read
A pilot acceptance plan for robotics data
Write acceptance, source-aware split boundaries and rework rules before expanding a robotics collection.
7 min read
Datasheets for datasets: a buyer’s template
Turn the academic datasheet idea into a practical diligence form. Blank fields become findings.
Related reading
Testable package walkthrough
5 min read
How to inventory an audio or video dataset
Build a file-level inventory before estimating volume, requesting a quote, or preparing a dataset for review. Count known durations and record the technical properties buyers need.
5 min read
Recorded hours vs usable dataset hours: how to count media volume
File-hours, unique activity-hours, speech-hours and accepted hours answer different questions. Define the unit before comparing collections or recording offers.
5 min read
How to write a dataset card buyers can evaluate
Document the dataset’s purpose, origin, collection process, limitations and permissions evidence. A useful card makes unknowns visible instead of hiding them behind marketing language.