Resources
Agentic trajectory and computer-use datasets, explained
Agents learn from recorded task sequences: what was on screen and what the user did. Here is what a trajectory contains, and why consent and PII decide whether you can use it.
Updated 2026-10-27 · 6 min read
An agentic trajectory links observations, actions and outcomes for a task in software. A screen recording alone may show behavior without recording the action schema or environment needed to replay and evaluate it.
Describe whether the source is a human workflow, an automated agent, a simulation or an edited demonstration. Keep native observations separate from inferred clicks, generated descriptions and success labels.
Define the observation/action record
An illustrative step schema can contain episode_id, step_id, timestamp, observation_ref, action_type, action_parameters, environment_version and outcome_status. These are planning fields, not an available Fiund dataset.
Specify action semantics: a click’s coordinate frame, a key event’s encoding, a browser target or an application command. Do not silently mix physical screen pixels and normalized coordinates. Keep any screenshots and logs linked to the actual step.
Document the environment and boundaries
State web, desktop or mobile environment, versions, accounts and reset procedure. Identify dependencies such as network state, permissions and starting files. An apparently successful video does not demonstrate reproducibility.
Record interruptions, failures, retries and whether the outcome was verified or merely reported by the operator or model. Keep raw events and edited derivatives distinct; an inferred action should not be presented as native telemetry.
Control sensitive information
Use an authorized environment and reviewed tasks. Screens, typed content and hidden fields can contain credentials, personal information or confidential business material. Establish a process for access, exclusion and review before capture.
A synthetic workflow can demonstrate schema without proving real-world performance. State its origin and limits. Do not reuse customer production accounts or claim permission merely because a worker can access them.
Evaluate a pilot and source-aware splits
Test parsing, observation/action alignment, environment setup and outcome checks. Group related workflows and repeated tasks deliberately when defining evaluation boundaries so derivatives do not leak into both sides.
Fiund confirms source availability, permitted use and deliverables after a brief. A public schema or format name is not a claim of an existing collection, reproducible benchmark or open paid capture assignment.
Frequently asked questions
Does this guide confirm a dataset is available?
No. Availability, permissions, included preparation and commercial terms require review of the actual proposed package.