Atomic Unit of Human-1M: Episode Requirements

Content

Each episode captures a proficient demonstrator executing economically useful physical activities in pursuit of a discrete goal.

• Economically Useful: The task has clear commercial value. It represents a specific job function or service for which a human would typically be compensated.
• Physical Activity: The work relies primarily on manual dexterity, bodily exertion, or the operation of tools and machinery, rather than cognitive, clerical, or administrative duties.
• Proficient Demonstrator: The individual executes the task with the competence, efficiency, and skill expected of a trained, paid professional in that specific role.
• Discrete Goal: A specific, measurable objective with a clear beginning and a defined end state, usually capable of being fully achieved within 5 to 10 minutes.

Video

• Minimum 30 FPS / 480p.
• The hands and task-relevant objects must remain sufficiently visible and in focus.
• Unaltered video speed.
• At least 95% of the episode with one or more hands in frame.

Task annotation

A 1–2 sentence high-level summary of the episode. It captures the task’s primary goal, the major actions required to achieve it, and the key objects involved.

• Primary Goal: The ultimate outcome or end state the demonstrator is working to achieve. Omitting it would leave the core purpose of the video unclear.
• Major Action: An essential step in the process. Omitting it would underrepresent or misrepresent how the task is actually completed.
• Key Object: An essential item, tool, or material being used. Omitting it would fail to capture what is required to complete the task.

Example:

The person prepares a sandwich by slicing bread, adding fillings, and placing it on a plate.

Subtask annotation

A contiguous sequence of 1–10-second segments spanning the length of the episode, breaking down the video’s primary goal into step-by-step actions. The guiding principles for subtask annotations are:

• Action Intent: Each segment is bounded by visual action-intent, a physical cue in the ego view that indicates the intention to begin or complete a task. Segments should begin at the earliest observable action-intent (usually hand movement toward an object) and end at the latest action-intent (usually hand/object disengagement) or the next discrete action-intent.
• Hand Laterality Specificity: All task-relevant hand-object interactions are annotated with explicit hand laterality to eliminate ambiguity about which hand performs each action (e.g., “hold the statue with the left hand while wiping with the right hand”).
• Specific, Contextually-Appropriate Verbs: Actions are described using specific, contextually-appropriate verbs to precisely capture the observed motion (e.g., “unscrew the lid” instead of “open the lid”).
• Unambiguous Targets: Objects of engagement are described using the minimum amount of detail necessary to disambiguate them from other objects in the environment (e.g., “the brown lid” instead of “the lid” when there are three).
• Self-Contained Segments: Each segment is annotated as an independent unit, fully interpretable without reference to preceding or subsequent segments.