AI Habitat 2.0

A simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios

Released in: Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Contributor:

Summary

Habitat 2.0 (H2.0) is a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. The platform makes comprehensive contributions to all levels of the embodied AI stack – data, simulation, and benchmark tasks. Specifically, the authors present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850x real-time) on an 8-GPU node, representing 100x speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, prepare groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. The authors find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from ‘hand-off problems’, and (3) SPA pipelines are more brittle than RL policies.

Platform

Images in dataset

2021

Year Released

Key Links & Stats

habitat-sim

@misc{szot2021habitat, title={Habitat 2.0: Training Home Assistants to Rearrange their Habitat}, author={Andrew Szot and Alex Clegg and Eric Undersander and Erik Wijmans and Yili Zhao and John Turner and Noah Maestre and Mustafa Mukadam and Devendra Chaplot and Oleksandr Maksymets and Aaron Gokaslan and Vladimir Vondrus and Sameer Dharur and Franziska Meier and Wojciech Galuba and Angel Chang and Zsolt Kira and Vladlen Koltun and Jitendra Malik and Manolis Savva and Dhruv Batra}, year={2021}, eprint={2106.14405}, archivePrefix={arXiv}, primaryClass={cs.LG} }

scenebox

Modalities

  1. Still Image
  2. Video
  3. RGB-D
  4. 3D Asset
  5. Point Cloud

Verticals

  1. Home/Office

ML Task

  1. Object Detection
  2. Semantic Segmentation
  3. Instance Segmentation
  4. Depth Estimation
  5. Object Recognition
  6. Object Tracking
  7. Scene Understanding

Related organizations

Facebook