🤖 Embodied Artificial Intelligence Seminar - Readings and Resources
Welcome to the CS6604 Embodied AI Seminar!
Below is a list of topics we’ll cover during the semester, along with recommended readings, and links to project pages or source code repositories where applicable.
For each paper, click on 📚 for the PDF version and on 🌍 for additional resources.
Topic 1: Benchmarks: Simulators, Environments, Datasets
- ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes 📚 🌍
- iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes 📚 🌍
- Matterport3D: Interpreting Visually-Grounded Navigation Instructions in Real Environments 📚 🌍
- CVDN: Vision-and-Dialog Navigation 📚
- Soundspaces: Audio-Visual Navigation in 3D Environments 📚 🌍
- AI2-THOR: An Interactive 3D Environment for Visual AI 📚 🌍
- Rearrangement: A Challenge for Embodied AI 📚
- Visual Room Rearrangement 📚 🌍
- ProcTHOR: Large-Scale Embodied AI Using Procedural AI Generation 📚 🌍
- ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills 📚 🌍
- Object Goal Navigation using Goal-Oriented Semantic Exploration 📚 🌍
- Embodied Question Answering in Photorealistic Environments with Point Cloud Perception 📚 🌍
- Alfred: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks 📚 🌍
- DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following 📚 🌍
- Alexa Arena: A User-Centric Interactive Platform for Embodied AI 📚 🌍
- VirtualHome: Simulating Household Activities via Programs 📚 🌍
- BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation 📚 🌍
- MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge 📚 🌍
Topic 2: Conceptual Framing, World Models, Behavioral and Performance Metrics
- World Models 📚 🌍
- Machine Theory of Mind 📚
- Recurrent World Models Facilitate Policy Evolution 📚 🌍
- Discovering and Achieving Goals via World Models 📚 🌍
- Planning to Explore via Self-Supervised World Models 📚 🌍
- Learning to Model the World with Language📚 🌍
- Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling 📚
- Dream to Control: Learning Behaviors by Latent Imagination 📚
- DayDreamer: World Models for Physical Robot Learning 📚 🌍
- Mastering Atari with Discrete World Models 📚 🌍
- Masked World Models for Visual Control 📚 🌍
- Structured World Models from Human Videos 📚 🌍
- Building Machines That Learn and Think Like People 📚
- Action and Perception as Divergence Minimization 📚
- Intrinsically Motivated Reinforcement Learning 📚
- Decision Transformer: Reinforcement Learning via Sequence Modeling 📚
- Curiosity-Driven Exploration of Learned Disentangled Goal Spaces 📚
- Encouraging and Evaluating Embodied Exploration 📚
- Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration 📚
- Learning to play with intrinsically-motivated, self-aware agents 📚
- On Evaluation of Embodied Navigation Agents 📚
- ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects 📚
- On the Evaluation of Vision-and-Language Navigation Instructions 📚
- A New Path: Scaling Vision-and-Language Navigation With Synthetic Instructions and Imitation Learning 📚
- Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation 📚
- Iterative Vision-and-Language Navigation 📚
- GRIDTOPIX : Training Embodied Agents with Minimal Supervision 📚 🌍
- On the Limits of Evaluating Embodied Agent Model Generalization Using Validation Sets 📚