The RAI institute's Perception team is focused on building the next generation of intelligent scene models that are grounded in the physics of the real world. This involves deconstructing scenes into fundamental components (geometry, semantics, physics, dynamics) and using this understanding to power generative world models and simulators that can predict future scene evolution. The goal is to enable future generations of intelligent machines that can perceive, reason about, and interact with the physical world.
Requirements
- Strong programming skills in Python and/or C++ and deep experience with a major deep learning framework (e.g., PyTorch).
- Experience applying research to robotic systems (e.g., using ROS 2, integrating models on physical hardware).
- Deep expertise in one or more of the following areas: (1). 3D Scene Understanding: (e.g., NeRFs, 3D Gaussian Splatting, 3D reconstruction, geometric deep learning). (2). Generative Modeling: (e.g., Diffusion Models, VAEs, LLMs, Multimodal Models) applied to 3D or dynamic data. (3). Physics-Informed AI: (e.g., Differentiable simulation, Physics-based modeling, System identification).
- Experience with large-scale model training, including distributed training techniques
- Familiarity with MLOps workflows and tools (e.g., Docker, cloud computing) for large-scale experiments
- Familiarity with causal representation learning or related techniques.
Responsibilities
- Research, design, and implement novel algorithms for understanding and modeling complex 3D scenes, with a focus on integrating geometry, semantics, and physics
- Develop and advance generative and physics-informed models capable of predicting realistic future states and dynamic interactions for applications like robotic manipulation
- Design and curate large-scale datasets to enhance model capabilities in physically-grounded and specialized applications
- Create robust 3D representations that capture not just object appearance, but also physical properties, affordances, and causal dymanics
- Incorporate principled uncertainty estimation into your models to ensure they are reliable and aware of their own limitations
- Collaborate with a multidisciplinary team of researchers and engineers to integrate your work into larger systems and build compelling demonstrations
- Share your innovative work with the community through publications at top-tier conferences (e.g., CoRL, CVPR, NeurIPS)
Other
- A PhD, or MS with equivalent research experience in Computer Science, Robotics, or a related field
- 5+ years of relevant post-graduate research or industry experience in robotics, computer vision, computer graphics, or machine learning
- A proven track record of creativity and research excellence, demonstrated by publications in leading ML and Robotics conferences.
- Mentor junior researchers and engineers, fostering a culture of technical excellence and collaborative innovation
- The latitude to pursue ambitious, long-term research goals in a culture that prioritizes scientific discovery
