The Meta Reality Labs Research Team is seeking to research and build the egocentric machine perception functionalities and digital twins of 3D scenes that will underpin future VR and contextual-AI-enabled AR devices.
Requirements
- Knowledge and hands-on experience working with 3D and projective geometry, in addition to image-space computer vision
- Hands-on experience implementing 3D computer vision algorithms as well as end-to-end training of ML models from data collection and design to evaluation
- Experience working within Python environments such as pytorch
- Experience working in a Unix environment
- Programming experience using python and pytorch.
- Basic experience with c++ codebase
- Experience with non visual sensors for location estimation and scene understanding : inertial, GPS, magnetometer, microphones, etc
Responsibilities
- Plan and execute cutting-edge research and development to advance the state-of-the-art in machine perception, state estimation, reconstruction of 3D objects and scenes.
- Collaborate with other researchers and engineers across machine perception teams at Meta to develop experiments, prototypes, and concepts that advance the state-of-the-art in AR/VR and AI systems.
- Work with the team to help design, setup, and run practical experiments and prototype systems related to large-scale high quality sensing and machine reasoning.
- Research novel object/scene reconstruction method to recover the physical properties or functionality of real-world objects/environments with egocentric machine perception.
- Build up the related dataset for physical scene understanding.
- Egocentric human states estimation and motion understanding with multimodal data.
- Egocentric 3D reconstruction and 3D scene understanding (3D segmentation, and 3D bounding box detection) of static and dynamic scenes.
Other
- Currently has, or is in the process of obtaining a PhD degree in the domain of computer-vision, 3D machine perception or machine learning, multimodal sensor fusion
- Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment
- 24 week internship availability
- Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as CVPR, ECCV/ICCV,, ICLR and NeurIPS, RSS, IROS
- Intent to return to a degree-program after the completion of the internship
