Home robot
Project Description:
A robotic assistant or agent is designed to assist you in various tasks, particularly those that are most bothersome in your daily life. This initiative draws inspiration from the Rearrangement Challenge, as presented by AI2-THOR and AI Habitat. The primary objective of this challenge is to create a model or agent capable of rearranging objects within a room to return them to a predefined initial configuration.
The project is devided in three: perception, navigation and manipulation.
The perception task involves developing a model capable of utilizing the robot s camera to identify objects and their respective positions. The Manipulation task consists of building a model than successfully grasps an object. For our project we will start with grasping a Mug/Glass
Algorithms:
- - ManiSkill2
- - AI2-THOR
- - AI Habitat
- - ViT
- - SAM (segment anything Model)
- - YOLO
Milestones:
- - Get familiar with the datasets
- - Explore the possibilities to generate more datasets using a 3D environments /Simulation (example: [ProcTHOR](https://procthor.allenai.org/)). change texture(colors), lighting, positions to generate more data
- - Prepare data for training. We will start with only one object and depending on our progress we can add more and more objects.
- - Chose an open-source Model)
- - Train your model, evaluate and iterate
- - (optional) Test your model with real data
Challenges:
- - Real robotic system needed
- - Real parts needed