Data Scientist

• Jan 2022 - Present

Research Intern (Vision, NLP and Robotics)

• Oct 2020 - Jun 2021

● Designed a data pipeline to collect 50,000 episodes of simulation data for complex robotic tasks from V-Rep (Coppeliasim) implementing inverse kinematics and planning on dual UR5 manipulators using Lua and Python. ● Trained FRCNN on the dataset for potential region proposals in the robot task to extract feature vectors as well as fine-tuned T5 and GPT-2 to create a large corpus of language data for further training. ● Created a novel multimodal attention-based deep learning model which took image proposals from FRCNN and dual-arm UR5 robot trajectories to describe the complex robotic task in Natural Language with an accuracy of 86%. ● Performed various ablation studies and comparisons with a state-of-the-art image captioning model (meshed memory transformer) to show that robot control improves the quality of the robot task description by 18%. Show less