Data Engineer

• Jul 2022 - Present

• Designing, developing, and testing pipelines using Python SDK that transform and process data from JDBC/HTTP source to ADLS to Snowflake for a full table and incremental snapshots. Automation of pipelines with CI/CD using Azure DevOps. Developed pipelines for batch and streaming scenarios. •Developed ETL pipelines using Glue Jobs, flask backend for an entity resolution MVP • Designing, developing, and testing pipelines using Python SDK that transform and process data from JDBC/HTTP source to ADLS to Snowflake for a full table and incremental snapshots. Automation of pipelines with CI/CD using Azure DevOps. Developed pipelines for batch and streaming scenarios. •Developed ETL pipelines using Glue Jobs, flask backend for an entity resolution MVP

Data Scientist

• Apr 2022 - Jul 2022

Senior Software Engineer

• Sep 2019 - Dec 2020

Worked as full stack developer for a client project at BHEL, one of the largest suppliers of Thermal power plant equipment and components in India. Involved at multiple stages in the development of our product, Remote Plant monitoring and management system, which enhances operations and maintenance of steel and thermal power manufacturing plants through artificial intelligence. Mainly responsible for the development and design of the User interface and providing the user with a seamless experience to visualize and gain key insights from data, moreover built server side micro services that serve data required by the user interface. Show less

Software Engineer

• Sep 2014 - Sep 2019

Worked as a Senior Engineer for a client project(View Glass Inc, USA). Following projects were undertaken for this client: 1. Resource Manager Application, a component which can be used to create and manage users, the resources the user has access to (for example: sites/zones/applications) and the privileges for those resources(for example: read/write/delete) 2. Analytics Platform, a rule engine that implements a set of rules and raises an alarm/exception if an anomaly is detected in the process. 3. Containerization, a project to standardize the build and deployment process of various applications to be scalable and reliable using docker, docker swarm and Nginx. The other project that was undertaken was for another client from Japan, MTES Neural Networks. 1. Image Server Application, an application for processing and storing image sent from end devices through MQTT Projects undertaken within organization: 1. Involved in building an admin GUI, where the IoT platform statistics such as number of devices connected, message sent/ recieved can be monitored. 2. Responsible for building a connector application, which is supposed to serve as a SDK to access LDAP, Bugzilla and few other open source APIs. Show less