Data Engineer

• Nov 2022 - Present

Data Developer

• Jan 2021 - Nov 2022

●Creating Common and reusable framework in ADF and Migrating the Existing ETLs (built using Data Stage) into ADF, which helps in meeting 80% of data needs ●Building re-usable Pipelines for extracting the data from ADLS(Azure Data Lake) to enterprise data pond using ADF, Data bricks etc. ●Work with Data modeler/BA in understanding the requirement and propose solutions ●Designed and Build number of ETL/ELT pipelines for Data ingestions. ●Take part in Safe Agile PI planning and provide estimates ●Work with data consumers, gather requirement and provide solution ●Responsible for code migration using Azure DevOps and GIT Hub. Show less

Big Data Developer

• Apr 2017 - Nov 2020

●Co-ordinated with business analysts and different interfacing team leads to gather requirements and prepare technical design documents●Designed and implemented number of Business logic in Hive.●Took part in project planning and provide estimates●Understood source systems and Analyze data to provide a design solution●Designed unit test strategy and review code and unit test cases to ensure positive, negative and boundary test scenarios are properly tested and documented●Supported business testing phase by closing gaps between business understanding and code generated. ● Managed offshore team and responsible for data lake releases and delivery. Show less

Big Data Administrator

• Sep 2013 - Mar 2017

●Created an ETL integration Toolkit to standardize, to integrate and to streamline data collection●Understood functional and technical requirements and performed POC to understand different source system and data extraction methods●Provided solution to client by extracting data from multiple utilities specific sources(GIS, Maximo and OSI PI) for analysis and smart decisions.●Identified issues/risk and impact on existing ETL toolkit functionality and Provide a design solution with minimal or no impact on the existing toolkitCreated ETL jobs and Python scripts for extracting data from various sources. Show less

Data Analyst

• Mar 2013 - Aug 2013

●Developed ETL code using Data Stage, SQL and Unix to cleanse and aggregate data based on report requirement ●Created a SQL and script-based reconciliation model which can validate source and translated target data ●Performed performance tuning by optimizing SQL queries and by tuning Data Stage jobs ●Supported SIT, UAT, and responsible for production deployment and warranty support ●Understanding the functional requirements and do the analysis of the requirements to assess feasibility of development through the Data Integration Toolkit and implementing the processes and logic to extract, transform, and distribute data across one or more data stores from a wide variety of sources ●To do the root-cause analysis of design and datasets from the business, to identify the critical inconsistencies. ●Risk audits, use issue logs, facilitate conflict resolution, and create forecasts. ●Measure the performance and provide tuning methodologies if required. Prepare unit testing scenarios and test cases to analyze the correctness of data. Show less