Engineering lead -Data

• Jan 2023 - Present

Data Engineer II

• Jun 2021 - Jan 2023

● Data mining and curation to meet analytics and data intelligence requirement using airflow, GKE, python ● Implemented data compliance according to Federal data regulations ● Lift and shift data from traditional source like MySQL, oracle to google big query ● Data ingestion and transformation from on premise Hadoop to google cloud (GCS- Big query) using Scala ● Enabled Airflow DAG alters for existing pipelines for success/failuer states ● Back filling of data in case of data loss. Data segment repair ● Uploading data from Avro, parquet, csv to big query sink ● Ingested parquet ,JSON,csv files using pyspark into hdfs ● Ingested data from mysql using JDBC connection and pandas into hdfs ● Created ETL for revenue analytics team using rasvelg job on VERTICA Show less

Architect - Big data (Full Time from cognizant)

• Jan 2021 - Jun 2021

● Data modeling, data Extraction (claims, members, provider, cases) ● Data Ingestion/Data Curation using spark Scala from different sources. ● Reading parquet /csv and loading it into hive ● Creating framework for file movement, spark job submission using Unix shell script ● Creating hive Tables, views using Hue (Cloudera) ● Querying analyzing data from non-relation database HBase ● Data modeling, data Extraction (claims, members, provider, cases) ● Data Ingestion/Data Curation using spark Scala from different sources. ● Reading parquet /csv and loading it into hive ● Creating framework for file movement, spark job submission using Unix shell script ● Creating hive Tables, views using Hue (Cloudera) ● Querying analyzing data from non-relation database HBase

Data Engineer (Full Time from cognizant)

• Nov 2015 - Dec 2020

● Data Modeling, Data Ingestion, Data curation using spark/scala ● Create and support Data engineering workflow (ETL) using SparkSQL, HQL ● Data ingestion using various file types including Json, CSV, parquet ,Avro in hsfs sink ● Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run MapReduce jobs in the backend ● Strong experience in analyzing large amounts of data sets writing PySpark scripts and Hive queries ● Developed python script to find long running jobs on YARN, monitor health of head and worker node of spark and Hadoop cluster ● Involved in Analyzing system failures, identifying root causes, and recommended course of actions, Documented the systems processes and procedures for future references. ● Good experience in using Data Modelling techniques to find the results based on SQL and PL/SQL queries ● Experience working with different databases, such as Oracle, SQL Server, MySQL and writing stored procedures, functions, joins, and triggers for different Data Models. ● Good knowledge of Apache NiFi to automate data movement between different Hadoop systems. ● Good knowledge of Azure data factory ● Good knowledge of REST services using python ● Good knowledge of AWS EC2 ,Elastic cache ,EMR, Load balancing services Show less

Architect(Full Time from cognizant)

• May 2013 - Oct 2015

 Design and development for Peoplesoft Strategic source module implementation  Implemented requisition workflow using custom application classes  Integration with vendor (Office Depot, Target) fulfillment services using REST webservices (PeopleCode)  Uploading various data sources (spreadsheet, comma separated files) to Oracle database  Performance tuning for existing pipelines written in Application Engine and SQR  Design and development for Peoplesoft Strategic source module implementation  Implemented requisition workflow using custom application classes  Integration with vendor (Office Depot, Target) fulfillment services using REST webservices (PeopleCode)  Uploading various data sources (spreadsheet, comma separated files) to Oracle database  Performance tuning for existing pipelines written in Application Engine and SQR

Peoplesoft Technical Lead (Full Time from cognizant)

• May 2011 - May 2013

 Technical and Functional Team Lead of PeopleSoft supply chain applications  Extensively worked on implementation, support and designing of business processes in the General Ledger  Facilitated requirement gathering sessions. Created Technical and functional design documents and implementation plan  Code Review and effort estimation for enhancements  Managed a 6-member team  Technical and Functional Team Lead of PeopleSoft supply chain applications  Extensively worked on implementation, support and designing of business processes in the General Ledger  Facilitated requirement gathering sessions. Created Technical and functional design documents and implementation plan  Code Review and effort estimation for enhancements  Managed a 6-member team

Sr Software Engineer

• Oct 2006 - Apr 2011

 Felicitated requirement gathering sessions. Created Technical and Functional design documents  Technical design preparation for HSBC banks general ledger system  Extensively worked on implementation, support and designing of business processes in the General Ledger  Build deployment and post production support  Felicitated requirement gathering sessions. Created Technical and Functional design documents  Technical design preparation for HSBC banks general ledger system  Extensively worked on implementation, support and designing of business processes in the General Ledger  Build deployment and post production support

Programmer Analyst

• Mar 2005 - Oct 2006

 Developed custom solution for Enterprise portal  Prepared technical design documents  Prepared unit test cases and ran positive and negative unit test cases  Build Deployment and post deployment monitoring and support  Developed custom solution for Enterprise portal  Prepared technical design documents  Prepared unit test cases and ran positive and negative unit test cases  Build Deployment and post deployment monitoring and support