Experience

MARSHALL TECHNOLOGIES INC

• United States
• IT Services and IT Consulting
• 1 - 100 Employee

• Sr. Data Eng.

  • Sep 2018 - Present

  • Performed performance tuning for Spark Streaming e.g. Setting right Batch Interval time, correct level of Parallelism, selection of correct Serialization & memory tuning. • End to end Critical Implementation (deployment, development, testing) for the Data obfuscation of PCI and PII data using Apache Airflow and Talend involving data across all lower environments (Dev/Test/Staging). • Developed and Implemented ETL Streaming process in Oracle, Redshift using spark-Kafka framework for batch and streaming pipelines for OLAP data using Spark Scala handling schema changes in the data stream. • Implemented Partitioning, Dynamic Partition and Buckets in Hive which resulted in an increase in performance as well as proper and logical organization of data. • Utilized Zookeeper and Spark interface for monitoring the proper execution of the Spark Streaming job. • Lead the data integration aspect of mission critical projects for the enterprise, including hands on delivery • Assists developers, analysts, and designers in conceptualizing and validating solutions that meet business requirements. • Configure Kafka producer with API endpoints using JDBC Autonomous REST Connectors • Designed and Developed ETL Jobs to extract data from Salesforce replica and load it in data mart in Amazon Redshift and Microsoft Sql Server. Show less


Earlham Savings Bank
• United States
• Banking
• 1 - 100 Employee
• Data Engineer

  • Jul 2016 - Sep 2018

UScellular
• United States
• Telecommunications
• 700 & Above Employee
• Data Engineer

  • Jul 2015 - Jun 2016

Deutsche Bank
• Germany
• Financial Services
• 700 & Above Employee
• Data Integration Engineer

  • Jan 2014 - Dec 2014