Big Data Engineer

• Jan 2017 - Present

Supporting both Data Science team and Core Dev team Managing the cloud infrastructure in AWS and Heroku Created streaming data flows using Kinesis Streams, used AWS Lambda for transformations of the streams and AWS S3 and ElasticSearch as sink Created data lake with AWS S3 as storage and used AWS Athena to create Data Mart on top of it Created lambda functions in AWS for ETL jobs in Java and node.js. Creating custom code pipelines for Continuous Integration Data Gathering, cleansing and analysis for decision making Training Machine Learning models for designing predictive systems. Evaluating those models and fine tuning for acceptance. Exposing the ML models written in R to other microservice using RestFul Web Services. Containerized deployment using dockers. Supporting the dev team by being involved in the mid-tier and backend development. Testing the REST endpoints for the acceptance using POSTMAN. Creating data pipelines using Kafka. Using DynamoDB, Cassandra as data storage and Redis for Cache. Performing feasibility analysis on the application stack or services. Show less

Spark Developer

• Jun 2016 - Dec 2016

Analyzing terabytes of data using Spark and creating predictive Machine Learning models using Spark Mlib and testing those models for the feasibility. Analyzing terabytes of data using Spark and creating predictive Machine Learning models using Spark Mlib and testing those models for the feasibility.

Big Data Developer

• Feb 2012 - Oct 2015