Senior Consultant

• Apr 2020 - Present

Data Engineer for Fashion industry client : - Pricing project: Creation of pipelines to build Data Gold Layer with features about products, stores, sales and inventory. - Promo project: Creation of pipelines to generate train and inference datasets as part of ML pipelines - Trinity Project: Setting up ETL/ELT frameworks on Databricks using PySpark and Spark SQL on AWS and GCP. - Creation of several pipelines to enrich the Feature Store. Data Engineer for Retail industry client : - Developed pipelines using Azure data factory, Azure data flow to load data in SQL Server. - Developed complex views on SQL Server. - Created triggers to load the data for Reporting Tool in an event based fashion. - Developed scripts using snowsql to load data in SNOWFLAKE Tech stack: - AWS ( AWS Glue, Sagemaker, Athena EMR), - GCP (Google Cloud Storage, Dataproc, Cloud Engine, Vertex-AI). - Azure ( Azure Data Factory) - Big Data Framework: Hadoop-Spark, Databricks, - Database technologies: Bigquery, Snowflake, SQL server, RedShift - Programming and data query languages: Python, PySpark, SparkSQL. - Extras: Airflow, git (GitHub), Jenkins Show less