Data Engineer

• Jan 2020 - Present

● Built ELT and ETL pipelines between internal data and external data stores for the broader data team ● Designed and implemented data storage solutions on Azure, including Azure SQL Database, Azure Cosmos DB, and Azure Blob Storage ● Optimized data processing performance by designing and implementing efficient data models, indexing strategies, and query optimization techniques ● Created and managed data integration solutions using Azure Data Factory, Azure Logic Apps, & Azure Functions ● Built and maintained data pipelines that support real-time data processing and streaming using Azure Event Hubs, Azure Stream Analytics, and Azure Functions ● Implemented data security measures such as data encryption, access control, and data masking to ensure data privacy and compliance with regulatory requirements ● Managed and optimized cloud resources to control costs and ensure optimal performance, including managing virtual machines, storage accounts, and networking resources ● Developed custom data processing solutions using programming languages such as Python ● Implemented data quality checks and data validation procedures to ensure data accuracy and consistency ● Collaborated with data scientists to design and implement machine learning solutions on Azure using tools such as Azure Machine Learning and Azure Databricks ● Designed and implemented data archiving and data retention policies to manage data storage and reduce costs ● Created and maintained documentation for data pipelines, data models, and data integration solutions ● Managed data version control using tools such as Git and Azure DevOps ● Implemented data backup and disaster recovery solutions to ensure data availability and continuity ● Collaborated with stakeholders to understand business requirements and design data solutions that meet their needs ● Involved in building data ingestion pipelines using Azure Data Factory to ingest structured and unstructured data Show less