Kotni Karthik

Databricks Engineer at Tek Star Corporation
  • Claim this Profile
Contact Information
us****@****om
(386) 825-5501
Location
Atlanta, Georgia, United States, GE

Topline Score

Topline score feature will be out soon.

Bio

Generated by
Topline AI

You need to have a working account to view this content.
Join now
You need to have a working account to view this content.
Join now

Credentials

  • Microsoft Certified: Azure Fundamentals
    Microsoft
    Feb, 2023
    - Oct, 2024
  • Tableau Desktop Specialist
    Tableau
    Oct, 2022
    - Oct, 2024
  • Microsoft Certified: Azure Data Engineer Associate
    Microsoft
    Feb, 2023
    - Oct, 2024

Experience

  • Tek Star Corporation
    • United States
    • IT Services and IT Consulting
    • 1 - 100 Employee
    • Databricks Engineer
      • Feb 2023 - Present

      Developed and maintained data pipelines using PySpark for real-time processing and analysis of large-scale datasets. - Implemented Spark Streaming applications for real-time data processing, cleaning, transformation, and aggregation. - Utilized AWS services like S3, Redshift, Glue, and EMR for data storage, processing, and transformation tasks. - Worked with Azure and GCP platforms for data processing and analytics, leveraging services like Azure Data Lake, Azure SQL Database, GCP BigQuery, and GCP Dataflow. - Collaborated with cross-functional teams to gather requirements and design efficient data storage and retrieval models. - Developed and maintained data warehouse pipelines and ETL processes using Snowflake, Azure Synapse Analytics, and Amazon Redshift. - Integrated data from various sources into data pipelines, including APIs, databases, and external systems. - Deployed machine learning models on Spark clusters for real-time predictions and insights. - Developed and maintained data visualization dashboards using Tableau to provide actionable insights. Show less

  • PROSPER INFORMATICS PRIVATE LIMITED
    • India
    • 1 - 100 Employee
    • Big Data Engineer
      • Jan 2020 - Jan 2023

      Analyzed large and critical data sets using Hadoop, HBase, MapReduce, Hive, Zookeeper, and Spark. - Loaded and transformed structured, semi-structured, and unstructured data using Hadoop/Big Data concepts. - Developed Spark scripts and UDFs using Scala and Spark SQL Query for connecting, querying, and writing back to RDBMS with Sqoop. - Utilized Databricks for transforming and manipulating large datasets using Spark's DataFrame or Dataset APIs. - Built and maintained Extract, Transform, Load (ETL) pipelines using Databricks. - Implemented event-driven architectures using AWS Lambda and Apache Kafka for seamless data ingestion and processing. - Worked with NoSQL databases like Cassandra for real-time data storage and analysis. - Collaborated with MLEngineers, data scientists, and business analysts to design and implement custom data models and analytics. - Developed and maintained serverless applications using AWS Lambda, DynamoDB, and S3. - Designed and built scalable, cloud-based data lakes on AWS and GCP using S3, Glue, and Dataflow. - Worked in DevOps environment and performed CI/CD deployments. - Designed and implemented cloud-based solutions in Azure, including Azure SQL database, Azure Data Factory, and Azure Analysis Services. - Utilized Terraform for managing cloud resources and scaling Spark clusters dynamically. Show less

  • HCL Technologies
    • India
    • IT Services and IT Consulting
    • 700 & Above Employee
    • Software Engineer
      • Mar 2017 - Jan 2020

      - Installed and configured Hadoop, MapReduce, HDFS, and AWS for multiple MapReduce jobs in Hive. - Implemented Spark using Scala and Spark SQL API for accelerated data processing. - Developed Nifi data types in a Docker container environment. - Designed data models for ODS, OLTP, and OLAP systems using ER Studio. - Installed and configured multi-node clusters in the cloud using Amazon Web Services (AWS) on EC2. - Wrote UNIX scripts and automated ETL processes using UNIX scripting. - Consumed and processed data from various sources using Hadoop, MapReduce Frameworks, HBase, and Hive. - Worked on batch processing using Apache Spark and ElasticSearch. - Ported PIG scripts and MapReduce programs to Spark Data framework APIs and Spark SQL APIs. - Created external Hive tables and implemented data pipelines using Oozie. - Worked with NoSQL databases like HBase for real-time data analysis. - Collaborated with stakeholders for data mapping, planning, and auditing. - Used Sqoop for importing data into HDFS/Hive from DB2. - Developed scripts for data import using Talend. - Handled data transfer from servers to HDFS using Apache Flume. - Improved Pig scripts for data analysis and refined data storage in DB2. - Worked on Hadoop log management and auditing. - Participated in project team meetings, provided technical support, and solved test problems. Show less

  • UV Infratech Systems Private Limited
    • India
    • 1 - 100 Employee
    • Associate Data Engineer
      • Dec 2014 - Feb 2017

      Participated in the software development life cycle (SDLC) and followed agile methodologies. - Developed logical and physical database models using ER Studio. - Worked with SQL, Oracle PL/SQL, and stored functions for data access and analysis. - Imported and cleaned data from various sources like Teradata, Oracle, flat files, and SQL Server. - Created DDL scripts using ER Studio for importing destination maps. - Developed dimensional data models using ER/Studio and Oracle databases. - Analyzed existing source systems and maintained metadata for effective data analysis. - Used Informatica for loading tables and implemented business data pipelines. - Developed and maintained data integration programs for Hadoop and RDBMS environments. - Wrote SQL queries and worked with SQL Server and Teradata. - Reverse-engineered DB2 and Teradata databases using ER Studio. - Analyzed data sources in JAD sessions and communicated data standards. - Prepared design processes for data phase objects and performed data mapping. - Worked on importing and cleaning data from multiple sources. - Created physical data models for Oracle and Teradata. - Collaborated with DBAs for logical-to-physical data model conversion. - Worked with ETL tools like Talend and Flume for data import. - Designed and maintained data warehouses using ER/Studio and SQL queries. Show less

Education

  • University of Hertfordshire
    Master's degree, Artificial Intelligence
    2020 - 2022
  • Visakha technical campus
    Bachelor's degree, Mechanical Engineering

Community

You need to have a working account to view this content. Click here to join now