Experience

Playkot

• Cyprus
• Computer Games
• 200 - 300 Employee

• Big Data Team Lead/DWH architect/Data Platform owner

  • Jun 2021 - Present

  Working with Data from differents sides. Main achievements: As a PO: * Planned quarterly and semi-annual goals with C-level management * Focused the team on achieving maximum business value * Analyzed similar solutions on the market and looked for their relevant features As an architect: * Collected information of the current situation with data * Designed data platform (general solution for the entire company (marketing, finance and all company products) * Designed a detailed roadmap of the migration process to the new data platform As an engineer: * Implemented a core of computing data platform on spark(scala + SQL API) + airflow + AWS s3 + parquet. * With DevOps, configured spark to work on k8s, as well as whole CD process * Optimized calculations: * spark: read optimization (listing optimization in s3) * spark: write optimization (committer change) * clickhouse: optimization of sorting keys, memory settings * Made a number of lectures for project analysts on theme diving deep into the tech part of data analysis As a team leader: * Assembled a team from scratch (4 date engineers, 2 dwh analysts): * team design * description of roles, vacancies, and assistance to recruiters with search * interview * Built onboarding process * Built a process of regular feedback meetings * Built process of sharing context and knowledge between analysts and engineers As an analyst: * Designed DWH using SCD 2 and layered architecture * Designed and developed a number of business metrics and dashboards * Implemented process of maintaining documentation on analytical events in the same format on all projects. Show less



Innotech

• Russian Federation
• IT Services and IT Consulting
• 700 & Above Employee

• Lead Big Data developer

  • Nov 2020 - Mar 2022

  I was in a tech leader role in a team of 4 engineers and write a lot of code. Tech. stack: - spark via scala + SQL - hdfs, yarn - airflow 1 - hive I work with buiseness analysts - we created data marts about SMB part of VTB. Also I interviewed candidates for DE. I was in a tech leader role in a team of 4 engineers and write a lot of code. Tech. stack: - spark via scala + SQL - hdfs, yarn - airflow 1 - hive I work with buiseness analysts - we created data marts about SMB part of VTB. Also I interviewed candidates for DE.



G5 Entertainment AB

• Sweden
• Entertainment Providers
• 300 - 400 Employee

• Big Data Team Lead

  • Oct 2018 - Nov 2020

  I was the team leader in the DE team. As a manager: -new engineers hiring -setting up education process for teammates -cross projects communications As an engineer: -Creation mini framework for integration with 20 different advertisement partners(like Adwords, Facebook, Adcolony, Unity and etc.) with different API(REST, Google Sheets, pushing to S3, SOAP) in similar way. Stack of technologies: AWS Lambda, AWS CloudWatch, AWS S3, AWS SAM, Java 8, REST, SOAP, Google API -Creation of infrastructure and tools for project analysts. I realized raw data parsing(MapReduce job), enrichment of data(MapReduce + Spark), finalization of target datasets structure(Apache ORC + distribute sorting by optimal fields), metadata creation and updating(AWS Glue), saving and daily updating of result data(AWS S3 + AWS Athena) and orchestration of all these actions(Oozie + EMR), linking R server with AWS Athena(R scripting + JDBC). There are about 2-3 billions records in a daily batch of raw data. - ML: Set up in production ml model(R + Xgboost + Spark(for distribute Xgboost model creation and predicting)) - Daily cluster computation optimization - Hive and Spark tuning, SQL scripts rewriting, migration to columnar binary formats(Apache ORC). Show less



Alfa-Bank

• Russian Federation
• Banking
• 700 & Above Employee

• Big Data Team Lead

  • Jun 2018 - Sep 2018



Sberbank-Technology

• Russian Federation
• IT Services and IT Consulting
• 700 & Above Employee

• Big Data developer

  • Apr 2016 - Jun 2018

  Building a data recording, storage and analytics system in the Hadoop ecosystem using the technology stack: hdfs, flume, kafka, solr, spark, yarn, oozie, hue, mq, ibus, livy, zookeeper, hbase, WebSphere Application server, wildfly. I have experience with hadoop from different sides: 1) Cluster design - server sizing, splitting file systems for various services, creating virtual servers in the openstack environment, preconfiguring servers using ansible 2) Deploying the cloudera cluster - installing the cluster via yum and via tarballs, setting up services via web-ui and cloudera rest api. 3) Cluster administration - incident resolution, cluster rebuilding - adding / removing nodes, migrating services, building your own parcels (parcels), monitoring services using jmx metrics, delimiting resource utilization on the cluster (YARN Resource & Scheduling Policy) 4) Development of application software - development of java applications: a) Client to write to kafka b) Streaming data processing (Spark Streaming + Kafka) c) Batch data processing (Spark) d) Development of a sink for writing to hdfs on a flume e) Development of agents to collect metrics and send to hdfs/solr to build monitoring. f) Development of sql similar search on sequence files in hdfs (apache livy + spark) 5) Working with authentication and authorization mechanisms: kerberos, ipa, ssl(tls). 6) Integration of MOM technologies with hdfs, namely IBM MQ and IBUS(broker) with hdfs. 7) Experience in managing development, implementation and communication with the customer. The development of this system was carried out in a small team (3-4 people), jira was used as a task tracker, bitbucket was used as a version control system, and jenkins was used as a CI environment. Show less