Big Data Engineer

• Oct 2019 - Present

I am currently working as Software Engineer for Santander (bank) Global Tech’s CIO area: -Development of Batch and NRT tools for the DataLake. Main technologies: Cloudera, Scala, Kafka, Spark, Java , Scala, Python, Akka. Previously, I worked in several projects: -Fast-Data Access framework for a consulting firm: Development of reusable and high scalable components for building Stream Processing solutions. Main technologies used: Scala, Spark (Streaming/ Continuous Processing), Kafka, Kafka Streams, MongoDB. -Operational Data Hub for an insurance company: Design of a MongoDB model to optimize the access to data. Design of the Microservices architecture: Main technologies: MongoDB, Java, Spring MVC, OpenShift. -NRT AI model server. Main technologies: Akka Streams, Spark ML. Show less

Big Data Engineer

• Oct 2018 - Oct 2019

I worked in several Big Data on-premise solutions for a bank: Real-Time Processing: Tool for real-time visualization of the bank liquidity (cash). Flink, Hazelcast. Batch: Improve the company’s Data Lake. Main Technologies used: Scala, Spark. I worked in several Big Data on-premise solutions for a bank: Real-Time Processing: Tool for real-time visualization of the bank liquidity (cash). Flink, Hazelcast. Batch: Improve the company’s Data Lake. Main Technologies used: Scala, Spark.

Big Data Engineer

• Mar 2016 - Oct 2018

As Big Data Architect and Developer I am in charge of the design, development and implementation of software solutions for processing large amounts of data, both in real time and batch. I worked in my three areas of expertise: Big Data, Cloud Development and Artificial Intelligence. My work in these areas can be summed up as follows: Big Data: - Development of a proprietary online analytical processing (OLAP) and data mining tool based on AWS (Redshift and DynamoDB). This system included several complex Python (Pandas, Luigi) ETL processes. - Real Time processing tool based on Kafka + Flink. This tool was developed in Scala. In this context a PoC of Apache Beam was developed. Cloud Development: - Development of a Java (Spring MVC/Cloud) based REST API. Part of the backend has been implemented following the microservices architectural style. Implementation of main cloud patterns (like Circuit Breaker), allowing fault tolerance and high availability. Artificial Intelligence: - Churn detection for an insurance company (Python Pandas, Numpy, Scikit-Learn). - Recomender System based on Spark (PySpark/Scala, MLlib). - Several PoCs where frameworks for Recommender Systems were tested (SeldonIO, PredictionIO). - Creation of a tool for datasets and pipelines versioning based on Pachyderm and Mleap. For ensuring the quality of the code, I follow good practices such as Continuous Integration (Jenkins) and Test Driven Development (TDD), Clean Code, and refactoring based on metrics obtained with tools like SonarQube. Show less

Software Architect and Developer in Innovation Projects

• Feb 2015 - Mar 2016

I worked in three areas: Big Data, Cloud Development and Artificial Intelligence. My work in these areas can be summed up as follows: Big Data: -Development of a solution for processing huge amounts of data coming from telecommunication systems making use of Hadoop, Hive, Pig and HBase. Cloud Development: -Development of a REST API (C# and .NET Framework) deployed on Azure. -Processes Automation by means of Azure Worker Roles and Web Roles. -Azure Blobs (HDFS), Azure HDInsight, Azure SQL Data Warehouse. -Big Data POC with Amazon EMR and S3. Artificial Intelligence: -Development of a tool for social networks analysis applying text mining, natural language processing, and machine learning techniques. Show less

Researcher and Sofware Architect

• Mar 2012 - Feb 2015

As a researcher at the Technical University of Madrid I have taken part in several international projects such as: - IMPONET: IMPONET ITEA 2 09030 - TSI-02400-2010-103: Intelligent Monitoring of Power Networks (Ministry of Industry, Tourism and Commerce). Project in which NoSQL Databases (MongoDB, Cassandra and HBase) and Big Data technologies (Hadoop MR) were studied and implemented. -NEMO&CODED ITEA2 08022 IDI-20110864: NEtworked MOnitoring & COntrol, Diagnostic for Electrical Distribution (CDTI). Project focused on distributed data processing by means of a Microservice Architecture. Complex Event Processing (CEP) techniques were also evaluated. As a software engineer I was in charge of the definition of architectures, software development and team management following agile methodologies (scrum) principles. Show less

Web Developer

• Sep 2011 - Oct 2011

Java Engineer

• Nov 2009 - May 2010