Senior Architect & Senior Backend Engineer

• 2012 - Present

Led and architected Carol.ai, an auto scalable, fault tolerant, big data and machine-learning platform written in Java. Highlights include: Fault-tolerance in auto-scaling environment: able to scale up or down without data loss or interruption to data processing. When preemptible instances were shutdown, Java workers will resume data processing after reconnecting to NATs streaming server. Horizontally-scalable data platform: able to scale data volume to petabytes with minimal supervision. Auto-scaling nodes in Elasticsearch cluster is based on disk and CPU watermarks. Master Data Management system with highly performant ETL: configurable ETL rules are compiled by Apache Velocity template engine into concise Java byte code and are executed during runtime without further parsing, leading to optimal performance. Flexible and powerful REST APIs: Swagger interface on top of Dropwizard framework enable developers and data scientists to build models, host applications and run custom algorithms on Carol.ai. Embedded JupyterLabs and custom plugins: Data scientists are able to create models, install packages and build Docker images to run on Carol.ai. These online or batch images are executed by APIs written in Go over KubeBuilder framework. Show less

Senior Backend Engineer

• Mar 2012 - Nov 2012

Built a python Django backend for a iPad catalog product to support a smooth viewing experience. Highlights include: Built a supervised machine learning model over Apache Mahout to make recommendations based on user activity, such as catalogs visited and the time users spend on a page. Mined data from Google Analytics which is used to create daily business intelligence reports. Built a python Django backend for a iPad catalog product to support a smooth viewing experience. Highlights include: Built a supervised machine learning model over Apache Mahout to make recommendations based on user activity, such as catalogs visited and the time users spend on a page. Mined data from Google Analytics which is used to create daily business intelligence reports.

Chief Data Officer & Senior Backend Engineer

• Oct 2000 - Apr 2012

Led and architected FoxHound, a crawler matrix that captures essential product information from all online retail websites. Highlights includes: Architected a highly elastic and scalable crawler matrix: All FoxHounds crawlers are light weight custom chrome plugins running on short-lived AWS spot instances, ranging from minutes to an hour, make them harder to be blocked or traced. FoxHound server was written in Java using Hibernate and spring data over Oracle databases. Architected and built a price intelligence data platform: The platform processes billions of scraped records daily, creating price intelligence reports for hundreds of clients including Amazon, Walmart, Home Depot, Lowes and other online retailers. This batch pipelines were written in Java and deployed over Hadoop cluster. Show less

Sr. Software Engineer & MSSQL Administrator

• Apr 1998 - Apr 2000

Built an insurance fraud detection platform that supports hundreds of field adjusters in real time. Designed data models and triggers that are capable of identifying suspicious claims between claimants, clinics and attorneys in real-time. Built an insurance fraud detection platform that supports hundreds of field adjusters in real time. Designed data models and triggers that are capable of identifying suspicious claims between claimants, clinics and attorneys in real-time.

Programmer Analyst

• Mar 1996 - Apr 1998

Designed, architected and built the company's flagship product, Maintenance Director, an aircraft logbook system which is used to track airworthy parts in an aircraft. It was commercially successful and was used by airlines such as Air Canada and HP executive private jets. Designed, architected and built the company's flagship product, Maintenance Director, an aircraft logbook system which is used to track airworthy parts in an aircraft. It was commercially successful and was used by airlines such as Air Canada and HP executive private jets.

Research Assistant

• Jun 1994 - Dec 1994

Software Engineer

• Sep 1990 - Jul 1992