AI and Data Engineer

I design and build AI agents, data platforms and the software around them.

About

I have worked in data and AI since 2017, across consulting, fintech, automotive and venture capital. I have led a data team, moved legacy pipelines to lakehouses, and shipped RAG systems and AI agents to production. Today I work independently on agent systems and the data behind them.

Clients and teams
Sapphire VenturesHella GutmannBIT CapitalAUDIMAN10Pearls
Services

How I can help

I take a project from first idea to production, or join your team to unblock the hard parts. You work directly with me.

AI agents and RAG systems

LLM applications that answer from your own data and take real actions. Built with clear structure, testing and monitoring so they hold up in production.

  • RAG
  • Pydantic AI
  • MCP
  • Vector search
  • LLM evals
  • Guardrails
  • Claude API
  • OpenAI API
  • LLMOps
  • MLflow

Data platforms and lakehouses

A modern data platform your teams can rely on. New builds or migrations from legacy warehouses, with clean models and governance from day one.

  • Databricks
  • Snowflake
  • dbt
  • Iceberg

Pipelines and cloud infrastructure

Reliable batch and streaming pipelines, set up as code on the cloud you already use. Repeatable, cost aware and easy to hand over.

  • Airflow
  • Terraform
  • AWS
  • GCP

Technical leadership

Interim data lead, architecture reviews and data strategy. I help small teams make good decisions early and mentor engineers along the way.

  • Architecture
  • Data strategy
  • Mentoring
Results

Selected outcomes

AI and data engineering results from client work and past roles.

AI Engineering
40%

Shorter deployment cycles for production AI models with automated MLOps and CI/CD.

  • Databricks
  • MLOps
  • CI/CD
Sapphire Ventures
AI Engineering
LLMs

Built LLM features into internal data analysis tools to automate routine data questions.

  • LLMs
  • Analytics
Hella Gutmann
Machine Learning
95%

Classification accuracy after improving a neural network model with sparse autoencoders.

  • Deep learning
  • Python
10Pearls
Data Engineering
60%

Faster data system performance after moving legacy pipelines to a modern lakehouse.

  • Databricks
  • AWS
Hella Gutmann
Data Engineering
50%

Less data delivery overhead across 6 teams with a data mesh and clear governance.

  • Data mesh
  • Governance
Hella Gutmann
Platform
3x

Faster deployments across dev and production after reworking the container setup.

  • Docker
  • gRPC
TrueOcean
Expertise

Tools I work with

Python is my main language. I pick the simplest tool that does the job and fits your stack.

AI and LLMs

  • LLMs
  • Generative AI
  • RAG pipelines
  • AI agents
  • Pydantic AI
  • MCP
  • Vector search
  • LLM evals
  • Guardrails
  • Claude API
  • OpenAI API
  • LLMOps
  • MLflow
  • Weights & Biases
  • PyTorch
  • TensorFlow
  • NLP

Data engineering

  • Databricks
  • Snowflake
  • dbt
  • Airflow
  • Spark
  • Kafka
  • Iceberg
  • Trino
  • Nessie
  • SQLMesh
  • Lakekeeper

Cloud and DevOps

  • AWS
  • Google Cloud
  • Azure
  • Terraform
  • Kubernetes
  • Docker
  • GitLab CI/CD
  • Linux
  • Nginx
  • Bash

Languages

  • Python
  • SQL
  • Go
  • Java
  • React
  • GraphQL

Databases

  • PostgreSQL
  • Aurora
  • MySQL
  • MongoDB
  • OpenSearch
  • Elasticsearch

Analytics and BI

  • Superset
  • Power BI
  • Tableau
  • ELK

Architecture

  • Lakehouse
  • Data mesh
  • Data Vault
  • Data modeling
  • Data governance
  • Large scale system design

Leadership

  • Team lead
  • Mentoring
  • Data strategy
  • Stakeholder work
  • Cross team delivery
  • Building new teams
Process

How we work together

Simple and transparent. You always know what is being built and why.

1. Intro call

We talk about your goal, your data and what is blocking you.

2. Clear plan

I send a short plan with scope, timeline and cost.

3. Build

I deliver in small steps you can see and test early.

4. Handover

Docs and knowledge transfer, so your team owns it.

Project

A defined outcome, such as a new pipeline, a migration or an AI feature.

Ongoing

A set number of days each month as part of your team.

Advisory

Architecture reviews and guidance for your own engineers.

Experience

Where I have worked

9+ years across consulting, fintech, automotive and venture capital.

  1. Sep 2024 to now

    Sapphire Ventures · Senior Data and AI EngineerFreelance

    Building AI infrastructure with Databricks and RAG, moving legacy pipelines to a lakehouse, and setting up MLOps with automated CI/CD.

  2. Jun 2023 to Dec 2025

    Hella Gutmann Solutions · Lead Data Engineer

    Led the BI and data warehouse team. Moved pipelines to Databricks and AWS, built a data mesh across 6 teams and added LLMs to internal analysis tools.

  3. Mar 2021 to May 2023

    BIT Capital · Senior Data Engineer

    Owned data solutions end to end. Built the internal data warehouse with Airflow, Snowflake, S3 and PostgreSQL, plus React and Power BI dashboards.

  4. Aug 2020 to Feb 2021

    TrueOcean · Senior Data and Analytics Engineer

    Designed gRPC service communication, ran scalable task orchestration with Cadence and sped up deployments 3x.

  5. Feb 2019 to Aug 2020

    Alten · Data Engineering Consultant

    Planned and built data warehouse and ETL projects for automotive clients including AUDI and MAN.

  6. May 2017 to Nov 2018

    10Pearls · Software Developer and Data Engineer

    Built Python and Java APIs and helped start the first data science team, winning work with clients like AARP and Telmate.

Writing

Latest articles

All articles

Credentials

Certifications

Research

Publications

  • Breast cancer HER2 subtype identification using histopathological imagesJournal of Digital Imaging
  • A robust strategy for automated negotiationsMultiagent and Grid Systems, 2017
  • B.S. Computer ScienceFAST-NUCES, Karachi

Have a data or AI project?

Tell me what you are building and where you are stuck. I will tell you honestly if and how I can help.

Book a 30 minute call

or email hello@faizankhan.me