Data Infrastructure Shouldn't Be Your Bottleneck. I Build Systems That Work.

I build modern data pipelines and automate processes that deliver three measurable outcomes: faster operations, higher accuracy, and reduced costs—using the best tools for your specific needs.

Let's Talk See My Work

"Lior doesn't just complete tasks, he digs deeper and often uncovers insights that others might miss. His ability to take initiative, solve complex problems, and see projects through to completion made a real impact on our team."

— Andrew Regen, Executive Director (Head of Enterprise Data), Lukka

7+ Years Experience
20+ Projects Delivered
100% Client Satisfaction
Scroll to explore

About Me

Modern Data Engineering for Business Impact

Lior Gefen - Senior Data Engineer

I solve data infrastructure problems that cost companies time and money. Most organizations struggle with manual processes, unreliable systems, and data workflows that don't scale. I build modern solutions that eliminate these bottlenecks.

My approach combines technical depth with business pragmatism. I work across the full data engineering stack—from API integration and ETL pipelines to cloud-native orchestration and streaming architectures—always focused on delivering measurable outcomes.

Recent Impact

Smith+Nephew (10,000+ employees)

Eliminated 8-9 FTEs through data engineering and automation across multiple business units. Built ETL pipelines, Power BI dashboards, and automated cross-team processes that enabled teams to focus on analysis rather than manual data work.

In recent roles at technology companies:

  • Modernized data orchestration platforms using Dagster, enabling self-service capabilities and reducing operational bottlenecks
  • Deployed Dagster on cloud-native infrastructure (Docker, Kubernetes)
  • Implemented event-driven data architectures improving reliability and processing efficiency
  • Designed configuration-driven systems enabling teams to operate more independently

What I Do

I'm a full-stack data engineer specializing in:

  • ✓ Building modern data pipelines (ETL, orchestration, streaming)
  • ✓ Automating manual processes to reduce operational costs
  • ✓ Implementing self-service capabilities that eliminate bottlenecks
  • ✓ Migrating legacy systems to modern architectures

My Background

PhD in data analysis + 7 years building data infrastructure across healthcare, fintech, crypto, and legal tech.

I've worked with 10,000+ employee enterprises and fast-growing scale-ups, delivering solutions that reduce costs, improve accuracy, and scale with growth.

Core Tech Stack: Dagster • Python • PySpark • AWS • Azure • PostgreSQL • Kafka • Docker/K8s • SQL

📍 Based in Panama, available globally (US/EU timezone overlap)

Aug 2025 - Present

Owner & President

Time of Data

  • Providing consultancy and advisory services for data engineering projects
  • Dagster pipeline implementation and optimization for clients
  • Best practices guidance for data orchestration and ETL workflows
  • Architecture design for cloud-based data solutions
  • Technical mentoring and team enablement
  • End-to-end data engineering solutions from design to deployment
Dagster Python PySpark AWS Azure Docker Kafka
Jan 2023 - Sep 2025

Senior Data Engineer

Lukka

  • Dagster implementation for data orchestration across multiple projects
  • Building ad-hoc ETL tools for complex data workflows
  • Docker/Kubernetes deployment and container orchestration
  • Streaming data pipelines with Kafka
  • Spark/PySpark on serverless Jupyter notebook solutions
  • Mentoring junior engineers and peers in Python, Dagster, and best practices
Python Dagster Docker Kubernetes Kafka PySpark Apache Polars Ibis
Dec 2022 - Dec 2023

Software Engineer

Climate Ireland

  • Built web scraper to collect news articles about environmental topics
  • Created necessary storage infrastructure in AWS
  • Implemented data pipelines for environmental data processing
  • Designed and deployed scalable cloud-based data solutions
Python AWS Web Scraping Data Pipelines
Feb 2022 - Dec 2022

Data Engineer

GFT Group

  • Architected Azure Data Factory pipelines for enterprise data warehouse (13-18 person team)
  • Led Azure Purview proof-of-concept; contributed to rollout implementation for data governance
  • Learned Terraform from scratch to collaborate meaningfully with DevOps team on IaC
  • SQL database development and optimization
Azure Data Factory SQL Terraform Azure Purview
May 2021 - Jan 2022

Data Analysis and Engineering

Factor Law (Legal Tech Startup)

  • Designed initial architecture for data warehouse supporting legal operations analytics
  • Client data analysis for legal touch-time estimation and process optimization
  • Built data ingestion pipelines with comprehensive logging and notifications
  • ETL tool development using Azure Data Factory and Python
  • Power BI dashboard development and automation tools (improved upon legacy VBA systems)
SQL Python Azure Azure Data Factory Power BI VBA
Jun 2018 - Apr 2021

Reporting Analyst

Smith & Nephew

  • Eliminated 8-9 FTE worth of manual work through automation across Order-to-Cash, Procure-to-Pay, and Record-to-Report teams
  • Built automated SAP data extraction and ETL pipelines for data warehouse project
  • Developed fleet management tracking system: end-to-end solution with Python ETL, MSSQL backend, Power BI dashboards
  • Created Power BI dashboards serving S&N executives and global business services teams
  • Self-taught C# to improve upon VBA automation solutions
Python VBA C# SQL Power BI SAP
Mar 2017 - May 2018

Collections Specialist

DXC Technology

Early career role in financial operations that demonstrated automation mindset— a pattern that defined my subsequent career. Even in a non-technical role, I identified and implemented automation opportunities.

  • Collaborated with Order-to-Cash Tower Lead to create global reporting systems
  • Automated global reports, receiving formal recognition for initiative and impact
  • Improved order release and Bill of Exchange processes for French operations

Education

🎓

Ph.D. in Jet Aeroacoustics

University of Roma Tre

2013 - 2017 | Rome, Italy

Development and application of advanced post-processing techniques for jet noise sources investigation. Specialized in wavelets, POD, and LSE techniques.

Publication: "Vortex dynamics and sound emission in excited high-speed jets" - Journal of Fluid Dynamics

🎓

M.Sc. in Mechanical Engineering

University of Toulouse III

2011 - 2013 | Toulouse, France

Specialization in Fluid Dynamics

🎓

B.Sc. in Mechanical Engineering

University of Le Havre

2007 - 2011 | Le Havre, France

Languages

French
Native/Bilingual
English
Native/Bilingual
Hebrew
Native/Bilingual

Technical Skills

Languages & Core

Python
SQL
MATLAB

Orchestration & Processing

Dagster
PySpark

Cloud Platforms

Azure
AWS

DevOps & Infrastructure

Docker
Kubernetes
Kafka

Services

Expert consultancy and advisory services for modern data engineering

"Lior is a very thorough and methodical engineer and I greatly recommend his end-to-end talents as a data engineer. He built robust data pipelines which support our pricing and reference data flows."
— Derek Landau, Senior Director at Lukka
⚙️

Dagster Consulting

Specialized consultancy for Dagster implementation and optimization. I've successfully migrated production systems from Talend and legacy tools to Dagster, reducing analyst workload by 3-5 hours/week and improving data reliability by 80%. Get expert guidance on building robust, scalable orchestration pipelines with proven best practices.

  • Dagster pipeline design and implementation
  • Migration from Talend, Airflow, and legacy orchestration tools
  • Performance optimization and debugging
  • Best practices and architectural guidance
🎯

Data Engineering Advisory

Strategic advisory services to help you make informed decisions about your data infrastructure, tooling, and processes. Get expert recommendations based on industry best practices.

  • Architecture review and recommendations
  • Technology stack selection guidance
  • Data strategy and roadmap planning
  • Team capability assessment and training
🔄

ETL/ELT Pipeline Development

Custom ETL/ELT solutions and consulting for complex data integration needs. From legacy system migrations to modern cloud-based data pipelines with best practices implementation.

  • Custom ETL pipeline design and development
  • Data transformation optimization
  • Integration architecture guidance
  • Performance tuning and troubleshooting
☁️

Cloud Data Architecture

Expert consulting for AWS, Azure, and cloud-native data infrastructure. Advisory on migrating on-premises systems to the cloud with modern governance, security, and scalability.

  • Multi-cloud migration strategy and execution
  • Azure Data Factory & AWS Glue consulting
  • Infrastructure as Code (Terraform) guidance
  • Data governance best practices
📊

Big Data & PySpark Consulting

Expert guidance on PySpark and distributed computing for large-scale data processing. Consulting on architecture, optimization, and best practices for handling massive datasets.

  • PySpark pipeline optimization
  • Distributed processing architecture
  • Performance tuning and troubleshooting
  • Best practices for scalable data processing
🚀

Streaming Data Solutions

Consulting on real-time data streaming architectures with Kafka and other streaming technologies. Advisory on building event-driven systems for immediate data insights.

  • Kafka architecture and implementation
  • Stream processing best practices
  • Real-time data integration strategies
  • Event-driven architecture design
👨‍🏫

Training & Mentoring

Technical mentoring and team enablement services. At Lukka, I mentored junior engineers in Python, Dagster, and modern data engineering practices. At Smith+Nephew, I trained team members on Power BI and VBA automation. I can help upskill your team with hands-on training tailored to your stack.

  • Dagster workshops and training
  • Python for data engineering
  • Best practices and code reviews
  • Team capability development

Recommendations

What colleagues and clients say about working with me

"
Lior is an exceptional data engineer with a rare combination of technical expertise and communication skills. He has a remarkable ability to design and build sophisticated data systems while explaining them clearly to both engineers and business stakeholders. What sets Lior apart is his problem-solving approach. When faced with any challenge, he dives deep into research and delivers results.
Ardavasd Ardhaldjian
Data Product Senior Analyst, Lukka
"
Lior is a very thorough and methodical engineer and I greatly recommend his end-to-end talents as a data engineer. Lior built robust data pipelines which support our pricing and reference data flows - and he isn't shy in learning new technologies and quickly became self-sufficient in intricacies of system architecture and pipelines. He led evaluation of several new technologies and proof-of-concepts, some of which solidified our design patterns for the foreseeable future.
Derek Landau
Senior Director, Pricing and Market Data Products, Lukka
"
I had the pleasure of working with Lior and he consistently impressed me with his curiosity, technical skill, and ability to deliver results. Lior is a highly inquisitive data engineer who doesn't just complete tasks, he digs deeper and often uncovers insights that others might miss. His ability to take initiative, solve complex problems, and see projects through to completion made a real impact on our team.
Andrew Regen
Executive Director (Head of Enterprise Data), Lukka
"
During his time working with me, Lior excelled as a data analyst and engineer. He consistently demonstrated strong technical proficiency, particularly in Python programming, which he leveraged to successfully automate complex data product operations. This work streamlined our processes and significantly improved efficiency. Beyond his technical skills, Lior is an outstanding communicator. He possesses a rare ability to translate intricate technical findings into clear, actionable insights for both technical and non-technical stakeholders.
Jill Kontogiorgos-Heintz
Head of Engineering, Lukka
"
Lior's expertise in linear statistics, programming, data processing, and finance resulted in delivery of many practical and very robust automations, Power BI dashboards and reports used by S&N executives as well as many members of GBS delivery teams! I truly enjoyed working with Lior and appreciated his professionalism and inquisitiveness at work.
Rafał Noculak
Senior Director of Group Financial Planning and Analysis, Smith+Nephew
"
I've been working with Lior for over 2.5 years in Smith & Nephew. He is kind of guy who will learn himself new tools or programming languages so he can do something better and faster. In his dictionary there is no such word as 'impossible'. He just needs to find out how and he is always managing to do it. Lior is also very detailed. He is always studying carefully the whole process. That allows him to easily identify the gaps and propose automations. In my career I have never met anyone who would be as passionate about data analysis as Lior is. He is a real gem in a team.
Iweta Adamczyk
Senior IT Business Analyst, GFT Technologies (worked together at Smith+Nephew)

Featured Projects

Real-world solutions delivering measurable business impact

On-the-Cloud Data Warehouse

Data Engineer @ GFT Group

Successfully migrated a complete on-premises data warehouse to Azure cloud infrastructure, implementing modern data orchestration, governance, and lineage tracking capabilities.

Duration: 11 months
Team Size: 13-18 people

Key Achievements:

  • Designed and implemented data pipelines for ETL processes
  • Collaborated with DevOps for Infrastructure as Code using Terraform
  • Deployed Azure Purview for data governance and lineage
  • Multiple deliverables completed within 2-3 week sprints
Azure Data Factory T-SQL Azure Purview Terraform Power BI

System Migration to Time-Series Database

Data Engineer

Led the migration of legacy data systems to a modern time-series database, handling complex data transformations and ensuring data integrity throughout the process. Successfully delivered within 3 months using PySpark for large-scale data processing.

Duration: 3 months

Key Achievements:

  • Transformed and migrated multiple datasets from legacy systems
  • Utilized PySpark for large-scale data processing
  • Implemented Polars for efficient data transformations
  • Adapted to evolving requirements with agile methodology
Python PySpark Polars Bash VMs

Fleet Tracking Dashboard

Senior Reporting Analyst @ Smith & Nephew

Developed a comprehensive Power BI dashboard for tracking medical device fleet, providing real-time insights into location, maintenance schedules, and lifecycle management.

Duration: 5 months (part-time)

Key Achievements:

  • Built ETL processes and internal database management system
  • Created T-SQL functions and procedures for data preparation
  • Designed interactive Power BI visualizations
  • Enabled stakeholders to track maintenance and decommissioning timelines
T-SQL Power BI Python Azure MSSQL

Reporting Automation Platform

Reporting Analyst @ Smith & Nephew

Independently initiated and delivered a complete automation solution for monthly Order To Cash reporting, eliminating manual processes and reducing error rates.

Duration: 6-9 weeks
Impact: Reduced manual work by 90%

Key Achievements:

  • Automated SAP data extraction using VBA and C#
  • Created Microsoft Access database for data management
  • Migrated visualizations from Excel to Power BI
  • Reduced human error and simplified team backup processes
VBA C# SQL Power BI Microsoft Access

Process Automation Initiative

Reporting Analyst @ Smith & Nephew

Delivered comprehensive process automation across multiple financial towers, significantly reducing manual workload and improving accuracy.

Impact: Reduced 8-9 FTE workload

Key Achievements:

  • Automated multiple financial reporting towers
  • Developed database for manufacturing operations
  • Created ETL pipelines and dashboards for fleet management
  • Integrated with SAP for seamless data flow
Python VBA SQL Power BI SAP

Ph.D. Research: Jet Aeroacoustics

Ph.D. Researcher @ AeroTraNet2

Developed advanced post-processing algorithms for analyzing jet noise sources using experimental data from microphone arrays and optical measurements.

Duration: 2013-2017

Key Achievements:

  • Developed algorithm to detect energetic events in turbulent flows
  • Applied advanced techniques: Wavelets, POD, and LSE
  • Published in Journal of Fluid Dynamics
  • Processed and analyzed large experimental datasets
MATLAB Python Data Analysis LaTeX Signal Processing

Get in Touch

Let's discuss how I can help with your data engineering needs

Contact Information

📧
📱
💼
🌍

Location

David, Chiriquí, Panama

Availability

Currently accepting new consultancy and advisory projects. Whether you're a 100-person scale-up or a 10,000-person enterprise with manual, slow, or error-prone data processes—let's talk.

Specializing in Dagster implementations, data pipeline optimization, and cloud data architecture. Available for both short-term engagements and long-term partnerships.