Case studyOne Click LCA · 2022—present

The Data Import Service

An automated pipeline that turns messy, inconsistent sustainability data from hundreds of international providers into clean, classified records the platform can actually use.

PythonFastAPIPostgreSQLDockerData Pipelines
30%+
manual effort removed
1000s
datasets processed
market coverage & adoption

01The problem

One Click LCA is the world's leading platform for life-cycle assessment — the calculations behind sustainable construction. Those calculations are only as good as the underlying data: EPDs (Environmental Product Declarations) published by hundreds of providers across the globe.

The catch: every provider structures that data differently. Onboarding a new dataset meant slow, manual mapping and classification — a bottleneck that capped how fast the platform could grow its coverage, and a drag on adoption.

02What I built

I architected and developed the Data Import Service (DIS) — a staged pipeline that automates ingestion, mapping, and classification of global datasets end to end, with humans looping in only where the data is genuinely ambiguous.

01
Ingest
Pull datasets from each EPD provider
02
Normalize
Reconcile formats into one schema
03
Map
Match fields to the platform model
04
Classify
Auto-categorize each record
05
Review
Human-in-the-loop for edge cases
06
Publish
Release into the LCA platform

providers → DIS → One Click LCA platform

03Key decisions

Provider-specific adapters, one shared core

Each EPD provider ships data differently. Rather than a monolith with endless special cases, I modularized ingestion into per-provider adapters feeding a shared normalization core — so onboarding a new provider became configuration, not a rewrite. That modularity is what let coverage (and revenue) expand.

Automate the 80%, keep humans on the 20%

Fully automated classification would trade accuracy for speed; fully manual was the bottleneck we started with. The pipeline auto-maps and auto-classifies the confident majority and routes only ambiguous records to human review — the balance that removed 30%+ of the manual effort without sacrificing data quality.

Idempotent, re-runnable stages

Datasets get corrected and re-published upstream, so each stage was designed to be safely re-run on the same input — making imports repeatable and recoverable instead of one-shot and fragile.

04Outcome

The pipeline cut manual effort by 30%+and scaled to process thousands of datasets, expanding the platform's market coverage and opening new revenue streams from international EPD providers. Automation plus a modular design turned data onboarding from a bottleneck into a repeatable, growing capability.

Want the deeper technical walk-through?