SSSamarth Sinha

Case study

From a Monolithic Data Platform to a Domain Data Mesh

Led the move of a large e-commerce company's central data platform into 34 domain-owned cloud projects, with shared policies and a source-quality rating.

Role
Led the data-platform function (11 direct reports)
Organisation
Tokopedia (GoTo Group)
Period
2019 — 2024
Status
Delivered, in production
4 → 34domain-owned GCP projects
11direct reports across five teams
17parameter source-quality rating
Stack
  • Google Cloud
  • BigQuery
  • GKE
  • Airflow
  • Kafka
  • Terraform
From a central platform to domain-owned projects under shared policies Central platform to Domain tagging. Domain tagging to Domain project A. Domain tagging to Domain project B. Domain tagging to Domain project N. Domain project A to Quality rating. Domain project B to Consumers. Domain project N to Access audits. Shared policies to Domain project N. Central platform one team owns everything Domain tagging owner per dataset Domain project A standard datasets Domain project B standard datasets Domain project N one per domain Quality rating score per source, alerts Consumers discover via catalog Access audits on a fixed schedule Shared policies replication, quality catalog, access
IllustrativeFrom a central platform to domain-owned projects under shared policiesScroll →

Context

Tokopedia's data platform was monolithic: one central platform, run by one team. Business domains depended on that team for new datasets, access and quality, and platform cost wasn't owned by the teams using it.

The goal was to give business domains ownership of their own data, without losing consistent governance across the company.

What I did

I led the programme as lead of the data-platform function, with 11 direct reports across data engineering, data science, data analytics and two internal teams.

My teams built it on the platforms we already ran: GKE, Airflow, Kafka streaming, Spark and Hadoop, with infrastructure as code (Terraform, SaltStack) and CI/CD.

Architecture and decisions

  • Ownership first. Every dataset was tagged with its business domain, then moved into a domain-owned GCP project with standardised BigQuery datasets, so every domain looks the same from the outside.
  • Shared policy, local ownership. Table replication, data-quality rules, catalog registration and access requests stayed shared policies rather than per-team choices.
  • Access you can audit. A data-access matrix, with audits on schedules from quarterly to annual.
  • Quality made visible. A data-schema rating model scored every source on 17 parameters (five-star scale), with daily and weekly alerts on quality and cost by team and service.

Outcome

  • 34 domain-owned projects with standardised datasets in production, up from 4.
  • Quality and cost visible per team and service, through daily and weekly alerts.