Data Mesh архитектура

Data Mesh — decentralized подход к data ownership и management. Alternative для monolithic data platforms.

4 принципа

Domain ownership: команды владеют своими данными end-to-end. Data as a product: datasets как продукты с users, SLAs, docs. Self-serve infrastructure: platform, не bespoke solutions. Federated computational governance: standards, но local implementation.

Domain data products

Wallet domain: transactions dataset. Game domain: bets, rounds datasets. KYC domain: verification statuses. Каждый team владеет и обслуживает свои. Discoverable через catalog. Consumable через standard interfaces (SQL, API).

Trade-offs

Pros: scales с organization, teams accountable. Cons: sophisticated cultural shift, требует mature engineering. Not for small teams. Best для enterprise с 100+ engineers. Иногда gradual adoption: hybrid между centralized и mesh.

Технологии

Storage: S3/Delta Lake. Sharing: open standards (Iceberg, Delta Sharing). Discovery: DataHub, Amundsen, Collibra. Access control: unified layer (Immuta, Privacera). Observability: Monte Carlo, Bigeye. Choice зависит от organization stage.