dbt для трансформаций данных

dbt (data build tool) — стандарт де-факто для transformation layer в modern data stack.

Core concepts

Models: SQL SELECT + Jinja. Sources: external tables. Refs: dependencies между models. Tests: assertions data quality. Documentation: auto-generated. Snapshots: SCD Type 2. Seeds: static reference data. Modular, version-controlled.

Workflow

Git-based development. dbt run: materialize models в warehouse. dbt test: quality checks. CI/CD: dbt Cloud или GitHub Actions. Environments: dev/staging/prod isolation. Documentation: dbt docs generate. Peer review culture.

Best practices

Layered architecture: staging → intermediate → marts. Naming conventions: stg_*, int_*, dim_*, fct_*. One responsibility per model. Testing: not_null, unique, relationships, accepted_values. Incremental models для big tables. Macros для reusability.

iGaming pipeline

Raw events → staging (typing, cleaning) → intermediate (joins) → marts (star schema для BI). Financial marts: separate schema с strict testing. Player marts: LTV, cohorts. Compliance: audit-focused marts. Tests obligatory для financial models.