A modern NBA analytics data platform powered by dbt, DuckDB, and Streamlit. Transforms raw basketball event data into a dimensional star schema (32,000+ games, 480,000+ shots from 2002 to 2026), persisting artifacts to Cloudflare R2 Object Storage with automated Gitea Actions CI/CD and self-hosted Coolify deployment.

🏛️ Architecture

flowchart LR
    PG[(Postgres Source)] -->|scripts/extract.sql| LOCAL_DB[(DuckDB 223 MB)]
    LOCAL_DB -->|dbt build| MARTS[Marts Star Schema]
    MARTS -->|db_storage.py upload| R2[(Cloudflare R2 Storage)]
    R2 -->|Smart Sync| COOLIFY[Coolify: streamlit.turbo-data.com]
    R2 -->|run_app.sh| LOCAL_APP[Local Streamlit Dashboard]

Medallion Data Modeling

Layer	Models	Description
Staging	`stg_games`, `stg_line_scores`, `stg_player_game_basic_stats`, `stg_player_game_adv_stats`, `stg_player_shot_charts`, `stg_team_game_basic_stats`, `stg_team_game_adv_stats`, `stg_season_thresholds`, `stg_team_season_thresholds`	Clean raw source data, compute seasonal percentile thresholds, type casting
Intermediate	`int_games_enriched`, `int_player_performance`, `int_team_performance`, `int_player_shots_enriched`	Entity resolution across team history maps, shot chart & free throw unification
Dimensions	`dim_teams`, `dim_players`, `dim_seasons`, `dim_dates`, `dim_arenas`, `dim_shot_zones`, `dim_player_game_archetypes`	Conformed entity dimensions
Facts	`fct_game_results`, `fct_team_game_stats`, `fct_player_game_stats`, `fct_quarter_scoring`, `fct_player_shots`, `fct_player_game_shooting`	High-performance analytical fact tables with surrogate keys

🚀 Quickstart & Local Setup

1. Prerequisites

uv (Python package manager)
DuckDB CLI (v1.3.0+)

2. Clone and Install Dependencies

git clone https://git.turbo-data.com/nprasad2077/Streamlit_NBA.git
cd Streamlit_NBA

# Install Python 3.12 and dependencies into .venv
uv sync

3. Configure Environment Variables

Copy the template and fill in your credentials:

cp .env.example .env

# Postgres Source Database (for ETL extraction)
POSTGRES_URL=postgresql://user:password@host:5432/nba

# Cloudflare R2 / S3 Object Storage
S3_ENDPOINT_URL=https://<ACCOUNT_ID>.r2.cloudflarestorage.com
S3_BUCKET_NAME=dbt-duckdb
S3_ACCESS_KEY_ID=your_r2_access_key
S3_SECRET_ACCESS_KEY=your_r2_secret_key
S3_REGION=auto
S3_DB_KEY=dbt_nba.duckdb

4. Launch the Streamlit Dashboard

./scripts/run_app.sh

The launcher automatically verifies and downloads the latest DuckDB database from Cloudflare R2 if it is not already present locally.

🗄️ Database Storage Management

Manage your analytical DuckDB file via the custom storage utility:

# Check remote R2 object vs local file status
uv run python scripts/db_storage.py status

# Download latest database (skips if ETag/size already matches)
uv run python scripts/db_storage.py download

# Force re-download from Cloudflare R2
uv run python scripts/db_storage.py download --force

# Upload local database to Cloudflare R2
uv run python scripts/db_storage.py upload

⚙️ Running the ETL Pipeline

To execute the complete ETL process locally (Extract from Postgres \rightarrow Run dbt transformations \rightarrow Upload artifact to Cloudflare R2):

./scripts/pipeline.sh

🛠️ dbt Development Commands

Run dbt models and tests directly using uv:

# Install dbt packages (dbt_utils)
uv run dbt deps

# Parse and validate project manifest
uv run dbt parse

# Build all models, seeds, and tests
uv run dbt build

# Build specific layer by tag
uv run dbt build --select "tag:staging"
uv run dbt build --select "tag:intermediate"
uv run dbt build --select "tag:marts"

# Run tests only
uv run dbt test

🚢 CI/CD & Deployment

Gitea Actions (.gitea/workflows/pipeline.yaml): Runs scheduled daily ETL builds (06:00 UTC) on a custom ubuntu-latest runner and updates Cloudflare R2.
Coolify Self-Hosting (streamlit.turbo-data.com): Production deployment built via Dockerfile with automated Let's Encrypt SSL and persistent storage caching.