← Back to Architecture

Data Architecture

Data flows, conceptual models, and information management patterns powering the Wellness Digital Twin

High-Level Data Flow

End-to-end data flow from wearable devices through processing, storage, and intelligent analysis

graph LR
    subgraph "Data Sources"
        Oura[Oura Ring<br/>Sleep, HRV, Activity]
        Dexcom[Dexcom CGM<br/>Glucose Data]
        User[User Input<br/>Meals, Goals, Mood]
        Calendar[Google Calendar<br/>Events, Schedule]
        Weather[Weather API<br/>Environmental Data]
    end

    subgraph "Data Ingestion & Normalization"
        Plugins[Plugin System<br/>Adapters & Connectors]
        Validation[Data Validation<br/>& Sanitization]
        Transform[Data Transformation<br/>& Enrichment]
    end

    subgraph "Data Storage Layer"
        Turso[(Turso Database<br/>LibSQL Cloud)]
        Cache[(Redis Cache<br/>Hot Data)]
        Files[Cloud Storage<br/>GCS Avatars]
    end

    subgraph "Data Processing & Intelligence"
        Memory[3-Tier Memory<br/>Episodic/Semantic/Procedural]
        Analytics[Pattern Recognition<br/>& Analytics Engine]
        Predictor[Predictive Models<br/>Energy, Meal Timing]
    end

    subgraph "Data Consumption"
        Agents[AI Agents<br/>Context-Aware Responses]
        API[REST API<br/>Client Access]
        Dashboard[Dashboard<br/>Visualizations]
    end

    Oura --> Plugins
    Dexcom --> Plugins
    User --> Plugins
    Calendar --> Plugins
    Weather --> Plugins

    Plugins --> Validation
    Validation --> Transform
    Transform --> Turso
    Transform --> Cache

    Turso --> Memory
    Turso --> Analytics
    Turso --> Predictor
    Cache --> Memory

    Memory --> Agents
    Analytics --> Agents
    Predictor --> Agents

    Turso --> API
    Cache --> API
    Files --> API

    API --> Dashboard

    style Oura fill:#e8f5e9
    style Dexcom fill:#e8f5e9
    style User fill:#e3f2fd
    style Calendar fill:#e8f5e9
    style Weather fill:#e8f5e9
    style Turso fill:#f3e5f5
    style Cache fill:#fff3e0
    style Files fill:#fce4ec
    style Memory fill:#e1f5fe
    style Analytics fill:#e1f5fe
    style Predictor fill:#e1f5fe
    style Agents fill:#ffebee
    style API fill:#e0f2f1
    style Dashboard fill:#e0f2f1

3-Tier Memory System

Advanced memory architecture enabling contextual understanding and personalized responses

📝

Episodic Memory

Stores specific events and interactions with temporal context

  • • Conversation history
  • • User interactions
  • • Meal logs with timestamps
  • • Exercise sessions
  • • Sleep records
  • • Mood journal entries
🧠

Semantic Memory

Knowledge graphs and learned facts about the user

  • • Dietary preferences
  • • Food allergies & restrictions
  • • Favorite exercises
  • • Health conditions
  • • Wellness goals
  • • Learned patterns

Procedural Memory

Behavioral patterns and habitual workflows

  • • Meal timing patterns
  • • Exercise routines
  • • Sleep schedule
  • • Response preferences
  • • Habit formations
  • • Workflow automation

Data Storage Technologies

Multi-layer storage strategy optimized for performance, scalability, and cost

Turso (LibSQL) - Primary Database

  • • SQLite-compatible edge database
  • • Low-latency global distribution
  • • Embedded replicas for speed
  • • Strong consistency guarantees
  • • Cost-effective at scale
  • • Direct SQL queries (no ORM)

Redis Cache - Hot Data Layer

  • • User preferences (30min TTL)
  • • Recent conversations (15min TTL)
  • • LLM responses (2hr TTL)
  • • Session state management
  • • 60-80% API cost reduction
  • • In-memory fallback for dev

Google Cloud Storage - Media

  • • User avatar images
  • • Smart scale photos
  • • Document uploads
  • • Signed URL access (1hr TTL)
  • • Auto-optimization (512x512)
  • • Local filesystem for dev

Database Indexing Strategy

  • • User ID indexes (all tables)
  • • Timestamp indexes for queries
  • • Composite indexes for joins
  • • 15+ strategic indexes
  • • Query optimization via indexes
  • • WAL mode for concurrency

Data Migration System

  • • SQL-based migration scripts
  • • Automated Python runner
  • • Version tracking in database
  • • Rollback support
  • • Auto-loads from .env
  • • No manual CLI auth needed

Data Security & Privacy

  • • Bcrypt password hashing
  • • JWT token authentication
  • • Role-based access control
  • • Data encryption at rest
  • • TLS encryption in transit
  • • GDPR/HIPAA compliance ready

Data Quality & Governance

Ensuring data accuracy, consistency, and compliance across the platform

Data Validation Pipeline

  1. 1.Schema validation via Pydantic models
  2. 2.Type checking and coercion
  3. 3.Range validation for biometric data
  4. 4.Anomaly detection for outliers
  5. 5.Data sanitization for security
  6. 6.Error logging and alerting

Data Governance Policies

  1. 1.User consent management
  2. 2.Data retention policies (90 days default)
  3. 3.Right to deletion (GDPR compliance)
  4. 4.Data portability and export
  5. 5.Audit logging for sensitive operations
  6. 6.Regular security audits