← Back to Architecture
Data Architecture
Data flows, conceptual models, and information management patterns powering the Wellness Digital Twin
High-Level Data Flow
End-to-end data flow from wearable devices through processing, storage, and intelligent analysis
graph LR
subgraph "Data Sources"
Oura[Oura Ring<br/>Sleep, HRV, Activity]
Dexcom[Dexcom CGM<br/>Glucose Data]
User[User Input<br/>Meals, Goals, Mood]
Calendar[Google Calendar<br/>Events, Schedule]
Weather[Weather API<br/>Environmental Data]
end
subgraph "Data Ingestion & Normalization"
Plugins[Plugin System<br/>Adapters & Connectors]
Validation[Data Validation<br/>& Sanitization]
Transform[Data Transformation<br/>& Enrichment]
end
subgraph "Data Storage Layer"
Turso[(Turso Database<br/>LibSQL Cloud)]
Cache[(Redis Cache<br/>Hot Data)]
Files[Cloud Storage<br/>GCS Avatars]
end
subgraph "Data Processing & Intelligence"
Memory[3-Tier Memory<br/>Episodic/Semantic/Procedural]
Analytics[Pattern Recognition<br/>& Analytics Engine]
Predictor[Predictive Models<br/>Energy, Meal Timing]
end
subgraph "Data Consumption"
Agents[AI Agents<br/>Context-Aware Responses]
API[REST API<br/>Client Access]
Dashboard[Dashboard<br/>Visualizations]
end
Oura --> Plugins
Dexcom --> Plugins
User --> Plugins
Calendar --> Plugins
Weather --> Plugins
Plugins --> Validation
Validation --> Transform
Transform --> Turso
Transform --> Cache
Turso --> Memory
Turso --> Analytics
Turso --> Predictor
Cache --> Memory
Memory --> Agents
Analytics --> Agents
Predictor --> Agents
Turso --> API
Cache --> API
Files --> API
API --> Dashboard
style Oura fill:#e8f5e9
style Dexcom fill:#e8f5e9
style User fill:#e3f2fd
style Calendar fill:#e8f5e9
style Weather fill:#e8f5e9
style Turso fill:#f3e5f5
style Cache fill:#fff3e0
style Files fill:#fce4ec
style Memory fill:#e1f5fe
style Analytics fill:#e1f5fe
style Predictor fill:#e1f5fe
style Agents fill:#ffebee
style API fill:#e0f2f1
style Dashboard fill:#e0f2f13-Tier Memory System
Advanced memory architecture enabling contextual understanding and personalized responses
📝
Episodic Memory
Stores specific events and interactions with temporal context
- • Conversation history
- • User interactions
- • Meal logs with timestamps
- • Exercise sessions
- • Sleep records
- • Mood journal entries
🧠
Semantic Memory
Knowledge graphs and learned facts about the user
- • Dietary preferences
- • Food allergies & restrictions
- • Favorite exercises
- • Health conditions
- • Wellness goals
- • Learned patterns
⚡
Procedural Memory
Behavioral patterns and habitual workflows
- • Meal timing patterns
- • Exercise routines
- • Sleep schedule
- • Response preferences
- • Habit formations
- • Workflow automation
Data Storage Technologies
Multi-layer storage strategy optimized for performance, scalability, and cost
Turso (LibSQL) - Primary Database
- • SQLite-compatible edge database
- • Low-latency global distribution
- • Embedded replicas for speed
- • Strong consistency guarantees
- • Cost-effective at scale
- • Direct SQL queries (no ORM)
Redis Cache - Hot Data Layer
- • User preferences (30min TTL)
- • Recent conversations (15min TTL)
- • LLM responses (2hr TTL)
- • Session state management
- • 60-80% API cost reduction
- • In-memory fallback for dev
Google Cloud Storage - Media
- • User avatar images
- • Smart scale photos
- • Document uploads
- • Signed URL access (1hr TTL)
- • Auto-optimization (512x512)
- • Local filesystem for dev
Database Indexing Strategy
- • User ID indexes (all tables)
- • Timestamp indexes for queries
- • Composite indexes for joins
- • 15+ strategic indexes
- • Query optimization via indexes
- • WAL mode for concurrency
Data Migration System
- • SQL-based migration scripts
- • Automated Python runner
- • Version tracking in database
- • Rollback support
- • Auto-loads from .env
- • No manual CLI auth needed
Data Security & Privacy
- • Bcrypt password hashing
- • JWT token authentication
- • Role-based access control
- • Data encryption at rest
- • TLS encryption in transit
- • GDPR/HIPAA compliance ready
Data Quality & Governance
Ensuring data accuracy, consistency, and compliance across the platform
Data Validation Pipeline
- 1.Schema validation via Pydantic models
- 2.Type checking and coercion
- 3.Range validation for biometric data
- 4.Anomaly detection for outliers
- 5.Data sanitization for security
- 6.Error logging and alerting
Data Governance Policies
- 1.User consent management
- 2.Data retention policies (90 days default)
- 3.Right to deletion (GDPR compliance)
- 4.Data portability and export
- 5.Audit logging for sensitive operations
- 6.Regular security audits