swarm / companies/starter/work/project/docs/onboarding-t10-sync-engine.md

raw file 1690 bytes
# Sync Engine Architecture: Onboarding Guide

Welcome to the Ferrite Systems sync engine. This document outlines the core architecture for moving HL7/FHIR data between legacy hospital systems. Our priority is reliability; we never lose a patient record.

## Core Components
1. **Rust Sync Engine (`src/sync/`)**: The heart of the system. It handles batch processing, deduplication, and conflict resolution. It is designed to be stateless where possible, relying on Postgres for state.
2. **Postgres Database**: The source of truth for sync state. Key tables include `sync_batches`, `patient_records`, and `sync_errors`.
3. **Adapters (`src/adapters/`)**: Modules for HL7 v2.x and FHIR R4. These parse incoming data and normalize it into our internal schema.

## Data Flow
1. **Ingestion**: Data arrives via HL7/FHIR endpoints.
2. **Normalization**: Adapters convert raw data into internal structs.
3. **Batching**: Records are grouped into batches for atomic processing.
4. **Sync**: The engine applies changes to the target system, ensuring idempotency.
5. **Error Handling**: Failed records are logged in `sync_errors` for manual review.

## Key Principles
- **Idempotency**: All sync operations must be idempotent to prevent duplicate records.
- **Atomicity**: Batches are processed atomically; a failure rolls back the entire batch.
- **Observability**: Every step is logged with correlation IDs for tracing.

## Getting Started
1. Read the Rust code in `src/sync/`.
2. Review the Postgres migrations in `migrations/`.
3. Run the test suite: `cargo test`.
4. Review the error handling in `src/sync/error.rs`.

Remember: When in doubt, prioritize data integrity over feature development.