Project Atlas AetherDev project hub
Atlas view

Data Integration

This domain groups systems that solve Data Integration problems - shared vocabulary, common interfaces, and repeatable operational constraints. Use this page to see which systems already exist, what they ingest and emit, and which patterns we repeatedly rely on in this problem space.

Domains are how we avoid reinventing the same solution in slightly different shapes. When a domain repeats, it’s a signal to standardize schemas, mapping layers, auth patterns, and operator workflows.

Domain guide

How to read this domain

Definition

This domain groups systems focused on data integration workflows, shared data models, and repeatable operational constraints.

What belongs

  • Systems where data integration is the primary problem space.
  • Integrations that move, validate, or enforce data in this domain.
  • Operational tools that reduce risk and manual work for this domain.

What doesn't

  • Generic tools where data integration is incidental metadata.
  • One-off experiments without domain-specific constraints.

Typical integrations and constraints

  • Source-of-truth feeds, vendor APIs, and scheduled extracts.
  • Schema drift, credential scope, and compliance requirements.
  • Role-based access and operational review cycles.

Common interfaces

  • REST/JSON APIs and vendor export endpoints.
  • CSV, SQL views, or Sheets-based boundary objects.
  • Auth boundaries (service accounts, OAuth, SSO).

Common patterns and playbooks

Csv Transform Extract Export Mapping Layer Ad Hoc Query Csv to Db Sync Sheet as Ui
Results 7 projects
Component · Active

csv_mapper

CSV Mapper and Transformer is a robust command-line tool written in Rust that allows you to map, transform, and filter CSV files using flexible, user-defined …

Reality Ingestion Human Interfaces Operational Adoption
Open project
Component · Maintenance

Database-Ingester

Enable safe, repeatable CSV ingestion into SQL Server tables with backups and dry-run visibility. Load config, optionally back up the target table, compare CSV …

Reality Ingestion Human Interfaces Operational Adoption Canonical Storage
Open project
System · Active

District Transfer Tracking

Centralizes, normalizes, and syncs district transfer data into a single Google Sheet for review and downstream reporting.

Reality Ingestion Canonical Storage Human Interfaces
Open project
Field Tool · Maintenance

mssql_query_to_csv

The **MSSQL Query to CSV** program is a command-line tool written in Rust that connects to a Microsoft SQL Server database, executes a SQL query, and exports …

Reality Ingestion Human Interfaces Operational Adoption Canonical Storage
Open project
Field Tool · Maintenance

mssql_to_csv

Run SQL files against MSSQL and export results to CSV, optionally through an SSH tunnel. It ingests TOML config, SQL file, MSSQL database and produces CSV …

Canonical Storage
Open project
Experiment · Active

SQL

Centralize operational queries for reporting, audits, and data movement workflows. It ingests Database tables (Aeries and related systems) and produces Query …

Canonical Storage
Open project
Field Tool · Maintenance

sql_to_csv

DB Query to CSV** is a flexible command-line tool written in Rust that connects to SQL databases (such as MSSQL, PostgreSQL, or MySQL) using SQLx, executes a …

Reality Ingestion Human Interfaces Operational Adoption Canonical Storage
Open project