Project Atlas AetherDev project hub
Atlas project deprecated

Rust SQLx Tooling (Integrated SSH Tunnel)

Make remote data extraction reproducible by managing SSH tunnel setup and teardown in the tool. It ingests SSH target; DB creds; query definitions and produces query results; artifacts (CSV/JSON); logs. [NOTE: Abandoned due to sqlx limitations. See alternates: sql-to-csv and mssql-to-csv]

Type
Component
Lifecycle
Deprecated
Last touched
2025-04-24
Visibility
Public
Why it's showcased

Demonstrates reproducible data extraction with managed SSH tunnel lifecycle built into the tool.

Purpose

Make remote data extraction reproducible by managing SSH tunnel setup and teardown in the tool.

Current state

Requirement is clear; implementation details are not yet documented beyond planned tunnel approaches.

Next step

Implement tunnel lifecycle with guaranteed teardown and readiness logging.

Interfaces

Inputs
  • SSH target
  • DB creds
  • query definitions
Outputs
  • query results
  • artifacts (CSV/JSON)
  • logs

Reality to Action trace

Reality Ingestion

Contributes in this stage.

Canonical Storage

Not in scope.

Automation Engines

Contributes in this stage.

Human Interfaces

Not in scope.

Operational Adoption

Contributes in this stage.

Core workflow

  1. Parse config for SSH target, DB creds, and query definitions.
  2. Establish the SSH tunnel and wait for port readiness.
  3. Run SQLx queries through the tunnel.
  4. Export results to CSV or JSON artifacts.
  5. Tear down the tunnel cleanly.
  6. Record logs and timing metrics.

Data integrity and contracts

Canonical schema definitions

  • Query definition files (SQL) and expected result columns.
  • Output CSV/JSON schema for extracts.
  • Config fields for SSH host, user, and DB connection info.

Source of truth rules

  • Remote database remains canonical for query results.
  • Query definitions define the operational extract view.
  • Artifacts are derived and not a system of record.

Data quality checks

  • Verify tunnel connectivity before running queries.
  • Validate query success and row counts.
  • Flag timeouts or connection failures for retries.

Safe handling

  • Protect SSH keys and DB credentials.
  • Avoid logging sensitive query parameters.
  • Store artifacts in restricted directories.

Downstream integration map

  • Reporting and analytics pipelines.
  • Downstream CSV-based integrations.

Operational notes

Reliability posture

Self-contained setup; tunnel created and managed by the app.

Observability

  • tunnel lifecycle logs
  • query timing metrics (recommended)

Security and privacy

Credentials handling and secret storage required.

Dependencies

Upstream
  • remote DB access
  • network reachability
Downstream
  • reporting
  • portals
  • integrations

Ownership

Owners

Josh Barton

Users

you, automation jobs, Josh Barton (owner)