Project Atlas AetherDev project hub
ASCIP Sync Engine Playbook icon
Playbooks

ASCIP Sync Engine Playbook

Operational runbook for syncing district staff data into ASCIP with diff/apply and audit logs.

System map

This playbook supports

ASCIP Sync Engine

Provide a deterministic, auditable way to keep district staff data aligned with ASCIP LMS by turning CSV extracts into a repeatable diff-and-apply workflow. Load config and mapping files, validate file paths, and establish API connectivity. Pull ASCIP reference data (groups, buckets, roles, timezones) to build local lookup maps. Read the district CSV, apply matching heuristics, and compute add/update/remove actions by group. Optionally run a dry run to generate reports without writes, then apply changes and emit audit/email reports.

Pipeline
Reality Ingestion Canonical Storage Automation Engines Operational Adoption
Domains
Vendor Integrations Identity SIS / HR Data Governance
Patterns
Diff Apply Audit Mapping Layer Observability First Identity Normalization Csv Boundary

Related Atlas entry

Purpose

Maintain a reliable, auditable sync between district staff data and ASCIP People, Groups, and Buckets. The goal is to replace manual reconciliation with a repeatable diff-and-apply workflow that logs every action and supports safe reruns.

When to use this playbook

  • Before running production syncs or applying new mapping files.
  • When reviewing audit logs after a major staff data refresh.
  • When investigating drift between district data and ASCIP state.

Signals to stop or escalate

  • Planned removals exceed an expected threshold.
  • API auth failures or repeated 4xx/5xx responses from ASCIP.
  • Mapping files no longer align with bucket/group definitions.

Audience and access

  • Primary operators: IT automation and data services staff.
  • Secondary reviewers: systems owner (you) and integration stakeholders.
  • Required access: ASCIP API credentials with permission to read/write People, Groups, and Bucket values.

System overview (what the sync actually does)

  • Pulls current ASCIP state for People, Groups, and Bucket definitions.
  • Fetches per-person detail records because list endpoints omit bucket data.
  • Normalizes identities (case-insensitive emails; match on work and personal emails when available).
  • Compares desired staff records from CSV to ASCIP state and produces a diff.
  • Applies only the delta (adds, updates, removals) and writes audit logs.
  • Enforces group membership rules (default district group and site/location group).

Patterns in use

Inputs

  • District staff CSV extract (source of truth).
  • Mapping files for bucket/group/value IDs.
  • ASCIP API credentials (HMAC auth).
  • Optional config flags for logging verbosity and dry-run mode (TBD if not implemented).

Outputs

  • ASCIP state changes (adds/updates/removals).
  • Diff report or action plan (if enabled).
  • Audit logs for each action and exceptions.
  • Exported CSV snapshots (if enabled).

Preconditions

  • Latest staff extract is validated and complete.
  • Mapping files are current and match ASCIP IDs.
  • Operator has verified credential scope and access.
  • Log destination is writable and rotation strategy is known.

Run steps

  1. Prepare inputs
    • Confirm staff CSV schema matches expected headers.
    • Validate mapping files (bucket/group IDs, value IDs, location groups).
  2. Snapshot current ASCIP state
    • Run the sync to fetch People, Groups, and Bucket definitions.
    • Confirm the per-person detail fetch executes (required for bucket data).
  3. Generate diff (dry-run if supported)
    • Review planned adds, updates, removals, and group membership changes.
  4. Apply changes
    • Run the apply step with rate-limiting if configured.
    • Watch logs for HTTP 411 or other vendor response issues.
  5. Review outputs
    • Validate counts: input records vs. actions performed vs. errors.
    • Review exception logs for mismatched groups or unmapped buckets.
  6. Archive artifacts
    • Store CSV inputs, mapping files, and run logs alongside the run timestamp.

Validation checklist

  • Confirm bucket assignments match expected values for a sample of users.
  • Confirm group memberships include both default district group and site group.
  • Confirm removed users are truly deprovisioned in the source extract.
  • Confirm no duplicate identities were created due to mismatched emails.

Monitoring and observability

  • Track total actions (adds, updates, removals, group assignments).
  • Track error rates and top failure reasons.
  • Capture timing per phase to detect slowdowns or API throttling.
  • Maintain a per-run summary for audit requests.

Failure modes and recovery

  • Missing bucket fields from list endpoints
    • Mitigation: ensure per-person detail fetch runs for every user.
  • HTTP 411 content-length requirement
    • Mitigation: avoid query-param POST/PUT; use correct body format.
  • Group membership desync
    • Mitigation: enforce group calls per group and re-run if drift detected.
  • Partial run failures
    • Mitigation: rerun with identical inputs; diff should reduce double-apply risk.
  • Mapping file drift
    • Mitigation: refresh mapping IDs before major runs; validate against ASCIP.

Security and privacy

  • PII is present in both inputs and logs. Avoid logging full records unless necessary.
  • Store logs in a restricted location with clear retention policy (TBD).
  • Use least-privilege API credentials scoped to required endpoints only.

Change management

  • Record changes to mapping files with timestamps and owner.
  • Document schema changes in the CSV extract definition.
  • Add a short run summary to the Atlas project page after major changes.

Open items and improvements

  • Add resumable batching and explicit rate-limit handling.
  • Add a first-class dry-run output if not already available.
  • Expand automated validation for bucket and group coverage.