Metarecord Service (MRS)

Appearance

Playbooks ​

Playbooks are standard operating procedures (SOPs) for MRS. Use calls.md for payload shape and required fields.

Surface availability (explicit) ​

  • API Gateway: Available (tenant surface).
  • Direct Lambda: Available (operator-only maintenance).
  • CLI: Available (g3n mrs ..., API Gateway + operator direct Lambdas).
  • MCP: Available.

Playbook: Create + retrieve a record (inline or presigned) ​

Goal: Store content in MRS with correct metadata, then retrieve it reliably.

Why this sequence:

  • Inline JSON is only allowed for small JSON payloads.
  • Large or non-JSON content must go through presigned S3 with gzip + MD5 validation.

Preconditions

  • Valid orgcode and a USM session_guid (or API key).
  • Container name that matches the MRS pattern.

SOP (happy path)

  1. Choose upload path (inline vs presign).
    • Reason: inline JSON is capped at 256 KB; presign is required for larger or non-JSON payloads.
  2. Inline create (POST /mrs/record) when JSON ≤256 KB.
    • Reason: simplest path with immediate persistence.
  3. Presign create (POST /mrs/record) when content is large or non-JSON.
    • Reason: returns an upload URL + content token for S3.
  4. Upload gzip payload to the presigned URL.
    • Reason: MRS requires gzip for presigned uploads and validates content length + MD5.
  5. Complete upload (POST /mrs/record/complete) with size/etag/md5.
    • Reason: finalizes the record and moves it to active.
  6. Read metadata (GET /mrs/record/meta) or read content (GET /mrs/record).
    • Reason: metadata for audits, content for consumption.
  7. Manage lifecycle as needed (POST /mrs/tag/add, /tag/remove, /ttl/set, /doom).
    • Reason: tags enable discovery; TTL/doom enforces retention.
  8. List records (GET /mrs/list) with filters/pagination.
    • Reason: inventory and discovery flows.

Example (include doomed in list)

bash
g3n mrs list --profile g3nretailstack --session-guid $SESSION --orgcode $ORG --container demo --status all --limit 5
bash
curl -sS -G "https://api.g3nretailstack.com/mrs/list" \
  -H "x-session-guid: $SESSION" \
  --data-urlencode "orgcode=$ORG" \
  --data-urlencode "container=demo" \
  --data-urlencode "include_doomed=true" \
  --data-urlencode "limit=5"

Outputs

  • Active record with metadata + optional content pointer.
  • Searchable by container, tag, prefix, or status.

Failure modes / remediation

  • gzip-required / missing-content-md5 / invalid-content-md5: recompute gzip + hex MD5 and retry.
  • missing-size / size-mismatch: recompute size bytes and retry complete.
  • md5-mismatch / md5-missing: re-upload and retry complete (ensure MD5 from gzip bytes).
  • doomed: record is terminal; create a new record instead.
  • idempotency-conflict: reuse the original response or choose a new idempotency key.

Cross-service relationships

  • USM: session/API key validation for tenant access.
  • OFM: org association is enforced (non-associated → 404).

Playbook: Operator maintenance (reconcile + tag scrub + cleanup) ​

Goal: Keep MRS indexes and storage consistent via operator-only maintenance.

Why this sequence:

  • Asynchronous delivery and large uploads can create drift between metadata and storage.
  • Tag indexes need periodic cleanup after doom/TTL.

Preconditions

  • IAM access to operator direct Lambdas (or CLI with --profile).
  • Target orgcode and container scope.

SOP (happy path)

  1. Reconcile (operator direct Lambda or CLI reconcile).
    • Reason: detects and repairs mismatches between DDB and S3.
  2. Tag scrub (operator direct Lambda or CLI tag-scrub).
    • Reason: removes doomed records from tag indexes.
  3. Cleanup (operator direct Lambda or CLI cleanup).
    • Reason: clears stale fan-out or rollup artifacts.
  4. Verify via tenant reads/lists.
    • Reason: confirms indexes and visibility are corrected.

Example (reconcile report-only; paginated)

bash
g3n mrs reconcile --profile g3nretailstack --orgcode $ORG --container demo --mode report-only --max-items 10
# if next_token returned:
g3n mrs reconcile --profile g3nretailstack --orgcode $ORG --container demo --mode report-only --max-items 10 --next-token "$NEXT_TOKEN"

Outputs

  • Consistent metadata ↔ storage state.
  • Clean tag index and reduced drift.

Failure modes / remediation

  • not-authorized: ensure IAM role and profile are correct.
  • not-found: confirm org/container scope before running operators.

Cross-service relationships

  • UTL: offboarding flows may invoke cleanup in export/purge windows (as applicable).