Retention Operations

Retention is configured per project. Project retention policy CRUD is implemented through GraphQL, the BFF bridge, control-plane storage, and the project settings UI. Deletion execution is a separate storage-maintenance boundary, not frontend or BFF behavior.

Current Product Behavior

Project admins can read and save the effective retention policy for a project. A policy has exactly one rule per data class, and updates replace the complete rule set using optimistic version checks.

Saved policies are durable configuration. The repository includes the storage-maintenance batch executor, scheduler loop, lease-aware fixture implementation, SurrealDB retention adapter, and service image/chart shape. Production deletion still depends on enabling the scheduler for the intended project IDs and validating the deployment with the opt-in SurrealDB retention tests for the data classes you plan to execute.

Data Classes

Data class	Default
TRACES	Delete after 30 days
LOGS	Delete after 30 days
METRICS	Delete after 30 days
AI_EVALS	Delete after 90 days
DATASETS	Retain
SCORERS	Retain
DASHBOARD_HISTORY	Retain
INGEST_CREDENTIAL_AUDIT	Delete after 365 days

Dashboard definitions, current project configuration, users, companies, memberships, and active ingest credential metadata are not deleted by telemetry retention.

Lifecycle

diagram

Operator Checks

When deletion execution is present, operators should check:

• project ID and data class in maintenance logs;
• matched, hard-deleted, soft-deleted, and final-deleted counts;
• policy version used by the batch;
• terminal errors and retry behavior;
• confirmation that no deletion crosses tenant/company/project boundaries.

Scheduler Configuration

The scheduler is disabled by default. In production-style SurrealDB builds, CLOUDGRID_STORAGE_ADAPTER=surrealdb wires storage-maintenance to the SurrealDB retention adapter; fixture builds use the in-memory fixture adapter for tests only.

Variable	Default	Purpose
CLOUDGRID_RETENTION_SCHEDULER_ENABLED	false	Enables scheduled deletion in storage-maintenance.
CLOUDGRID_RETENTION_SCHEDULER_INTERVAL_SECONDS	3600	Tick cadence, from 5 minutes to 24 hours.
CLOUDGRID_RETENTION_SCHEDULER_PROJECT_IDS	none	Comma-separated project IDs for the first production wave.
CLOUDGRID_RETENTION_BATCH_LIMIT	1000	Maximum rows per project/data-class batch.
CLOUDGRID_RETENTION_LEASE_SECONDS	900	Lease duration per project/data class.
CLOUDGRID_STORAGE_ADAPTER	surrealdb in SurrealDB builds	Selects the compiled storage-maintenance adapter.

When enabled, the scheduler iterates configured project IDs and every retention data class. One failed project/data class does not stop the rest of the tick. Multiple storage-maintenance replicas coordinate with a lease key shaped as retention:{projectId}:{dataClass}.

Do not add custom runbook commands around retention until the executable exposes them. The private batch contract is storage_maintenance.retention.execute_batch.

Safety Rules

• Retention deletion must be project-scoped.
• Soft-deleted records are hidden from normal GraphQL reads.
• retain means no automatic deletion for that data class.
• The BFF and frontend must not implement deletion loops.
• Storage-write remains the normal telemetry mutator; retention deletion is a maintenance mutation boundary.

Next Step

Review the concept page: Retention and alerts.