# SelfService Portal Restructure ToDo

This file tracks the planned restructure from the current single-template deployment model to a generic configuration composition platform for on-prem, hybrid, and cloud workloads.

## Status Discipline

- Keep this file updated after every implementation step.
- Reflect completed work, deferred cleanup, migration status, and DemoData changes here before moving to the next package.
- Keep `Context/DemoData.cs` aligned with model/API changes so the API can be tested after every migration.

## Goals

- Support reusable configuration building blocks for many services, such as Active Directory, SQL, SharePoint, Exchange, Teams, Azure, and M365.
- Allow shared catalog data, such as domains and environments, to be linked instead of duplicated.
- Compose one environment deployment from multiple deployments, for example AD, SQL, SharePoint, and cloud policy workloads.
- Keep deployments reproducible by referencing immutable template versions.
- Keep the worker independent from a single renderer so PowerShell DSC v2 and DSC v3 JSON can both be supported.

## Work Packages

### 1. Architecture Baseline

Status: Completed

- Define the target domain terms: Environment, Domain, Service, Target, DeploymentGroup, Deployment, Template, TemplateVersion, TemplateSelection, DeploymentTarget, QueueJob, Artifact.
- Document the intended relationships.
- Decide how `JsonData` should be stored and validated.
- Define compatibility rules for the existing API and database.
- Output: `Docs/Architecture/DeploymentModel.md`

### 2. Template Versioning

Status: Completed

- Add `TemplateVersionModel`. Completed.
- Move mutable template document fields from `TemplateModel` into `TemplateVersionModel`. Completed for new writes while legacy fields remain for compatibility.
- Keep `TemplateModel` as the stable catalog entry. Completed.
- Add `JsonHash`, `SchemaVersion`, `IsPublished`, `PublishedAt`, and `PublishedBy`. Completed.
- Add unique constraints for `TemplateId + Version`. Completed.
- Migrate existing `Templates.JSONData` into an initial `TemplateVersions` row. Completed in migration.
- Keep API responses backward compatible at first. Completed.
- Add Contoso demo data for template versions. Completed.
- Add reset/seed migration for local development data. Completed and applied locally.
- Defer removal of legacy `TemplateModel.Version` and `TemplateModel.JSONData` until Web/Worker no longer depend on them.

### 3. JSON Document Contract

Status: Completed

- Define the canonical JSON document shape for templates and deployments. Completed in `Docs/Architecture/JsonDocumentContract.md`.
- Add schema versioning, for example `schemaVersion`. Completed for template documents and tolerant deployment overrides.
- Add server-side JSON validation. Completed through `JsonDocuments/ConfigurationDocumentValidator.cs`.
- Add database checks where supported, for example `ISJSON(...)`. Completed for `TemplateVersions.JsonData`; other JSON columns remain validated in code until they are migrated.
- Add content hash calculation for version comparisons. Completed for `TemplateVersionModel.JsonHash`.
- Decide which document fields should be projected into queryable columns. Completed in the contract documentation.

### 4. Deployment Composition

Status: Completed

- Add `DeploymentTemplateSelectionModel`. Completed.
- Allow a deployment to reference multiple template versions with role and sort order. Completed through deployment batch composition endpoints.
- Add `DeploymentParameterValueModel` for explicit user overrides. Completed.
- Add target assignment model for deployment targets. Completed as `DeploymentTargetAssignmentModel`; generic `TargetModel` follows in Step 5.
- Keep old `DeploymentGroup.TemplateId` as legacy until Web and Worker are migrated. Completed for this slice.
- Update Web GUI to test deployment composition directly. Completed in `DeploymentBatchDetailsPage`.

### 5. Generic Target Model

Status: Completed

- Generalize beyond virtual machines. Completed with `TargetModel`, `TargetController`, `TargetRepository`, and target DTOs.
- Introduce target types such as `VirtualMachine`, `Tenant`, `Subscription`, `ResourceGroup`, `User`, `Group`, `Site`, and `PolicyScope`. Completed through `TargetType`.
- Link targets to environments/domains where applicable. Completed for optional domain links; environment targeting remains composition/worker-specific.
- Preserve VM-specific details in `MetadataJson` or a specialized extension table. Completed with generic `MetadataJson`; no extension table yet.
- Add provider scoping for on-prem, Azure, Microsoft 365, Teams, and related target providers. Completed through `ProviderType`.
- Add migration `20260707211957_AddGenericTargetModel`; migration script generated at `buildcheck/AddGenericTargetModel.sql`.
- Update rudimentary Web GUI to use `/targets` and expose target type/provider fields. Completed.

### 6. Worker Pipeline Refactor

Status: Pending

- Load deployment composition from API/database.
- Convert JSON documents to ordered PowerShell hashtables for the existing Merge/Resolve modules.
- Add a renderer abstraction:
  - `PowerShellDscV2Renderer`
  - `DscV3JsonRenderer`
  - future cloud/action renderers
- Store generated artifacts and logs.

### 7. Queue Hardening

Status: Pending

- Add worker claim/lock semantics based on `LockedBy`, `LockedUntil`, and a concurrency token.
- Add `CorrelationId`, `Priority`, `ScheduledAt`, `HeartbeatAt`, and `WorkerName`.
- Add useful indexes for pending/running jobs.
- Persist step and target output metadata.

### 8. Web/API Migration

Status: Pending

- Update API endpoints to expose template versions and deployment composition.
- Update Web to create DeploymentGroups with multiple Deployments.
- Add GUI support for sealed parameters and sealed template blocks.
- Add preview mode based on `Resolve-DSCConfigurationData -SkipSecrets`.
- Add promotion flow from Test to QA/Prod.

### 8a. Interactive GUI Preview

Status: Deferred

- Keep current GUI changes rudimentary while backend work packages are implemented.
- Add an interactive parameter editor that can show effective target values.
- Add a deployment JSON preview for the composed deployment document.
- Add a Merge/Resolve preview action that displays the merged and resolved configuration data.
- Redesign deployment and deployment composition views after the backend model has stabilized.

### 9. Legacy Cleanup

Status: In Progress

- Remove legacy direct template references after migration.
- Remove old DTO fields that expose mutable `Template.JSONData`.
- Remove obsolete SMA Runbook/Event/Job API and data model. Completed.
- Keep read-only migration endpoints only if required.
- Archive or migrate old deployment records.

## First Implementation Slice

1. Finish `Docs/Architecture/DeploymentModel.md`. Completed.
2. Add `TemplateVersionModel` and EF migration. Completed.
3. Migrate existing `Templates.JSONData` into `TemplateVersions`. Completed.
4. Update Template API to return the current published version. Completed.
5. Add tests/build checks for the new model. Build and migration script checks completed; automated tests are still pending.
6. Add Contoso DemoData and reset/seed migration. Completed.

## Next Step

Start work package 6 by refactoring the worker pipeline to load deployment composition from the API/database and render artifacts through a DSC v2/v3-capable abstraction.