DotNet-Projekte/Microsoft.SelfService.Portal.Core.API
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add queue hardening by modifying DeploymentJobTargets, DeploymentJobSteps, and DeploymentJobs tables

Browse Source 
- Drop existing indexes on DeploymentJobTargets and DeploymentJobSteps
- Alter Status column to nvarchar(450) in DeploymentJobTargets, DeploymentJobSteps, and DeploymentJobs
- Add new columns: Finished, OutputMetadataJson, Started to DeploymentJobTargets; ErrorMessage, Finished, OutputMetadataJson, Started to DeploymentJobSteps; CorrelationId, HeartbeatAt, Priority, RowVersion, ScheduledAt, WorkerName to DeploymentJobs
- Create new indexes for improved query performance
- Add constraints to ensure OutputMetadataJson is valid JSON
- Record migration in __EFMigrationsHistory
This commit is contained in:
Torsten Brendgen
2026-07-09 08:50:09 +02:00
parent 97238c28c9
commit 01c2a5df2e
45 changed files with 8198 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
ToDo.md
View File

@@ -79,31 +79,34 @@ Status: Completed
### 6. Worker Pipeline Refactor
Status: Pending
Status: Completed
- Load deployment composition from API/database.
- Convert JSON documents to ordered PowerShell hashtables for the existing Merge/Resolve modules.
- Add a renderer abstraction:
- `PowerShellDscV2Renderer`
- `DscV3JsonRenderer`
- future cloud/action renderers
- Store generated artifacts and logs.
- Load deployment composition from API/database. Completed in Worker `DeploymentCompositionLoader`.
- Convert JSON documents to ordered PowerShell hashtables for the existing Merge/Resolve modules. Completed as `PowerShellDscV2Renderer`, which writes `ConfigurationData.psd1` with `[ordered]@{...}` output for the composed deployment document.
- Add a renderer abstraction. Completed through `IDeploymentRenderer`.
- `PowerShellDscV2Renderer`. Completed.
- `DscV3JsonRenderer`. Completed.
- future cloud/action renderers. Extension point added.
- Store generated artifacts and logs. Completed as filesystem artifacts under configurable `WorkerPipeline:ArtifactRoot`; structured logs are written through `ILogger`.
- Replace the old dry-run provisioning provider path with `DeploymentPipeline`. Completed.
- Note: actual invocation of `Merge-DSCConfigurationData` / `Resolve-DSCConfigurationData` can now be added inside the DSC v2 renderer when the worker moves from render-only to execute mode.
### 7. Queue Hardening
Status: Pending
Status: Completed
- Add worker claim/lock semantics based on `LockedBy`, `LockedUntil`, and a concurrency token.
- Add `CorrelationId`, `Priority`, `ScheduledAt`, `HeartbeatAt`, and `WorkerName`.
- Add useful indexes for pending/running jobs.
- Persist step and target output metadata.
- Add worker claim/lock semantics based on `LockedBy`, `LockedUntil`, and a concurrency token. Completed with atomic `ExecuteUpdateAsync` claim logic in the worker and SQL `rowversion` on deployment jobs.
- Add `CorrelationId`, `Priority`, `ScheduledAt`, `HeartbeatAt`, and `WorkerName`. Completed in model, DTOs, API mapping, and Web TypeScript contracts.
- Add useful indexes for pending/running jobs. Completed for queue claim lookup, worker lookup, correlation id, target status, and step status/sort order.
- Persist step and target output metadata. Completed with JSON metadata columns and SQL `ISJSON` constraints.
- Migration `20260708194227_AddQueueHardening` and SQL script `buildcheck/AddQueueHardening.sql` generated.
### 8. Web/API Migration
Status: Pending
Status: In Progress
- Update API endpoints to expose template versions and deployment composition.
- Update Web to create DeploymentGroups with multiple Deployments.
- Update API endpoints to expose template versions and deployment composition. Completed for deployment batch list/details, version-aware create, and queue payloads.
- Update Web to create DeploymentGroups with multiple Deployments. Completed for the rudimentary create flow: users select a template version, targets become target assignments, and legacy executions stay available for compatibility.
- Add GUI support for sealed parameters and sealed template blocks.
- Add preview mode based on `Resolve-DSCConfigurationData -SkipSecrets`.
- Add promotion flow from Test to QA/Prod.
@@ -139,4 +142,4 @@ Status: In Progress
## Next Step
Start work package 6 by refactoring the worker pipeline to load deployment composition from the API/database and render artifacts through a DSC v2/v3-capable abstraction.
Continue work package 8 with sealed parameter/block visibility, a backend preview endpoint, and the later promotion flow. Keep rich interactive preview UX in Step 8a.