Pending work checklist — 2026-05-18 snapshot
Authored: 2026-05-18.
Purpose: Single-page inventory of every open work item across Plans 0-7, follow-ups (FU-1..FU-53), and engineering hardening for scale. Used to drive priority + ensure nothing falls through.
How to use:
- Hard blockers at the top — track these to dated deadlines.
- Per-plan sections below — pick up the next-actionable item per plan.
- Cross off (or change status) as items close; commit each closure with the evidence file in
compliance/audit/.
- This doc is a snapshot — when the situation changes meaningfully, re-snapshot under a new dated filename rather than mutating this one. Lets us see history.
🔴 HARD BLOCKERS (dated deadlines)
| ID | Item | Deadline | Owner | Evidence/Runbook |
|---|
| FU-51 | Provision real SendGrid prod live key via AWS SSM /hmp/prod/sendgrid/api-key + apply hmp-third-party-secrets ExternalSecret | Before first-property onboarding (~2026-06-15) | Operator | hmp-infrastructure/compliance/follow-ups.md#fu-51 |
| Plan 6 Q4 | On-call rotation set up with 3 named approvers (2-of-N gate needs ≥3 humans; today have 1) | 2026-06-15 | VP Eng + SRE Lead | hmp-docs/docs/platform-registry/plan-6-decision-log.md#q4 |
| Plan 6 Q5 | Backup retention recommendation sign-off (current 30d prod / 7y CloudTrail satisfies SOC 2 + DPDPA) | 2026-06-15 | Legal/DPO + CISO + SRE Lead | …plan-6-decision-log.md#q5 |
| Reloader prod validation | Force-rotate test cluster to validate end-to-end, then monitor prod natural rotation | 2026-05-24 (next prod RDS rotation) | Operator | hmp-infrastructure/monitoring/runbooks/fu-52-reloader-rollout.md |
| FU-46 | RS256 JWT keys for staging+prod; remove ALLOW_HS256 opt-out (security gap) | Before first-property onboarding | Operator + Eng | hmp-infrastructure/compliance/follow-ups.md#fu-46 |
🟡 Plan 4 — Demo tenant on prod
| Task | Status | Action |
|---|
| Task 24 prod demo-tenant seed | Job YAML authored at hmp-infrastructure/kubernetes/jobs/seed-runner/prod-demo/ | Operator: get 2-of-N approval per hmp-infrastructure/monitoring/runbooks/plan-4-seed-prod-demo.md → apply via kubectl |
🟡 Plan 5 — Cutover (in flight)
| Item | Status | Action |
|---|
api.keynova.ai + live.keynova.ai | ✅ Live on prod EKS ALB (verified 2026-05-17 with ResAvenue handler 400-on-bad-XML) | — |
| 7-day prod burn-in observability | 🟢 In flight (day 2 of 7) | Daily check via hmp-infrastructure/monitoring/runbooks/resavenue-cert-reissuance.md "7-day burn-in observability" section |
| ResAvenue first hotel onboarding | 🟡 Waiting on first hotel's data | Operator: replace placeholders in hmp-infrastructure/seeds/seed-resavenue-prod-connection.sql + apply |
| FU-6 ResAvenue cert v3 closure | 🟡 Closes after burn-in completes | ~2026-05-24 |
🟡 Plan 6 — Prod-launch baseline
| ID | Item | Status | Action |
|---|
| Baseline re-run | 🔴 Operator must re-run YAML after AWS profiles + flags fixed → expect 17-22 PASS (was 1 PASS / 7 FAIL / 13 NOCREDS / 2 AUTOMODE on first run) | Operator | |
| B-17 | Renovate.json propagation to 25 repos | 🟡 Template + script authored | Operator runs hmp-infrastructure/scripts/propagate-renovate-config.sh |
| B-18 | First quarterly access review | 🟡 Runbook + 2026-Q2 schedule shipped | First review due 2026-06-30 — hmp-infrastructure/monitoring/runbooks/quarterly-access-review.md |
| B-19 | DR drill execution (failover ap-south-1 → ap-south-2) | 🔴 Not executed | Operator (SRE) — author DR drill script next |
| B-20 | Backup-restore drill against keynova-prod-vault | 🔴 Not executed | Operator (SRE) |
| B-21 | Pentest report | 🔴 Vendor not engaged | External — 2-4 week procurement lead time |
| B-23 | CIC-1..5 sign-offs (all currently signed=0) | 🔴 Open | Product / Security / SRE / CS / Payments stakeholders |
| B-24 | Spec §14 5 decisions (see Plan 6 decision log) | 🟡 Recommendations drafted | Sign-offs pending (Q4+Q5 are hard blockers per top of doc) |
🟡 Plan 7 — Legacy retirement (in progress)
| Phase | Status | Action |
|---|
| Phase 1 inventory + 2-of-N approval | ✅ Authored | — |
| Phase 2 data-snapshot | Skipped per operator confirmation (seed-only data) | — |
| Phase 3 destroy | 🟢 Operator manually deleting legacy account 730335256638 via AWS Console (in progress) | Continue Console deletions until all resources gone |
Phase 4 code cleanup (git rm legacy Terraform stacks + compose files + deploy scripts) | 🟡 Pending Phase 3 completion | Author single cleanup commit when Phase 3 done |
| Phase 5 cost verification (T+7d post-destroy) | 🟡 Pending | Operator pulls AWS Cost Explorer 7 days after final destroy |
| FU-3 legacy module dedup | ✅ Closes when Phase 4 commits | — |
🔵 Engineering hardening for scale (preparing for 100s of hotels)
Discussed 2026-05-18 in response to "what happens during RDS rotation when 100s of hotels are using it?"
| Level | Item | Why | Effort | Status |
|---|
| L1 | Add preStop hook (sleep 10) to 20 base Deployments | Eliminates ALB target-deregistration race → ~95% of rolling-restart 502s gone | 30 min | 🔴 Not started |
| L1 | Bump terminationGracePeriodSeconds to 60s on all Deployments | Cleaner pod shutdown alignment with ALB drain | 15 min (same commit) | 🔴 Not started |
| L2 | Tune ALB deregistration_delay.timeout_seconds=30 (currently default 300) | Faster target removal; tighter rolling-restart window | 30 min (per-Ingress annotation) | 🔴 Not started |
| L2 | Bump prod HPA minReplicas from 2 to 3 on critical services | Rotation window only affects 1/3 not 1/2 | 30 min | 🔴 Not started |
| L3 | Migrate to RDS IAM auth | Eliminates password-rotation impact entirely (15-min auto-renewing tokens via IRSA) — Plan 2 already enabled enabled_iam_authentication = true on prod RDS; app needs to use it | ~2 days engineering | 🔴 Not started |
| L4 | AWS RDS Proxy (optional, transparent rotation + pool) | Cleanest credential-rotation isolation; +~$15/mo per proxy | ~1 day | 🔴 Not started |
| Op | Schedule prod RDS rotation for off-peak (02:00 IST) | Operator-controlled timing instead of AWS automatic | 1h ops setup | 🔴 Not started |
Recommendation: L1 + L2 land BEFORE first-property onboarding. L3 lands before scaling beyond 10 hotels. L4 + Op are optional polish.
🟡 Other open follow-ups (compliance/follow-ups.md)
| ID | Item | Owner |
|---|
| FU-2 | Tooling installation runbook (tflint / tfsec / checkov local install guide) | Plan 0 carry-over |
| FU-5 | AWS Config delivery channel + S3 Object Lock incompatibility (needs separate bucket) | Plan 6 |
| FU-11 | IDC OIDC ArgoCD app — operator console registration | Operator |
| FU-12 | ArgoCD per-env PAT population in Secrets Manager | Operator |
| FU-13/14 | Windows DNS poisoning + TCP parallelism (workaround documented; CI runners on Linux unaffected) | Workaround in place |
| FU-18 | Kustomize v5 namespace transformer ID conflict (whole-tree builds fail; per-overlay works) | Architecture follow-up |
| FU-25 | Migrate deploy notifications Slack → Teams | Q3 sign-off + operator config |
| FU-26 | ArgoCD ci-bot tokens are non-expiring + no revocation runbook | Plan 6 hardening |
| FU-27 | ArgoCD helm_release timeout false-negative on ConfigMap-only changes | Workaround documented |
| FU-28 | Self-approval allowed on staging/prod during pilot (closes when Q4 lands) | — |
| FU-43 | Per-env auth-service-runtime-env propagation to qa/staging/prod | Operator runtime secret rotation |
| FU-45 | Service runtime secret rotation policy + Lambda | Plan 6 hardening |
| FU-46 | RS256 JWT keys for staging+prod; remove ALLOW_HS256 | Security gap — should be before first-property (also listed in hard blockers above) |
| FU-48 | Asset-service migration history reset | Operator playbook at hmp-infrastructure/monitoring/runbooks/fu-48-asset-migration-reset.md |
| FU-49 | Notification-service device_tokens orphan migration | Same playbook as FU-48 with substitutions |
| FU-53 | metrics-server install on test + qa (staging + prod already have it) | Engineering authoring (~30 min) — mirror FU-52 pattern |
🟡 Sales & Marketing sign-offs (per 2026-05-16 playbook)
| Item | Owner |
|---|
| VP Sales + CFO sign-off on pricing tiers | Stakeholder |
| CMO + CEO sign-off on marketing plan | Stakeholder |
| Board sign-off on KPIs + org/comp plan | Stakeholder |
Reference: hmp-docs/docs/sales-and-marketing/ (25 files, 10 sections).
🔴 Property #1 Readiness Audit (2026-04-28 verdict: NO-GO)
| Severity | Count | Status |
|---|
| P0 | 47 | 🔴 Closure rate not currently tracked in this snapshot |
| P1 | 41 | 🔴 |
| P2 | 39 | 🔴 |
Mandatory reading per project CLAUDE.md before any PR on audited code paths:
hmp-docs/docs/platform-registry/property-1-readiness-audit-2026-04-28.mdhmp-docs/docs/platform-registry/property-1-gold-standard-fix-plan-2026-04-28.md
Recommended: schedule a dedicated session to triage current closure rate + prioritize remaining items relative to first-property timing.
✅ Recently completed (for context)
| Item | Date | Notes |
|---|
| Plan 3 D.2 (CI/CD pipelines, 24/24) | 2026-05-17 | All envs deploying via Azure Pipelines |
Plan 5 PROD DNS cutover (api.keynova.ai, live.keynova.ai) | 2026-05-17 | Verified end-to-end with ResAvenue handler |
| FU-52 Reloader deployed all 4 envs | 2026-05-17 18:06-18:13 IST | compliance/audit/2026-05-17-fu-52-reloader-shipped-*.txt |
| FU-50 promote-image.sh rebase-retry | 2026-05-17 | Permanent fix shipped + self-tested live |
FU-7 Kyverno disallow-aws-credentials-env | 2026-05-11 | forEach + AnyIn rewrite; cluster-wide enforcement restored |
| Plan 6 baseline checklist execution (1 PASS / 7 FAIL / 13 NOCREDS / 2 AUTOMODE) | 2026-05-17 | Operator must re-run after AWS profile + flag fixes |
| Plan 4 Tasks 17/19/20/21 | 2026-05-13 | Foundation + 5 cross-service guards |
| Schema/Seed Audit Phases A/B/C/D | 2026-05-15 | 9 migrations + 3 CI gates + 13 demo-data generators |
| Sales & Marketing playbook folder | 2026-05-16 | 25 files / 10 sections / ~5k lines |
Top 5 recommended next actions (priority order)
- Force-rotate test cluster to validate Reloader works end-to-end (10 min, zero customer risk)
- Ship Level 1 hardening — preStop hooks + grace period on all 20 Deployments (1h authoring)
- FU-51 SendGrid prod key provisioning (operator AWS Console step — blocks first-property)
- Q4 on-call rotation setup + Q5 Legal sign-off (stakeholder asks — blocks first-property by 2026-06-15)
- FU-53 metrics-server install for test + qa (small authoring task; closes the HPA-failing-metrics warnings)
Re-snapshot triggers
Re-author this checklist (as a new dated file, don't mutate this one) when any of these happen:
- Any hard blocker closes — recalibrate priorities
- Plan 7 Phase 3 destroy completes — reflect Plan 7 progress + close FU-3
- 2026-05-24 prod RDS rotation event — validates or invalidates FU-52 fix
- Property #1 Readiness Audit re-run completes — new P0/P1/P2 numbers
- Plan 4 Task 24 prod demo seed executes — closes Plan 4
- Plan 5 7-day burn-in completes — closes Plan 5 + opens Plan 7 code cleanup