Architecture Deployment API

Operational View

Describes how the system is operated, monitored, and maintained once deployed. Covers health checking, error handling, configuration management, and known operational gaps.

System Health Checks

Container orchestration (Docker Compose in dev, Kubernetes in production) continuously polls each service’s dedicated health check. The API’s /health endpoint is the top-level signal for the application layer. Each infrastructure service exposes its own probe — PostgreSQL via pg_isready, Redis via redis-cli ping, MQTT via mosquitto_sub, and Keycloak via its /health/ready endpoint. All five must report healthy for the platform to be considered fully operational; a failure in any one of them will block dependent service startup.

flowchart TD
  subgraph HealthChecks["Health Check Endpoints"]
    HC["GET /health\n(ASP.NET Core built-in)"]
    PGH["pg_isready -U postgres\n(postgres-dev)"]
    RH["redis-cli ping\n(redis-dev)"]
    MQH["mosquitto_sub -t $$SYS/broker/uptime\n(mqtt-dev)"]
    KCH["curl /health/ready\n(keycloak-dev)"]
  end

  ORCH["Container Orchestrator\n(Docker Compose / Kubernetes)"] -->|polls| HC
  ORCH --> PGH
  ORCH --> RH
  ORCH --> MQH
  ORCH --> KCH

Sub-domain Mapping — Health Checks

Health Check Business Sub-domain What breaks if it fails
GET /health (API) All sub-domains Entire API unavailable — no bouquet discovery, orders, or vendor management
pg_isready (PostgreSQL) All sub-domains No persistence; all commands and queries fail immediately
redis-cli ping (Redis) Identity & Tenancy, Customer Purchasing, Payments Token blacklist, idempotency, and query cache all unavailable; duplicate orders possible
mosquitto_sub (MQTT) IoT / MQTT Vase telemetry (heartbeats, sensor data) not ingested; freshness scores stale
/health/ready (Keycloak) Identity & Tenancy Vendor and customer login fails; dev-token mode unaffected

Request Error Handling Pipeline

flowchart TD
  REQ["Incoming HTTP Request"] --> GEX["GlobalExceptionMiddleware\n(catches all unhandled exceptions)"]
  GEX -->|unhandled exception| PD["ProblemDetails response\n500 Internal Server Error"]
  GEX --> AUTH["Authentication\n(401 Unauthorized if JWT invalid/missing)"]
  AUTH --> TENANT["TenantResolutionMiddleware\n(403 if claims missing for protected routes)"]
  TENANT --> BLACKLIST["TokenBlacklistMiddleware\n(401 if JTI revoked in Redis)"]
  BLACKLIST --> AUTHZ["Authorization Policies\n(403 if role/policy mismatch)"]
  AUTHZ --> RATE["RateLimiter\n(429 Too Many Requests)"]
  RATE --> IDEMP["IdempotencyMiddleware\n(replays cached response if key exists)"]
  IDEMP --> CTRL["Controller → MediatR Handler"]
  CTRL -->|Result.Error| MAP["Error string → HTTP status mapping\n(400 / 404 / 409 / 422 etc.)"]
  MAP --> RESP["HTTP Response"]

Sub-domain Mapping — Error Pipeline

Middleware layer Business Sub-domain Error it handles
GlobalExceptionMiddleware All sub-domains Uncaught exceptions — wraps as 500 ProblemDetails
Authentication (401) Identity & Tenancy Missing or invalid Keycloak/dev JWT token
TenantResolutionMiddleware (403) Identity & Tenancy JWT present but lacks required vendor_id / customer_id claims
TokenBlacklistMiddleware (401) Identity & Tenancy Revoked JWT jti — logged-out session attempting reuse
Authorization Policies (403) All sub-domains Role/policy mismatch (e.g., Customer accessing a Vendor endpoint)
RateLimiter (429) All sub-domains Abuse protection; applies globally across all routes
IdempotencyMiddleware (replays) Customer Purchasing / Payments Duplicate POST /orders or payment confirmation with same idempotency key
Controller error mapping All sub-domains Result.Error string prefixes (bouquet-unavailable, vendor-mismatch, …) mapped to 400 / 404 / 409

Logging

The API uses ASP.NET Core’s built-in structured logging with RequestIdMiddleware injecting X-Request-ID on every request, enabling log correlation across services.

Log Level Logged By
Information Request start/end (LoggingBehaviour), domain event dispatch
Warning Validation failures, cache misses, token blacklist hits
Error Unhandled exceptions (GlobalExceptionMiddleware), Keycloak admin API failures

Log files are written to src/backend/FlowerShop.API/logs/ (mapped via Docker volume api_dev_logs).

Operational Runbook — Common Procedures

Starting the platform (dev)

# Start infrastructure
docker-compose -f docker/docker-compose.dev.yml up -d

# Start the API (hot reload)
cd src/backend/FlowerShop.API
dotnet watch run

# Start portals (separate terminals)
cd src/portals/FlowerShop.AdminPortal  && dotnet run   # :5001
cd src/portals/FlowerShop.VendorPortal && dotnet run   # :5002
cd src/portals/FlowerShop.CustomerApp  && dotnet run   # :5003

Running database migrations

cd src/backend/FlowerShop.Infrastructure
dotnet ef migrations add <MigrationName> --startup-project ../FlowerShop.API

cd ../FlowerShop.API
dotnet ef database update

Running tests

dotnet test    # runs all 126 tests across 3 test projects

Checking Swagger

Navigate to http://localhost:8080/swagger — all 23 controllers and their endpoints are documented.

Sub-domain Mapping — Monitoring Signals

Signal Business Sub-domain Meaning
/health → unhealthy All sub-domains DB or Redis unreachable — immediate action required
outbox_messages accumulating All sub-domains OutboxDispatcherService stalled or InMemoryEventBus failing — domain events not delivered
SmartVase.ConnectionStatus = Offline spike IoT / MQTT MQTT broker issue or network partition affecting vases; freshness scoring degraded
Redis memory growing Identity & Tenancy, Customer Purchasing Token blacklist or idempotency key TTL misconfigured; check key expiry policies
Keycloak admin API errors Identity & Tenancy Keycloak unreachable; vendor onboarding (VendorOnboardingService) will fail silently

Known Operational Gaps (active stubs)

Gap Impact Planned Fix
RabbitMQEventBus.PublishAsync returns Task.CompletedTask Domain events are silently dropped; InMemoryEventBus is active instead EP-09 S-09-06
AnalyticsController calls VendorId.New() in 4 actions All analytics data is wrong (random GUID instead of tenant’s VendorId) EP-10 T-10-001
VendorPortal reads VendorId from session instead of JWT claim Vendor isolation broken for portal users EP-12 T-12-003
VendorPortal POSTs photos to hardcoded http://localhost:8080/api/mock-vase/... Photo upload fails in any non-localhost environment EP-12 T-12-004
INotificationServiceClient is a stub FCM push notifications not delivered to mobile devices EP-06
IStripeService is a stub Payment not wired in CreateOrderCommandHandler EP-04
Cart + OrderLine DB tables do not exist Cart and multi-line order endpoints return DB errors until migration is run Run AddCartAndOrderLines migration
CachingGeocodingService uses IMemoryCache Geocoding cache is process-local; lost on restart; not shared across instances Intentional for now; revisit in EP-09

Monitoring Signals

Signal Source Meaning
/health → unhealthy API process Immediate attention — DB or Redis unreachable
outbox_messages rows with processed_at IS NULL accumulating PostgreSQL OutboxDispatcherService not running or event bus failing
SmartVase.ConnectionStatus = Offline spike PostgreSQL MQTT broker issue or network partition affecting vases
Redis memory usage growing Redis Token blacklist or idempotency key TTL misconfigured
Keycloak admin API errors in logs API logs Keycloak service unreachable; vendor onboarding will fail

Sub-domain Mapping — Operational Gaps

Gap Business Sub-domain Impact
RabbitMQEventBus stub All sub-domains (async events) Domain events dropped in production; InMemoryEventBus is the active fallback
AnalyticsController calls VendorId.New() Vendor Management / Platform Administration All analytics responses return data for a random, non-existent vendor
VendorPortal reads VendorId from session Vendor Management Vendor isolation broken in the portal — any session can access any vendor’s data
INotificationServiceClient stub Customer Purchasing FCM push notifications not delivered to any mobile device
IStripeService stub Payments Orders complete without actual payment processing
Cart + OrderLine tables missing Customer Purchasing Cart and multi-line order endpoints return DB errors until migration is run

Security Operations

flowchart LR
  subgraph TokenLifecycle["JWT Token Lifecycle"]
    ISSUE["Keycloak issues JWT\n(sub, role, vendor_id, vendor_type, customer_id)"]
    USE["API validates JWT\nTenantResolutionMiddleware reads claims"]
    REVOKE["POST /auth/revoke\nJTI added to Redis blacklist"]
    CHECK["TokenBlacklistMiddleware\nchecks Redis on every request"]
  end

  ISSUE --> USE --> REVOKE --> CHECK --> USE
  • Token revocation is synchronous and Redis-backed — revoked tokens are blocked on the next request.
  • Multi-tenancy isolation is enforced at the MediatR layer via TenantValidationBehavior — handlers cannot accidentally access another tenant’s data.
  • Rate limiting is applied globally via ASP.NET Core’s built-in RateLimiter middleware.
  • Idempotency prevents duplicate orders even under concurrent client retries.