Skip to content
eSHARS Operations

Investigation: SSRS Report Viewer Connectivity Failure

Date: 2026-03-25 Status: Resolved Outcome: Root cause identified — SSRS VM’s public IP removed, breaking App Service outbound route. Fixed via VNet integration.

Context

Report viewing in the eSHARS web app (Historical Data → SSRS reports) was broken across all nonprod environments after infrastructure changes. The error shown was:

Service unavailable! Unable to connect to the remote server

The changes preceding the failure: the public IP was removed from vm-sql16-np (nonprod SSRS VM) as part of a security posture improvement.

Architecture (relevant)

The eSHARS report viewer uses two hops:

Browser
  → wa-eshars-nonprod (App Service)
      → MvcReportViewer.aspx (renders in same App Service)
          → SSRS (vm-sql16-np) via server-side HTTP call
              → Eshars.ReportViewer (ASP.NET WebForms on IIS on the SSRS VM)
                  → SSRS via localhost

The MvcReportViewer NuGet package (v0.8.3) makes the server-side call from the App Service to SSRS. It reads the SSRS URL from an app setting. Before this investigation, that setting pointed to the old public IP.

Findings

Finding 1: Two app setting keys required

The MvcReportViewer package reads MvcReportViewer.ReportServerUrl (dot-notation). C# code in ReportsController, SAFormService, and ReferralFormService reads MvcReportViewerReportServerUrl (camelCase). Both must be updated together. Updating only the camelCase key produced a partial fix that was confusing to diagnose.

See INFRA-0004 for the full decision record.

Finding 2: App Service cannot reach private IP without VNet integration

Azure App Services run in a managed compute environment that is not natively connected to your VNet. The App Service had no route to 10.1.0.4 (SSRS VM private IP) without explicit VNet integration. This is not obvious from the Azure portal — the App Service and VM appear to be in the same subscription/region but are not network-adjacent by default.

Finding 3: VNet integration is per-slot, not per App Service or App Service Plan

VNet integration must be configured on each deployment slot individually. It is not inherited from the parent App Service or the App Service Plan. wa-eshars-nonprod has 5 slots (dev, qa, regression, uat, trn) — each required its own VNet integration configuration.

Finding 4: Training slot uses a different pattern

The trn slot routes to SSRS via App Gateway (trainingreports.myeshars.com) rather than direct private IP. It uses Key Vault references in app settings pointing to kv-eshars-training. This pattern was pre-existing and working. Changing it to match the other slots risked breaking training — it was left as-is.

See INFRA-0003.

Finding 5: Config source hierarchy

wa-eshars-nonprod (App Service) app settings override Web.config values. Key Vault references are used only for specific settings (e.g., training slot). The Eshars.ReportViewer app on the SSRS VM uses configBuilders="AzureKeyVault" which overlays ALL appSettings from Key Vault at startup — a different pattern from the main web app.

Finding 6: Regression slot had wrong SSRS instance

Regression was initially configured pointing to QAReportServer instead of REGReportServer. Fixed to http://10.1.0.4/REGReportServer.

Finding 7: Removal of configBuilders from ReportViewer Web.config crashes the app

During troubleshooting, configBuilders="AzureKeyVault" was temporarily removed from Eshars.ReportViewer/Web.config to isolate variables. This caused the app to crash on startup (502 Bad Gateway on nonprodreports.myeshars.com) because Key Vault-sourced settings were no longer injected. Reverted by restoring the attribute and recycling the IIS app pool.

Resolution

  1. Created subnet sn-webapp-eshars-np (10.1.3.0/24) in rg-eshars-nonprod VNet, delegated to Microsoft.Web/serverFarms
  2. Added VNet integration to all 5 slots of wa-eshars-nonprod → sn-webapp-eshars-np
  3. Updated both app settings on each slot to use SSRS private IP:
    • MvcReportViewer.ReportServerUrl → http://10.1.0.4/{instance}
    • MvcReportViewerReportServerUrl → http://10.1.0.4/{instance}
  4. Left training slot unchanged (App Gateway pattern working correctly)

All 5 nonprod slots confirmed working post-fix.

Follow-up Actions