Defender: Internet-Exposed VM Remediation

Status: In Progress Created: 2026-02-27 Jira: DEVOPS-35, DEVOPS-123, DEVOPS-128, DEVOPS-142 Trigger: Daily Defender attack path alerts since 2026-01-27 — “Internet exposed Azure VM with high severity vulnerabilities” Subscription: 9639039c-86c6-4859-b014-d28688863cd5

Summary

Defender has identified VMs reachable from the internet with high severity vulnerabilities allowing remote code execution. Investigation found:

2 deleted VMs with orphaned resources (public IPs, NSGs, NICs, disks still allocated)
13 orphaned NSGs with wide-open rules (* source) not attached to any NIC
4 deallocated/stopped VMs with public IPs and open NSGs (will be exposed if started)
7 running VMs with public IPs and NSGs allowing inbound from the entire internet

Phase 1: Orphaned Resource Cleanup

No dev input needed — these resources have no VM behind them. Safe to delete.

1.1 — `hisd-test-vm` (hisd-testing)

VM confirmed deleted — only orphaned resources remain
Delete NIC hisd-test-vm218
Delete public IP hisd-test-vm-ip (52.180.95.103)
Delete NSG hisd-test-vm-nsg (had RDP 3389 + SQL 1433 + HTTP 80 from *)
Delete disk hisd-test-vm_298a3764bc5c4d4caddb6223c749b4d3
Check if hisd-testing RG can be deleted entirely

1.2 — `vm-eshars-work` (ops-esh-sbox)

VM confirmed deleted — only orphaned resources remain
Delete NIC vm-eshars-work722
Delete public IP vm-eshars-work-ip (13.82.69.62)
Delete NSG vm-eshars-work-nsg (had RDP 3389 from *)
Check if ops-esh-sbox RG can be deleted entirely

1.3 — Orphaned NSGs (not attached to any NIC)

All have open inbound rules from *. None are attached to a network interface.

#	NSG	Resource Group	Open Ports	Deleted?
1	vm-chris-deploytest-nsg	vm-chris-deploytest_group	RDP 3389	[ ]
2	vm-cwl-test-nsg	vm-chris-deploytest_group	RDP 3389	[ ]
3	vm-cwl-testmv-nsg	rg-new-infra-southcentralus	RDP 3389	[ ]
4	8-year-cleanup-worker-vm-nsg	rg-eshars-prod	RDP 3389	[ ]
5	vm-cwl-test2-nsg	rg-eshars-prod	RDP 3389	[ ]
6	vm-eshars-build-nsg	rg-eshars-prod	8080	[ ]
7	vm-historic-cleanup-worker-nsg	rg-eshars-prod	RDP 3389	[ ]
8	vm-tmhp-prod2-nsg	rg-eshars-prod	RDP 3389	[ ]
9	vm-tmhp-prod3-nsg	rg-eshars-prod	RDP 3389	[ ]
10	vm-tmhp2-prod-nsg	rg-eshars-prod	RDP 3389	[ ]
11	vm-cwltest-2-nsg	rg-eshars-nonprod	RDP 3389	[ ]
12	vm-keyvault-test-nsg	rg-eshars-nonprod	RDP 3389	[ ]
13	vm-api-connect-nsg	rg-eshars-nonprod	RDP 3389	[ ]

Investigate: vm-tmhp-prod-2-nsg (rg-eshars-nonprod) IS attached to NIC vm-tmhp-prod-2151 — do not delete without checking what it serves.

Phase 2: Deallocated/Stopped VMs

These VMs exist but are not running. They still have public IPs allocated (costing money) and open NSGs that will expose them if anyone starts them.

DEV TEAM: Please confirm for each VM — is it still needed? If yes, why?

2.1 — `vm-eshars-dev` (rg-eshars-dev) — DEALLOCATED

Detail	Value
Public IP	20.81.233.176
NSG open from `*`	HTTP 80, FTP 20, FTP 21, passive FTP 60000-65000, Seq 5341
Power state	Deallocated

Dev response: Is this VM still needed? What is it used for?
Decision: Keep / Delete / Keep but lock down NSG
Action taken: _________________

2.2 — `import-staging-deletion-agent` (rg-eshars-historical-data-deletion) — DEALLOCATED

Detail	Value
Public IP	20.29.49.122
NSG open from `*`	RDP 3389
Power state	Deallocated

Dev response: Is this VM still needed? Was this a one-time historical data deletion job?
Decision: Keep / Delete / Keep but lock down NSG
Action taken: _________________

2.3 — `vm-tmhp-prod1` (rg-eshars-prod) — STOPPED

Detail	Value
Public IP	~~40.122.162.156~~ Removed 2026-03-21
NSG rules from `*`	RDP 3389 (blocked by DenyAllInbound_Isolation at pri 100)
Power state	Stopped (still billing for reserved resources)

Dev response: Is this decommissioned? vm-tmhp-prod is running — is prod1 a predecessor?
Decision: Keep / Delete / Keep but lock down NSG
Action taken: Public IP removed, orphaned IP deleted. Bastion allow rule added. (2026-03-21)

2.4 — `vm-tmhp-int` (rg-eshars-prod) — DEALLOCATED

Detail	Value
Public IP	~~52.165.170.168~~ Removed 2026-03-21
NSG open from `*`	~~20150-25150~~ Set to Deny 2026-03-21
Power state	Deallocated

Dev response: Is this still needed? What are ports 20150-25150 for?
Decision: Keep / Delete / Keep but lock down NSG
Action taken: Public IP removed, passive FTP rule set to Deny, Bastion allow rule added. Zero network traffic — likely legacy/standby. (2026-03-21)

Phase 3: Running VMs — Nonprod

These VMs are running with public IPs and NSG rules allowing inbound from * (the entire internet).

DEV TEAM: For each VM, please answer:

Does this VM need a public IP, or can it be accessed via Bastion/VPN?

If it needs open ports, what source IPs should be allowed instead of *?

Why is this port open? What service uses it?

3.1 — `vm-db-api` (rg-eshars-nonprod) — RUNNING

Detail	Value
Public IP	13.86.64.151
NSG open from `*`	RDP 3389
Risk	CRITICAL — RDP brute force from internet

Dev response: Why does a DB API VM need RDP from the internet?
Recommended: Remove * rule, use Bastion or restrict to office/VPN IP
Action taken: Public IP removed, AllowBastionRDP rule added, NAT gateway for outbound. (2026-03-21)

3.2 — `vm-successed-nonprod` (rg-eshars-nonprod) — RUNNING

Detail	Value
Public IP	20.15.225.23
NSG open from `*`	FTP 21, SSH 22, ports 5001-5051
Risk	HIGH — FTP is unencrypted, SSH brute force

Dev response: What service runs on this VM? Why FTP instead of SFTP? What are ports 5001-5051?
Recommended: Replace FTP with SFTP, restrict all source IPs
Action taken: Public IP removed, AllowBastionRDP rule added, NAT gateway for outbound. VM is no longer internet-exposed, but existing NSG open port rules still allow access from within the VNet/peered networks; inbound NSG tightening remains required. (2026-03-21)

3.3 — `vm-jobscheduler` (rg-eshars-nonprod) — RUNNING

Detail	Value
Public IP	20.114.116.215
NSG open from `*`	HTTPS 443, WinRM 5986
Risk	HIGH — WinRM allows remote management from internet

Dev response: Why does the nonprod job scheduler need HTTPS from the internet? What uses WinRM 5986?
Recommended: Restrict source IPs, remove WinRM * rule
Action taken: VM on separate VNet (southcentralus) — disk copy to centralus in progress. Will rebuild on main nonprod VNet, remove public IP. Old VM kept as fallback. (2026-03-21 — IN PROGRESS)

3.4 — `vm-sql16-1-np` (rg-eshars-nonprod-centralUS) — RUNNING

Detail	Value
Public IP	40.122.54.1
NSG open from `*`	HTTP 80, HTTPS 443
Risk	HIGH — SQL Server VM serving web traffic directly?

Dev response: Why does a SQL Server VM have HTTP/HTTPS open from the internet? What web service runs here?
Recommended: Put behind App Gateway or restrict source IPs
Action taken: _________________

Note (2026-03-03): DNS nonprodreports.myeshars.com resolves to 40.113.238.4 (vm-sql16-np). These VMs likely host SSRS + Eshars.ReportViewer for nonprod — same pattern as prod. Check MvcReportViewerReportServerUrl in kv-eshars-qa/kv-eshars-uat/kv-eshars-regression for HTTP URLs pointing at these VMs on port 80.

3.5 — `vm-sql16-np` (rg-eshars-nonprod) — RUNNING

Detail	Value
Public IP	40.113.238.4
NSG open from `*`	HTTP 80, HTTPS 443
Risk	HIGH — same concern as vm-sql16-1-np

Dev response: Same question — why HTTP/HTTPS from internet on a SQL VM?
Recommended: Put behind App Gateway or restrict source IPs
Action taken: _________________

Note (2026-03-03): This is the nonprod SSRS VM. nonprodreports.myeshars.com → 40.113.238.4. See prod findings in Phase 4.1 — same MvcReportViewerReportServerUrl HTTP pattern likely applies here.

Phase 4: Running VMs — Prod

Highest care required. Changes here affect production services.

4.1 — `vm-ssrs16-prod` (rg-eshars-prod) — RUNNING

Detail	Value
Public IP	23.99.180.223
NSG open from `*`	HTTP 80, HTTPS 443
Risk	HIGH — SSRS reports accessible from entire internet
DNS	`prodreports.myeshars.com` → `23.99.180.223` (no App Gateway)

Investigation Findings (2026-03-03)

Port 443 (HTTPS) — required for user-facing access. SSRSReportServerUrl in all Key Vaults already uses HTTPS:

Key Vault	Value
`kv-eshars-prod`	`https://prodreports.myeshars.com`
`kv-eshars-qa`	`https://nonprodreports.myeshars.com`
`kv-eshars-uat`	`https://nonprodreports.myeshars.com`
`kv-eshars-training`	`https://trainingreports.myeshars.com`
`kv-eshars-regression`	`https://nonprodreports.myeshars.com`

Port 80 (HTTP) — required because of server-to-server calls. MvcReportViewerReportServerUrl in kv-eshars-prod is:

http://23.99.180.223/ReportServer

This is used by wa-eshars-prod (Eshars.WebUI) for server-side PDF rendering:

ReportsController.cs:724-730 — PDF downloads
SAFormService.cs:371 — SA form rendering
ReferralFormService.cs:323 — referral form rendering

The Eshars.ReportViewer app is deployed on this VM alongside SSRS. It calls SSRS via http://localhost/ReportServer (from kv-eshars-reportviewer), which is internal to the VM and unaffected by NSG rules.

Architecture:

User browser ──HTTPS 443──▶ vm-ssrs16-prod (IIS)
                                ├── Eshars.ReportViewer ──http://localhost:80──▶ SSRS
                                └── SSRS ReportServer

wa-eshars-prod ──HTTP 80──▶ vm-ssrs16-prod:80/ReportServer  ← PROBLEM
  (WebUI App Service)         (plain HTTP over public internet)

Remediation Steps

Verify https://prodreports.myeshars.com/ReportServer responds correctly
Update MvcReportViewerReportServerUrl in kv-eshars-prod to https://prodreports.myeshars.com/ReportServer
Restart wa-eshars-prod to pick up new config
Test: PDF downloads, SA forms, referral forms, progress notes
Close port 80 on NSG, verify reports still work
Repeat for nonprod vaults (kv-eshars-qa, kv-eshars-uat, kv-eshars-regression, kv-eshars-training)
Restrict port 443 source IPs (currently *) — at minimum to wa-eshars-prod outbound IPs

4.2 — `vm-eshars-sftp` (vm-eshars-sftp_group) — RUNNING

Detail	Value
Public IP	40.77.98.52
NSG open from `*`	SSH 22
Risk	MEDIUM — SFTP by nature needs external access, but should be restricted

Dev response: Which external partners/IPs connect to this SFTP? (HISD, TMHP, others?)
Recommended: Restrict SSH 22 source to known partner IP ranges
Action taken: _________________

Remediation Log

Date	Item	Action	By
2026-02-27	Investigation	Audited all VMs, public IPs, and NSGs in subscription	Leon
2026-03-03	vm-ssrs16-prod	Root cause found: `MvcReportViewerReportServerUrl` in `kv-eshars-prod` uses `http://23.99.180.223/ReportServer` (HTTP). User-facing `SSRSReportServerUrl` already HTTPS. Fix: update KV secret to HTTPS, then close port 80	Leon
2026-03-21	Infrastructure	Deployed Azure Bastion (`bastion-eshars-prod`, Developer SKU) to prod VNet for RDP/SSH without public IPs	Leon
2026-03-21	Infrastructure	Deployed NAT Gateway (`natgw-eshars-prod` + `pip-natgw-prod`) on default subnet for outbound internet after public IP removal	Leon
2026-03-21	vm-jobscheduler-prod	Removed public IP (40.86.93.124), deleted orphaned IP resource. Now private-only (10.0.0.9)	Leon
2026-03-21	vm-tmhp-int	Removed public IP (52.165.170.168), deleted orphaned IP resource. Now private-only (10.0.0.8). Set passive FTP rule (20150-25150) to Deny	Leon
2026-03-21	vm-tmhp-prod	Removed public IP, deleted orphaned IP resource. Now private-only (10.0.0.7)	Leon
2026-03-21	vm-rns-prod	Removed public IP (20.84.144.199), deleted orphaned IP resource. Now private-only (10.0.0.10)	Leon
2026-03-21	vm-tmhp-prod1	Removed public IP (40.122.162.156), deleted orphaned IP resource. Now private-only	Leon
2026-03-21	NSGs (x6)	Added `AllowBastionRDP` rule (pri 200, TCP 3389+22 from 10.0.1.0/26 + 168.63.129.16) to: vm-rns-prod-nsg, vm-jobscheduler-prod-nsg, vm-tmhp-int-nsg, vm-ssrs16-prod, vm-tmhp-prod1-nsg, vm-eshars-sftp-nsg	Leon
2026-03-21	Patch assessment	All 4 assessed VMs fully patched (0 missing): vm-rns-prod, vm-jobscheduler-prod, vm-tmhp-prod, vm-tmhp-int. vm-eshars-sftp assessment failed (agent issue)	Leon
2026-03-21	vm-tmhp-prod	Confirmed still active — 2.2 GB/day network traffic. Do not decommission	Leon

References

Defender Findings Register
Jira: DEVOPS-35, DEVOPS-123, DEVOPS-128, DEVOPS-142
Defender attack path subscription scope: 9639039c-86c6-4859-b014-d28688863cd5

Defender: Internet-Exposed VM Remediation

Summary

Phase 1: Orphaned Resource Cleanup

1.1 — hisd-test-vm (hisd-testing)

1.2 — vm-eshars-work (ops-esh-sbox)

1.3 — Orphaned NSGs (not attached to any NIC)

Phase 2: Deallocated/Stopped VMs

2.1 — vm-eshars-dev (rg-eshars-dev) — DEALLOCATED

2.2 — import-staging-deletion-agent (rg-eshars-historical-data-deletion) — DEALLOCATED

2.3 — vm-tmhp-prod1 (rg-eshars-prod) — STOPPED

2.4 — vm-tmhp-int (rg-eshars-prod) — DEALLOCATED