Azure Automation Runbooks

Exported from aa-eshars-prod (and related accounts) in rg-eshars-prod. Scripts live in eshars-ops/scripts/azure-runbooks/.

Automation Accounts

Current (v2, Central US) — see the v2 page for detail:

Account	Resource group	Purpose
`aa-eshars-nonprod-cus`	`rg-eshars-ops-nonprod-cus`	Self-service restore/seed for dev, qa, regression
`aa-eshars-prod-cus`	`rg-eshars-ops-prod-cus`	Self-service restore for uat, training, prod

Legacy (v1, East US 2) — the rest of this page:

Account	Runbooks	Purpose
`aa-eshars-prod`	25	Database lifecycle — copies, scrubs, exports, monitoring
`aa-eshars-prod-azmodule`	1	SQL database scaling
`JobLogsCleanup`	5	Purge old Quartz job log records (per environment)
`PeriodicCleanup`	5	General application data cleanup (per environment)

Server	Hosts	Resource Group
`db-eshars-prod`	Production, UAT, Regression databases	`rg-eshars-prod`
`db-eshars-nonprod`	QA, Training databases	`rg-eshars-nonprod`
`db-eshars-developeraccess`	Scrubbed dev copies	`rg-eshars-nonprod`

Account	Container	Purpose
`sahisddatabackup`	`hisdbackups`	Monthly exports
`sahisddatabackup`	`hisdbackups-weekly`	Weekly exports (13-day retention)
`sahisddatabackup`	`hisd-prod-snapshots-iep`	UAT/fiscal-year snapshots
`esharsdbstorage`	`db-storage`	On-demand developer .bacpac files

Runbook	Source	Target	Notes
`aa-eshars-prod-to-prodtest`	prod	developeraccess/prodtest	Main DB for dev testing
`aa-eshars-prod-audit-to-prodtest-audit`	prod-audit	developeraccess/prodtest-audit	Copies at S3, scales to P2
`aa-eshars-prod-to-regression-audit`	prod-audit	nonprod/regression-audit01	Legacy RunAs auth
`aa-eshars-prod-to-regression-log`	db-log-prod	prod/regression-log01	Same-server copy
`aa-qa-to-regression`	nonprod/qa	prod/regression-ds	Cross-server copy
`ar-eshars-regressiondb-copy`	prod	prod/regression-ds	Same-server copy
`ar-eshars-uatdb-copy`	prod	prod/uat-data	Same-server copy
`ar-eshars-database-copy`	(parameterized)	(parameterized)	Generic copy with overwrite flag

These runbooks copy production data to a developer-accessible server, scrub all PII, and export a .bacpac. See PII Scrub Pipeline for full details.

Runbook	Status	Auth	Notes
`ar-eshars-database-scrub`	Current	Managed Identity	Auto-cleanup after export. Runs on Hybrid Worker `run-bacpacs-group`
`ar-eshars-proddb-copy-and-scrub-for-devs`	Legacy	Managed Identity + RunAs	Hardcoded storage key — do not use
`Test-latestprodcopyto-sqldeveloperaccess`	Legacy	RunAs	Hardcoded storage key — do not use
`ar-eshars-drop-db-eshars-backupscrub`	Legacy	RunAs	Misleading name — actually a full copy+scrub, not a drop
`aa-eshars-prod-to-qa-scrub`	Active	RunAs	Prod → QA/Regression with environment-aware scrub
`ar-eshars-qa-copy-for-dev`	Active	Managed Identity	QA → dev; audit-table-only scrub

See Backup & Export Schedule for schedule, retention, and storage layout.

Runbook	Databases	Storage	Retention
`ar-eshars-weekly-db-export`	prod, prod-audit	hisdbackups-weekly	13 days
`ar-eshars-weekly-logdb-export`	db-log-prod	hisdbackups-weekly	13 days (cleanup commented out)
`ar-eshars-monthly-db-export`	prod, prod-audit, db-log-prod	hisdbackups	None (overwrites)
`ar-eshars-uat-db-fiscal-update`	prod, prod-audit, db-log-prod	hisd-prod-snapshots-iep	7 years
`ar-eshars-uat-db-bacpac-update`	uat-data, uat-audit, uat-log	hisd-prod-snapshots-iep	None

Runbook	Source	Target	Notes
`ar-eshars-db-restore-from-bacpac`	hisd-prod-snapshots-iep/*.bacpac	prod/uat-data	Imports at P4 tier. Hardcoded bacpac filename
`ar-eshars-uat-fiscal-update-test`	hisd-prod-snapshots-iep/*.bacpac	prod/uat-data	Drops existing UAT, imports at P6

Runbook	Purpose
`ar-eshars-elasped-time-prod`	Measures prod DB query latency, posts to Slack `#proj-eshars-devops`
`ar-eshars-elasped-time-nonprod`	Measures QA DB query latency, posts to Slack `#proj-eshars-devops`
`ar-eshars-clear-logs`	Deletes ELMAH error logs older than 31 days. Parameterized (server, DB, creds)
`TriggerRnSFlow`	Runs `ruby main.rb` on `vm-rns-prod` via `Invoke-AzVMRunCommand`
`ScaleAzureSQLDB`	Scales a SQL DB to a specified DTU tier (Basic, S0–S12, P1–P15)
`eSHARS-JobScheduler-OOM-Restart`	Triggered by Azure Monitor alert on Event 1026. Restarts `eSharsJobScheduler_PRODJOB` on `vm-jobscheduler-prod` via Hybrid Worker, then sends Teams alert

Calls sp_CleanupOldJobData on each environment’s log database. Server/database names are hardcoded in each script.

Variant	Server	Database	Auth
`CleanRecords_PROD`	db-eshars-prod	db-log-prod	AutomationPSCredential
`CleanRecords_QA`	db-eshars-nonprod	db-log-qa	AutomationPSCredential
`CleanRecords_Regression`	db-eshars-prod	db-eshars-regression-log01	AutomationPSCredential
`CleanRecords_Training`	db-eshars-nonprod	db-log-trn	AutomationPSCredential
`CleanRecords_UAT`	db-eshars-prod	db-eshars-uat-log	AutomationPSCredential

Calls sp_CleanupData on each environment’s main application database. Server/database names pulled from Azure Automation Variables.

Misleading filename — ar-eshars-drop-db-eshars-backupscrub.ps1 is actually a full copy+scrub+export pipeline, not a simple database drop.
Hardcoded dates — ar-eshars-uat-db-bacpac-update and ar-eshars-uat-db-fiscal-update have hardcoded date suffixes. Manual update required each run.
Legacy auth — ~50% of runbooks still use AzureRunAsConnection (deprecated). Migration to Managed Identity is partially complete (see below).
Weekly log export retention disabled — ar-eshars-weekly-logdb-export has the blob cleanup line commented out.

Pattern	Runbooks	Status
`Connect-AzAccount -Identity` (Managed Identity)	~13	Current
`AzureRunAsConnection` (Service Principal)	~12	Deprecated — migrate
`Get-AutomationPSCredential` (SQL auth only)	10	Expected for SQL connections