Audit Azure VM Storage and Ownership Fast with this KQL Query

Ever

Ever been asked to pull a report of specific Azure VMs, their OS versions, who owns them, and exactly how much storage they are chewing up? Clicking through the Azure Portal to piece this together is a massive time sink.

This is where Azure Resource Graph (ARG) shines. I use the KQL query below whenever I need to quickly audit environments for cost allocation, cleanup projects, or basic inventory tracking.

Why this query is useful:

  • Calculates real storage: It grabs the OS disk and iterates through all attached data disks to give you a true TotalCombinedStorageGB metric. It also safely handles edge cases, like VMs with zero data disks, without dropping them from the report.
  • Makes it readable: It joins backend tables to swap the ugly Subscription ID for the human-readable Subscription Name.
  • Filters like a pro: It extracts the Owner tag and the OSVersion, allowing you to easily filter for specific teams or operating systems using fuzzy matches and lists.

How to use it:

  1. Open Resource Graph Explorer in the Azure Portal.
  2. Paste the KQL query below.
  3. Update the YOUR FILTERS section with your target OS and Owner names.
  4. Hit Run and export your clean table directly to a CSV!
Resources
| where type =~ 'microsoft.compute/virtualmachines'
| extend VMName = tostring(name)
| extend Region = tostring(location)
| extend ResourceGroup = tostring(resourceGroup)
| extend OSType = tostring(properties.storageProfile.osDisk.osType)
| extend OSVersion = strcat(tostring(properties.storageProfile.imageReference.offer), " ", tostring(properties.storageProfile.imageReference.sku))
| extend Owner = tostring(tags['Owner'])
// --- YOUR FILTERS ---
| where OSVersion contains "<ENTER_OS_VERSION_HERE>"
// The in~ operator checks against a list of values
| where Owner in~ ("<OWNER_1>", "<OWNER_2>", "<OWNER_3>")
// --------------------
| extend OSDiskSizeGB = toint(properties.storageProfile.osDisk.diskSizeGB)
| extend OSDiskSizeGB = iff(isnull(OSDiskSizeGB), 0, OSDiskSizeGB)
| extend DataDisks = properties.storageProfile.dataDisks
| extend DataDisks = iff(isnull(DataDisks) or array_length(DataDisks) == 0, dynamic([{"diskSizeGB": 0}]), DataDisks)
| mv-expand DataDisks
| extend DataDiskSizeGB = toint(DataDisks.diskSizeGB)
| extend DataDiskSizeGB = iff(isnull(DataDiskSizeGB), 0, DataDiskSizeGB)
| summarize TotalDataDiskSizeGB = sum(DataDiskSizeGB) by id, VMName, ResourceGroup, subscriptionId, Region, OSType, OSVersion, Owner, OSDiskSizeGB
| extend TotalCombinedStorageGB = OSDiskSizeGB + TotalDataDiskSizeGB
| join kind=leftouter (
    ResourceContainers
    | where type =~ 'microsoft.resources/subscriptions'
    | project subscriptionId, SubscriptionName = tostring(name)
) on subscriptionId
| project
    VMName,
    ResourceGroup,
    SubscriptionName,
    Region,
    OSType,
    OSVersion,
    Owner,
    OSDiskSizeGB,
    TotalDataDiskSizeGB,
    TotalCombinedStorageGB
| order by VMName asc

Kafka Fundamentals

Apache Kafka is a powerhouse for real-time data streaming, acting like a superhighway for data that never sleeps. For database engineers dipping their toes into streaming, it’s your bridge from batch processing to instant insights.

Kafka Basics

Picture Kafka as a distributed post office for massive data volumes. Producers (apps sending data) drop messages into topics—logical channels like mailboxes. These topics split into partitions across brokers (Kafka servers) for scalability and parallelism. Consumers subscribe to topics, pulling messages at their pace, with replication ensuring no data loss even if servers fail.

(see the generated image above)

This setup delivers high throughput (millions of messages/second), low latency, and fault tolerance—perfect for evolving from SQL queries to event streams.

Core Components

  • Producers: Push data from sources like databases or sensors into Kafka topics.
  • Brokers and Partitions: Brokers store data durably on disk; partitions enable horizontal scaling.
  • Consumers: Read from topics independently, supporting multiple apps per stream.
  • Connect and Streams: Kafka Connect links external systems (e.g., databases via CDC); Kafka Streams or Flink processes data in-flight for transformations.

As DBAs, think JDBC/CDC plugins feeding Kafka for real-time replication, sidestepping laggy ETL jobs.

Key Benefits

Kafka shines in durability (disk-backed logs for replays), pub-sub flexibility (one producer, many consumers), and seamless scaling.

(see the generated image above)

It integrates with your stack—PostgreSQL CDC to Kafka, then to Redshift or Elasticsearch—boosting monitoring like PMM/Grafana with live metrics.

Healthcare Wins

In healthcare, Kafka streams patient vitals from wearables for instant alerts, aggregates EHR logs for fraud detection, or pipes CDC from hospital databases to analytics for outbreak tracking—all HIPAA-compliant with encryption and audits. For DB engineers, it’s CDC gold: capture changes from MySQL/SQL Server in real-time, feeding ML models without downtime, unlike traditional replication.

Use Cases

  • Real-time monitoring (e.g., ICU telemetry).
  • Data integration (EHR to billing systems).
  • CDC for compliant syncing.
  • Event-driven apps (appointment reminders via microservices).

Azure PostgreSQL Flexible Server: Inactive Replication Slots Eating Your Storage (And How to Fix It)

Inactive replication slots in Azure Database for PostgreSQL Flexible Server can silently fill your disk with WAL files. Here’s how to spot, drop, and prevent them.

The Problem: WAL Explosion from Orphaned Slots

Replication slots ensure WAL retention so consumers (CDC tools, read replicas) don’t miss changes. Inactive slots—created by stopped CDC jobs, deleted replicas, or failed experiments—pin old WAL indefinitely, consuming storage until it fills. Azure Flexible Server has safeguards like auto-grow, but slots can still cause outages.

Spot the Culprits

Run these to identify storage hogs:

-- WAL retained by each slot (biggest first)
SELECT slot_name, plugin, slot_type, active, 
       pg_size_pretty(pg_wal_lsn_diff(restart_lsn, '0/0')) AS retained_wal
FROM pg_replication_slots
ORDER BY pg_wal_lsn_diff(restart_lsn, '0/0') DESC;
-- Current lag relative to WAL head
SELECT slot_name, 
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS lag_size
FROM pg_replication_slots;
-- Inactive slots only
SELECT * FROM pg_replication_slots WHERE NOT active;
-- Check active physical replication (HA/replicas)
SELECT pid, state, sent_lsn, replay_lsn, write_lag 
FROM pg_stat_replication;

Focus on inactive logical slots (slot_type='logical'active=false).

Clean Them Up Safely

Drop one by one—never active or Azure HA slots like azure_standby:

SELECT pg_drop_replication_slot('your_inactive_slot_name');

Storage recovers as WAL checkpoints recycle old segments (minutes to hours). Verify with Azure Metrics > Disk Used.

Prevention: Best Practices

PracticeActionWhy
Monitor slotsAlert on inactive slots >24h or WAL >20% diskCatches issues early 
Limit WAL retentionSet max_slot_wal_keep_size = '20GB' (PG13+)Auto-invalidates lagging slots 
Config for CDC/replicaswal_level=logicalmax_replication_slots >= replicas + CDC + 4 (HA)Reserves space 
Cleanup workflowDrop slot before stopping CDC job/replicaNo orphans 
Azure limitsCheck portal Server Parameters; monitor replicasHA needs ~4 slots 

Real-World Traps

  • CDC tools (Debezium, DMS, Fivetran) create per-task slots; drop on job stop.
  • Deleted read replicas leave slots; check Azure portal first.
  • HA failover recreates azure_standby; ignore it.

Automating SQL Server User Removal with PowerShell and dbatools

Introduction

When an employee leaves or a service account is retired, it’s essential to remove their access cleanly and consistently from SQL Server.
Manually revoking access across multiple databases can be error-prone and time-consuming — especially in large environments.

In this post, we’ll look at how to use the dbatools PowerShell module to automatically remove a user from all databases (except system ones) and drop the server-level login, with full logging for audit purposes.


Prerequisites

  • Install dbatools (if not already installed): Install-Module dbatools -Scope CurrentUser -Force
  • Ensure you have sysadmin rights on the SQL instance.
  • Have the login name ready (domain or SQL account).

The PowerShell Script

<#
.SYNOPSIS
Removes a SQL Server login and its users from all user databases.
Works for both domain and SQL logins, with logging.
#>

param(
    [Parameter(Mandatory = $true)]
    [string]$SqlInstance,
    [Parameter(Mandatory = $true)]
    [string]$Login,
    [string]$LogFile = "$(Join-Path $PSScriptRoot ("UserRemovalLog_{0:yyyyMMdd_HHmmss}.txt" -f (Get-Date)))"
)

if (-not (Get-Module -ListAvailable -Name dbatools)) {
    Write-Error "Please install dbatools using: Install-Module dbatools -Scope CurrentUser -Force"
    exit 1
}

function Write-Log {
    param([string]$Message, [string]$Color = "White")
    $timestamp = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
    $logEntry = "[$timestamp] $Message"
    Write-Host $logEntry -ForegroundColor $Color
    Add-Content -Path $LogFile -Value $logEntry
}

Write-Log "=== Starting cleanup for login: $Login on instance: $SqlInstance ===" "Cyan"

$UserDatabases = Get-DbaDatabase -SqlInstance $SqlInstance | Where-Object { -not $_.IsSystemObject }

foreach ($db in $UserDatabases) {
    try {
        $dbName = $db.Name
        $user = Get-DbaDbUser -SqlInstance $SqlInstance -Database $dbName -User $Login -ErrorAction SilentlyContinue
        if ($user) {
            Write-Log "Removing user [$Login] from [$dbName]" "Red"
            Remove-DbaDbUser -SqlInstance $SqlInstance -Database $dbName -User $Login -Confirm:$false -ErrorAction Stop
            Write-Log "✅ Removed from [$dbName]" "Green"
        }
        else {
            Write-Log "User [$Login] not found in [$dbName]" "DarkGray"
        }
    }
    catch {
        Write-Log "⚠️ Failed in [$dbName]: $_" "Yellow"
    }
}

try {
    $loginObj = Get-DbaLogin -SqlInstance $SqlInstance -Login $Login -ErrorAction SilentlyContinue
    if ($loginObj) {
        $loginType = $loginObj.LoginType
        Write-Log "Removing server-level login [$Login] ($loginType)" "Red"
        Remove-DbaLogin -SqlInstance $SqlInstance -Login $Login -Confirm:$false -ErrorAction Stop
        Write-Log "✅ Server-level login removed" "Green"
    }
    else {
        Write-Log "No server-level login [$Login] found" "DarkGray"
    }
}
catch {
    Write-Log "⚠️ Failed to remove login [$Login]: $_" "Yellow"
}

Write-Log "=== Completed cleanup for [$Login] on [$SqlInstance] ===" "Cyan"
Write-Log "Log file saved to: $LogFile" "Gray"


How It Works

  • Get-DbaDatabase lists all user databases.
  • Get-DbaDbUser / Remove-DbaDbUser checks for and removes the user from each DB.
  • Get-DbaLogin / Remove-DbaLogin cleans up the login from the instance.
  • All actions are written to a timestamped .txt log for compliance or auditing.

Example Usage

.\Remove-DbUserFromAllDatabases.ps1 -SqlInstance "SQLPROD01" -Login "Contoso\User123"

You can also specify a custom log path:

.\Remove-DbUserFromAllDatabases.ps1 -SqlInstance "SQLPROD01" -Login "appuser" -LogFile "C:\Logs\UserCleanup.txt"


Key Takeaways

  • Fully automated and non-interactive — perfect for offboarding workflows.
  • Handles both Windows and SQL logins gracefully.
  • Creates a detailed audit log for every action taken.
  • Safe to re-run — it skips users or logins that don’t exist.

Automate SQL Server Database Role Assignment with PowerShell and dbatools

Introduction

As a SQL Server DBA, one of the most repetitive administrative tasks is granting user access to multiple databases — especially in environments with dozens or even hundreds of databases.
Instead of manually connecting to each database and assigning roles, you can automate the process with the dbatools PowerShell module.

In this post, we’ll walk through how to automatically grant a user db_datareader and db_datawriter roles across all user databases, while excluding system databases.


Prerequisites

Before running the script:

  • Install the dbatools PowerShell module: Install-Module dbatools -Scope CurrentUser -Force
  • Ensure your account has sufficient permissions (sysadmin or equivalent).
  • Know the SQL instance name and the login you want to grant permissions to.

The PowerShell Script

# Requires dbatools
# Install-Module dbatools -Scope CurrentUser -Force

$SqlInstance = "MyServer\MyInstance"  # Replace with your SQL Server instance
$Login = "MyDomain\MyUser"            # Replace with your Windows or SQL login

# Get all user databases (excluding system DBs)
$UserDatabases = Get-DbaDatabase -SqlInstance $SqlInstance | Where-Object { -not $_.IsSystemObject }

foreach ($db in $UserDatabases) {
    Write-Host "Processing database: $($db.Name)" -ForegroundColor Cyan

    try {
        # Create the user if not already present
        $user = Get-DbaDbUser -SqlInstance $SqlInstance -Database $db.Name -User $Login -ErrorAction SilentlyContinue
        if (-not $user) {
            New-DbaDbUser -SqlInstance $SqlInstance -Database $db.Name -Login $Login -Username $Login -Confirm:$false | Out-Null
        }

        # Grant roles
        Add-DbaDbRoleMember -SqlInstance $SqlInstance -Database $db.Name -Role db_datareader -User $Login -Confirm:$false -ErrorAction Stop
        Add-DbaDbRoleMember -SqlInstance $SqlInstance -Database $db.Name -Role db_datawriter -User $Login -Confirm:$false -ErrorAction Stop

        Write-Host "✅ Granted db_datareader and db_datawriter in $($db.Name)" -ForegroundColor Green
    }
    catch {
        Write-Warning "Failed to process $($db.Name): $_"
    }
}

Write-Host "Completed assigning roles for $Login on all user databases." -ForegroundColor Green


Explanation

  • Get-DbaDatabase retrieves all databases and filters out system ones.
  • New-DbaDbUser ensures the login exists as a user in each DB.
  • Add-DbaDbRoleMember grants the necessary roles.
  • The script is non-interactive (-Confirm:$false), making it perfect for automation or CI/CD pipelines.

Example Usage

.\Grant-DbRoles.ps1 -SqlInstance "SQL01" -Login "Contoso\User123"


Key Takeaways

  • Save hours by automating repetitive access management tasks.
  • dbatools provides robust error handling and clean PowerShell syntax.
  • Works seamlessly with both Windows and SQL logins.
  • Ideal for onboarding new users or service accounts.