Loading millions of records into a cloud database can be a frustratingly slow task—unless you identify where your bottlenecks are. In this post, I will share how we significantly improved our insertion speeds on Azure Database for PostgreSQL Flexible Server by adjusting a single, often-overlooked setting: provisioned IOPS.
The Challenge: Slow Inserts Despite Low CPU
We were running a large data migration from Databricks to Azure Database for PostgreSQL Flexible Server. Our setup:
- Instance: Memory Optimized, E8ds_v4 (8 vCores, 64 GiB RAM, 256 GiB Premium SSD)
- Insert Method: 8 parallel threads from Databricks, each batching 50,000 rows
Despite this robust configuration, our insert speeds were disappointing. Monitoring showed:
- CPU usage: ~10%
- Disk IOPS: 100% utilization
Clearly, our CPU wasn’t the problem—disk I/O was.
The Bottleneck: Disk IOPS Saturation
Azure Database for PostgreSQL Flexible Server ties write performance directly to your provisioned IOPS (Input/Output Operations Per Second). PostgreSQL is forced to queue up write operations when your workload hits this limit, causing inserts to slow down dramatically.
Key signs you’re IOPS-bound:
- Disk IOPS metric at or near 100%
- Low CPU and memory utilization
- Inserts (and possibly other write operations) are much slower than expected
The Fix: Increase Provisioned IOPS
We increased our provisioned IOPS from 1,100 to 5,000 using the Azure Portal:
- Go to your PostgreSQL Flexible Server in Azure.
- Select Compute + storage.
- Adjust the IOPS slider (or enter a higher value if using Premium SSD v2).
- Save changes—no downtime required.
Result:
Insert speeds improved immediately and dramatically. Disk performance no longer throttled the database, and we could fully utilize our CPU and memory resources.
Lessons Learned & Best Practices
- Monitor your bottlenecks: Always check disk IOPS, CPU, and memory during heavy data loads.
- Scale IOPS with workload: Azure lets you increase IOPS on the fly. For bulk loads, temporarily raising IOPS can save hours or days of processing time.
- Batch and parallelize wisely: Match your parallel threads to your vCPU count, but remember that IOPS is often the true limiter for bulk writes.
- Optimize indexes and constraints: Fewer indexes mean fewer writes per insert. Drop non-essential indexes before bulk loads and recreate them afterward.
Conclusion:
If your PostgreSQL inserts are slow on Azure, check your disk IOPS. Increasing provisioned IOPS can unlock the performance your hardware is capable of—sometimes, it’s the simplest tweak that makes the biggest difference.
Database Blog 