Scaling SaaS Infrastructure: From 1k to 1M Users
Scaling a SaaS platform from 1,000 to 1,000,000 users isn't just about adding more servers — it requires a fundamental rethinking of your data layer, caching strategy, and infrastructure topology. When you're billing in Dollars, paying infrastructure in Euros, and serving customers in Pounds, every millisecond of latency and every wasted compute cycle directly impacts your bottom line.
📊 The 8 KPIs That Matter Most
Before optimizing, you need to measure. These are the metrics that separate unicorns from failures:
| KPI | Target (1k users) | Target (100k users) | Target (1M users) |
|---|---|---|---|
| P99 Latency | <200ms | <150ms | <100ms |
| Uptime SLA | 99.9% | 99.95% | 99.99% |
| Error Rate | <1% | <0.1% | <0.01% |
| MTTR | <1 hour | <15 min | <5 min |
| DB Query Time | <50ms | <20ms | <10ms |
| Cache Hit Ratio | >80% | >90% | >95% |
| Monthly Churn | <5% | <3% | <1.5% |
| Infra Cost/User | $2.50 | $0.50 | $0.08 |
🗄️ 1. Database Sharding Strategies
Your database is almost always the first bottleneck. Vertical scaling only gets you so far — at scale, you must go horizontal.
🔑 Tenant-Based Sharding
Each tenant gets their own shard. Simple to implement, excellent data isolation, and makes GDPR compliance trivial.
✅ Best for: B2B SaaS with <10k tenants
🌍 Geographic Sharding
Split data by region: US-East, EU-West, APAC. Reduces latency for local users and helps with data sovereignty laws.
✅ Best for: Global platforms with regulatory requirements
📅 Time-Based Sharding
Partition data by time period. Old shards can be archived to cold storage, reducing active dataset size dramatically.
✅ Best for: Analytics-heavy platforms, logs, IoT
🔀 Hash-Based Sharding
Hash a key (user ID) to distribute records evenly. Consistent hashing prevents massive reshuffling when adding nodes.
✅ Best for: B2C SaaS with millions of users
⚡ 2. Multi-Layer Caching Architecture
A well-designed caching strategy can reduce database load by 90%+ and deliver sub-10ms response times.
In-process caches with microsecond access. Best for config values, feature flags, session data.
Shared across all instances. Sub-millisecond access with Redis Cluster. Use for API responses, user profiles.
Cache at edge locations worldwide. A UK user gets content from London, not Virginia. Reduces origin load by 70-80%.
| Strategy | Pattern | Consistency | Best For |
|---|---|---|---|
| Cache-Aside | App checks cache → miss → reads DB | Eventual | Read-heavy |
| Write-Through | App writes to cache → cache writes to DB | Strong | Consistency critical |
| Write-Behind | App writes to cache → async batch to DB | Weak | High-throughput writes |
⚖️ 3. Advanced Load Balancing
DNS-Level (Route 53)
- • Geographic routing
- • Weighted round-robin
- • Latency-based routing
- • Failover configs
L7 (ALB/NGINX)
- • Path-based routing
- • Header-based routing
- • Sticky sessions
- • WebSocket support
L4 (NLB/HAProxy)
- • TCP/UDP balancing
- • Ultra-low latency
- • Static IP support
- • gRPC workloads
🏗️ 4. Architecture Evolution Roadmap
Monolith + Single DB
Single PostgreSQL, single server, vertical scaling. ~$200/month.
Modular Monolith + Read Replicas + Redis
Add read replicas, Redis for cache, CDN for static assets. ~$2,000/month.
Microservices + Sharded DB + Event-Driven
Extract high-traffic services, shard DB, introduce Kafka. ~$15,000/month.
Multi-Region + CQRS + Auto-Scaling
Active-active multi-region, CQRS, K8s auto-scaling. ~$50,000+/month.
💰 5. Cost Optimization
💡 Quick Wins
- ✅ Right-size instances (most over-provision by 40%)
- ✅ Reserved instances for baseline (save 30-60%)
- ✅ Spot instances for batch jobs (save 70-90%)
- ✅ S3 Intelligent Tiering for storage
⚠️ Anti-Patterns
- ❌ Running dev/staging 24/7
- ❌ Uncompressed cross-region data transfer
- ❌ DEBUG logging in production
- ❌ N+1 queries hitting the database
📈 6. Monitoring & Observability Stack
| Layer | Tool | Purpose |
|---|---|---|
| Metrics | Prometheus + Grafana | Infrastructure & app metrics |
| Tracing | Jaeger / OpenTelemetry | Distributed request tracing |
| Logging | ELK Stack / Loki | Centralized log aggregation |
| Alerting | PagerDuty / OpsGenie | Incident management |
| APM | Datadog / New Relic | Application performance monitoring |
❓ Frequently Asked Questions
When should I start sharding my database?
When your single instance consistently exceeds 80% CPU/memory, or query latency degrades beyond acceptable SLAs. Typically around 50k-100k active users.
Is Redis or Memcached better for SaaS caching?
Redis wins in almost every scenario in 2026. It supports complex data structures, persistence, clustering, and Lua scripting.
How much should I budget for infrastructure at 100k users?
A well-optimized SaaS at 100k users should cost $3,000-$8,000/month on AWS/GCP. More means over-provisioning.
Should I use multi-region from day one?
No. Start single-region with a CDN. Only go multi-region when you have significant user bases across continents or regulatory requirements.
Need a Scalability Architect?
Whether you're scaling from 1k to 100k or planning your journey to a million users, Aqib Mustafa specializes in SaaS infrastructure optimization, database sharding, and cost-efficient cloud architectures.