I Found 18x Over-Provisioning in My K8s Lab Cluster (Yours Is Probably Worse)

This week, I ran a simple audit on my standard Kubernetes lab cluster.

Two commands. 30 seconds. One shocking discovery.

Command 1: See what pods REQUEST

kubectl describe nodes | grep -A 5 "Allocated resources"

Command 2: See what pods ACTUALLY USE

kubectl top nodes

Here’s what I found:

Requested: 18% of cluster capacityActually using: 1% of cluster capacity

Gap: 18x over-provisioned

Translation: I could fit 18x more workloads on the same hardware.

Or said differently: I’m wasting 93% of what I requested.

Before you think “that’s just a standard K8s lab cluster,” let me be clear:

This is the industry standard.

I’ve audited 30+ clusters now. Startups, mid-size companies, enterprises.

The pattern is consistent:

My 18x? Right in the middle.

Most teams have no idea this is happening.

It’s not your fault.

Real examples from my cluster:

CoreDNS:

kube-proxy:

metrics-server:

These are SYSTEM PODS from Kubernetes itself.

If the defaults are this wasteful, what chance do you have?

The Kubernetes docs tell you WHAT requests and limits are.

They don’t tell you HOW to size them.

You’ll read things like:

Okay. But HOW?

How do I know what my application needs?

Do I guess? Copy someone else’s manifest? Use the defaults?

The docs don’t say.

So everyone does what’s easiest: copy-paste and move on.

Here’s what actually happens:

The step everyone skips: measuring actual resource usage and adjusting.

Why?

Because nobody told you to do it.

Because you don’t know how.

Because you’re busy with “more important” things.

Because the default worked, so why mess with it?

And that’s how you end up with 18x over-provisioning.

“My cluster is small. 18x waste doesn’t matter, right?”

Let me show you.

Current state:

At this small scale, I’m already at minimum HA config (2 nodes). Can’t reduce nodes further. Waste is “locked in.”

Cost impact now: $0

But watch what happens when this scales...

Over-provisioned scenario:

Right-sized scenario:

Difference: $30/month = $360/year

Over-provisioned: 6 nodes = $180/monthRight-sized: 2 nodes = $60/month

Difference: $120/month = $1,440/year

Over-provisioned: 11 nodes = $330/monthRight-sized: 3 nodes = $90/month

Difference: $240/month = $2,880/year

The pattern scales linearly.

Fix it at 10 pods, save $3,000/year at 200 pods.

While looking at CPU, I noticed something else.

Memory situation (opposite of CPU):

Requested: 540Mi (8% of capacity)Actually using: 998Mi (15% of capacity)

Memory is UNDER-provisioned by 85%.

This is dangerous.

Most developers:

Result:

The fix: Measure BOTH resources.

Stop reading. Run these two commands.

kubectl describe nodes | grep -A 5 "Allocated resources"Look for the percentage:cpu 700m (18%) 0 (0%)

That 18% is what the scheduler sees.

kubectl top nodesLook at CPU%:NAME CPU(cores) CPU%node-1 20m 1%

That 1% is reality.

Scheduler view: 18%Reality: 1%

Gap: 18 ÷ 1 = 18x over-provisioned

If your gap is:

1-3x. : You’re doing better than 95% of clusters.

5-10x. : Normal. Room for optimization, not urgent.

10-30x. : Industry average. Significant savings available.

30-50x.: Extreme waste. Fix this immediately.

50x+. : Something is very wrong.

Reply with your numbers - I’ll tell you what they mean.

Here’s what you’re going to do.

I’ve broken it down by day. Time estimates included.

Follow the order. Don’t jump around.

Run this right now:

kubectl top pods --all-namespaces --sort-by=cpu

Look at the output. Pick the 5 pods using the most CPU.

Grab a notepad. Write down:

Do this before you close this email.

One measurement tells you nothing. You need the pattern.

For each of your 5 pods, run this throughout the week:

kubectl top pod <pod-name> -n <namespace>

Check it at different times. Traffic patterns matter.

I usually check:

Write down the peak. That’s what matters.

Now do the math.

Take your peak usage. Multiply by 1.5. That’s your new request.

Example:

Simple.

Why 1.5x?

You need headroom for:

But you don’t need 10x. You’re not running a nuclear reactor.

1.5x keeps you safe without the insane waste.

Open a spreadsheet. Track all 5 pods. You’ll need this.

Lets make some changes.

Update the requests for each pod like below.

Current:

resources: requests: cpu: 500m # Copied from some blog post memory: 256Mi # Someone's guess

Change to :

resources: requests: cpu: 60m # Based on actual × 1.5 memory: 120Mi # Based on actual × 1.5

Important: Start with ONE pod. Not all 5.

Test one. Make sure it works. Then move to the next.

Deploy your first optimized pod to staging.

Wait 48 hours. Watch for problems:

CPU throttling?Requests are too low. Bump them up by 20%. Try again.

OOM kills?Memory is too low. Bump it up by 20%. Try again.

Everything running normally?You nailed it. Move to production.

Once it’s stable in production for 3 days, start on pod #2.

Repeat until all 5 are done.

Total timeline: 2-3 weeks for all 5 podsTotal time investment: 4-6 hoursPotential savings: $360-$2,880/year

Start today. Run that first command.

Let me show you what this looks like.

Before optimization:

After optimization:

Savings: $30/month = $360/year

Time to implement: 4-6 hours over 2 weeks

ROI: Pays for itself in 2 months

Without early optimization:

With early optimization:

Difference: $240/month = $2,880/year

The work you do at 50 pods saves 10x at 200 pods.

I hear the same objections every time.

By the time you “need” to optimize, you have:

Result: It never happens.

Better approach: Fix it now when you have 10-50 pods. Takes 4 hours, not 4 weeks.

Today: $30/month (annoying, not urgent)

In 6 months at scale: $240/month (urgent, but now 10x harder to fix)

The optimization is the same effort.

Do it when it’s easy, not when it’s desperate.

Do you?

4 hours of work = $360-$2,880/year saved.

That’s $90-720/hour ROI.

Show me the “bigger problem” with better ROI.

Run the audit. Reply with your numbers.

Format:

Requested: X%Actual: Y%Gap: Zx

I’ll tell you:

And if your gap is over 30x, I’ll show you exactly where to start.

Want to stop wasting 93% of your Kubernetes capacity?

Start with the 30-second audit.

Then fix your top 5 pods.

4 hours of work. $2,880/year saved.

Or keep doing what everyone else does:

Copy-paste defaults. Deploy. Never look back. Wonder why AWS bills keep growing.

Your choice.

- Naveen

Thanks for reading BeyondOps Newsletter! This post is public so feel free to share it.