That article is magic. Here is the most important part:
“But what about when we scale to 100,000 users?”
“Then we’ll have the revenue to hire a dedicated infrastructure team. Right now, we have eight engineers and we’re spending 60 hours a week managing Kubernetes instead of shipping features.”
That excerpt clicks so many of the Paul Graham boxes. Common sense aside I am also a huge fan Docker Compose. It is stupid simple on a hard to fathom scale.
Unfortunately the tech available to us today is not designed to be understandable, it is made to generate consulting fees, training tuition and support contracts.
Kubernetes is really not overkill. Kubneretes is exactly what you make it.
Throw in a bunch of "magic" operators that no one understands, configure complex SDN overlay networks, and use invasive security sidecars. Make Kubernetes extremely customized to your environment. And, surprise, it's extraordinarily difficult to learn and use.
On the other hand, if you practice some basic restraint, Kubernetes is simply a distributed init system. Easy and fast to use. Kubernetes gives you base functionality you'll never get out of Docker Compose. Like, automatic load-balancing between pods, automatic node failover and recovery.
I'd take a simple 3-node k3s cluster over hacked-togther compose files and glue scripts any day. And, in doing that, it makes moving into something like EKS much easier in the future should you need "web scale" (mongo is web scale (sorry)).
>In our Docker Compose world, this problem didn’t exist. Services ran where we told them to run.
This is really interesting.
One of the big selling points of Kubernetes is that it takes care of scheduling on its own, distributes replicas and so on. This is especially useful when you are autoscaling pods.
But when you don't need autoscaling, especially if you have a limited amount of microservices, you may as well deploy your applications on the nodes you want them to run on. And running a script on a single node or 3 doesn't really make a difference (even better if you can parallelize, but maybe it's not even necessary).
Yes you could do the same with a mix of labels and advanced scheduling configurations, but if this is the main (or only) reason you use Kubernetes, and you don't really need autoscaling, Docker Compose or something similar makes sense.
Good article, sometimes a bit hard to read with all the links to other blog posts.
However there are some good nuggets in this article like this one:
"That’s when I realized: we’d built a dependency on one person’s specialized knowledge. And that knowledge had nothing to do with our actual product."
I see that at more smaller orgs, where they want to have technology X but fail to realize it requires a small team. Not because its a 3 man job to operate but because if 1 leaves or is unavailable the knowledge is gone. The knowledge can be acquired but it takes some time, and that in between period can be painful. When you tell them that and then they start to calculate the costs, it can be sobering.
The advice to keep the tech boring and widely embedded in the organization is something I agree with.
It depends on the business for sure. Kube is overkill until you have someone on your team whose specialization is infra. Then that person will probably be spearheading kube anyway :)
Sounds like an overreaction by the OP out of frustration by not understanding how something works and how to debug it. Instead of learning and accepting the growing pains, decides to throw the baby out with the bath water, shame.
And then he goes on mentioning two thousand steps nobody takes because those using K8 know what CI/CD is. Those changes that took him 2 hours take 30 seconds in my setup. Is it really k8s fault?
Then he proceeds rambling that he spend 3 hours on an OOM error because some junior didn't configure upper Memory limits. Kubernetes doesn't mean you can use it without learning the basics.
I never complain that python is garbage when I never read a book about python and vibe coded something with Claude. Why would I complain that writing in python takes a lot of time?
That article is magic. Here is the most important part:
“But what about when we scale to 100,000 users?”
“Then we’ll have the revenue to hire a dedicated infrastructure team. Right now, we have eight engineers and we’re spending 60 hours a week managing Kubernetes instead of shipping features.”
That excerpt clicks so many of the Paul Graham boxes. Common sense aside I am also a huge fan Docker Compose. It is stupid simple on a hard to fathom scale.
Even if you have the money, Docker Compose and similar solutions are still a good option for 100,000 users.
Conversely no tech is good for any number of users if you don't understand it.
Unfortunately the tech available to us today is not designed to be understandable, it is made to generate consulting fees, training tuition and support contracts.
Kubernetes is really not overkill. Kubneretes is exactly what you make it.
Throw in a bunch of "magic" operators that no one understands, configure complex SDN overlay networks, and use invasive security sidecars. Make Kubernetes extremely customized to your environment. And, surprise, it's extraordinarily difficult to learn and use.
On the other hand, if you practice some basic restraint, Kubernetes is simply a distributed init system. Easy and fast to use. Kubernetes gives you base functionality you'll never get out of Docker Compose. Like, automatic load-balancing between pods, automatic node failover and recovery.
I'd take a simple 3-node k3s cluster over hacked-togther compose files and glue scripts any day. And, in doing that, it makes moving into something like EKS much easier in the future should you need "web scale" (mongo is web scale (sorry)).
>In our Docker Compose world, this problem didn’t exist. Services ran where we told them to run.
This is really interesting.
One of the big selling points of Kubernetes is that it takes care of scheduling on its own, distributes replicas and so on. This is especially useful when you are autoscaling pods.
But when you don't need autoscaling, especially if you have a limited amount of microservices, you may as well deploy your applications on the nodes you want them to run on. And running a script on a single node or 3 doesn't really make a difference (even better if you can parallelize, but maybe it's not even necessary).
Yes you could do the same with a mix of labels and advanced scheduling configurations, but if this is the main (or only) reason you use Kubernetes, and you don't really need autoscaling, Docker Compose or something similar makes sense.
Good article, sometimes a bit hard to read with all the links to other blog posts.
However there are some good nuggets in this article like this one: "That’s when I realized: we’d built a dependency on one person’s specialized knowledge. And that knowledge had nothing to do with our actual product."
I see that at more smaller orgs, where they want to have technology X but fail to realize it requires a small team. Not because its a 3 man job to operate but because if 1 leaves or is unavailable the knowledge is gone. The knowledge can be acquired but it takes some time, and that in between period can be painful. When you tell them that and then they start to calculate the costs, it can be sobering.
The advice to keep the tech boring and widely embedded in the organization is something I agree with.
[dead]
> We moved to Docker Compose
Then you didn't need Kubernetes in first place.
> The initial setup took two weeks of his time. Full-time.
It takes 1 day max for such a simple setup.
> Load balancers: $180/month (one per service because of how we’d configured ingress)
> t3.medium
This can't be real
>Then you didn't need Kubernetes in first place.
I think that's precisely the point the author was trying to make.
It depends on the business for sure. Kube is overkill until you have someone on your team whose specialization is infra. Then that person will probably be spearheading kube anyway :)
Sounds like an overreaction by the OP out of frustration by not understanding how something works and how to debug it. Instead of learning and accepting the growing pains, decides to throw the baby out with the bath water, shame.
I agree with this article in part but also wonder if this wouldn't be an issue if people had spent time training on kubernetes.
Why should they spend time on training on a solution that is overkill for their use case?
Im curious why eng orgs adopt a team wide tech without team wide kt or upskilling. Its a persistent management problem afaict not a tech problem.
Its an indictment of modern tech mgmt tbh. Its the principals job to elevate the team technically. Instead all I see lately is C suite ass kissing
[dead]
Seems clickbait to me.
"Here’s what it looked like with Kubernetes:"
And then he goes on mentioning two thousand steps nobody takes because those using K8 know what CI/CD is. Those changes that took him 2 hours take 30 seconds in my setup. Is it really k8s fault?
Then he proceeds rambling that he spend 3 hours on an OOM error because some junior didn't configure upper Memory limits. Kubernetes doesn't mean you can use it without learning the basics.
I never complain that python is garbage when I never read a book about python and vibe coded something with Claude. Why would I complain that writing in python takes a lot of time?