Say you have a single section of your code that's very compute intensive. If it's part of a larger monolith then you would spin up multiple instances of the entire monolith to keep up with traffic. To take a concrete example, say you have an app that does login, some social stuff, some billing, pulling from an external API and finally your special sauce - rendering a PDF of what your user has created. The last step is a bottle neck but none of the others are. You could increase the number of instances of renderers and your entire system would chug along.
You've asked the right question though. Is independent scaling "necessarily" better? No, it's a trade-off.
* To get this feature you've sacrificed the simplicity of having a single codebase leading to a single binary that can be deployed with ease.
* You've added extra components within your system like load balancers.
* What would earlier have been a function call is now RPC with network and serialisation overhead.
* Your system might be a little more difficult to debug unless you collect logs in one place and it's possible to follow a single user across multiple components.
* It's possible for your system to fail in ways that weren't possible before. For instance there might be a network issue between two specific nodes and it's hard to figure that out unless you have proper monitoring in place.
Of course there are advantages other than failure isolation and independent scaling but I haven't gone into those.
Note that your example seems to only matter if I use differently spec'd instances for different services. If they all have the same ratio of e.g. memory to cpu to disk, I'm not sure what slicing my instances across services really gets me. If you slice it across services, you add a few instances, and your system chugs along. If you don't, you still add a few instances, and it still chugs along.
I could maybe see if you have some services that take huge amounts of memory and others that take huge amounts of CPU. If you have a standard "monolithic app instance", and had to scale up your CPU by 10x but memory only by 1x because of your pdf renderer, you will likely be wasting large amounts of memory. But unless you have huge disparities in memory vs cpu use (and services don't vary in use together) I don't really know what sort of cost savings you can get here - wouldn't this be the actual value add of being able to independently scale - less cost because you can more accurately hit your resource (cpu vs memory vs disk and so on) requirements for your project?
In contrast, from my experience, separating your web/app/database/cache layer from eachother tends to be extremely beneficial for independent scaling because they almost always vary widely in how they consume which resources (memory vs cpu vs disk and so on). They also tend to be written with this in mind and so it is basically free to do so.
An aside, but many of your downsides apply not to just services, but also having to scale an application horizontally. If the name of the game is scaling, then many of these you will pay for regardless of the question of monolith vs. services.
All that above aside, I definitely feel the other benefits. But I really don't get it - every article I read about services seems to mention independent scaling, when it seems like a fairly suspect benefit. Maybe I just haven't worked on the correct project.
Coming from Square (which is mostly SOA, but with an old monolithic service), we had quite a few services which really needed to be separated for performance reasons:
- One needed a large in-process cache in order to deliver good performance; it would have consumed to much memory on each instance of the monolith.
- Some services used large ML models, which also would have consumed too much of memory on monolith instances.
- A lot of our payment-related services had hourly or daily batch jobs. Anything with big resource spikes probably shouldn't share a machine with latency-sensitive code (like online payment processing or just web handlers).
- Related to the above, some jobs had to be done by a master instance. If the monolith did them, they would have disproportionately affected a single instance of the monolith.
CPU vs memory is one area, but also being blocked on IO and network saturation. An endpoint that is essentially a glorified proxy is going to scale differently than one that does real CPU work. Doing all of that in a synchronous platform us going to require massive memory for the threaded IO, or it's going to be really expensive on an event loop. So build two separate systems, put them on specialized hardware (make sure you have 10G network on the proxy).
Each thing takes some resources. Application performance tends to get worse over time, usually because the focus is on adding features, rather than improving efficiency (sweeping generalization). So eventually you'll find yourself in a spot where your monolithic app doesn't have much headroom on existing machines. You switch to a new auth system that's 20% more expensive, and the whole system is running out of memory.
This is a contrived example. I mostly agree with you. I think you do hit a point at the application level where the big monolithic app is a little bit to big, and that's a tough spot to be in.
If you can keep things small, light and efficient mono will work forever. But always keep in mind that it can outgrow your instance sizes. So prepare to either move up to the next size instance (tough if you're running your own hardware), or start thinking about splitting out parts of the app.
Resource utilization is generally a function of traffic. Why is adding more machines to a specific service's load balancer pool better than adding more machines to the monolith's load balancer pool? You're saving maybe a little bit of memory by not loading unnecessary code, but that seems inconsequential given OS level caching.
One possible argument is that you can reserve capacity by service, so a computationally expensive but unimportant endpoint doesn't accidentally gobble up the resources you added to alleviate the starvation that was slowing down a different endpoint.
But you could also do that with smarter load balancing - deploy a monolith to all hosts, but partition traffic between separate pools depending on endpoint at the load balancer level.
I don't think performance isolation is a good argument for microservices. I don't think failure isolation is either - the interactions are likely to create more, trickier failures than the isolation will prevent.
The real argument is about scaling the organization. Much easier to work on and frequently deploy small codebases with small numbers of commits and commiters per day, communicating across team boundaries via Thrift IDL files, than to have thousands of engineers on one codebase and thousands of potentially breaking changes introduced between every deploy.
Scaling your app isn't just about turning knobs. It's about making design tradeoffs based on your usage pattern. Most apps are very framework dependent and mixing and matching frameworks within a monolith is a nightmare. Having more flexibility around platform allows you to scale independently.
This seems like a great example of how seductive it can be to over-fragment a system, and suddenly have hard-to-debug microservices for all those little pieces, when really all that's needed in a case like this is a front-end app that does login, sessions, billing, everything except the rendering work which goes off to a queue-based worker service.
You've asked the right question though. Is independent scaling "necessarily" better? No, it's a trade-off.
* To get this feature you've sacrificed the simplicity of having a single codebase leading to a single binary that can be deployed with ease.
* You've added extra components within your system like load balancers.
* What would earlier have been a function call is now RPC with network and serialisation overhead.
* Your system might be a little more difficult to debug unless you collect logs in one place and it's possible to follow a single user across multiple components.
* It's possible for your system to fail in ways that weren't possible before. For instance there might be a network issue between two specific nodes and it's hard to figure that out unless you have proper monitoring in place.
Of course there are advantages other than failure isolation and independent scaling but I haven't gone into those.