Horizontal vs Vertical Scaling
Horizontal scaling is when you add more machines and distribute the work across them - parallelization
Vertical scaling is when you improve performance of the current machines - better CPUs, GPUs with more RAM, lower latency so on.
Practically if a system is horizontally scalable it is better as it is easier to do - you just buy new machines. Improving current ones requires research/optimization and so on.
Horizontal scaling
- more complex management
- no required downtime, no need to stop current machines when you add more
- higher fault tolerance
Vertical scaling
- easier management
- required downtime
- single point of failure
- limited by hardware capacity