Most systems have fixed-size thread pools, apache httpd, php-fpm, gunicorn, gocraft, asynq, and machinery. And those systems work great and help quickly scale without solving the problem from the ground up. Still, with enough scenarios, you would start noticing minor issues like using only about 25% of the CPU, and still, your queue lag is increasing, or your httpd server is giving 503 even though it has enough resources. Or you tried to give 2x CPU and 2x memory expecting your containers will start processing 2x but were surprised to see that it is not.
Autoscaling Thread Pools
Autoscaling Thread Pools
Autoscaling Thread Pools
Most systems have fixed-size thread pools, apache httpd, php-fpm, gunicorn, gocraft, asynq, and machinery. And those systems work great and help quickly scale without solving the problem from the ground up. Still, with enough scenarios, you would start noticing minor issues like using only about 25% of the CPU, and still, your queue lag is increasing, or your httpd server is giving 503 even though it has enough resources. Or you tried to give 2x CPU and 2x memory expecting your containers will start processing 2x but were surprised to see that it is not.