It seems to randomly stop working [Caddy, which is used as a reverse proxy is responding with "503 Service Unavailable"] with no apparent reason. A restart of all docker containers fixed it – until the next time.
Websites that have PHP handler set as FPM application served by Apache or FPM application served by nginx at Domains > example.com > PHP Settings experience one of the following issues:
- load slowly
- not accessible [permanently or periodically] with a 50x error in a web-browser:
502 Bad Gateway
503 Service Temporarily Unavailable
504 Gateway Time-out - The following message appears in the PHP-FPM error logfile
/var/log/plesk-php*XX*-fpm/error.log
[where XX is a PHP version used by the websites]: WARNING: [pool example.com] server reached max_children setting [*], consider raising it - The message above is usually accompanied with this error: unable to read what child say: Bad file descriptor [9]
- If nginx is enabled on the server, one of the following error messages may appear in domain's error logfile
/var/www/vhosts/system/example.com/logs/proxy_error_log
:[error] 11456
0: *118348 connect[] to unix:///var/www/vhosts/system/example.com/php-fpm.sock failed [11: Resource temporarily unavailable] while connecting to upstream, client: 203.0.113.2, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:///var/www/vhosts/system/example.com/php-fpm.sock:", host: "example.com"
[error] 20058
0: *4725139 upstream prematurely closed connection while reading response header from upstream, client: 203.0.113.2, server: example.com, request: "POST /path/to/some/file HTTP/2.0", upstream:
Cause
The PHP-FPM pm.max_children
has been reached by the website[s].
Resolution
Increase the pm.max_children
limit for the affected domains:
- Connect to the Plesk server via SSH.
- Find affected domains using the command below:
Note: The number on the left indicates how many times the
pm.max_children
limit has been reached by the website. If a domain is using a dedicated FPM application and has a custom vhosts location, change the path /var/www/vhosts/system/ in the command below: # grep -r "server reached max_children setting" /var/log/*php*-fpm/error.log | cut -d' ' -f5 | tr -d ']' | sort | uniq -c | sort -nr 100 example.com 50 example.net 25 example.org - Log in to Plesk.
- Go to Domains > example.com > PHP Settings > PHP-FPM Settings.
Increase the pm.max_children
value for the domain and apply the changes. Increase this value for all domains listed in step 2.
Note: In this example, we are setting pm.max_children
to 20. To fine tune this setting, visit this KB article.
If you have a busy server your php-fpm may crash out with the following error in the logs [ /var/log/phpX-fpm.log or /var/log/plesk-phpXX-fpm/error.log , where X is a PHP version]:
tail -f /var/log/php-fpm/error.log
Look for the following entries:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
Your website is not accessible with 503 Service Temporarily Unavailable or 502 Bad Gateway error.
Check that php-fpm service is running:
ps aux | grep fpm
Expected output should be running [once]:
php-fpm: master process [/opt/plesk/php/5.6/etc/php-fpm.conf]
Check number of allowed running php-fpm processes
ps afvx | grep domain.com
Expected output:
7075 pts/0 S+ 0:00 19 155 103148 884 0.1 \_ grep domain.com
7018 ? S 0:02 852 3394 375249 56132 8.3 \_ php-fpm: pool domain.com
7019 ? S 0:02 1778 3394 374469 55152 8.2 \_ php-fpm: pool domain.com
7021 ? S 0:01 735 3394 372729 52908 7.9 \_ php-fpm: pool domain.com
7022 ? S 0:04 716 3394 377661 59236 8.8 \_ php-fpm: pool domain.com
Resolution: Edit the conf file with the following:
vi /etc/php-fpm.conf
or
vi /etc/php-fpm.d/[domain.com].conf
For Plesk follow the instructions on the header of the conf file to override default php-fpm values by creating a custom php.ini file with specific fpm directives in:
/var/www/vhosts/system/mydomain.tld/conf/php.ini
Note that if the file is non-existent and you may need to create it.
Add the following entries:
[php-fpm-pool-settings]
pm = dynamic
pm.max_children = 25
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 498
Or for Plesk optionally just add the following entries:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
0
Restart php-fpm service:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
1
For Plesk you need to update PHP settings to apply the changes:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
2
If that fails you can try to recompile domains settings:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
3
or for one domain:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
4
Then restart php-fpm56 service [or php-fpmXX where X is a PHP version used in Plesk]:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
5
Verify the max_children limit has increased using the above commands.
If you need to calculate and change these values based on the amount of memory on the system the following command will help us to determine the memory used by each [PHP-FPM] child process:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
6
The RSS column shows non-swapped physical memory usage by PHP-FPM processes in kilo Bytes. If on an average each PHP-FPM process takes ~85MB of RAM on your server, appropriate value for pm.max_children can be calculated as:
pm.max_children = Total RAM dedicated to the web server / Max child process size
The server has 8GB of RAM, so:
pm.max_children = 6144MB / 85MB = 72
You need to take into account any other services running on the machine while calculating memory usage. Then change the settings as follow:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
7
You can check an average memory usage by single PHP-FPM process with this handy command:
WARNING: [pool example.com] server reached max_children setting [5], consider raising it
ERROR: unable to read what child say: Bad file descriptor [9]
8
You can use the same steps above to calculate the value for MaxClients for Apache web server - just substitute the php-fpm with httpd.