Apache Error: “semget: No space left on device”
The error indicates that apache failed and will not start again, check the error log If you see an error similar to the following, it could indicate that your server has run out of semaphores and apache cannot be started. Error log file under /usr/local/apache/error_log reports the following errors:
[emerg] (12)Cannot allocate memory: mod_fcgid: Create process manager error
[error] (28)No space left on device: Cannot create SSLMutex Configuration Failed
The errors mean that there is no available IPC (inter-process communication) resources in the system, such as semaphores or shared memory segments.
How much IPC resources are used can be found using ‘ipcs’ command:
# ipcs -a
To solve this problem you can restart Apache, Postgres and other services that consumer many IPC resources or increase limit of the resources in the system using ‘sysctl’. When you stop all services the semaphores and shared memory segments have to be removed, if not, and you still able to see them using ‘ipcs’ command, try to remove them manually using ‘ipcrm’ command. For example to remove semaphore:
# ipcs -a
—— Semaphore Arrays ——–
key semid owner perms nsems
0x00000000 201293824 apache 600 1
…
# ipcrm -s
On older servers the above command may not work. So you may need to do the following.
/sbin/service httpd stop
ipcs -s | grep nobody | gawk ‘{ print $2 }’ | xargs -n 1 ipcrm sem
/sbin/service httpd start
See ‘man ipcrm’ for more information.
Below is the example how to increase number of semaphores on Fedora Core 4.
get current semaphores value:
# /sbin/sysctl -a | grep sem
kernel.sem = 200 32000 32 128
set new value:
# /sbin/sysctl -w kernel.sem=250
OR add new value into /etc/sysctl.conf in order the changes persist after system boot.
kernel.sem = 250 128000 32 512
Then load the new settings into the kernel:
sysctl -p
Note: This post assumes you are running Apache on a Linux server and are familiar with the command line.
