Enable core dumps in RedHat.
- February 16th, 2011
- Posted in Documentation
- Write comment
I used the following procedures to enable core dumps in RHEL6.
# vi /etc/security/limits.conf
…
#* soft core 0
* soft core unlimited
…
# vi /etc/sysctl.conf
…
kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t
fs.suid_dumpable = 2
…
Note:
%e is the filename
%g is the gid the processs was running under
%p is the pid of the process
%s is the signal that caused the dump
%t is the time the dump occurred
%u is the uid the process was running under
# vi /etc/sysconfig/init
…
DAEMON_COREFILE_LIMIT=’unlimited’
…
# sysctl -p
To enable core dumps for an xinetd process, I needed to install abrt. On my minimum install of RHEL6, I needed to install the following to get core dump files created:
1:dbus
abrt
abrt-addon-ccpp
abrt-addon-kerneloops
abrt-addon-python
abrt-cli
abrt-libs
abrt-plugin-logger
abrt-plugin-rhtsupport
abrt-plugin-sosreport
ConsoleKit
ConsoleKit-libs
eggdbus
elfutils
elfutils-libs
libtar
polkit
sos
xmlrpc-c
xmlrpc-c-client
xz
yum-utils
I must still be missing something, because I get the following when I try to list crashes.
abrt-cli –list
dbus error: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
error requesting DBus name com.redhat.abrt, possible reasons: abrt run by non-root; dbus config is incorrect; or dbus daemon needs to be restarted to reload dbus config
And I am not able to start the daemon (abrtd). I get the following when I try to start the daemon:
Starting abrt daemon: abrtd: Failed to start: timeout waiting for child
It looks they are probably related, and there are a few bugzilla reports on the issue, but no clear solution. I just needed to get the core dumps created, and I have that for now.
I am fairly certain that the mods to sysctl.conf do not do anything as I have never gotten a core file in /tmp as a result of this change. I don’t know if the limits settings are needed or not, I left them in place.
The key to getting core dumps in RHEL6 is abrtd. In my case I had to:
* make sure that the entries for abrt were in /etc/passwd and /etc/group, eg:
abrt:x:499:499::/etc/abrt:/sbin/nologin (for passwd)
abrt:x:499: (for /etc/group)
* make sure that abrtd would start. In my case I had to turn it on with /sbin/chkconfig. It also would not start correctly via the /etc/init.d/abrtd script until I added the passwd and group entries for the user/group. Check your syslogs for “abrt” information about startup errors.
* get the /etc/abrt/abrt.conf file configured correctly. In my case I set:
OpenGPGCheck = no
ProcessUnpackaged = yes
MaxCrashReportsSize = 0 (for unlimited cores)
comment out all entries for ActionsAndReporters
Once I did these steps, I could fire up a process, do a “kill -3” on the process from another window, and have it say “Abort (core dumped)”. Another scan of the syslogs told me that the core stuff got written to /var/spool/abrt. A directory ccpp-[numbers] got created there with a file “coredump” inside. At that point I could do “gdb [path to executable] coredump” and get going on debugging. Finally!
What idiot at Redhat made this the default behavior for application core dumps? While abrtd is a good idea, don’t force it on us to do standard unix core debugging.
@Jeff Earickson
Thanks for the input. In my case, the core dumps were being generated in the /tmp directory with the format as specified in the kernel.core_pattern entry. This only worked for daemons, not for anything started from xinetd. Eventually, I did get everything creating dumps in abrtd, by making similar changes to the /etc/abrt/abrt.conf.
#OpenGPGCheck = yes
OpenGPGCheck = no
…
#ProcessUnpackaged = no
ProcessUnpackaged = yes
…
#MaxCrashReportsSize = 1000
MaxCrashReportsSize = 10000
If anybody finds the idiots name in Redhat who made the decision to F up a simple thing like core file creation; let u know. I’d like kick his arse… What cooking mess …
@Jopo
They have made a number of such changes recently that left me wondering why too.
I’m having this error when creating database in red hat 6 Linux ;
An unexpected error has been detected by Hotspot Virtual Machine :
SIGSEGV (0*b) at PC=0*9f40945c, pid=6141, tid=3079419584
Java VM:Java Hotspot(TM) client VM (1.5.0_17-b02 mixed mode)
problematic frame:
c {libnnZ11.so+0*3c45c}
An error report file with more information is saved as hs_err_pid6141.log
Aborted(core dumped)
please help me out .
@makula
This looks like an issue with Oracle’s Java in RHEL6. It might be an issue with the version. I gave on java in Redhat Linux a long time ago.