Jun's Blog

Output, activities, memo and etc.

Feodra: Where is my core file?

I am trying to make a friend with binary and debug reading 2 books "BINARY HACKS" and "DEBUG HACKS" written in Japanese.

This time I would write about the segmentation fault and core file to debug on Fedora 33.

Change the maximum size of the created core files.

You can see the size by ulimit -c (-c The maximum size of core files created).

$ ulimit -c
0

If the command shows 0, we need to change it to unlimited or a number.

$ ulimit -c unlimited

$ ulimit -c
unlimited

Make segmentation fault happen.

Here is a sample file in the book "DEBUG HACKS" to make a segmentation fault happen. On the line *a = 0x1;, a segmentation fault happens by referring the NULL pointer.

$ cat segfault.c
#include <stdio.h>

int main(void)
{
    int *a = NULL;

    *a = 0x1;
    return 0;
}

Compile by gcc without -g

$ gcc segfault.c -o segfault

$ ./segfault 
Segmentation fault (core dumped)

But I did not see the core file on the current directory. So, where is the core file?

First check the kernel.core_pattern that is how the core file is created. I see the core file is executed by pipe + a command: |/usr/lib/systemd/systemd-coredump.

$ sysctl -a | grep ^kernel.core_pattern
...
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
...

By seeing the manual of systemd-coredump.

$ man systemd-coredump

I found the core file created at /var/lib/systemd/coredump.

$ ls /var/lib/systemd/coredump
core.segfault.1000.e8e24ac0a2264b208eee28f934beb62c.891505.1617826339000000.zst

Seeing the [1], I knew that there was coredumpctl command.

$ man coredumpctl

I could see the list of core list by coredumpctl list like this.

$ coredumpctl list --since=2021-04-06
TIME                            PID   UID   GID SIG COREFILE  EXE
Wed 2021-04-07 22:12:19 CEST 891505  1000  1000  11 present   /home/jaruga/git/binary-debug-test/segfault

$ coredumpctl list segfault
TIME                            PID   UID   GID SIG COREFILE  EXE
Wed 2021-04-07 22:12:19 CEST 891505  1000  1000  11 present   /home/jaruga/git/binary-debug-test/segfault

So, how to get the actual core file to the current directory. In this case, we can dump the segfault.coredump file to the current directory.

$ coredumpctl -o segfault.coredump dump segfault
           PID: 891505 (segfault)
           UID: 1000 (jaruga)
           GID: 1000 (jaruga)
        Signal: 11 (SEGV)
     Timestamp: Wed 2021-04-07 22:12:19 CEST (33min ago)
  Command Line: ./segfault
    Executable: /home/jaruga/git/binary-debug-test/segfault
 Control Group: /user.slice/user-1000.slice/session-2.scope
          Unit: session-2.scope
         Slice: user-1000.slice
       Session: 2
     Owner UID: 1000 (jaruga)
       Boot ID: e8e24ac0a2264b208eee28f934beb62c
    Machine ID: 7919e0e789f64910aad5d7d544336ab1
      Hostname: localhost.localdomain
       Storage: /var/lib/systemd/coredump/core.segfault.1000.e8e24ac0a2264b208eee28f934beb62c.891505.1617826339000000.zst
       Message: Process 891505 (segfault) of user 1000 dumped core.
                
                Stack trace of thread 891505:
                #0  0x0000000000401116 n/a (/home/jaruga/git/binary-debug-test/segfault + 0x1116)
                #1  0x00007fac6cb3a1e2 __libc_start_main (libc.so.6 + 0x281e2)
                #2  0x000000000040104e n/a (/home/jaruga/git/binary-debug-test/segfault + 0x104e)

We can not see the actual line in the file that the segmentation fault happened. To see the line number, we need to compile with gcc -g option.

$ gdb -c segfault.coredump ./segfault
GNU gdb (GDB) Fedora 10.1-2.fc33
...
Core was generated by `./segfault'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000401116 in main ()
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.32-4.fc33.x86_64

Before creating the core file by gcc -g, install the missing debug info package by:

$ sudo dnf debuginfo-install glibc-2.32-4.fc33.x86_64

Compile by gcc with -g

$ gcc -g segfault.c -o segfault-g

$ ./segfault-g
Segmentation fault (core dumped)
$ ls /var/lib/systemd/coredump
core.segfault.1000.e8e24ac0a2264b208eee28f934beb62c.891505.1617826339000000.zst
core.segfault-g.1000.e8e24ac0a2264b208eee28f934beb62c.892539.1617829143000000.zst
$ coredumpctl list segfault-g
TIME                            PID   UID   GID SIG COREFILE  EXE
Wed 2021-04-07 22:59:03 CEST 892539  1000  1000  11 present   /home/jaruga/git/binary-debug-test/segfault-g
$ coredumpctl info segfault-g
           PID: 892539 (segfault-g)
           UID: 1000 (jaruga)
           GID: 1000 (jaruga)
        Signal: 11 (SEGV)
     Timestamp: Wed 2021-04-07 22:59:03 CEST (1min 58s ago)
  Command Line: ./segfault-g
    Executable: /home/jaruga/git/binary-debug-test/segfault-g
 Control Group: /user.slice/user-1000.slice/session-2.scope
          Unit: session-2.scope
         Slice: user-1000.slice
       Session: 2
     Owner UID: 1000 (jaruga)
       Boot ID: e8e24ac0a2264b208eee28f934beb62c
    Machine ID: 7919e0e789f64910aad5d7d544336ab1
      Hostname: localhost.localdomain
       Storage: /var/lib/systemd/coredump/core.segfault-g.1000.e8e24ac0a2264b208eee28f934beb62c.892539.1617829143000000.zst
       Message: Process 892539 (segfault-g) of user 1000 dumped core.
                
                Stack trace of thread 892539:
                #0  0x0000000000401116 n/a (/home/jaruga/git/binary-debug-test/segfault-g + 0x1116)
                #1  0x00007f43deeed1e2 __libc_start_main (libc.so.6 + 0x281e2)
                #2  0x000000000040104e n/a (/home/jaruga/git/binary-debug-test/segfault-g + 0x104e)
$ coredumpctl -o segfault-g.coredump dump segfault-g

Here is the result by gdb with core file.

$ gdb -c segfault-g.coredump ./segfault-g
GNU gdb (GDB) Fedora 10.1-2.fc33
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./segfault-g...
[New LWP 892539]
Core was generated by `./segfault-g'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000401116 in main () at segfault.c:7
7     *a = 0x1;

So, this time, we can see the actual line number and the content in the file.

#0  0x0000000000401116 in main () at segfault.c:7
7     *a = 0x1;

We can check the code around the line causing the segmentation fault, by specifying the line number 6 in this case.

(gdb) l 6
1 #include <stdio.h>
2 
3 int main(void)
4 {
5     int *a = NULL;
6 
7     *a = 0x1;
8     return 0;
9 }

References