Jun's Blog

Output, activities, memo and etc.

Top > Info-tech > Debugging OpenSSL with GDB and ltrace

Debugging OpenSSL with GDB and ltrace

Info-tech

This article is to note to debug OpenSSL with GDB. I use the case of this issue on OpenSSL making a segmentation fault on OpenSSL version 3.0.1, then fixed in OpenSSL version 3.0.2.

My environment

Fedora 37

GCC version 12.2.1

$ gcc --version
gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4)
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

GDB 12.1

$ gdb --version
GNU gdb (GDB) Fedora Linux 12.1-6.fc37
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

OpenSSL 3.0.8

$ openssl version
OpenSSL 3.0.8 7 Feb 2023 (Library: OpenSSL 3.0.8 7 Feb 2023)

An example of debugging OpenSSL

Initially I was trying to reproduce the issue with OpenSSL 3.0.1 built from the source. However I saw one runtime error related to one of the dependency library of OpenSSL when running the binary with GDB, I just explain with the system OpenSSL 3.0.8. The segmentation fault issue doesn't happen on the version. However it just explain how to debug on the OpenSSL.

Here is a minimal reproducing script to check the issue above.

$ cat repro.c 
#include <openssl/hmac.h>

#define CF_CHECK_EQ(expr, res) if ( (expr) != (res) ) { goto end; }
#define CF_CHECK_NE(expr, res) if ( (expr) == (res) ) { goto end; }

int main(void)
{
    const unsigned char key[24] = {0};
    const unsigned char ct[166] = {0};

    const EVP_MD* md = NULL;
    EVP_PKEY *pkey = NULL;

    EVP_MD_CTX* ctx = EVP_MD_CTX_new();
    EVP_MD_CTX* ctx_tmp = NULL;

    CF_CHECK_NE(pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, NULL, key, sizeof(key)), NULL);
    CF_CHECK_EQ(EVP_DigestSignInit(ctx, NULL, EVP_sha256(), NULL, pkey), 1);

    ctx_tmp = EVP_MD_CTX_new();
    CF_CHECK_EQ(EVP_MD_CTX_copy(ctx_tmp, ctx), 1);
    EVP_MD_CTX_free(ctx);
    ctx = ctx_tmp;

    CF_CHECK_EQ(EVP_DigestSignUpdate(ctx, ct, sizeof(ct)), 1);
end:
    return 0;
}

Compile with GCC

When I debug a code, I find the following command is useful.

$ gcc -g3 -ggdb3 -gdwarf-5 $(pkgconf --cflags --libs openssl) -o repro repro.c

$ echo $?
0

$ ls -l repro
-rwxr-xr-x. 1 jaruga jaruga 413600 Feb 17 17:46 repro*

Let's see each part of the command line one by one. The -g* options are for debug. You can check it by man gdb or on the GDB official manual page - Debugging Options.

In my opinion, specifying the number of the debug options are a good practice, because you can be conscious about the which level (for -glevel and -ggdblevel) or version (-gdwarf-version) of the debug options. In new gcc version, the number can be default value. However when you work with a variety of gcc versions, specifying the level or version is useful.

To make this sustainable, you can set the alias below in your .bashrc.

$ alias gcc-debug='gcc -g3 -ggdb3 -gdwarf-5'

You can check the used DWARF version by the command below.

$ objdump -W ./repro | grep 'DWARF Version'
  DWARF Version:               5

For the pkgconf --cflags --libs openssl, the --cflags is to print the -I options, and --iibs is to print the -l options of the compiler options built to create the package.

You can see like this.

$ pkgconf --list-all | grep openssl
openssl                        OpenSSL - Secure Sockets Layer and cryptography libraries and tools

$ pkgconf --cflags --libs openssl
-lssl -lcrypto

When running the reproducer with system OpenSSL 3.0.8, it exists successfully without error.

$ ./repro

$ echo $?
0

Build OpenSSL 3.0.1 from the source

Next, let's install OpenSSL 3.0.1 to compare the issue by running the reproducing script. Check the INSTALL.md on the OpenSSL repository.

$ git clone https://github.com/openssl/openssl.git

$ cd openssl

$ git checkout openssl-3.0.1

$ ./Configure \
  --prefix=${HOME}/local/openssl-3.0.1 \
  linux-x86_64

$ make

You can run the make test optionally. I saw some test failures. However, I moved forward.

$ make test
...
Test Summary Report
-------------------
80-test_ssl_new.t                (Wstat: 256 (exited 1) Tests: 30 Failed: 1)
  Failed test:  12
  Non-zero exit status: 1
Files=242, Tests=3281, 364 wallclock secs ( 6.11 usr  0.58 sys + 296.88 cusr 57.21 csys = 360.78 CPU)
Result: FAIL
make[1]: *** [Makefile:3252: run_tests] Error 1
make[1]: Leaving directory '/home/jaruga/var/git/openssl'
make: *** [Makefile:3248: tests] Error 2
$ make install

$ ~/local/openssl-3.0.1/bin/openssl version
OpenSSL 3.0.1 14 Dec 2021 (Library: OpenSSL 3.0.8 7 Feb 2023)

Let's compile with OpenSSL 3.0.1.

$ gcc -g3 -ggdb3 -gdwarf-5 -lssl -lcrypto -I$HOME/local/openssl-3.0.1/include -L$HOME/local/openssl-3.0.1/lib64 -o repro-with-openssl-3.0.1 repro.c

$ echo $?
0

$ ls -l repro-with-openssl-3.0.1 
-rwxr-xr-x. 1 jaruga jaruga 412960 Feb 17 18:10 repro-with-openssl-3.0.1*

Check the link info. As a default, the binary links to the system OpenSSL 3.0.8.

$ ldd repro-with-openssl-3.0.1 
    linux-vdso.so.1 (0x00007fff1c4e1000)
    libssl.so.3 => /lib64/libssl.so.3 (0x00007f6407811000)
    libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f6407200000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f6407635000)
    libz.so.1 => /lib64/libz.so.1 (0x00007f64071e6000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f64078cd000)

With the LD_LIBRARY_PATH, it links to the OpenSSL 3.0.1.

$ LD_LIBRARY_PATH=$HOME/local/openssl-3.0.1/lib64 ldd repro-with-openssl-3.0.1
    linux-vdso.so.1 (0x00007fff32507000)
    libssl.so.3 => /home/jaruga/local/openssl-3.0.1/lib64/libssl.so.3 (0x00007f12fcea2000)
    libcrypto.so.3 => /home/jaruga/local/openssl-3.0.1/lib64/libcrypto.so.3 (0x00007f12fca00000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f12fc824000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f12fcf4c000)

The binary exits successfully without error.

$ ./repro-with-openssl-3.0.1

$ echo $?
0

And the binary exists with segmentation fault as expected.

$ LD_LIBRARY_PATH=$HOME/local/openssl-3.0.1/lib64 ./repro-with-openssl-3.0.1
Segmentation fault (core dumped)

Prepare GDB setting

When debugging the binary with system OpenSSL 3.0.8 on the GDB, it can be like this.

$ gdb -q -ex "set breakpoint pending on" -ex "set debuginfod enabled on" --args ./repro
Reading symbols from ./repro...
(gdb) b main
Breakpoint 1 at 0x401191: file repro.c, line 8.
(gdb) r
Starting program: /home/jaruga/doc/memo/20230227_day_of_learning/openssl_test/repro 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, main () at repro.c:8
8       const unsigned char key[24] = {0};

However, to make this step easy and sustainable, you can set the init files. The set startup-quietly on is the same with the gdb -q. See man gdbinit.

$ cat ~/.config/gdb/gdbinit 
set breakpoint pending on
set debuginfod enabled on

$ cat ~/.config/gdb/gdbearlyinit 
set startup-quietly on

Or the following alias might also be useful.

$ alias gdb-args='gdb -q -ex "set breakpoint pending on" -ex "set debuginfod enabled on" --args'

Debug with GDB

So, you can run the gdb simply. I ran b main (break main), r (run), then c (continue), and bt (backtrace).

$ gdb --args ./repro
Reading symbols from ./repro...
(gdb) b main
Breakpoint 1 at 0x401191: file repro.c, line 8.
(gdb) r
Starting program: /home/jaruga/doc/memo/20230227_day_of_learning/openssl_test/repro 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, main () at repro.c:8
8       const unsigned char key[24] = {0};
(gdb)

I faced the following issue when debugging with the source-built OpenSSL 3.0.1. It seems an issue related to openldap. So, I couldn't debug this case.

$ LD_LIBRARY_PATH=$HOME/local/openssl-3.0.1/lib64 gdb --args ./repro-with-openssl-3.0.1
gdb: symbol lookup error: /lib64/libldap.so.2: undefined symbol: EVP_md2, version OPENSSL_3.0.0

$ echo $?
127

Debug with ltrace

The ltrace is library call tracer. It's useful when you debug libraries.

Here is the installed version on my environment.

$ sudo dnf install ltrace

$ rpm -q ltrace
ltrace-0.7.91-46.fc37.x86_64

$ ltrace --version
ltrace 0.7.91
Copyright (C) 2010-2013 Petr Machata, Red Hat Inc.
Copyright (C) 1997-2009 Juan Cespedes <cespedes@debian.org>.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Then run the commands.

The producer exists successfully with the system OpenSSL 3.0.8 libraries.

$ ltrace ./repro
EVP_MD_CTX_new(1, 0x7ffd7cc7c3f8, 0x7ffd7cc7c408, 0x403df8) = 0x17f92a0
EVP_PKEY_new_mac_key(855, 0, 0x7ffd7cc7c2a0, 24)     = 0x180e530
EVP_sha256(7, 0x17f9010, 6170, 10)                   = 0x7ff59b3e4ec0
EVP_DigestSignInit(0x17f92a0, 0, 0x7ff59b3e4ec0, 0)  = 1
EVP_MD_CTX_new(0x1818710, 0, 0, 0x18246d0)           = 0x1824750
EVP_MD_CTX_copy(0x1824750, 0x17f92a0, 0x17f92a0, 0x1824750) = 1
EVP_MD_CTX_free(0x17f92a0, 0x18245b0, 24, 0x1824ba0) = 0
EVP_DigestSignUpdate(0x1824750, 0x7ffd7cc7c1f0, 166, 0x7ffd7cc7c1f0) = 1
+++ exited (status 0) +++

The segmentation fault happens with the OpenSSL 3.0.1 libraries.

$ LD_LIBRARY_PATH=$HOME/local/openssl-3.0.1/lib64 ltrace ./repro
EVP_MD_CTX_new(1, 0x7fff121e58d8, 0x7fff121e58e8, 0x403df8) = 0xbee2a0
EVP_PKEY_new_mac_key(855, 0, 0x7fff121e5780, 24)     = 0xc027f0
EVP_sha256(7, 0xbee010, 3117, 10)                    = 0x7f07b4815ba0
EVP_DigestSignInit(0xbee2a0, 0, 0x7f07b4815ba0, 0)   = 1
EVP_MD_CTX_new(0xc2cc00, 0, 0, 0xc472d0)             = 0xc02e20
EVP_MD_CTX_copy(0xc02e20, 0xbee2a0, 0xbee2a0, 0xc02e20) = 1
EVP_MD_CTX_free(0xbee2a0, 0, 0xbee2a0, 0xc02e20)     = 0
EVP_DigestSignUpdate(0xc02e20, 0x7fff121e56d0, 166, 0x7fff121e56d0 <no return ...>
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++
$ LD_LIBRARY_PATH=$HOME/local/openssl-3.0.1/lib64 ltrace ./repro-with-openssl-3.0.1
EVP_MD_CTX_new(1, 0x7fff718b5828, 0x7fff718b5838, 0x403df8) = 0x8a22a0
EVP_PKEY_new_mac_key(855, 0, 0x7fff718b56d0, 24)     = 0x8b67f0
EVP_sha256(7, 0x8a2010, 2273, 10)                    = 0x7fbdd2015ba0
EVP_DigestSignInit(0x8a22a0, 0, 0x7fbdd2015ba0, 0)   = 1
EVP_MD_CTX_new(0x8e0c00, 0, 0, 0x8fb2d0)             = 0x8b6e20
EVP_MD_CTX_copy(0x8b6e20, 0x8a22a0, 0x8a22a0, 0x8b6e20) = 1
EVP_MD_CTX_free(0x8a22a0, 0, 0x8a22a0, 0x8b6e20)     = 0
EVP_DigestSignUpdate(0x8b6e20, 0x7fff718b5620, 166, 0x7fff718b5620 <no return ...>
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++

That's all. Happy debugging!