000018275 - How to enable SSL debug output during the handshake

Document created by RSA Customer Support Employee on Jun 16, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000018275
Applies ToBSAFE/SSL-C
BSAFE/SSL-C ME
BSAFE/MES
IssueHow to add debug code to an application to dump raw SSL packets
How to enable SSL debug output during the handshake
Resolution

The following code defines the callbacks which do the actual data dumping.   
Read the comments for explanations of what the functions do.  Implementations of these callbacks are included in the sample code in the samples/source/common directory.

/*
 * Global BIO for handling debug output
 */
BIO *bio_err;

/*
 * BIO debug callback definition 

 * This callback dumps the raw data sent to and from an SSL object.  
 * It is set into the BIO object that the SSL object uses internally.
 */
long MS_CALLBACK bio_data_dump_cb(BIO *bio, int cmd, char *argp, int argi,
    long argl, long ret)
{
    BIO *out = (BIO *)BIO_get_cb_arg(bio);
    char *mode = NULL;
   
    if (out == NULL)
    {
        return(ret);
    }

    if (cmd == (BIO_CB_READ | BIO_CB_RETURN))
    {
        mode = "read";
    }
    else if (cmd == (BIO_CB_WRITE | BIO_CB_RETURN))
    {
        mode = "write";
    }
    else
    {
        return(ret);
    }
   
    BIO_printf(out, "%s to %08X [%08lX] (%d bytes => %ld (0x%X))\n", mode,
        bio, argp, argi, ret, ret);
    BIO_dump(out, (unsigned char *)argp, (int)ret);
   
    return(ret);
}

/*
 * SSL_CTX object state callback
 * This callback is set against either an SSL_CTX or an SSL object.
 * If it's set against the SSL_CTX then all SSL objects created
 * from that SSL_CTX inherit this callback.
 *

 * It prints out the current state of the connection during the
 * SSL handshake. 

 */
void SSL_CTX_state_cb(SSL *s, int where, int ret)
{
    char *str;
    int w;
    w = where& ~SSL_ST_MASK;
    if (w & SSL_ST_CONNECT)
        str="SSL_connect";
    else if (w & SSL_ST_ACCEPT)
        str="SSL_accept";
    else
        str="undefined";

    if (where & SSL_CB_LOOP)
    {
        BIO_printf(bio_err, "%s:%s\n", str,
            SSL_state_string_long(s));
    }
    else if (where & SSL_CB_ALERT)
    {
        str=(where & SSL_CB_READ)?"read":"write";
        BIO_printf(bio_err, "SSL3 alert %s:%s:%s\n",
            str,
            SSL_alert_type_string_long(ret),
            SSL_alert_desc_string_long(ret));
    }
    else if (where & SSL_CB_EXIT)
    {
        if (ret == 0)
            BIO_printf(bio_err, "%s:failed in %s\n",
                str, SSL_state_string_long(s));
        else if (ret < 0)
        {
            BIO_printf(bio_err, "%s:error in %s\n",
                str, SSL_state_string_long(s));
            BIO_free(bio_err);
        }
    }
}


The following code sets the above BIO callback's into their appropriate objects. 


/*
 * Create the BIO for outputting the debug data
 */
bio_err = BIO_new_fp(stderr, BIO_NOCLOSE);

/*
 * After a call to SSL_CTX_new and before the call to SSL_new, add this code to setup the
 * state callback.
 */
SSL_CTX_set_info_cb(ssl_ctx, SSL_CTX_state_cb);

/*
 * This code sets the raw data output callback into the underlying BIO used by an
 * SSL object.
 *
 * Note: if you are creating a BIO to connect to the peer and setting that into
 * the SSL object you can apply this callback directly to that BIO.  Replace
 * 'sslBIO' with the BIO you are setting.
 *
 * Note 2: the bio_err is created with the following code.  This makes it easy
 * to redirect the output to a file or another stream (i.e. stdout).
 *   BIO_new_fp (stderr, BIO_NOCLOSE);
 * 
 */
BIO *sslBIO = SSL_get_rbio   (ssl);
BIO_set_cb(sslBIO , bio_data_dump_cb);
BIO_set_cb_arg(sslBIO, (char *)bio_err);

NotesThe purpose of these callbacks is to see exactly what is happening during the SSL handshake.  This is a fantastic tool to debug handshake problems.
If the code makes a call to SSL_set_fd, the code to install the bio_data_dump_cb callback must come after the call to SSL_set_fd.  SSL_set_fd resets the underlying BIO so the callback has to be installed after SSL_set_fd.
Legacy Article IDa35516

Attachments

    Outcomes