000023478 - RKM C# client encrypted or HMAC data is too long  or is base64 encoded twice

Document created by RSA Customer Support Employee on Jun 16, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000023478
Applies ToKey Manager Client 1.5.2
C# or .NET
IssueRKM C# client encrypted data is too long, or is base64 encoded twice
Error message, "error getting keys from KMS, error from server, access denial, error code 4780018."
Encrypted data is too long, or HMAC value is too long
Decrypts or HMAC's are failing for no apparent reason
Cause

The C# or .NET code is probably double base64 encoding the encrypted or HMAC'd output. 

In RKM 1.5.2 we introduced base64 support to the client.  This was controlled via API changes in the functions KMSEncryptData and KMSHMACData.  The base64 argument, when true or nonzero, instructs the client to base64 encode the output.  KMSDecryptData autodetects base64 encoded data so it?s API did not change. 

The C# samples demonstrate how to call into the KMClient.dll function.  These samples were written before the base64 API changes were introduced and use the .NET Base64 class to base64 encode and decode data.  These samples were not updated to reflect the base64 API change.  The result is that C# code calling through our KMClientWrapper.cs sample interface do not explicitly pass a value to the new base64 argument so it?s mostly passing nonzero garbage, which the DLL interprets as a true value.  It returns base64 encoded data, which in turn is base64 encoded again by the sample code.   This double encoding is the source of the problem.  Most likely the customer code used our samples as a template and copied the bug. 

Resolution

The solution is to update their code to use the new argument, and to remove the code that explicitly base64 encodes the encrypted output.

Update the kmclientWrapper.cs file with the following update function signatures.  Then remove the explicity base64 code from the problem code and update the KMSEncryptData or KMSHMACData call to pass ?true? as a final argument.  This results in the cleaner code and you?ll only base64 encode your data once.


This code contains the new base64encode argument.  When set to true, outputted data is base64 encoded.

        [DllImport("kmclient.dll")]
        public static extern int KMSEncryptData(int handle,
               [MarshalAs(UnmanagedType.LPStr)] string keyclass,
               [MarshalAs(UnmanagedType.LPStr)] string clearText,
                                                int clearLen,
             [MarshalAs(UnmanagedType.LPArray)] byte[] cipherText,
                                                int cipherSize,
                                                ref int cipherLen,
                                                bool base64encode);

        [DllImport("kmclient.dll")]
        public static extern int KMSHMACData(int handle,
            [MarshalAs(UnmanagedType.LPStr)] string keyClass,
            [MarshalAs(UnmanagedType.LPStr)] string keyID,
            [MarshalAs(UnmanagedType.LPStr)] string text,
                                             int textLen,
          [MarshalAs(UnmanagedType.LPArray)] byte[] hmac,
                                             int hmacSize,
                                             ref int hmacLen,
                                             bool base64encode);

NotesUpdated C# samples are attached to Defect 57998.
Legacy Article IDa37183

Attachments

    Outcomes