000033022 - Recommendations for performance testing scripts for RSA Adaptive Authentication on Prem (AAoP)

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000033022
Applies ToRSA Product Set: Adaptive Authentication (OnPrem)
RSA Product/Service Type: Adaptive Authentication (OnPrem)
 
IssueCustomer wants to know the right parameters for doing performance testing and/or in some cases is getting unusual results, such as lockouts, when doing performance testing.
ResolutionPlease see the Notes section for more information on performance or load testing with customers and/or in customer environments.
When pressed for recommendations in scenarios where testing is failing or not proceeding as expected, we have provided some very limited input.  We originally suggested at least three changes which might need to be made to improve the testing scenarios.  Please see below:
  • Need to use multiple device Prints (i.e., several different ones, 25+).  Too few device prints will not approximate real world usage of AAoP.
  • Need three to four blocks of 256 unique IP addresses.  Too few IP addresses will not approximate real world usage of AAoP.
  • Should use 100-500 user IDs (this is not the same as virtual test users or vUsers).  Too few user IDs will very likely cause issues described below.
The last suggestion is highlighted as it is likely related to the cause of the lockouts seen in challenge question only testing sequences.  After consulting with one of the most senior implementation Engineers to get his input, we have summarized his comments below.
 
If you are running challenge only testing sequences with this limited number of user IDs, then there would be parallel threads using the same user ID.  AA will see multiple challenges for the same user using different session IDs.  If the abandoned challenge count parameter is enabled, it is a recipe for causing lockouts.  Increasing the challenge count threshold before lock out parameter and/or disabling the abandoned challenge count setting only provides a temporary relief or workaround for the issue.  The indicated solution would be to create many more AAoP user IDs than the number of virtual users configured in the testing platform.  This would remove the likelihood of the above described circumstance occurring during a testing sequence. 
 
An example “formula” may be helpful, if a little simplistic:  # of vUsers / # of AA UserIDs / # of Transactions per Second (TPS)  = # of concurrent uses of same UserID, e.g., 50 / 14 (3.57) /25 TPS (1.79) (i.e., 50% of vUser population using 14 AA UserIDs evenly distributed each second) =  each AA UserId in use 1.79 times each second.  Again, the chance of this causing lockout issues, with AAOP functioning as designed, is of a very high probability.

 
NotesPLEASE NOTE:  Neither RSA Technical Support (CS) nor RSA Engineering will engage in performance or load testing with customers and/or in customers' environments.  RSA QA Engineering has provided a performance guide for major version releases (e.g., the RSA Adaptive Authentication (On Premise) 7.2 Performance Guide).  This document can, and should, be shared with customers who request assistance with performance or load testing.  It will give them general guidelines as to environment configuration and specific tuning parameters for various application servers and database platforms.  The document also contains detailed testing results which were achieved by RSA QA for a given version. 
When pressed for recommendations in scenarios where testing is failing or not proceeding as expected, we have provided some very limited input.  We originally suggested at least three changes which might need to be made to improve the testing scenarios.

Attachments

    Outcomes