000039431 - Search index optimization of file repository bogs down indexing process for very large indexes in Archer 6.x

Document created by RSA Customer Support Employee on Nov 4, 2020
Version 1Show Document
  • Comment • 0
  • View in full screen mode

Article Content

Article Number000039431
Applies ToProduct Set: Archer
Version/Condition: 6.x
 
IssueLucene, the search indexing plugin that is used by Archer, performs a process called optimization on the file repository index at a default interval of three minutes. If the index is very large, optimization can take longer than three minutes, so the process repeats as soon as it finishes. Optimization uses a lot of memory and CPU so it can slow down the indexing and search process. Also, while optimization is being done, the index cannot be searched.


 
CauseReview the queuing service log file to see how long optimization is taking for the file repository directory.  If the optimization restarts soon after it completes, this can interfere with indexing and searching.  

In the example below, optimization of the file repository takes about seven minutes, and restarts as soon as it completes.
 

10/23/2020 12:10:29 AM - Optimizing index path \\<host>\SearchIndex\50000\RepositoryFiles\
...
10/23/2020 12:17:30 AM - Optimized index path \\<host>\SearchIndex\50000\RepositoryFiles\
...
10/23/2020 12:17:32 AM - Optimizing index path \\<host>\SearchIndex\50000\RepositoryFiles\
...
10/23/2020 12:26:33 AM - Optimized index path  \\<host>\SearchIndex\50000\RepositoryFiles\


 
ResolutionThe frequency of optimizing the repository file index can be increased in the queuing service configuration file.  Locate the file on the server running the queuing service, typically at the following path:
 

C:\Program Files\RSA Archer\Services\Archer.Services.Queuing.exe.config


Locate this setting:

<add key="repositoryOptimizeThreshold" value="3" />
<!-- How many minutes elapse before a file repository optimization runs (if documents have been added to the index file), minimum value is 1, max value is 1440, defaults to 3 -->


And change it to the maximum value, 1440. (There is no benefit to optimizing the index more frequently).

<add key="repositoryOptimizeThreshold" value="1440" />
<!-- How many minutes elapse before a file repository optimization runs (if documents have been added to the index file), minimum value is 1, max value is 1440, defaults to 3 -->


Restart the queuing service to load the changed configuration file.




 

Attachments

    Outcomes