000038772 - Frequent Search Index Rebuilds needed for RSA Archer

Document created by RSA Customer Support Employee on Apr 28, 2020Last modified by RSA Customer Support Employee on Sep 9, 2020
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000038772
Applies ToRSA Product Set: RSA Archer
RSA Version/Condition:  All
Issue
  • The RSA Archer Search Index breaks frequently or is often slow.
  • Quick Searches are no longer working for new records.
CauseCorruption of the RSA Archer Search Index can be caused by a number of known reasons:
  • A recent upgrade.
  • Whenever backing up and restoring an instance database.
  • Locks on the Search Index file path and more.
Here we will walk through a number of scenarios and configurations that can cause frequent issues with the Search Index.
ResolutionHow to determine if the Search Index is truly broken:
  1. Quick Searches are no longer working for new records.
  2. tblSearchMessageQueue is backlogged:
    • Run the following SQL query and observe the count returned: 


SELECT COUNT(*) FROM dbo.tblSearchMessageQueue smq WHERE smq.priority_type_id = 3


  • If the count returned is above 0, run the query again in 5 minutes. If the count returned is higher after 5 minutes, this could point to a broken Search Index.

  1. Check the Search Index file path that is specified in the ACP and ensure the timestamps are being updated. If not, this could also point to a broken Search Index. 
If all 3 items are true, you will need to continue with a full Search Index rebuild using the steps outlined in KB 000030392, located here https://community.rsa.com/docs/DOC-45540


Common configuration items that can cause frequent Search Index issues:

  1. On the Queuing Server, verify that the virus protection software excludes the Search Index File Location from scanning.
  2. Ensure that the RSA Archer Queuing Service is enabled and running on only one server.
  3. Have the Search Index File Location local to the Queuing Server like C:\ArcherFiles\Index.  Share this folder path like \\MyServer\Archer\Index and update the Search Index Path in the ACP with the UNC path.  This prevents issues from things like DNS name resolution, share permissions, network latency, and so forth.
  4. If the Queuing Server also handles the Job Engine and/or Web processes, you may want to consider dedicating a server for the Queuing Service.
  5. Ensure that the Service Account running the Queuing Service has full access to the Search Index file path.
  6. Ensure that no two RSA Archer Instances are using the same Search Index file path.

Other common issues:

  1. write.lock errors continue to occur in Queuing Service logs (example below), despite following all the above-noted recommendations.

    Error closing the index path: \\ServerName\archer\indexes\50000\quicksearch\, Lucene.Net.Store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@\\ServerName\archer\indexes\50000\quicksearch\write.lock
    at Lucene.Net.Store.Lock.Obtain(Int64 lockWaitTimeout)
    at Lucene.Net.Index.DirectoryReader.AcquireWriteLock()
    at Lucene.Net.Index.IndexReader.DeleteDocument(Int32 docNum)
    at Lucene.Net.Index.IndexReader.DeleteDocument(Int32 docNum)
    at Lucene.Net.Index.IndexReader.DeleteDocuments(Term term)
    at ArcherTech.Services.Queuing.IndexWriterFacade.DeleteAllDocumentsByContentId()
    at ArcherTech.Services.Queuing.IndexWriterFacade.Flush(Boolean deleteBeforeFlush)
    at ArcherTech.Services.Queuing.IndexFactory.Close(String indexPath, Boolean deleteBeforeFlush)
    7/24/2019 1:59:31 PM - Started Optimizing \\ServerName\archer\indexes\50000\quicksearch\
    7/24/2019 1:59:32 PM - Error optimizing the index path: \\ServerName\archer\indexes\50000\quicksearch\, Lucene.Net.Store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@\\ServerName\archer\indexes\50000\quicksearch\write.lock
    at Lucene.Net.Store.Lock.Obtain(Int64 lockWaitTimeout)
    at Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean create, Boolean closeDir, IndexDeletionPolicy deletionPolicy, Boolean autoCommit, Int32 maxFieldLength, IndexingChain indexingChain, IndexCommit commit)
    at Lucene.Net.Index.IndexWriter..ctor(String path, Analyzer a, Boolean create)
    at ArcherTech.Services.Queuing.IndexFactory.Optimize(String fullIndexPath)
    7/24/2019 1:59:33 PM - Finished Optimizing \\ServerName\archer\indexes\50000\quicksearch\

  2. Lower the Flush Factor value from 4000 to 1000 in \\ArcherInstallDirectory\Services\Archer.Services.Queuing.exe.config file. This will allow the index files to merge. Lowering the flushFactor setting forces the segment files to be smaller in size to allow smaller files to be merged at any given point, reducing memory and performance issues.
Workaround  

Attachments

    Outcomes