|Applies To||Access Manager 6.0|
All web agents
|Issue||Information about Web Crawlers and RSA Access Manager and Cleartrust|
"Web Crawler" is a generic name for any script or process that can be used for automating maintenance tasks on a website, such as checking to see if links are correct or for validating HTML code. Microsoft refers to them as "Web Spiders", and they are also known as web robots, ants, automatic indexes, bots, worms and scutters.
More general description information on:
RSA does not make any recommendations on which 3rd party product to select, but the following notations are applicable:
1) With Cleartrust 5.5.3 and Access Manager 6x products, you should be able to use any web crawler as long as it supports http cookies.
|Legacy Article ID||a40668|