Customers might state that they have missing data, comparing two systems in same environments where they see data differences, missing clickstreams or simply want to view raw data in their shards.
These need to be investigated in the actual log files located in the /var/opt/silvertail/data directory.
The directories are arranged by the following:
Do a webex session with the customer and ask them to go to the data in question to obtain the shard files.
Determine the shard that needs to be extracted. This will depend on the issue, however, in general the shards are arranged by the number of bits specified. The shards are distributed by IP's across the shards. When you list the directory, the name of the shard will contain a bit value.
The customer or CS Engineer will use a built-in utility to decrypt the targeted data files called logcat. This utility is not officially supported but can be used.
The command below will decrypt and move to a temp folder for collection.
logcat -f conf FILE
The location of conf file is required and is usually in two places var/opt/silvertail/etc/logcat.conf or universal_conf.txt. For example:
Further analysis on the shard files using getshard.
getshard –i <ip> -b 8
The 8 is seen in the naming of the shards is the number of bits in size. For example:
getshard -b 8 -i 192.168.1.2
Explore shards with imported utility loggrep. The loggrep utility can be obtained by FRI CS Engineers and can be used to look through the contents of a single shard. For example:
# var/opt/silvertail/bin/loggrep –I <ip address> -m POST
Example Cat the shard and run loggrep to find a particular IP address and REQUEST in the contents, do a count and print the unique entries. (This will show if there are too many counts for one page, or too many unique pages because page normalization has not been performed.)
There are many ways to approach the analysis of shard files but this is an introductory to using these tools.
Decrypted shards will contain raw data and may contain sensitive information. These files should be handled in a secure fashion.
Use of getshard
[root@wtd etc]# /var/opt/silvertail/bin/getshard
Silver Tail shard generator version 22.214.171.124
getshard -i <ip> -b <bits> [-H] -s <string> [-v]
-i <ip> IP address to show the shard (required)
-b <bits> Number of shard bits. Default 0.
-H Display the hash value of ip (optional)
-s <string> Show the hash of string.
-v Display the version