I'm using a very simple query to pull back all hosts from NetWitness for offline analysis. You can see a small sample of the results below. The query for alias.host returns all values, which have many redundant entries.
The same query using Investigator will not display the redundant entries but rather show each alias.host entry only once.
Is there any way to duplicate this using the REST API?
Thanks!
Hi Rob,
Yes, there is. Look at the values message instead of the query one.
Something like /sdk?msg=values&size=1000&fieldName=alias.host&where=<yourwhereclause>&flags=2305
To sort by descending count as in Investigator.
Flags are described in the NwSDK documentation and copied below for ease of access.
Flags = 0x0001 - Total session count
Flags = 0x0002 - Total session size in bytes
Flags = 0x0004 - Total packet count
Flags = 0x0008 - Total field count
Flags = 0x0100 - Sort by Total
Flags = 0x0200 - Sort by Value
Flags = 0x0400 - Order Ascending
Flags = 0x0800 - Order Descending
Flags = 0x8000 - Estimate Total
Hope that helps!
Rui
Rui,
Thanks for the response. Using your method versus a query seems to time out with simple request to bring back 100 values. Using a WHERE clause to restrict the time to an hour window didn't seem to make a difference either.
http://XX.XX.XX.XX:50105/sdk?msg=values&size=100&fieldName=alias.host&flags=2305
| 334829 | 2012-Dec-09 13:59:11 | SDK-Values | audit | User admin (session 3083966, XX.XX.XX.XX:63913) has issued values (pipe 3084063): flags=2305 fieldName=alias.host size=100 |
| 334830 | 2012-Dec-09 13:59:42 | Rest | audit | User admin (session 3083966, XX.XX.XX.XX:63913) has a running REST request that has timed out, returning HTTP 408 |
| 334831 | 2012-Dec-09 13:59:44 | SDK-Values | audit | User admin (session 3083966, XX.XX.XX.XX:63913) has canceled values (pipe 3084063, elapsed time 00:00:33): flags=2305 fieldName=alias.host size=10 |
| 334870 | 2012-Dec-09 14:09:35 | SDK-Values | audit | User admin(session 3085244, XX.XX.XX.XX:63919) has issued values (pipe 3085250): flags=2305 "where time = '2012-Dec-09 08:00:00'-'2012-Dec-09 09:00:00' && service=80" fieldName=alias.host size=1000 |
| 334874 | 2012-Dec-09 14:10:06 | Rest | audit | User admin(session 3085244, XX.XX.XX.XX:63919) has a running REST request that has timed out, returning HTTP 408 |
| 334875 | 2012-Dec-09 14:10:08 | SDK-Values | audit | User admin(session 3085244, XX.XX.XX.XX:63919) has canceled values (pipe 3085250, elapsed time 00:00:33): flags=2305 "where time = '2012-Dec-09 08:00:00'-'2012-Dec-09 09:00:00' && service=80" fieldName=alias.host size=1000 |
The query returned values immediately, even for 10-20,000 size settings.
Regards,
Rob
Rob,
Try adding "&expiry=0" to the URL, that should avoid the timeouts. I'm not sure why your query is causing a timeout even for samll time windows.
Hope that helps!
Rui
Rui - you nailed it!
The lack of the "&expiry=0" was the issue. The values lookup performance versus the query lookup performance is very different. Since the host.alias is indexed and the lookup was over a short timeframe, I'm very surprised at the time the lookup took to complete. The query was MUCH faster, although it did return duplicate values.
Example URL bringing back 20,000 values for service=80 and timeframe 12/8/12 08:00 - 12/9/12 09:00
Also if you are reading this thread and it's helpful, post some useful examples of using the API for others to learn
Thanks - Rob
Rob,
Try adding "&expiry=0" to the URL, that should avoid the timeouts. I'm not sure why your query is causing a timeout even for samll time windows.
Hope that helps!
Rui