Has anyone had any experience creating a custom feed based around IPv6 addresses?
Specifically, I want to do something very similar to what is in the examples that show how to create department listings using IP ranges, I just want to set up a similar feed using our IPv6 ranges. A couple of the methods that I have used have failed, and I am not sure if it is in how I am structuring the XML file, or if it is a format issue in the CSV file in how I am listing the IPv6 ranges.
Any ideas?
Thanks
Upload your CSV and we'll take a look to see if we can spot the problem.
These are just examples, but basically my CSV looks like:
2001:0db8:85a3:0042::/64,Dept 1
2001:0db8:85a3:0043::/64,Dept 2
2001:0db8:85a3:0044::/64,Dept 3
2001:0db8:85a3:0045::/64,Dept 4
2001:0db8:85a3:0046::/64,Dept 5
My XML looks something like this:
<FDF> <FlatFileFeed name="IPv6 Source Dept Listing" path="dept_list_ipv6.csv" separator="," comment="#"> <MetaCallback name="dept source" valuetype="IPv6"> <Meta name="ipv6.src"/> </MetaCallback> <LanguageKeys> <LanguageKey name="deptipv6.src" valuetype="Text"/> </LanguageKeys> <Fields> <Field index="1" type="index" range="cidr"/> <Field index="2" type="value" key="deptipv6.src" /> </Fields> </FlatFileFeed> </FDF>
First of all, do you have a meta key in your index for deptipv6.src and .dst?
Second I've not done a feed with ipv6 before, and I'm not certain that CIDR would apply.
To keep it simple, use ranges. Here is a sample XML that could use low and high ranges. The trick will be to define the boundary IP addies in your csv.
<FlatFileFeed name="NetName" path="acmenetworks.txt" separator=","
comment="#">
<LanguageKeys>
<LanguageKey name="netname" valuetype="Text"
srcname="netname.src" destname="netname.dst" />
</LanguageKeys>
<Fields>
<Field index="1" type="index" range="low"/>
<Field index="2" type="index" range="high"/>
<Field index="3" type="value" key="netname"/>
</Fields>
</FlatFileFeed>
Note in this example it calls for a netname keys for your index, but you can change that to your deptipv6 key instead.
Give this a try and let me know if you are successful.
The "System Administration Guide Version 9.8", pages 232 says on IPv6 addresses: "Does not support CIDR notation". The guide is from July 2012, but the statement probably still holds.
However, the template of Fielder doesn't work either. It seems like the attribute "range" requires IPv4 addresses. If I try to compile the feed with IPv6 ranges, I get the following output:
Creating feed IPv6 Source Dept Listing... done. 0 entries, 4 invalid records All feeds complete.
Thank you both. I am farther than I was yesterday.
Fielder, I tried that XML and updated my csv to use ranges:
2001:0db8:85a3:0042:0000:0000:0000:0000,2001:0db8:85a3:0042:ffff:ffff:ffff:ffff,Dept 1
2001:0db8:85a3:0043:0000:0000:0000:0000,2001:0db8:85a3:0043:ffff:ffff:ffff:ffff,Dept 2
etc,
And I got the same error that Johannes is getting.
The thought behind my original XML I posted above was that I needed to define in the feed that I wanted to look at ipv6 addresses. I didn't think SA/NW would understand that the indexes were "ip's" like it does with v4 addresses.
Once I got the failure fure from the xml above, I tweaked my original to look like this:
<FDF> <FlatFileFeed name="IPv6DeptList" path="dept_listing_ipv6.csv" separator="," comment="#"> <MetaCallback name="IPv6 Source Address" valuetype="IPv6"> <Meta name='ipv6.src'/> </MetaCallback> <LanguageKeys> <LanguageKey name="ipv6dept.src" valuetype="Text" /> </LanguageKeys> <Fields> <Field index="1" type="index" range="low"/> <Field index="2" type="index" range="high"/> <Field index="3" type="value" key="ipv6dept.src"/> </Fields> </FlatFileFeed> </FDF>
Once I did that and retried creating the feed, it accepted all of my entries (minus 1, but I think I know why).
Haven't checked to see if its "working" yet, but this is farther than I have gotten before.
So if this works, my next question is, how would the MetaCallback along with my new ipv6dept key name, and then specify ipv6dept.src and ipv6dept.dst in the same file, instead of having to call them out in separate feeds? If that isn't possible when using the MetaCallback, thats fine. Still just looking to create the feed as efficiently as possible.
Since its IP v6, I don't think you need a metacallback. And like in my example above, you simply specify a language key of ipv6dept. You use two index keys of ipv6dept.src and dept.dst.
In the example I posted above this would create meta in two keys- both src and dst. No need for two feeds
Hmm, then I am not sure why it will not accept the entries in the csv when I use your method. Format issue in the csv somehow?
I didn't change the csv when I created the metacallback version, but all of my entries were invalid otherwise. I did try using the valuetypes of Text, as well as IPv6.
I did a couple of tests with Investigator 9.7.5.9 and the the compiler of 9.8.5.5:
Here's a working feed-definitions.xml based on the feed of aebay:
<FDF> <FlatFileFeed name="IPv6DeptList" path="dept_list_ipv6.csv" separator="," comment="#"> <MetaCallback name="meta" valuetype="IPv6"> <Meta name='ipv6.src'/> <Meta name='ipv6.dst'/> </MetaCallback> <LanguageKeys> <LanguageKey name="ipv6dept" valuetype="Text" srcname="ipv6dept.src" destname="ipv6dept.dst" /> </LanguageKeys> <Fields> <Field index="1" type="index" range="low"/> <Field index="2" type="index" range="high"/> <Field index="3" type="value" key="ipv6dept"/> </Fields> </FlatFileFeed> </FDF>
Thanks Johannes! Just made that change, and will be watching for that key to start populating. Still new to feed writing, and wasn't sure how to tackle more than one Meta field.
I'll post again if I have any issues, but that looks like the solution right now.
Thanks again all,
-A
Is it possible to create meta on ALL ipv6dept in the above example? The above example produces .src and .dst meta but I just want to see ipv6dept meta for any sessions that have a source or destination in my CSV. Does that make sense?
I have created a new Key and want to see meta generated along the lines:
ip6dept
Dept1 (#sessions), Dept2 (#sessions),
and on the Brokers and Concentrators update the index-XXX.xml to include:
<key description="IPv6 Department" name="ipv6dept" format="Text" level="IndexValues" />
Also, is there a document somewhere that details all the options for the LanguageKeys? There is not much detail in the NWAdministrator guide on this topic.
Just remove the dstname and srcname attributes from the language key.
The feed documentation is great compared to Lua parsers. There are a couple of restriction though that are undocumented. One thing that cost me a couple of hours: Never have overlapping IP ranges in one feed. They are very common in practice, but not supported by Netwitness.
Not that anyone else is still trying to do this, but I just wanted to post a follow up. This is actually still not working. If I use the above and specify individual IPv6 addresses, it will show up in my feed. But increasing the low,high to any kind of range (even just 2 or 3 addresses) breaks it.
Working with support, but at this point I don't have any warm fuzzies that what I am trying to do is possible.
I can confirm that the feed does not work for ranges. I would be very much interested in the response of the support team.
I just received the following response from support:
I learned from Dev currently range lookups are only supported with IPV4. Support for IPV6 is being added to IP Feeds in version 10.2 which is due out in a few weeks.
I just received the following response from support: