Decoder: Map IP Address to Service Type

Document created by RSA Information Design and Development on Oct 23, 2017Last modified by RSA Information Design and Development on Nov 7, 2017
Version 2Show Document
  • View in full screen mode
  

This topic describes the procedure to map an IP address to a service type for log parsing.

The Log Collector discovers event source type on a per-message basis. If the correct parser is not used for the specific event source, the messages that are common between event source types are misclassified. The misidentified messages will not populate service rules and alerts, and the reports will not have proper information. Also, if there are multiple services associated with an IP address, it can be difficult for the parsers to identify the exact service from which the log is generated. 

If you map an IP address to its services, the log decoder can identify the service from which the log is generated. When messages come into the log decoder from a mapped service, the assigned parsers are loaded to find event matches. 

You can assign service types to IPV4, IPV6 or hostname value of the event source. You can also assign multiple service types to a single IP address. You can also use the CollectorID when different service types with the same IP address are sent to different collectors.

Procedure

To map an IP address to a service type, do the following:

  1. In the Security Analytics menu,  select Administration > Services.
  2. In the Services view, select a Log Decoder, and in the Actions column, select Actions menu cropped > View > Explore.
  3. Go to /decoder/parsers node, right-click parsers, and select Properties.
  4. In the Properties view, specify the ipdevice command with the following parameters:
    op=add/remove entries="ipaddress=service” (for example, op=add entries="10.100.201.300=ciscoasa")
  5. Click Send.

IPdevice Command

In the ipdevice command, three operations are available:

  • add: This operation adds or updates entries in the ipdevice map. Multiple space delimited address/type pairs may be specified.
    op=add entries="<address>=<service type>"
  •  remove: This operation removes entries from the ipdevice map. Multiple space delimited address/type pairs may be specified.
    op=remove entries="<address>"
  • describe: This operation returns the values currently in the ipdevice map.

Time Zone Support

The Log Decoder currently has the ability to configure the system so that a given log device source can be associated with a time zone so that the event can be correctly converted to UTC across all devices.

Three time zone formats are currently accepted and are shown in the following examples:

  1. Olson format:
    America/Anguilla
  2. POSIX formats:
    EST5EDT
    AST2:45ADT0:45,M4.1.6/1:45,M10.5.6/2:45
  3. Offset by Hours formats:
    EST
    -500

Note: Offset by Hours time zone formats do not change for Daylight Savings Time.

Result

Security Analytics maps the IP address to a time zone in the log decoder. Event time meta is updated according to their respective mappings.

Procedure

To map an IP address to a time zone, do the following:

  1. In the Security Analytics menu, select Administration > Services.
  2. In the Services view, select a Log Decoder, and in the Actions column, select Actions menu cropped> View > Explore.
  3. Go to /decoder/parsers node, right click Parsers, and select Properties.
  4. In the Properties view, specify the iptmzone command with the following parameters:
    op=add entries="ipaddress=timezone" (for example, op=add entries="10.10.10.10=Africa/Addis_Ababa")
  5. Click Send.

iptmzone Command

In the iptmzone command, three operations are available:

  • add: This operation adds or updates entries in the iptmzone map. Multiple space delimited address/type pairs may be specified.
    op=add entries="<address>=<time zone>"
  • remove: This operation removes entries in the iptmzone map. Multiple space delimited address/type pairs may be specified.
    op=remove entries="<address>"
  • describe: This operation returns the values currently in the iptmzone map.

Examples

The following examples provide instances for mapping IP addresses to time zones:

  • If you want to map two different entries with different IPV4 values and time zone, enter the following parameter in the iptmzone command and click Send
    "op=add entries=”10.10.10.10=America/Anguilla 10.10.10.11=Pacific/Rarotonga”
  • If you want to remove an entry for a single IPV4 value and time zone, enter the following parameter in the iptmzone command and click Send.

"op=remove entries=10.5.245.9"

  • If you want to create a single entry for an IPV6 value and time zone, enter the following parameter in the iptmzone command and click Send.

op=add entries=”2001:DB8:85A3::8A2E:370:7334=America/Anguilla”

  • If you want to map a single device to a time zone or offset, you can create an entry by using:

    op=add entries="<address>=<time>"
    Where <address> is an IPV4, IPV6, or hostname and where <time> is an integer offset or a time zone Olson, or POSIX format. Enter the following parameter in the iptmzone command and click Send.

    For example:
    op=add entries="10.168.0.2=EST5EDT"

    Alternately, you can enter
    the following parameter in the iptmzone command and click Send.

    For example:
    op=add entries="10.168.0.2=America/Anguilla 2001:DB8:85A3::8A2E:370:7334=0500 nwappliance21=EST5EDT,M3.2.0/2,M11.1.0"

Event Time Support

A parameter mdformat has been added to the iptmzone message in /decoder/parsers on a Log Decoder.
The Log Decoder currently has the ability to change the dates in the logs to have the following format:
mdy or dmy (month/day/year or day/month/year)

By default, the value for the date is set to none.

Note:
. Event time meta in a parser does not account for dmy and mdy.
. The day/month/year change can only be done from the Rest API. There is no UI capability to do this.
. Currently, there is no functionality to detect this scenario and adjust parsing automatically.

Result

Security Analytics maps the IP address along with the date format in the Log Decoder. Event time meta is updated according to their respective mappings.

Procedure

To change the date format, do the following:

  1. In the Security Analytics menu, select Administration > Services.
  2. In the Services view, select a Log Decoder, and in the Actions column, select Actions menu cropped > View > Explore.
  3. Go to /decoder/parsers node, right click Parsers, and select Properties.
  4. In the Properties view, select iptmzone from the drop-down list, and specify the command with the following parameters:
    op=add entries="ipaddress" mdformat="dmy or mdy or none"(for example, op=add entries="1.1.1.1" mdformat=dmy)
    Where,
    entries is the Device IP address, and
    mdformat can be dmy, mdy, or none.
  5. Click Send.

event time support

Missing Year Support

If there is no year associated with a format string, the parser attempts to populate the year heuristically. In most cases, the value is populated based on the current year. The current year is assigned as the message year (UTC), as per the clock on the Log Decoder.

The following are the various other scenarios:

  1. Latent log during transition to new year.
  2. Logs from forward time zones (or skewed clocks) just before new year transition.
  3. Latent logs received where a leap day cannot be successfully assigned to an appropriate leap year.

Latent log during transition to new year

If the day is more than 31 days into the future, the year is decremented.

For example,

The message date is Dec-31. The current date is 2018-Jan-1. The temporary assigned date, 2018-Dec-31, will be more than 31 days into the future.

This will cause the year component to be decremented to 2017, resulting in a reasonable message time.

Logs from forward time zones (or skewed clocks) just before new year transition

If the day is more than 334 days into the past, the year is incremented.

For example,

The message date is Jan-1. The current date is 2017-Dec-31. The temporary assigned date, 2017-Jan-1, will be more than 334 days into the past.

This will cause the year component to be incremented to 2018, resulting in a reasonable message time.

Latent logs received where a leap day cannot be successfully assigned to an appropriate leap year

If the day (Feb 29) is invalid (say, the assigned year is NOT a leap year), the relative position to the current time cannot be calculated. To do this, the year is decremented in an attempt get a valid time stamp.

Because the position of the leap day relative to the transition to the new year, along with the expectation that no logs would be received this far into the future, we do not ever make an attempt to increment the year to produce a valid time stamp.

After the year is decremented, the same logic is then followed (refer to statement 1 and 2). If the result is still an invalid day. The parser throws and a valid message time cannot be parsed.

Limitations

Note: Currently, there is no mechanism to assign a year to the import of logs.

As this pertains to leap days, if you find the correct leap year, an inconsistent data would result. For example, if a set of messages is two years old and during a leap year, only a single day would remain which is accurate. Because of the heuristic described above, the remaining set of messages would have time stamps one year too.

For data consistency, usually you cannot find valid leap years beyond 1 month and less than 11 months.

Examples

The following examples provide instances for logs with year and logs without year:

  • If the log is with year, event time is generated as below:

%emcavamar: 1704^^2017-04-07^^07:50:02^^1^^<event-source NodeID="avamar" ProgramName="com.avamar"/>^^1270626^^SYSTEM^^ERROR^^OK^^MCS:DPN_Proxy^^Internal server error^

<MESSAGE

id1="1:01"

id2="1"

eventcategory="1605020000"

functions="&lt;@msg:*PARMVAL($MSG)&gt;&lt;@event_time:*EVNTTIME($MSG,'%W-%G-%F %H:%U:%O',fld2,fld3)&gt;"

content="&lt;fld1&gt;^^&lt;fld2&gt;^^&lt;fld3&gt;^^&lt;fld4&gt;^^&lt;&lt;event-source NodeID=&quot;&lt;hostname&gt;&quot; ProgramName=&quot;&lt;fld8&gt;&quot;/&gt;^^&lt;fld9&gt;^^&lt;category&gt;^^&lt;severity&gt;^^&lt;event_type&gt;^^&lt;agent&gt;^^&lt;event_description&gt;" />

  • If the log is without year, event time is still generated, The current year is assigned as the message year (UTC), as per the clock on the Log Decoder.

%emcavamar: 1704^^04-07^^07:50:02^^1^^<event-source NodeID="avamar" ProgramName="com.avamar"/>^^1270626^^SYSTEM^^ERROR^^OK^^MCS:DPN_Proxy^^Internal server error^

<MESSAGE

id1="1:01"

id2="1"

eventcategory="1605020000"

functions="&lt;@msg:*PARMVAL($MSG)&gt;&lt;@event_time:*EVNTTIME($MSG,'%G-%F %H:%U:%O',fld2,fld3)&gt;"

content="&lt;fld1&gt;^^&lt;fld2&gt;^^&lt;fld3&gt;^^&lt;fld4&gt;^^&lt;&lt;event-source NodeID=&quot;&lt;hostname&gt;&quot; ProgramName=&quot;&lt;fld8&gt;&quot;/&gt;^^&lt;fld9&gt;^^&lt;category&gt;^^&lt;severity&gt;^^&lt;event_type&gt;^^&lt;agent&gt;^^&lt;event_description&gt;" />

You are here
Table of Contents > Additional Procedures > Map IP Address to Service Type

Attachments

    Outcomes