rwpmapbuild(1) SiLK Tool Suite rwpmapbuild(1)NAMErwpmapbuild - Create a binary prefix map from a text file
SYNOPSISrwpmapbuild [--input-file=FILENAME] [--output-file=FILENAME]
[--mode={ipv4|ipv6|proto-port}] [--dry-run] [--ignore-errors]
[--note-add=TEXT] [--note-file-add=FILENAME]
rwpmapbuild--help
rwpmapbuild--version
DESCRIPTION
Prefix maps provide a way to map field values (specifically either IP
addresses or protocol-port pairs) to string labels based on a user-
defined map file. rwpmapbuild reads textual input to create a binary
prefix map file. The syntax of this input is described in the "INPUT
FILE FORMAT" section below.
As described in pmapfilter(3), you can partition, count, sort and
display SiLK flow records based on the string labels defined in the
prefix map. To view the contents of a prefix map file, use
rwpmapcat(1). To query the contents of a prefix map, use
rwpmaplookup(1).
The textual input is read from the specified input file, or from the
standard input when the --input-file switch is not provided. The
binary output is written to the named output file, or to the standard
output when the --output-file switch is not provided and the standard
output is not connected to a terminal.
OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an
exact match for an option. A parameter to an option may be specified
as --arg=param or --arg param, though the first form is required for
options that take optional parameters.
--input-file=FILENAME
Read the textual input from FILENAME. You may use "stdin" or "-"
to represent the standard input. When this switch is not provided,
the input is read from the standard input. The input file format
is described below.
--output-file=FILENAME
Write the binary prefix map to FILENAME. You may use "stdout" or
"-" to represent the standard output. When this switch is not
provided, the prefix map is written to the standard output unless
the standard output is connected to a terminal.
--mode={ipv4|ipv6|proto-port}
Specify the type of the input, as if a mode statement appeared in
the input stream. The value specified by this switch must not
conflict with an explicit mode statement appearing in the input.
--dry-run
Do not write the output file. Simply check the syntax of the input
file.
--ignore-errors
Write the output file regardless of any errors encountered while
parsing the input file.
--note-add=TEXT
Add the specified TEXT to the header of the output file as an
annotation. This switch may be repeated to add multiple
annotations to a file. To view the annotations, use the
rwfileinfo(1) tool.
--note-file-add=FILENAME
Open FILENAME and add the contents of that file to the header of
the output file as an annotation. This switch may be repeated to
add multiple annotations. Currently the application makes no
effort to ensure that FILENAME contains text; be careful that you
do not attempt to add a SiLK data file as an annotation.
--help
Print the available options and exit.
--version
Print the version number and information about how SiLK was
configured, then exit the application.
INPUT FILE FORMAT
The input file format consists of any number of input lines of the
forms described below. Note that there is not a form that accepts a
single IP address and a label; therefore, to provide a label for a
single IP address you must append "/32" to a single IPv4 address (or
"/128" to a single IPv6 address).
Blank lines in the input file are ignored, as are comments. Comments
begin with the first "#" character on a line and extend to the end of
the line.
rwpmapbuild maps ranges to string labels. These string labels may be
created either explicitly via the label statement or implicitly by
specifying text after a range, but a single input file must use only
one method to create labels. When the label statement is used, all
labels must be pre-declared in the label statement prior to their use
in the default statement or an range statements.
In the following, the label-value represents either a numerical label
identifier that was created with the label statement or label-text.
NOTE: Unlike many SiLK input files, there is no explicit delimiter
between the range and the string label. The range and string label are
separated by whitespace. The first non-whitespace character after the
range begins the label.
label-text is a textual string that begins at the first non-whitespace
character and extends to the final non-whitespace character on that
line that does not appear in a comment. The label-text may include
embedded whitespace and non-alphanumeric characters. While a comma
(",") is legal in the label-text, using a comma prevents the label from
being used by the --pmap-src and --pmap-dest switches in rwfilter(1).
The following statements are supported:
map-name simple-string
Creates a name for the data in this prefix map file. The simple-
string cannot contain whitespace, a comma, or a colon. When the
prefix map file is used by rwfilter(1), the simple-string is used
to generate the filtering switch names. When the prefix map file
is used by rwcut(1), rwgroup(1), rwsort(1), rwstats(1), or
rwuniq(1), the simple-string is used to generate the field names.
See pmapfilter(3) for details.
label num label-text
Associate the numeric identifier num with the given label text
label-text. It is an error if num or label-text appear in any
other label statement. The maximum allowed value for num is
2147483647, but note that rwpmapbuild creates an empty label for
all the unassigned numeric identifiers that are less than the
maximum identifier used in the input file. The label statement
must appear before the default statement and before range
definitions. When a label statement appears in the input,
rwpmapbuild will complain if you attempt to use a label-value that
was not previously defined in a label statement.
default label-value
Make the given label identifier or label text the default value for
any ranges not explicitly mentioned in this input file. The
default statement must appear before any ranges are specified. If
the default statement does not appear in the input, the label
"UNKNOWN" is automatically defined and used as the default.
mode { ipv4 | ipv6 | proto-port | ip }
Specify how to process the file. The mode statement must appear
before any ranges are specified. The mode can also be set using
the --mode command line switch. When both the mode statement and
the --mode switch are given, their values must match. When neither
the mode statement nor the --mode switch is provided, rwpmapbuild
processes the input in IPv4 address mode. The ip mode is
deprecated; it is an alias for ipv4.
Address Mode
When rwpmapbuild is in IPv4 address mode, any IPv6 address in the input
file will raise an error.
cidr-block label-value
Associate the given label identifier or label text with this CIDR
block. The CIDR block is composed of an IP address in canonical
notation (e.g, dotted-decimal for IPv4), a slash "/", and the
number of significant bits.
low-ip high-ip label-value
Associate the given label identifier or label text with this IP
range, where low-ip and high-ip are in canonical notation.
low-int high-int label-value
Treat low-int and high-int as 32-bit values, convert the values to
IPv4 addresses, and associate the given label identifier or label
text with the IPv4 range.
Protocol/Port Mode
proto/port proto/port label-value
Associate the given label identifier or label text with all
protocols and port numbers between these two values inclusive.
Note that while port is not meaningful for all protocols
(specifically, it is meaningful for TCP and UDP and may contain
type/code information for ICMP), this file allows port numbers to
be given for any protocol.
proto proto label-value
Associate the given label identifier or label text with all
protocols between these two values.
NOTES
The IP Address input file can contain nested CIDR blocks. They should
be ordered with the more general blocks first, and the more specific
blocks last. That is, use:
10.0.0.0/8 My-network
10.1.0.0/16 Special-Subnet-1
10.1.2.0/24 Special-Subnet-2
Likewise, the protocol/port data can be nested:
6 6 TCP
6/0 6/1024 TCP/Generic reserved
6/22 6/22 TCP/SSH
6/25 6/25 TCP/SMTP
6/80 6/80 TCP/HTTP
EXAMPLE
In the following examples, the dollar sign ("$") represents the shell
prompt. The text after the dollar sign represents the command line.
Lines have been wrapped for improved readability, and the back slash
("\") is used to indicate a wrapped line.
Reading and writing to a file:
$ echo "10.1.2.3/32 my favorite host" > fav.txt
$ rwpmapbuild-i fav.txt -o fav.pmap
Reading and writing to stdin and stdout:
$ echo "10.9.8.128/27 suspicious subnet" \
| rwpmapbuild --input-file=stdin --output-file=stdout > suspicious.pmap
Complex IP File
# Numerical mappings of labels
label 0 non-routable
label 1 internal
label 2 external
# Default to "external" for all un-defined ranges.
default external
# Force IP-mode
mode ip
# Create a name
#
# This will add --pmap-src-network and --pmap-dst-network
# switches to rwfilter, and src-network and dst-network
# fields to rwcut, rwgroup, rwsort, rwstats, and rwuniq
map-name network
## Reserved and non-routable blocks ###########################
# Addresses in this block refer to source hosts on "this"
# network. Address 0.0.0.0/32 may be used as a source
# address for this host on this network; other addresses
# within 0.0.0.0/8 may be used to refer to specified hosts
# on this network [RFC1700, page 4].
0.0.0.0/8 non-routable
# This block is set aside for use in private networks. Its
# intended use is documented in [RFC1918]. Addresses within
# this block should not appear on the public Internet.
10.0.0.0/8 non-routable
# This block is assigned for use as the Internet host
# loopback address. A datagram sent by a higher level
# protocol to an address anywhere within this block should
# loop back inside the host. This is ordinarily
# implemented using only 127.0.0.1/32 for loopback, but no
# addresses within this block should ever appear on any
# network anywhere [RFC1700, page 5].
127.0.0.0/8 non-routable
# This is the "link local" block. It is allocated for
# communication between hosts on a single link. Hosts
# obtain these addresses by auto-configuration, such as when
# a DHCP server may not be found.
169.254.0.0/16 non-routable
# This block is set aside for use in private networks. Its
# intended use is documented in [RFC1918]. Addresses within
# this block should not appear on the public Internet.
172.16.0.0/12 non-routable
# This block is assigned as "TEST-NET" for use in
# documentation and example code. It is often used in
# conjunction with domain names example.com or example.net
# in vendor and protocol documentation. Addresses within
# this block should not appear on the public Internet.
192.0.2.0/24 non-routable
# This block is set aside for use in private networks.
# Its intended use is documented in [RFC1918]. Addresses
# within this block should not appear on the public Internet.
192.168.0.0/16 non-routable
# 240.0.0.0/4 - This block, formerly known as the Class E
# address space, is reserved. The "limited broadcast"
# destination address 255.255.255.255 should never be
# forwarded outside the (sub-)net of the source. The
# remainder of this space is reserved for future use.
# [RFC1700, page 4]
255.255.255.255/32 non-routable
# -- Below this line, would add any mappings appropriate to
# -- the local network.
Complex Protocol/Port File
# Default to a hyphen ("-") for all un-defined ranges.
default -
# Force Protocol/Port-mode
#
# This MUST be present, since IP mode is the default.
mode proto-port
# Protocol Overview
1 1 ICMP
6 6 TCP
17 17 UDP
50 50 ESP
58 58 ICMPv6
# TCP -- Specific Ports
6/0 6/1024 TCP/Generic Reserved
6/21 6/21 TCP/ftp
6/22 6/22 TCP/ssh
6/25 6/25 TCP/smtp
6/53 6/53 TCP/dns
6/80 6/80 TCP/http
6/6000 6/6063 TCP/X11
# UDP -- Specific Ports
17/53 17/53 UDP/dns
# ICMP -- Specific Type/Code
#
# To convert a type/code to a "port" value as stored in SiLK:
# (type << 8) | code OR (type * 256) + code
# so 3/3 (Destination Unreachable/Port Unreachable) becomes:
1/771 1/771 ICMP/Destination Unreachable/Port Unreachable
ENVIRONMENT
SILK_CLOBBER
The SiLK tools normally refuse to overwrite existing files.
Setting SILK_CLOBBER to a non-empty value removes this restriction.
SEE ALSOpmapfilter(3), rwfilter(1), rwfileinfo(1), rwpmapcat(1),
rwpmaplookup(1), rwcut(1), rwgroup(1), rwsort(1), rwstats(1),
rwuniq(1), silk(7)SiLK 3.11.0.1 2016-02-19 rwpmapbuild(1)
