rwpdu2silk(1)

NAME
rwpdu2silk - Convert NetFlow v5 records to SiLK Flow records

SYNOPSIS
  rwpdu2silk [--silk-output=PATH] [--print-statistics]
        [--log-destination={stdout | stderr | none | PATH}]
        [--log-flags={none | { {all | bad | default | missing
                                | record-timestamps} ...} } ]
        [--note-add=TEXT] [--note-file-add=FILE]
        [--compression-method=COMP_METHOD]
        {--xargs | --xargs=FILENAME | PDUFILE [PDUFILE...]}

  rwpdu2silk --help

  rwpdu2silk --version

DESCRIPTION
rwpdu2silk reads NetFlow v5 PDU (Protocol Data Units) records from one or more files, converts the records to the SiLK Flow format, and writes the SiLK records to the path specified by --silk-output or to the standard output when --silk-output is not provided. Note that rwpdu2silk cannot read from the standard input.

rwpdu2silk expects its input files to be a the format created by Cisco's NetFlow Collector: The file's size must be an integer multiple of 1464, where each 1464 octet chunk contains a 24 octet NetFlow v5 header and space for thirty 48 octet NetFlow v5 records. The number of valid records per chunk is specified in the header.

rwpdu2silk reads NetFlow v5 records from the files named on the command line when --xargs is not present. If an input file name ends in ".gz", the file is uncompressed as it is read. When the --xargs switch is provided, rwpdu2silk reads the names of the files to process from the named text file or from the standard input if no file name argument is provided to the switch. The input to --xargs must contain one file name per line.

OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--silk-output=PATH
    Write the binary SiLK Flow records to PATH, where PATH is a filename, a named pipe, the keyword "stderr" to write the output to the standard error, or the keyword "stdout" or "-" to write the output to the standard output. If PATH names an existing file, rwpdu2silk exits with an error unless the SILK_CLOBBER environment variable is set, in which case PATH is overwritten. If this switch is not given, the output is written to the standard output. Attempting to write the binary output to a terminal causes rwpdu2silk to exit with an error.

--print-statistics
    Print, to the standard error, the number of records that were written to the SiLK output file. See also --log-destination.

--log-destination={none | stdout | stderr | PATH}
    Write more detailed information to the specified destination. The default destination is none which suppresses messages. Use stdout or stderr to send messages to the standard output or standard error, respectively. Any other value is treated as a file name in which to write the messages. When an existing file is specified, rwpdu2silk appends any messages to the file. Information that is written includes the following:

    •   For each input stream, the number of PDU records read, number of SiLK records generated, number of missing records (based on the NetFlow v5 sequence number), and number of invalid records.

    •   Messages about each NetFlow v5 packet that was rejected due a bad version number or having a record count of 0 or more than 30.

    •   Additional messages enabled by the --log-flags switch.

--log-flags=FLAGS
    Write additional messages regarding the NetFlow v5 data to the --log-destination, where FLAGS is a comma-separated list of names specifying the type messages to write. When this switch is not specified, the default value for FLAGS is "none". This switch takes the same values as the log-flags setting in the sensor.conf(5) file. This manual page documents the values that are relevant for NetFlow v5 data. Since SiLK 3.10.0.

    all Log everything.

    bad Write messages about an individual NetFlow v5 record where the packet or octet count is zero, the packet count is larger than the octet count, or the duration of the flow is larger than 45 days.

    default
        Enable the default set of log-flags used by sensor.conf: bad, missing. Despite the name, this is not the default setting for this switch; none is.

    missing
        Examine the sequence numbers of NetFlow v5 packets and write messages about missing and out-of-sequence packets.

    none
        Log nothing. It is an error to combine this log-flag name with any other. This is the default setting for --log-flags.

    record-timestamps
        Log the timestamps that appear on each record. This produces a lot of output, and it is primarily used for debugging.

--note-add=TEXT
    Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.

--note-file-add=FILENAME
    Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.

--compression-method=COMP_METHOD
    Specify the compression library to use when writing output files. If this switch is not given, the value in the SILK_COMPRESSION_METHOD environment variable is used if the value names an available compression method. When no compression method is specified, output to the standard output or to named pipes is not compressed, and output to files is compressed using the default chosen when SiLK was compiled. The valid values for COMP_METHOD are determined by which external libraries were found when SiLK was compiled. To see the available compression methods and the default method, use the --help or --version switch. SiLK can support the following COMP_METHOD values when the required libraries are available.

    none
        Do not compress the output using an external library.

    zlib
        Use the zlib(3) library for compressing the output, and always compress the output regardless of the destination. Using zlib produces the smallest output files at the cost of speed.

    lzo1x
        Use the lzo1x algorithm from the LZO real time compression library for compression, and always compress the output regardless of the destination. This compression provides good compression with less memory and CPU overhead.

    snappy
        Use the snappy library for compression, and always compress the output regardless of the destination. This compression provides good compression with less memory and CPU overhead. Since SiLK 3.13.0.

    best
        Use lzo1x if available, otherwise use snappy if available, otherwise use zlib if available. Only compress the output when writing to a file.

--xargs
--xargs=FILENAME
    Read the names of the input files from FILENAME or from the standard input if FILENAME is not provided. The input is expected to have one filename per line. rwpdu2silk opens each named file in turn and reads records from it as if the filenames had been listed on the command line.

--help
    Print the available options and exit.

--version
    Print the version number and information about how SiLK was configured, then exit the application.

ENVIRONMENT

SILK_CLOBBER
    The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.

SILK_COMPRESSION_METHOD
    This environment variable is used as the value for --compression-method when that switch is not provided. Since SiLK 3.13.0.

SEE ALSO
rwfileinfo(1), rwflowpack(8), sensor.conf(5), silk(7), zlib(3)

BUGS
rwpdu2silk cannot read from the standard input.