omhttpfs: Hadoop HTTPFS Output Module

sskaje <sskaje@gmail.com>

omhttpfs: Hadoop HTTPFS Output Module

Module Name:
omhttpfs

Available Since:
8.10.0

Author:
sskaje <sskaje@gmail.com>

Purpose
This module is an alternative to omhdfs via Hadoop HDFS over HTTP.

Dependencies

libcurl

Configuration Parameters

Note
Parameter names are case-insensitive.

Action Parameters

Host

type
default
mandatory
obsolete legacy directive

word
127.0.0.1
no
none

HttpFS server host.

Port

type
default
mandatory
obsolete legacy directive

integer
14000
no
none

HttpFS server port.

User

type
default
mandatory
obsolete legacy directive

word
hdfs
no
none

HttpFS auth user.

https

type
default
mandatory
obsolete legacy directive

binary
off
no
none

Turn on if your HttpFS runs on HTTPS.

File

type
default
mandatory
obsolete legacy directive

word
none
yes
none

File to write, or a template name.

isDynFile

type
default
mandatory
obsolete legacy directive

binary
off
no
none

Turn this on if your file is a template name.
See examples below.

Template

type
default
mandatory
obsolete legacy directive

word
RSYSLOG_FileFormat
no
none

Format your message when writing to file. Default: RSYSLOG_FileFormat

Configure
./configure --enable-omhttpfs

Examples

Example 1
module(load="omhttpfs")
template(name="hdfs_tmp_file" type="string" string="/tmp/%$YEAR%/test.log")
template(name="hdfs_tmp_filecontent" type="string" string="%$YEAR%-%$MONTH%-%$DAY% %MSG% ==\n")
local4.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on")
local5.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on" template="hdfs_tmp_filecontent")

See also
Help with configuring/using Rsyslog:

Mailing list - best route for general questions
GitHub: rsyslog source project - detailed questions, reporting issues
that are believed to be bugs with Rsyslog

Module Name:	omhttpfs
Available Since:	8.10.0
Author:	sskaje <sskaje@gmail.com>