Logmuxd

Run Time Access
Event Handler

Introduction

Logmuxd is a daemon that collects logs from several sources, filters the messages, rewrites the messages, and outputs the messages to several destinations. It is released under the GPL, written in C, and uses the RTA package for configuration and management.

Logmuxd can be used in conjunction with syslogd or, for light duty work, as a replacement for syslogd.

Data path is: input, filter,
rewriting, output

The Problem
Logmuxd Overview
A Database Interface
Sources and Destinations
Advantages
Some Effort...

The Problem

Logmuxd solves two problems for a Linux appliance designer. First, it can filter, rewrite, and redirect log messages on an appliance. The ability to rewrite a message is important if you want to give your users real feedback on system status and system events. Consider the example of a Linux-based USB print server. Logmuxd could translate the cryptic log message "kernel: usb_control/bulk_msg: timeout" into something more useful to the user, something like "Please verify that the USB printer is connected and is on-line".

Second, the wide variety of messages sources and destinations makes logmuxd ideal for inter-process communication of events. Instead of creating programs of your own, you can use syslog() as the basis of all event routing. For example, a Linux-based telephone answering machine could just use syslog to log the caller ID and let the various user interfaces, such as a front panel LCD, capture and rewrite the message as needed.

Logmuxd Overview

There are four processing steps in the data path of a log message. It is input, filtered, rewritten, and output.

Input

Logmuxd accepts inputs from several sources including a syslog-like Unix socket, a UDP socket, the equivalent of a 'tail -f' on a file, a FIFO, and TCP connections which can both be initiated (TCP-Out) and accepted (TCP-In).

Input messages can be delimited by a Carriage Return character (e.g. tail -f) or by the output from each read call (e.g. reading from a UDP socket).

Filters

Filters are regular expressions that describe which log messages to process. The regular expression can contain tags that are saved for use in the rewriting step. The filter data structure includes the regex match pattern, a printf format string for rewriting, and a string which says which output gets the rewritten message. You can also filter on priority and facility if the source uses a syslog format.

The output specification includes the type of output as well as, optionally, the instance of that output. For example you can specify that some messages go to all SNMP trap destinations while other messages go to just one instance of the trap destinations.

Rewriting

If an input message matches a filter the message is rewritten before being passed to the output. Rewriting is controlled by a printf format string. The parameters to the string include the date and time, the original message, and up to ten matching patterns from the original message. The new message can contain all new text with nothing from the original message if desired.

Outputs

Logmuxd can route messages to several types of output including disk files, FIFO's, UDP sockets, orginated and accepted TCP connections, SNMP traps, e-mail, and PostgreSQL tables. (At least this is the *planned* list of outputs. Most of these are not available yet.)

The configuration of the output is very specific to the type of the output. Further, messages must be rewritten to meet the requirements of the output processor. For example, rewritten SNMP trap messages must include the version number to use for the trap (-v 1, -v 2c, or -v 3). Information like this is more easily placed in the rewritten message than tied to the specific trap destination.

A Database Interface

Logmuxd uses the Run Time Access (RTA) package for configuration and management. RTA makes arrays of structures in logmuxd look like tables in a PostgreSQL database. Thus we can use all of the PostgreSQL tools to manage logmuxd while it is running. RTA is described at http://www.linuxappliancedesign.com/projects/rta/index.html.

The essense of a database interface is that all of our status, statistics, and configuration is placed in arrays of structures which can be seen and edited as DB tables. Fields in the tables which are writable are configuration and read-only fields are status and statistics. RTA includes a generic PHP-based table editor and you can go directly to the tables in a running application by clicking here.

The RTA library includes several built-in tables. They are

Table Name	Description
rta_tables	The table of all tables in the system.
rta_columns	The list of all columns in all tables along with their attributes.
rta_dbg	Configuration of debug logging. Logging can be directed to syslog or to stderr using this table.
rta_stat	Usage and error counts for the rta package.

Logmuxd is built using a prototype daemon called empd. The empty daemon also has several built-in tables:

Table Name	Description
Fd_Desc	Table of file descriptors being serviced by the select() loop.
Config	The saved and current working configuration of the daemon
Logit	A table giving the names of sections of code, and giving the log level threshold for each section.
Timer	A linked list of the current timers in the system.
Child	A linked list of the current children spawned.
UIConns	Data about TCP connections from UI frontend programs

The tables of most interested to us are, of course, the ones related to message inputs, filters, and outputs.

Table Name	Inputs
MuxIn	The table of log input sources. These sources include named pipes (FIFOs), Unix sockets, the 'tail -f' of a specified file, TCP outgoing connections, and incoming TCP and Unix sockets.
Accpt	The table of log input connections from a TCP or Unix socket.
Rawlog	Collected raw log messages

Table Name	Filters & Rewriting
Filters	A linked list of filters to be applied to incoming log messages. A log messages that matches the regular expression is rewritten and given to the output specified.

Table Name	Outputs
FileOut	The table of file-type log destinations. These include named pipes (FIFOs), and disk files.
MailOut	Table of email addresses for log output. The destination email address can be a comma separated list of addresses but the resulting string is limited to [A-Za-z0-9_-,@], that is, alphanumeric with underscore, dash, and at-sign, All other characters are illegal. A write callback does the above sanity checking.
NetOut	Table of network destinations for log messages. These include TCP connections in, TCP connections out, and UDP destinations.
AccptOut	The table of log input connections from a TCP or Unix socket.
PgdbOut	Table of edatabase addresses for log output. The destination edatabase address can be a comma separated list of addresses but the resulting string is limited to [A-Za-z0-9_-,@], that is, alphanumeric with underscore, dash, and at-sign, All other characters are illegal. A write callback does the above sanity checking.
SnmpOut	Table of edatabase addresses for log output. The destination edatabase address can be a comma separated list of addresses but the resulting string is limited to [A-Za-z0-9_-,@], that is, alphanumeric with underscore, dash, and at-sign, All other characters are illegal. A write callback does the above sanity checking.
TblOut	Collected output log messages. These are the log messages that are visible in the web interface.