1----------- 2logparse(8) 3----------- 4 5.. _synopsis: 6 7======== 8Synopsis 9======== 10 11`logparse` [options] 12 13.. _description: 14 15=========== 16Description 17=========== 18 19Logparse is a simple and extensible log analyser which parses and summaries medium-term server logs (up to around 30 days old). It reports notable events and statistics reported from common server programs, and has a comprehensive API to allow users to write their own parsers for custom services. 20 21Logparse is also integrated with systemd's `logrotate` to optionally rotate logs only after they have been summarised, and the user may choose to get an email (requires postfix) or a static HTML/plaintext file with the log summary. As an added feature, IP addresses from ssh/samba/apache logs may be resolved to either hostnames or FQDNs. 22 23Configuration is through the file `/etc/logparse.conf`, in INI format. A description of the configuration variables is shown below at :ref:`configuration`. 24 25Some features require logparse to be run as root (primarily log rotation). It is recommended to set it up on a cron job on a weekly basis. 26 27.. _usage: 28 29====== 30Usage 31====== 32.. argparse:: 33 :module: logparse.interface 34 :func: get_argparser 35 :prog: logparse 36 :nodescription: True 37 38.. _parsers: 39 40======= 41Parsers 42======= 43 44The program is based on a model of independent **parsers** (consisting of Python modules) which analyse logs from a particular service. Logparse comes with a range of these built in, but additional parsers can be written in Python and placed in `/usr/share/logparse/parsers`. At the moment, the built-in parsers are: 45 46- cron (DEPRECATED) - number of commands, list commands (root user only) 47- cron-journald - number of commands, list commands, list commands per user (requires libsystemd) 48- httpd - list requests, clients, user agents, bytes transferred, no. of errors 49- mem - get installed/usable/free memory 50- postfix - list recipients and bytes sent 51- smbd - number of logins, list users & clients 52- sshd (DEPRECATED) - logins by user/hostname, attempted root logins, invalid users 53- sshd-journald - logins by user/hostname, attempted root logins, invalid users (requires libsystemd) 54- sudo - number of sessions, list users and commands 55- sysinfo - hostname, OS, OS version, platform, processors 56- temperature - instantaneous temperatures of motherboard, CPU, cores, disks 57- zfs - zpool scrub reports, disk usage 58 59.. _configuration: 60 61============= 62Configuration 63============= 64 65Logparse can be configured with various options in the file `/etc/logparse/logparse.conf`. Alternatively, the configuration file may be placed elsewhere and referenced with the `-c|--config` command line option. 66 67Configuration files are written in INI format and interpreted using the standard Python :py:mod:`configparser` module. The file is divided into sections denoted by a line with the section name in square brackets, e.g. `[logparse]` for the section called logparse (this is the main section where global options are placed). Other sections are defined for each parser, e.g. `[httpd]` for the section of configuration values for the httpd parser. For more information on configuration syntax, see <https://docs.python.org/3/library/configparser.html#supported-ini-file-structure>. 68 69############################################ 70General configuration (`[logparse]` section) 71############################################ 72 73output 74 File to which the logparse output (HTML or plaintext) is saved. If empty, no output is written. Analogous to the command line option -d|--destination which overrides this config value. Default: empty 75overwrite 76 Whether to automatically overwrite an existing output file. Default: false 77title 78 Title for HTML and plaintext, shown at the top of the output. This string supports the standard variables which are substituted for various metadata in the program (see :ref:`variables`). Default: logparse 79maxlist 80 Maximum number of regular text items to show in bulleted lists, e.g. user agents for the httpd parser. Default: 10 81maxcmd 82 Maximum number of command items to show in bulleted lists, e.g. recently executed commands in the cron parser. Default: 6 83resolve-domains 84 Global setting for resolution of IP addresses into domains. If set to `ip`, logparse will never resolve IP addresses into domains. If set to `fqdn`, IPs will be resolved to show full the full hostname and domain. If set to `fqdn-implicit`, FQDNs will be shown except for local addresses which just show the hostname. If set to `host-only`, only the first segment of the FQDN is shown (i.e. the hostname). This configuration value is passed to the :py:meth:`logparse.util.resolve` function which can be implemented by parsers. This option can also be set in individual parsers, where it is called `resolve-domains-x` where x is the name of the parser. These parser-specific declerations override the global one. Default: fqdn 85rotate 86 Whether to rotate logs using systemd's logrotate utility. Analogous to the command line option -r|--rotate and -nr|--no-rotate which override this configuration value. Default: false 87verbose 88 High level of debug output. Analogous to the command line option -v|--verbose. Default: false 89quiet 90 No output to stdout except for printing the resulting output (enabled with the `-p|--print` command line option) and critical errors. Analogous to the command line option -q|--quiet. Default: false 91hostname-path 92 Path to file containing the hostname of the current machine (FQDN or not). This is used to identify the machine in the log output. Default: /etc/hostname 93parsers 94 Space-separated list of parsers to enable. If empty, all the included parsers are run except for deprecated ones. Analogous to the command line option -l|--logs. Default: empty 95ignore-parsers 96 Space-separated list of parsers to ignore (i.e. not execute). If empty, no parsers are ignored. Analogous to the command line option -nl|--ignore-logs. Default: empty 97 98############################################## 99HTML specific configuration (`[html]` section) 100############################################## 101 102header 103 Path to a header template which will be prepended to HTMl files and HTML emails. This template includes the entire <head> section and the <body> tag, and by default it outputs a title and some basic metadata at the top of the page. The template file can include variables for substitution, see :ref:`variables`. Default: /etc/logparse/header.html 104css 105 Path to a CSS stylesheet which will be linked or converted to inline tags for all HTML output (files and email). Default: /etc/logparse/main.css 106embed-styles 107 Whether to convert the stylesheet to inline tags. This is enforced for all HTML emails, and this option allows this same conversion for standalone files. This is done using the :py:mod:`premailer` package. Default: false 108css-relpath 109 If embed-styles is set to false, this option will convert the URL of the stylesheet (supplied by the css option) into a path relative to the output directory (if writing to a file) or the current directory (if outputting to stdout). Otherwise, the stylesheet is linked to exactly as supplied in the css option. Default: true 110 111##################################################### 112Plain-text specific configuration (`[plain]` section) 113##################################################### 114plain 115 Enable plaintext output instead of HTML. Analogous to the -p|--print command line option. Default: false 116 117linewidth 118 Limit regular text lines to this number of characters. Note tables do not obey this variable (this is on the todo list). Default: 80 119 120##################################### 121Log file locations (`[logs]` section) 122##################################### 123 124This section defines paths to log files used for each parser. These paths may vary depending on the operating system and service configuration. Any value can be added to this section, so user-supplied parsers can get their logfile locations from here as well. However, there is currently no mechanism of setting default values for user-supplied parsers, so defaults must be set in the parser files themselves. 125 126auth 127 Path to the system's authentication log. Default: /var/log/auth.log 128cron 129 Path to the cron log. Default: /var/log/cron.log 130cpuinfo 131 Path to the system's CPU info file in procfs. Default: /proc/cpuinfo 132meminfo 133 Path to the system's memory info file in procfs. Default: /proc/meminfo 134sys 135 Path to the system's primary syslog. Default: /var/log/syslog 136smbd 137 Path to the directory for Samba logfiles. Default: /var/log/samba 138zfs 139 Path to a file which contains the output of `zpool status`. Default: /var/log/zpool.log 140postfix 141 Path to the postfix log. Default: /var/log/mail.log 142httpd-access 143 Path to Apache's access log. Default: /var/log/apache2/access.log 144httpd-error 145 Path to Apache's error log. Default: /var/log/apache2/error.log 146 147###################################### 148Email configuration (`[mail]` section) 149###################################### 150 151Email is sent using the default mail transfer agent (usually Postfix). 152 153to 154 Recipient address. If empty, no email is sent. Analogous to the -t|--to command line option. Default: empty 155from 156 Sender address. If empty, default sender address is determined by the MTA. Default: empty 157subject 158 Text to use as the subject of the message. Variables may be substituted with metadata (see :ref:`variables`). Default: logparse from $hostname 159mailbin 160 Path to the MTA binary (usually Postfix). Default: /usr/bin/mail 161 162.. _variables: 163 164===================== 165Variable substitution 166===================== 167 168In some configuration options, variables may be used to substitute values at runtime. Variable syntax is simply a dollar sign ($) followed by the variable name. The following variables are supported: 169 170`$css` 171 Path to the CSS stylesheet as determined by the css configuration option 172`$date` 173 Date when processing was started 174`$time` 175 Time when processing was started 176`$hostname` 177 Hostname as specified by the hostname configuration option 178`$title` 179 Value of the title variable as determined by the title option 180`$version` 181 Version of logparse 182 183 184.. _api: 185 186============= 187API structure 188============= 189 190Due to the extensible nature of logparse, many of the core functions can be used in user-supplied parsers. Each parser is written in Python as a single-file module which defines a derivative of the Parser class (see `logparse.load_parsers.Parser` below). Therefore parser classes which inherit the base class have access to all the normal formatting and I/O functions which are documented below. 191 192If logparse is to be imported as a Python package into another project for some reason, you can control the general operations with the `logparse.interface` module. 193 194################## 195logparse.interface 196################## 197 198.. automodule:: logparse.interface 199 :members: 200 201##################### 202logparse.load_parsers 203##################### 204 205.. automodule:: logparse.load_parsers 206 :members: 207 208################### 209logparse.formatting 210################### 211 212.. automodule:: logparse.formatting 213 :members: 214 215================ 216More information 217================ 218| Readme https://git.lorimer.id.au/logparse.git/about 219| Source https://git.lorimer.id.au/logparse.git/tree 220| Contact mailto:bugs@lorimer.id.au