logging24
log everything · query anything
Every query returns filtered and aggregated results from a scan over your stored logs. Filtering proceeds by customer selection (in case you have access to multiple organizations), log stream selection, time range and finally a full-text regex search.
Each log stream has one text label along four separate dimensions. Typical setups will use the first label for hostname and the second for log file or service name. When querying, you specify a (potentially empty) prefix for each of those four dimensions. Only log streams whose labels start with the prefix you gave are considered during the search. Consequently, if you don't specify any prefixes, all streams match the filter and are included in the search.
Most log analysis will be restricted to a specific time range. We use the specified time-range to pre-filter stored log segments before running the regex search. Queries on shorter time ranges will therefore be faster than those on longer intervals. All times shown in the UI are UTC.
Only log lines which match the given regex in their entirety are returned from a search. This implies that you'll nearly always want to put .* at the start and end of your queries.
Our regex engine supports the following constructs:
| a | A single "a" |
| . | Any single character |
| [abc] | "a", "b" or "c" |
| [^abc] | Any single character, but not "a", "b", or "c" |
| [a-z] | "a", "b", ... "y" or "z" (ranges based on ASCII encoding) |
| \. | Single character escape (full list below) |
| a? | Nothing or one "a" |
| a* | Nothing or some "a"s, e.g. "", "a", "aaaaaaa" |
| a+ | At least one "a", e.g. "a", "aaaaaaa" |
| a{N} | Exactly N times "a" |
| a{N,} | At least N times "a" |
| a{N,M} | At least N, and at most M times "a" |
| (ab) | "ab" but with parsing priority, e.g. "(ab){3}" matches "ababab" |
| (.*foo.*)|(.*bar.*) | Any string matching .*foo.* or .*bar.*, i.e. with "foo" or "bar" somewhere |
| (.*foo.*)&(.*bar.*) | Any string matching .*foo.* and .*bar.*, i.e. with "foo" somewhere and "bar" somewhere (else) |
| (.*foo.*)~ | Any string not matching .*foo.*, i.e. with no "foo" anywhere |
| (?int) | Capture integer for numeric analysis |
| (?num(3)) | Capture floating point (with three digits precision) for numeric analysis |
| (?txt(subexpression)) | Capture up to 31 bytes for textual analysis |
| (?<x>subexpression) | Capture numerics from subexpression as x-coordinate |
| (?<y>subexpression) | Capture numerics from subexpression as y-coordinate |
| (?<k>subexpression) | Capture subexpression as (grouping) key |
The following single character escapes exist:
| \a | Alert or bell, ASCII 0x07 |
| \d | [0-9] |
| \D | [^0-9] |
| \h | Horizontal space, i.e. \t or space |
| \H | [^\h] |
| \e | Escape, ASCII 0x1b |
| \f | Form feed, ASCII 0x0c |
| \n | New line, ASCII 0x0a (note that matching is always per single log event) |
| \N | [^\n] |
| \r | Carriage return, ASCII 0x0d |
| \s | Space, i.e. [\h\v] |
| \S | [^\s] |
| \t | Horizontal tab, ASCII 0x09 |
| \v | Vertical space, i.e. new line, carriage return, vertical tab, or form feed |
| \V | [^\v] |
| \w | Word character, i.e. [0-9a-zA-Z_] |
| \W | [^\w] |
| \xAB | Single byte, specified in hexadecimal |
| \\ | Any non-numeric, non-word character following \ is taken verbatim, e.g. \., \\, \(, \[ |
If no numerical information is captured during the regex search, the UI will show the number of matching lines over time with a list of the latest matching events below it.
If any (?int) or (?num(N)) is used, the UI switches to histogram mode shows a scatter plot of the matching log lines. As more matches are found for the same resulting coordinate the corresponding pixel will get darker.
Due to the way single-pass regex matching works, capture of numeric information cannot backtrack but is eagerly initiated as soon as the query expression encounters the capturing clause. In particular it is not possible to use .*(?<x>(?int)) is the number for X.* on a string like "5 is the number for X, 7 is the number for Y" as the .* can match "5 is the number for X, " and then (?int) matches the "7". Instead, try to identify a prefix uniquely leading up to the numeric information you need, e.g. .*the number for X is (?<x>(?int)).* is fine.
In many contexts, it is useful to split search results along some textual key. We support this via (?<k>(?txt(subexpression))) which extracts the key to split on.