Sumoshell is collection of utilities to improve analyzing log files written in Go. grep can't
tell that some log lines span multiple individual lines. Parsing out fields is cumbersome.
Aggregating is basically impossible, and there is no good way to view the results. In Sumoshell,
each individual command acts as a phase in a pipeline to get the answer you want. Sumoshell
brings a lot of the functionality of Sumo Logic to the command line.
Commands should start with sumo search [filter] which will transform logs into the json format
sumoshell uses. Commands should end with render or graph which render the output to the terminal.
Each operator is a stand-alone binary allowing them to be easily composed.
Usage
Like SumoLogic, sumoshell enables you pass log data through a series of transformations to get
your final result. Pipelines start with a source (tail, cat, etc.) followed by the sumo operator.
An example pipeline might be:
tail -f logfile | sumo search "ERROR" | sumo parse "thread=*]" | sumo count thread | render
This would produce a count of log messages matching ERROR by thead. In the basic renderer, the
output would look like:
_Id _count thread
0 4 C
1 4 A
2 1 B
The sumo search operator
sumo search takes an optional filter parameter to allow for basic searching. The sumo operator
performs 3 steps:
Break a text file into logical log messages. This merges things like stack traces into a
single message for easy searching.
Allow basic searching.
Transforms the log message into the sumoshell internal json format.
The sumo json operator
For JSON logging, the sumo json operator will automatically parse JSON from your logs and
extract key value pairs.
Displaying results
After using the sumo operator, the output will be in JSON. To re-render the output in a
human-readable form, | the results of your query into one of the three render operators.
render: Capable of rendering aggregate and non-aggregate data.
Add nowraw to drop the raw data when an aggregate isn't present.
Aggregates are updated in place using terminal escape sequences, with a limit of 20 shown.
Add all to remove the limit. Aggregates will be rendered when the stream ends (ctrl+c)
graph: Curses based renderer for rendering tabular data as a bar chart.
Parsing Data
sumoshell supports a basic parse operator similar to the parse operator in SumoLogic. Queries
take the form:
... | sumo parse "[pattern=*] pattern2:* morePatterns=(*)" as pattern, pattern2, more | ...
Filtering Data
sumoshell supports a filter operator similar to the where operator in SumoLogic. Queries take
the form:
... | sumo parse "[host=*]" as host | sumo filter host = server1
This will drop any log lines that don't have server1 as the host.
Aggregating Data
sumoshell currently supports 3 aggregate operators:
count Example queries:
... | sumo count # number of rows
... | sumo count key # number of rows per key
... | sumo count key value # number of rows per the cartesian product of (key, value)
sum Example queries:
... | sumo sum k # sum of all k's
... | sumo sum v by k # sum of all v's by k
average Example queries:
... | sumo average k # average of all k's
... | sumo average v by k # average of all v's by k
|
Forgot your password?
New user?
python3-jenkins-1.8.2-1.lbn36.noarch
Builder
