Overview

 

This topic includes some tips and tricks to help with analyzing logs at a linux shell

 

---

Negate

A lot of times we aren't quite sure what to look for but can easily choose log entries that we know we can ignore.  There are several ways to do this but the most straight forward while at a shell in linux is to use grep -v

 

Examples

Look at everything but messages from bpgc:

 

cat tsunami.log | grep -v bpgc | less

 

Now you want to also filter out ostd messages:

 

cat tsunami.log | grep -v bpgc | grep -v ostd | less

 

---

Get Columns

Alot of times it is helpful to only display specific columns.  This is especially helpful if you need to do sorting later.

 

One way to get columns is with awk.  By default awk separates a line of text into columns using a space as a delimiter.  You can specify your own delimiter using -F and then continue to pipe out to additional additional delimiters.

 

The basic syntax to print a specific column (where $1 is the first column) : awk '{print $1}'

 

Examples

We'll use the following log entry from tsunami and grab it by grepping the timestamp:

 

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log

ERROR  - 12/28/13-20:41:25 - Triggerd ReplicationAPI.cpp(7097) [replicationd] replicateObject() - [TID 3839378] Error replicating /snfs/ddup/shares/clt-bkup-srv2/D2D/IMG000023/clt-arc08-srv2.vmdk.lck/D51993.lck: No such file or directory(2)

 

Get just the timestamp and the logger:

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $3, $5}' 

12/28/13-20:41:25 Triggerd

 

Get the severity, timestamp and file name:

[/stornext/sesrepo/tmp/nascar/SR1653020/scratch/collect/node1-collection/app-info]-[1107]

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $1, $3, $14}'

ERROR 12/28/13-20:41:25 /snfs/ddup/shares/clt-bkup-srv2/D2D/IMG000023/clt-arc08-srv2.vmdk.lck/D51993.lck:

 

Change the delimiter from a space to a - and get the section that includes the logger, code file with line number, process and function:

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk -F - '{print $4}'

 Triggerd ReplicationAPI.cpp(7097) [replicationd] replicateObject() 

 

Grab just the file name without that path:

[/stornext/sesrepo/tmp/nascar/SR1653020/scratch/collect/node1-collection/app-info]-[1115]

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $14}'|awk -F/ '{print $9}'

D51993.lck:

 

Notce that the : is included.  Sed is the best option to remove it but for now we'll stick with Awk and add another filter:

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $14}'|awk -F/ '{print $9}'| awk -F: '{print $1}'

D51993.lck

 

 

---

Replace/Remove

When generating a list or just trying to make the output of your log parsing look better you can use Sed.  The syntax is as follows:

 

sed s/patternToFind/PatternToReplace/

 

Examples

Using the previous example with the file name that was followed by a colon, we'll replace the colon with nothing using /://

 

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $14}'|awk -F/ '{print $9}'|sed s/://

D51993.lck

 

Say we want a list of file names without displaying the .lck.  We can replace .lck: with nothing like so:

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $14}'|awk -F/ '{print $9}'|sed s/.lck://

D51993

 

Say for some reason we need to rename the files form .lck files to .tmp files:

--> grep 'ERROR  - 12/28/13-20:41:25' tsunami.log|awk '{print $14}'|awk -F/ '{print $9}'|sed s/.lck:/.tmp/

D51993.tmp

 

---

Search Everything

Sometimes when you find something that you want to research it's good to search the entire bundle for it.  Grep is good for this

Examples

 

Find all instances of error, case insensitive.  This takes much longer:

 

grep -ri error ./ | less

 

Find all errors in just the tsunami files:

 

grep ERR tsunami.log*

 

Find all errors in the tsunami files but don't list the file name where the entry was found in the output:

 

grep -h ERR tsunami.log*