Advanced Grepping

Shell & Grep: Where we count certain strings in a log file by hour.

for var in `seq 10 23`;
do
echo 'Feb 28 '$var;
grep 'Feb 28 '$var /log/cli/Hanson/2015-02-28.log | grep 'Duplicate order cancel' | wc -l;
done

How can we see: Number of Errors in several files. 

Output as: Number Filename

 grep -RIci "ERROR" . | awk -v FS=":" -v OFS="\t" '$2>0 { print $2, $1 }' | sort -hr
 

Directory wide search replace

Change foo to bar for all php files.


find . -name "*.php" -print | xargs sed -i 's/foo/bar/g'


Grep

Grep + Find
Recursive grep for files containg the word 'jsp'

find . | xargs grep -i jsp
grep -i = ignore case




Ignore grepping
Counts the number of lines containg 'fish' but not containing 'haddock' in my_food.log

grep 'fish' my_food.log | grep -v 'haddock' | wc -l




Grepping but not keeping the whole line
Counts the number of times 'fish' appears in the file

grep 'fish' my_food.log | grep -o 'fish' | wc -l
grep -o 'x' = only keep the string 'x' not the whole line 'x' is on.




Grepping and finding unique lines:

grep 'fish' my_food.log |sort | uniq



Grepping with RegEx:
Returns the last word in each line.
\w doesn't work


grep -oG '[a-z0-9A-Z\-\_\-]'*$ products

Note: add -P for more regex terms allowing you to use \d & \s [Search for 50X errors in http logs]

grep -P '\s50\d\s' 2014-05-20.log



Grep with many values from a file
I want to grep for all these values in a file in 1 go:


Use file with patterns:
cat baddata.txt
1
3
5

cat test.txt
1
2
3
4
5
6
7

grep -v -f baddata.txt test.txt
2
4
6
7



Grep and get the line above / below the target line
grep -AN = grep and get N lines After the target line. ( -B = Before)


cat data.txt
haddock
salmon
tuna
prawn
mackeral
crabs

grep -A1 -B1 mackeral data.txt
prawn
mackeral
crabs

Grep for all IP addresses in directory and report number of times they occur:

egrep -oh '(\d{1,3}\.){3}\d{1,3}' * | uniq -c

1 192.168.1.1

1 0.0.0.0

2 127.0.0.1

Grep for all urls in nginx. Count & Sort them .
cat  2014-04-05.log | cut -f 5 | sort | uniq -c | sort -rn > sorted_urls


Nice grep example:
   cat -n: print line number.
   use of -B and -A to look at lines 'near' the problem line

cat -n 2014-02-01.log | grep -i rball |grep -B10 -A50 1680071