Friday 10 July 2009

Summing all the numbers in a file

This works on a file of the form

a=1
bbbb=2
cc=4
and will output '7'

 awk '{  split($1, ar, "="); sum += ar[2]; } END { print sum; }' < ab2.txt


....and while we are here in Awk:
$0 = the whole line
$1..n = the line after a split by space
END must be in capitals

.. Awk in a shell script using strings not files:
This takes the string s1 and splits it (by space) and stores the first 2 columns.
(to split by something other than space use -F) see here
col1=`echo $s1 | awk  '{print $1}'`
col2=`echo $s1 | awk  '{print $2}'`


Awk with ls
Split by underscore, only show unique names
List files -> Keep part of the filename before '_' -> only keeps unique names
ls | awk '{ split($1, ar, "_"); print ar[1] }' | uniq

Awk for http logs. Find all 400 errors in this todays log:
head -100 2014-02-13.log | awk -v FS='\t' '{ print $6,$0 }' | grep ^40* 
Alternate way of finding 500 errors: 
cat /log/nginx_access/current | awk '$9 == "500" { print $0 }' | less
 

Not technically awk - Shell script that greps & sorts logs by each hour:

 
start=$(date --date '5 apr 2014 0:00' +%s)
stop=$(date --date '5 apr 2014 23:00' +%s)
#These are expressed in seconds. You can use a for loop on their values, increasing them by 1 hour (3600 seconds):
for t in $(seq ${start} 3600 ${stop})
do
        d=$(date --date @${t} +'%d/%b/%Y:%H')
        d_pretty=$(date --date @${t} +'%d-%b-%Y-%H')
        #echo $d
        grep $d 2014-04-05.log | cut -f 8 | sort | uniq -c| sort -rn | head -20 > out_$d_pretty.txt
done
 

Grep can count occurrences in a file:

 grep -c RuntimeError 2015-10-* 

1 comment:

  1. using awk to get pids:

    ps -ef | rg internal-wat | awk '{print $2 }'

    ReplyDelete