This post will be related to the use of grep, sed, and awk to process text streams and files.
GREP
Is a command searches a file for lines containing a match to the given strings and prints the matching lines.
It is mainly used in the following combinations:
grep 'some string' given_file cat given_file | grep 'some string' command | grep 'some string'
Example:
We want to see the line containing our ip address, so we run ifconfig and look for inetsting:
root@server:~# ifconfig eth0 | grep inet inet addr:10.1.0.1 Bcast:10.1.255.255 Mask:255.255.0.0 root@server
Example:
We would like to see if there is a user named cristian on our system. We list the content of the file and then search for cristian in /etc/passwd:
root@server:~# cat /etc/passwd | grep cristian cristian:x:1000:1000:Cristian Huza,,,:/home/cristian:/bin/bash root@server:~#
Example:
Let’s say that we have a file named students.txt and we want to see if we have a student named Anne.
This is the content of the file:
- John
- Danny
- Mary
- Bill
- Anne
Now, we will grep for “anne”:
root@server:~# grep anne students.txt root@server:~#
We have no result, but why? well, we searched for anne, but all the names in the file begin with a capital letter. We will have to make the search case insensitive using -i parameter:
root@server:~# grep -i anne students.txt Anne root@server:~#
One other interesting option is -v. It inverses the match, so if you search for John in a certain text file, it will provide you all the lines that DO NOT contain John:
root@server:~# grep -v John students.txt Danny Mary Bill Anne root@server:~#
SED
Is defined as a stream editor. Actually it is mostly used to replace, substitute, delete a certain string.
Example:
root@server:~# echo “John has three apples” | sed s/John/Anne/
Anne has three apples
root@server:~#
Now, if we have a certain string that appears multiple times in a stream, we will have to use the global replacement parameter g, otherwise only the first occurrence will be replaced:
root@server:~# echo "John has three apples. John has 5 candies" | sed s/John/Anne/ Anne has three apples. John has 5 candies root@server:~# echo "John has three apples. John has 5 candies" | sed s/John/Anne/g Anne has three apples. Anne has 5 candies root@server:~#
You can also use -i parameter to make the substitution inline for example we can replace John with Mike in a certain text file:
root@server:~# cat students.txt John Danny Mary Bill Anne root@server:~# sed -i 's/John/Mike/' students.txt root@server:~# cat students.txt Mike Danny Mary Bill Anne root@server:~#
For more options please check the man page.
AWK
Is defined as a pattern scanning and text processing language and it is used in combination with grep and cut or in standalone scripts.
We would like to print only the Month and the year of the current date for example:
root@server:~# date
Mon Aug 1 23:07:09 EEST 2011
root@server:~# date | awk '{print $2 " " $6}'
Aug 2011
root@server:~#
Or, to print the line number and then the actual line:
root@server:~# awk '{printf("%5d : %s\n", NR,$0)}' students.txt
1 : Mike
2 : Danny
3 : Mary
4 : Bill
5 : Anne
root@server:~#
For more information check out the man page for awk/gawk.