Useful UNIX Shell Commands

Useful UNIX Shell Commands
=========================================

It seems I keep bumping into situations which repeat. I list below some UNIX shell commands I've used on many occasions.


grep
----

Case insensitive search:
grep -i
Demo: grep -i bizVigyan danny.txt

List files which contain pattern:
grep -l
Demo: grep -l bizVigyan *.txt

Return lines which do not match:
grep -v
Demo: grep -v bizVigyan billy.txt


find
----

Find may be used to find a file. Also I use it to generate lists of files which
fit some kind of pattern. The lists can than be fed to other shell commands.

Self Explanatory Demos:
---------------------------------

bizVigyan04-oracle-% /bin/find . -name initbizVigyandev.ora -print

./dbs/initbizVigyandev.ora
bizVigyan04-oracle-%


bizVigyan04-oracle-% /bin/find . -name 'init*.ora' -print
./dbs/initbizVigyandev.ora
./dbs/initbizVigyantest.ora
./dbs/initbizVigyanrc.ora
./hs/admin/inithsodbc.ora
bizVigyan04-oracle-%


bizVigyan04-oracle-% /bin/find . -type l -print
./bin/runInstaller
./lib/libagtsh.so
./lib/libobk.so
./lib/libclntsh.so
./precomp/public/SQLCA.H
bizVigyan04-oracle-%


bizVigyan04-oracle-% /bin/find . -type l -exec /bin/ls -l {} \;

lrwxrwxrwx 1 oracle dba 56 Jul 18 2000 ./bin/runInstaller ->

disk/u01/app/oracle/product/oui/install/runInstaller.sh

lrwxrwxrwx 1 oracle dba 15 Jul 18 2000 ./lib/libagtsh.so -> libagtsh.so.1.0
lrwxrwxrwx 1 oracle dba 13 Jul 18 2000 ./lib/libobk.so -> libdsbtsh8.so
lrwxrwxrwx 1 oracle dba 16 Jul 18 2000 ./lib/libclntsh.so -> libclntsh.so.8.0
lrwxrwxrwx 1 oracle dba 7 Jul 18 2000 ./precomp/public/SQLCA.H -> sqlca.h
bizVigyan04-oracle-%


I think of the 'xargs' keyword as a magic wand which moves a group of tokens
from the left to the right and squish them onto one line.

Here is ASCII art which describes my mental picture:

aabbcc
xxyyzz
112233 -> xargs -> aabbcc xxyyzz 112233 uu bye
uu
bye

The 'xargs' keyword works well with the 'find' command because the 'find' command
lists file names in a single column.

This demo uses 'xargs' keyword to do the same thing as the
/bin/find . -type l -exec /bin/ls -l {} \;
demo.

bizVigyan04-oracle-% /bin/find . -type l -print | xargs /bin/ls -l
lrwxrwxrwx 1 oracle dba 56 Jul 18 2000 ./bin/runInstaller ->

disk/u01/app/oracle/product/oui/install/runInstaller.sh lrwxrwxrwx 1 oracle dba 15 Jul 18
2000 ./lib/libagtsh.so -> libagtsh.so.1.0
lrwxrwxrwx 1 oracle dba 16 Jul 18 2000 ./lib/libclntsh.so -> libclntsh.so.8.0
lrwxrwxrwx 1 oracle dba 7 Jul 18 2000 ./precomp/public/SQLCA.H -> sqlca.h
bizVigyan04-oracle-%


Notice how /bin/ls does the opposite of xargs; it takes a list of
files on a single command line and transforms the list into a column of file names.

Now we demonstrate 'find' sending a bunch of file names to grep:

/bin/find . -type f -print | xargs | grep -l bizVigyan

The above command finds all files (in a tree!) which contain the string 'bizVigyan'.

The 'find' command is also useful for finding files based on date.

The command below finds all files younger than 7 days:

/bin/find . -type f -mtime -7 -print

The command below finds all files older than 7 days:

/bin/find . -type f -mtime +7 -print

Since 'find' can find files based on date, it works well with cpio:

/bin/find . -type f -mtime -7 -print | /bin/cpio -o > /tmp/newerFiles.cpio

The above shell command could be considered a low-tech incremental backup.
To check the contents of the backup, we'd do something like this:

cat /tmp/newerFiles.cpio | /bin/cpio -it



perl
----

Perl is a great language. Often, I use it to edit collections of files.

If I want to edit just one file I'd do something like this:

perl -e 's/Smith/Bikle/gi' -p -i.bak aFileIwantChanged.txt

If I want to edit all the files in a tree of directories, I'd use find, grep, and xargs to locate the names and then
feed the list to a perl command similar to the one above:

/bin/find . -type f -print | xargs /bin/grep -l Smith | xargs perl -e 's/Smith/Bikle/gi' -p -i.bak

Note that if perl changes any files it will back up the original copies to files with a '.bak' suffix. If we are confident the perl edits are okay, we may remove the .bak files with a command like

this:

/bin/find . -type f -name '*.bak' -print | xargs /bin/grep -l Smith | xargs /bin/rm

Yes, perl is a useful tool; I may put together a directory full of useful perl scripts when I get some free time.

Miscellaneous
--------------

If I want to edit a file and put the results in another file, I'd use sed rather than perl:

cat initSMITH.ora|/bin/sed '1,$s,SMITH,bizVigyan,g'>initbizVigyan.ora

Here is a command I use to look at some hardware configuration information on a Solaris host:

/usr/platform/sun4u/sbin/prtdiag

Here is a demo command I use to kill a set of processes which fit a grepable pattern:

/bin/ps -ef |/bin/grep oraclebizVigyan|/bin/awk '{print $2}' | xargs kill

Sometimes /var/adm/messages can contain interesting information:

/bin/tail /var/adm/messages

Netstat contains connection information between your host and others.
For example it can help answer, "Who is connected on port 1521?":

/bin/netstat -a|/bin/grep 1521

Here is another awk demo which I've not actually used yet but does
demonstrate 2 good ideas. The first idea is that I can use the '-F' option to specify a field separator in the file I'm passing to awk.
For example, the /etc/passwd file contains fields separated by ':'.

The second idea is that awk has some SQL-like functionality. For example, if I want to select (perhaps display would be a better verb)
the value of the first column for all lines in /etc/passwd where the third column value is "0", I'd type this:

cat /etc/passwd | /bin/awk -F: '$3 == "0" {print $1}'

A rough SQL equivalent would be this:

SELECT column1 FROM /etc/passwd WHERE column3 = '0';

Here is another awk demo which acts like the SQL function named SUM().

cat /etc/group | /bin/awk -F: '{sum+=$3}END{print sum}'

In SQL terms it would be like this:

SELECT SUM(column3) FROM /etc/group;



A Recap
-------

/bin/find . -name 'init*.ora' -print

/bin/find . -type l -print

/bin/find . -type l -exec /bin/ls -l {} \;

/bin/find . -type l -print | xargs /bin/ls -l

/bin/find . -type f -print | xargs grep -l bizVigyan

/bin/find . -type f -mtime -7 -print

/bin/find . -type f -mtime +7 -print

/bin/find . -type f -mtime -7 -print | /bin/cpio -o > /tmp/newerFiles.cpio

cat /tmp/newerFiles.cpio | /bin/cpio -it

perl -e 's/Smith/Bikle/gi' -p -i.bak aFileIwantChanged.txt

/bin/find . -type f -print | xargs /bin/grep -l Smith | xargs perl -e 's/Smith/Bikle/gi' -p -i.bak

/bin/find . -type f -name '*.bak' -print | /bin/grep -l Smith | xargs /bin/rm

cat initSMITH.ora|/bin/sed '1,$s,SMITH,bizVigyan,g'>initbizVigyan.ora

/usr/platform/sun4u/sbin/prtdiag

/bin/ps -ef |/bin/grep oraclebizVigyan|/bin/awk '{print $2}' | xargs kill

/bin/tail /var/adm/messages

/bin/netstat -a|/bin/grep 1521

cat /etc/passwd | /bin/awk -F: '$3 == "0" {print $1}'

cat /etc/group | /bin/awk -F: '{sum+=$3}END{print sum}'