melp.nl

< Return to main page

bash: Getting started, finally...

I've found bash scripting to be one of the gems I (re)discovered this year. When you're like me, you like to automate stuff. There's of course loads of scripting languages available to get it done, but frankly, using bash for lots of filesystem and/or configuration related tasks makes more sense than for example PHP or Python.

Here are some pointers to get you on track. As always feel free to ask questions.

Streams and redirection

First of all, you need to know (and understand) what streams are, and how you connect the output of one program to the input of another. For this, you'll use pipes. A pipe connects the output of a program to the input of the other.

Simple example:

#!bash
# redirect the output of echo to the input of mysql
echo "SELECT 1" | mysql

The "SELECT 1" string is input for the mysql program, so mysql will execute the query.

Then there's redirection. You have output redirection, and input redirection. You can use this as follows:

#!bash
# write the output of echo to the file myquery.sql
echo "SELECT 1" > myquery.sql
# use the file myquery.sql as input for mysql, 
# redirect output to the results.txt file and
# redirect errors to the errors.txt file
mysql < myquery.sql > ./results.txt 2> ./errors.txt

Output redirection is mostly used to ignore output of scripts by writing errors and output to /dev/null (the black hole of nothingness), input redirection is not that common.

Tests

You can use if for tests. The basic structure is

#!bash
if condition
then
    # do something
elif someothercondition
then
    # do something else
else
    # do even more stuff
fi

You can shorthand the code by inserting a semi-colon ; wherever there's a newline, except after "then", so

#!bash
if [ "1" ]; then echo "w00t"; fi

would work fine. I prefer writing the "if" and the "then" on one line. There are lots of tests and evaluations available, but the most commonly used are the following:

Further reading about test constructs

case statements

These provide similar constructs to switch statements in e.g. PHP or C.

#!bash
case $1 in
    "a" ) echo "We'll do a";;
    * ) echo "Ok we'll do nothing";;
esac

Note the double semicolons terminating each case. These are particularly useful for check for simple arguments.

For-loops

The most common loop you would use are for-loops. With for loops you can walk through a collection of items, separated by space or newline.

#!bash
for i in a b c
do
    echo $i
done

prints:

a
b
c

Back-ticks and evaluations

You can use back-tick quotes to evaluate a piece of script to a string. This can be as simple as saving a variable.

Example

#!bash
DIR=`pwd`
cd /
echo $DIR;

Or using it in a for loop:

#!bash
for i in `ls -d *`
do
   echo $i
done

prints every file in the current directory. Which is useless, of course ;)

You can also use $( ... ) to evaluate a command and use that in a for loop, which makes it easier to nest evaluations

#!bash
for i in $( ls -d `pwd`/* )
do
    echo $i
done

xargs

xargs is used to use input as a parameter for the given command. Examples explain best, so:

#!bash
ls -d * | xargs echo

Does about the same as the previous script.

find

Find is probably the most powerful tool in shell scripting. It finds files for you, based on almost any condition you can think of. The most commonly used though, are by type, and by name, or a combination of both. The first argument is always the directory where you will be starting your search.

Example

#!bash
# find all php files in /var/www
find /var/www -type f -name "*php" 
# find all html files in /var/www (case insensitive)
find /var/www -type d -iname "*html"

A very commonly used option is -print0, to print the results terminated with a \0 character in stead of a newline. This is useful in conjunction with xarg's -0 option, when the results contain spaces, so the filenames aren't cut in half while processing.

Read the find man pages for more information, or just type man find

The processing commands

cat Simply output a file's contents.

#!bash
cat /etc/passwd

grep: find a certain pattern in a string or a file, and print matching lines.

Mostly used options: -n (print line number), -e (use regular expressions), -o (only print matching part)

Examples

#!bash
# print the rowing containg my name from /etc/passwd
grep `whoami` /etc/passwd 
# find a php file in my homedir containing my name
find ~ -type f -print0 | xargs -0 grep `whoami`

awk: An inline scripting language to process text.

Mostly used: awk '{print $2}', where '2' is a column number

Example

#!bash
# Show column names from mysql.user table
mysql -Ne "DESCRIBE mysql.user" | awk '{print $1}'

sed: Stream editor, process input and render output.

Mostly used: sed 's/replace this/with that/' in a pipe

#!bash
cat /etc/hosts.allow | sed 's/192.168.178.10/192.168.178.11/' > /etc/hosts.allow.new

cut: Cut parts of an input string for each line

Mostly used: cut -c -5, cut -c 5, cut -c 5-, cuts the last 5, from the 5th, and the first 5 of a string respectively and prints the rest.

#!bash
svn status | cut -c 9-

Prints only the files, not the status. This is especially useful if you can't trust the output to be space-separated columns, to use with awk.

A sample script file

A script file is a text file, containing a first line pointing to the interpreter (the shebang)

#!bash
#!/bin/bash

Or for freebsd users:

#!bash
#!/usr/local/bin/bash

and the following lines are just simply executes as if it were command lines.

You can use numbered arguments, with $0 pointing to the script file itself. So here's a little sample file giving you an impression of how to use that:

#!bash
#!/bin/bash

TARGET="/tmp"

# If first argument is not empty, write it to TARGET
if [ "$1" != "" ]; then
   TARGET=$1
fi

# check if $TARGET is a dir
if ! [ -d $TARGET ]; then
   # write error to stderr 
   echo "Sorry, target directory $TARGET does not exist!" >&2
   exit -1
fi

echo "woohah" > "$TARGET/tmp.txt"

Now, go combine, mix, and match, and most of all, learn more on the way. Read more about bash scripting on tldp.org and get comfortable with the bash readline shortcuts, consider downloading and printing a cheatsheet. And don't forget the man pages for built-in shell commands.

Final tip: use forward slash to search inside a manpage and 'n' to scroll to the next search match.

In my experience, the easiest problems aren't easy to solve by simply googling, so if you run into anything don't hesitate to ask here.

Read the entire tutorial & comments at drm.tweakblogs.net.


< Return to main page


You're looking at a very minimalistic archived version of this website. I wish to preserve to content, but I no longer wish to maintain Wordpress, nor handle the abuse that comes with that.