bash: Getting started, finally...
I've found bash scripting to be one of the gems I (re)discovered this year. When you're like me, you like to automate stuff. There's of course loads of scripting languages available to get it done, but frankly, using bash for lots of filesystem and/or configuration related tasks makes more sense than for example PHP or Python.
Here are some pointers to get you on track. As always feel free to ask questions.
Streams and redirection
First of all, you need to know (and understand) what streams are, and how you connect the output of one program to the input of another. For this, you'll use pipes. A pipe connects the output of a program to the input of the other.
Simple example:
#!bash
# redirect the output of echo to the input of mysql
echo "SELECT 1" | mysql
The "SELECT 1" string is input for the mysql program, so mysql will execute the query.
Then there's redirection. You have output redirection, and input redirection. You can use this as follows:
#!bash
# write the output of echo to the file myquery.sql
echo "SELECT 1" > myquery.sql
# use the file myquery.sql as input for mysql,
# redirect output to the results.txt file and
# redirect errors to the errors.txt file
mysql < myquery.sql > ./results.txt 2> ./errors.txt
Output redirection is mostly used to ignore output of scripts by writing errors and output to /dev/null (the black hole of nothingness), input redirection is not that common.
Tests
You can use if for tests. The basic structure is
#!bash
if condition
then
# do something
elif someothercondition
then
# do something else
else
# do even more stuff
fi
You can shorthand the code by inserting a semi-colon ; wherever there's a newline, except after "then", so
#!bash
if [ "1" ]; then echo "w00t"; fi
would work fine. I prefer writing the "if" and the "then" on one line. There are lots of tests and evaluations available, but the most commonly used are the following:
[ -d "dir" ]check if "dir" is a directory[ -f "file" ]check if "file" is a file( program )See if program runs succesfully (i.e. doesn't return an exit code)[ "$var" == "" ]check if $var is empty.
Further reading about test constructs
case statements
These provide similar constructs to switch statements in e.g. PHP or C.
#!bash
case $1 in
"a" ) echo "We'll do a";;
* ) echo "Ok we'll do nothing";;
esac
Note the double semicolons terminating each case. These are particularly useful for check for simple arguments.
For-loops
The most common loop you would use are for-loops. With for loops you can walk through a collection of items, separated by space or newline.
#!bash
for i in a b c
do
echo $i
done
prints:
a
b
c
Back-ticks and evaluations
You can use back-tick quotes to evaluate a piece of script to a string. This can be as simple as saving a variable.
Example
#!bash
DIR=`pwd`
cd /
echo $DIR;
Or using it in a for loop:
#!bash
for i in `ls -d *`
do
echo $i
done
prints every file in the current directory. Which is useless, of course ;)
You can also use $( ... ) to evaluate a command and use that in a for loop, which makes it easier to nest evaluations
#!bash
for i in $( ls -d `pwd`/* )
do
echo $i
done
xargs
xargs is used to use input as a parameter for the given command. Examples explain best, so:
#!bash
ls -d * | xargs echo
Does about the same as the previous script.
find
Find is probably the most powerful tool in shell scripting. It finds files for you, based on almost any condition you can think of. The most commonly used though, are by type, and by name, or a combination of both. The first argument is always the directory where you will be starting your search.
Example
#!bash
# find all php files in /var/www
find /var/www -type f -name "*php"
# find all html files in /var/www (case insensitive)
find /var/www -type d -iname "*html"
A very commonly used option is -print0, to print the results terminated with a \0 character in stead of a newline. This is useful in conjunction with xarg's -0 option, when the results contain spaces, so the filenames aren't cut in half while processing.
Read the find man pages for more information, or just type man find
The processing commands
cat Simply output a file's contents.
#!bash
cat /etc/passwd
grep: find a certain pattern in a string or a file, and print matching lines.
Mostly used options: -n (print line number), -e (use regular expressions), -o (only print matching part)
Examples
#!bash
# print the rowing containg my name from /etc/passwd
grep `whoami` /etc/passwd
# find a php file in my homedir containing my name
find ~ -type f -print0 | xargs -0 grep `whoami`
awk: An inline scripting language to process text.
Mostly used: awk '{print $2}', where '2' is a column number
Example
#!bash
# Show column names from mysql.user table
mysql -Ne "DESCRIBE mysql.user" | awk '{print $1}'
sed: Stream editor, process input and render output.
Mostly used: sed 's/replace this/with that/' in a pipe
#!bash
cat /etc/hosts.allow | sed 's/192.168.178.10/192.168.178.11/' > /etc/hosts.allow.new
cut: Cut parts of an input string for each line
Mostly used: cut -c -5, cut -c 5, cut -c 5-, cuts the last 5, from the 5th, and the first 5 of a string respectively and prints the rest.
#!bash
svn status | cut -c 9-
Prints only the files, not the status. This is especially useful if you can't trust the output to be space-separated columns, to use with awk.
A sample script file
A script file is a text file, containing a first line pointing to the interpreter (the shebang)
#!bash
#!/bin/bash
Or for freebsd users:
#!bash
#!/usr/local/bin/bash
and the following lines are just simply executes as if it were command lines.
You can use numbered arguments, with $0 pointing to the script file itself. So here's a little sample file giving you an impression of how to use that:
#!bash
#!/bin/bash
TARGET="/tmp"
# If first argument is not empty, write it to TARGET
if [ "$1" != "" ]; then
TARGET=$1
fi
# check if $TARGET is a dir
if ! [ -d $TARGET ]; then
# write error to stderr
echo "Sorry, target directory $TARGET does not exist!" >&2
exit -1
fi
echo "woohah" > "$TARGET/tmp.txt"
Now, go combine, mix, and match, and most of all, learn more on the way. Read more about bash scripting on tldp.org and get comfortable with the bash readline shortcuts, consider downloading and printing a cheatsheet. And don't forget the man pages for built-in shell commands.
Final tip: use forward slash to search inside a manpage and 'n' to scroll to the next search match.
In my experience, the easiest problems aren't easy to solve by simply googling, so if you run into anything don't hesitate to ask here.