Bash¶

Streams, Redirection, and Control Structures¶

Streams¶

  • STDIN
  • STDOUT
  • STDERR

Output Redirection¶

  • The greater than symbol (>), is used to redirect output
    • With no additional symbols, this redirects STDOUT to the specified location
    • 1> also redirects STDOUT to the specified location, but this form is not normally used
    • 2> redirects STDERR to the specified location
    • &> redirects both STDOUT and STDERR to the same specified location
    • >> appends STDOUT to the specified file
In [15]:
echo "Hello" > hello.txt
In [16]:
more hello.txt
Hello
In [17]:
echo "World" >> hello.txt
In [18]:
more hello.txt
Hello
World
In [19]:
gcc no_file.c
gcc: error: no_file.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.

In [20]:
gcc no_file.c 2> gcc_errors.txt

In [21]:
more gcc_errors.txt
gcc: error: no_file.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
In [22]:
more out_and_err.py
#!/usr/bin/python
from __future__ import print_function
import sys

def eprint(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

print("I am on STDOUT")
eprint("I am on STDERR")
In [23]:
./out_and_err.py > out 2> err
In [24]:
more out
I am on STDOUT
In [25]:
more err
I am on STDERR

Input Redirection¶

  • The less than symbol (<) is used to redirect input to STDIN
    • Not many variations of this, but....
    • Two less than operators (<<) are used to create a here document, which will have its own slide
In [26]:
more simple.py
#!/usr/bin/python
number = int(raw_input("Please enter a number: "))
print "The square of %d is %d" % (number, number * number)
In [27]:
./simple.py < numbers.txt
Please enter a number: The square of 40 is 1600

Here Documents¶

  • A here document takes any string and allows it to be passed to a command as if it were coming from STDIN
    • For commands that take multiple arguments, you may see the dash (-) being used to explicitly indicate which argument should use STDIN
    • The << must be followed by a delimiter that is used to mark the end of the HERE document
    • Using <<- will remove leading tabs, which can be useful for formatting nice looking scripts

Here Strings¶

  • If all you want to redirect is a single line, you can use three less than symbols (<<<) with no delimiter to indicate a here string
    • Any variables in a here string (or here document) are expanded before being redirected
In [28]:
more numbers.txt
40
1
2
3

In [29]:
diff - numbers.txt <<EOF
    40
    1
    2
    3
EOF
1,4c1,5
<     40
<     1
<     2
<     3
---
> 40
> 1
> 2
> 3
> 

In [30]:
diff - numbers.txt <<< "Hello"
1c1,5
< Hello
---
> 40
> 1
> 2
> 3
> 

Pipes¶

  • Many times the output of one command will function as the input to a second command
  • Rather than redirect output to a tempoarary file and then use that file as input, use the pipe command (|)
    • The STDERR stream can be redirection along with the STDOUT stream using |&
In [31]:
ls -lh | wc -l
54
In [32]:
find ~/ -size +100M | head
/home/bryan/atom/.git/objects/pack/pack-9b1de18828423dccad81d95b642b9f7059665561.pack
/home/bryan/gloveTest4.bin
/home/bryan/wacky1
/home/bryan/bryan/wiki.log
/home/bryan/bryan/google1Deps10000.csv
/home/bryan/bryan/word_projections-1600.txt
/home/bryan/bryan/wikiSpaceDeps.pkl
/home/bryan/bryan/wikiSpace.pkl
/home/bryan/bryan/googleDeps1Sided+Nouns.pkl
/home/bryan/bryan/wikiSpaceDepsAdj.pkl
find: `standard output': Broken pipe
find: write error

Tee¶

  • The tee command takes in a stream as input, and outputs that stream both to STDOUT and to the specified file
    • Used following a pipe operator
In [33]:
pip2 install -U scipy |& tee scipy.log
Requirement already up-to-date: scipy in /usr/local/lib/python2.7/dist-packages
Collecting numpy>=1.8.2 (from scipy)
  Using cached numpy-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl
Installing collected packages: numpy
  Found existing installation: numpy 1.13.0
    Uninstalling numpy-1.13.0:
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/usr/lib/python2.7/shutil.py", line 303, in move
    os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/local/bin/f2py'
In [ ]:
more scipy.log
Requirement already up-to-date: scipy in /usr/local/lib/python2.7/dist-packages
Collecting numpy>=1.8.2 (from scipy)
  Using cached numpy-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl
Installing collected packages: numpy
  Found existing installation: numpy 1.13.0
    Uninstalling numpy-1.13.0:
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in
 main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 34
2, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 778, in
 install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 754
, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 1
15, in remove
    renames(path, new_path)

Redirecting From Multiple Commands¶

  • Sometimes you may need to combine the output of multiple commands and pass this on to a third or fourth command
  • You could use temporary files, but process substitution fills this need nicely
  • The syntax is <(command)
    • This relies on certain operating system features, so isn't truly portable, but can be assumed to be
In [1]:
diff <(ls -lh .) <(ls -lh ~/CMSC331)
1,54c1,99
< total 5.3M
< drwxrwxr-x 3 bryan bryan 4.0K Sep 12 12:11 433Fall17
< -rw-rw-r-- 1 bryan bryan 2.0M Sep  7 10:17 airline_tweets.tsv
< -rw-rw-r-- 1 bryan bryan  125 Sep 12 09:05 anchored.pl
< -rw-rw-r-- 1 bryan bryan    0 Sep 19 10:14 an_empty_file
< -rwxrwxr-x 1 bryan bryan  24K Sep 19 08:46 a.out
< -rw-rw-r-- 1 bryan bryan  132 Sep 12 09:13 bad_alt.pl
< -rw-rw-r-- 1 bryan bryan  127 Sep 12 09:14 capture.pl
< -rwxrwxr-x 1 bryan bryan  176 Sep 14 09:48 cla_examples.sh
< -rw-rw-r-- 1 bryan bryan   15 Sep 19 11:58 err
< -rw-rw-r-- 1 bryan bryan  137 Sep 12 08:46 fast.pl
< -rw-r--r-- 1 bryan bryan  37K Sep  4 21:07 fb_messenger.png
< -rw-r--r-- 1 bryan bryan  24K Sep  4 20:15 fb_verify.png
< -rw-rw-r-- 1 bryan bryan  106 Sep 19 11:58 gcc_errors.txt
< -rw-rw-r-- 1 bryan bryan  132 Sep 12 09:12 good_alt.pl
< -rw-rw-r-- 1 bryan bryan  139 Sep 12 09:09 greedy.pl
< -rwxrwxr-x 1 bryan bryan   31 Sep 14 10:38 hello.sh
< -rw-rw-r-- 1 bryan bryan   19 Sep 14 10:37 hello_simple.sh
< -rw-rw-r-- 1 bryan bryan   12 Sep 19 11:57 hello.txt
< -rw-rw-r-- 1 bryan bryan 258K Aug 31 10:59 Lecture00.html
< -rw-rw-r-- 1 bryan bryan 990K Aug 31 15:15 Lecture00.ipynb
< -rw-rw-r-- 1 bryan bryan 366K Sep  5 15:43 Lecture01.html
< -rw-rw-r-- 1 bryan bryan  85K Sep  5 15:31 Lecture01.ipynb
< -rw-rw-r-- 1 bryan bryan 314K Sep 14 10:34 Lecture02.html
< -rw-rw-r-- 1 bryan bryan  61K Sep  7 12:22 Lecture02.ipynb
< -rw-rw-r-- 1 bryan bryan 287K Sep 14 10:34 Lecture03.html
< -rw-rw-r-- 1 bryan bryan  24K Sep 12 12:13 Lecture03.ipynb
< -rw-rw-r-- 1 bryan bryan 302K Sep 19 10:47 Lecture04.html
< -rw-rw-r-- 1 bryan bryan  38K Sep 14 14:27 Lecture04.ipynb
< -rw-rw-r-- 1 bryan bryan 287K Sep 19 10:47 Lecture05.html
< -rw-rw-r-- 1 bryan bryan  32K Sep 19 12:08 Lecture05.ipynb
< -rw-rw-r-- 1 bryan bryan  683 Sep 15 17:37 Lecture06.ipynb
< -rw-rw-r-- 1 bryan bryan  567 Sep 15 17:38 Lecture07.ipynb
< -rw-rw-r-- 1 bryan bryan  565 Sep 15 17:38 Lecture08.ipynb
< -rw-rw-r-- 1 bryan bryan  129 Sep 12 09:15 noncapture.pl
< -rw-rw-r-- 1 bryan bryan  140 Sep 12 09:09 nongreedy.pl
< -rw-rw-r-- 1 bryan bryan   10 Sep 19 09:02 numbers.txt
< -rw-rw-r-- 1 bryan bryan   15 Sep 19 11:58 out
< -rwxrwxr-x 1 bryan bryan  191 Sep 19 08:51 out_and_err.py
< -rw-rw-r-- 1 bryan bryan  176 Sep 19 09:23 part1.tsv
< -rw-rw-r-- 1 bryan bryan  106 Sep 19 09:25 part2.csv
< -rw-rw-r-- 1 bryan bryan   68 Sep  4 22:57 re_example.pl
< drwxrwxr-x 3 bryan bryan 4.0K Sep 12 10:40 regex_starter_code
< -rw-r--r-- 1 bryan bryan 9.2K Sep  5 10:04 registers.png
< -rw-rw-r-- 1 bryan bryan  17K Sep  4 23:15 rolling_stone_500_greatest_2010.txt
< -rw-rw-r-- 1 bryan bryan 1.3K Sep 19 12:06 scipy.log
< -rwxrwxr-x 1 bryan bryan  128 Sep 19 09:02 simple.py
< -rw-rw-r-- 1 bryan bryan  127 Sep 12 08:46 slow.pl
< -rw-rw-r-- 1 bryan bryan   19 Sep 14 10:09 to_sort1.txt
< -rw-rw-r-- 1 bryan bryan   23 Sep 14 10:09 to_sort2.txt
< -rw-rw-r-- 1 bryan bryan   15 Sep 14 10:10 to_sort3.txt
< -rw-rw-r-- 1 bryan bryan   90 Sep 14 10:15 to_sort4.txt
< -rw-rw-r-- 1 bryan bryan  124 Sep 12 09:05 unanchored.pl
< -rw-rw-r-- 1 bryan bryan  313 Sep  4 16:14 Untitled.ipynb
---
> total 27M
> -rw-r--r-- 1 bryan bryan  30K Sep 14 13:30 aa.png
> -rw-r----- 1 bryan bryan 164K Aug 29 18:57 ambparse.jpg
> -rw-r--r-- 1 bryan bryan  30K Sep 14 14:01 bwilk1@linuxserver1.cs.umbc.edu
> -rw-r----- 1 bryan bryan 4.7K Aug 29 18:57 capDFA.svg
> -rw-rw-r-- 1 bryan bryan 112K Aug 29 18:57 compilation.jpg
> -rw-rw-r-- 1 bryan bryan  551 Aug 29 18:56 complex.lua
> -rw-r----- 1 bryan bryan  24K Aug 29 18:56 DFA1.svg
> -rw-r----- 1 bryan bryan  12K Aug 29 18:57 DFA2.svg
> -rw-r----- 1 bryan bryan  13K Aug 29 18:57 ending.svg
> -rw-rw-r-- 1 bryan bryan  537 Sep 14 13:02 example.l
> -rwxrwxr-x 1 bryan bryan  24K Sep 18 15:34 example_lex
> -rw-rw-r-- 1 bryan bryan  46K Sep 18 15:33 example_lex.c
> -rw-rw-r-- 1 bryan bryan  19K Aug 29 18:57 FinalReivew.ipynb
> -rw-rw-r-- 1 bryan bryan 212K Aug 29 18:56 FinalReview.html
> -rw-rw-r-- 1 bryan bryan  142 Aug 29 18:57 final.scm
> -rw-r--r-- 1 bryan bryan 3.5K Aug 29 18:56 hw2.html
> -rw-rw-r-- 1 bryan bryan 7.1K Aug 29 18:56 hw3.html
> -rw-rw-r-- 1 bryan bryan  18M Aug 29 18:57 HW3.ipynb
> -rw-rw-r-- 1 bryan bryan 1.2K Aug 29 18:57 hw3.lua
> -rw-rw-r-- 1 bryan bryan 3.9K Aug 29 18:56 hw3Script.lua
> -rw-rw-r-- 1 bryan bryan  87K Aug 29 18:56 hybrid.jpg
> -rw-rw-r-- 1 bryan bryan  96K Aug 29 18:57 interpreter.jpg
> -rw-r----- 1 bryan bryan  27K Aug 29 18:56 ipDFA.svg
> drwxrwxr-x 2 bryan bryan 4.0K Aug 29 18:57 java
> drwxrwxr-x 2 bryan bryan 4.0K Aug 29 18:57 js
> -rw-rw-r-- 1 bryan bryan 278K Aug 30 10:27 Lecture00.html
> -rw-rw-r-- 1 bryan bryan  14K Aug 31 15:13 Lecture00.ipynb
> -rw-rw-r-- 1 bryan bryan  53K Aug 30 13:58 Lecture00.pdf
> -rw-rw-r-- 1 bryan bryan 220K Aug 29 18:57 Lecture00.slides.html
> -rw-rw-r-- 1 bryan bryan 255K Sep  5 15:55 Lecture01.html
> -rw-rw-r-- 1 bryan bryan 9.3K Sep  5 15:56 Lecture01.ipynb
> -rw-rw-r-- 1 bryan bryan 257K Sep  7 14:02 Lecture02.html
> -rw-rw-r-- 1 bryan bryan 197K Aug 29 18:57 Lecture02.html.bak
> -rw-rw-r-- 1 bryan bryan  12K Sep 11 15:18 Lecture02.ipynb
> -rw-rw-r-- 1 bryan bryan 265K Sep 18 11:23 Lecture03.html
> -rw-rw-r-- 1 bryan bryan  16K Sep 18 11:23 Lecture03.ipynb
> -rw-rw-r-- 1 bryan bryan 262K Sep 14 14:01 Lecture04.html
> -rw-rw-r-- 1 bryan bryan  15K Sep 18 15:34 Lecture04.ipynb
> -rw-rw-r-- 1 bryan bryan 197K Aug 29 18:57 Lecture05.html
> -rw-rw-r-- 1 bryan bryan 7.2K Sep 19 11:51 Lecture05.ipynb
> -rw-rw-r-- 1 bryan bryan  14K Sep 19 11:37 Lecture05.ipynb.bak
> -rw-rw-r-- 1 bryan bryan 210K Aug 29 18:56 Lecture06.html
> -rw-rw-r-- 1 bryan bryan  17K Aug 29 18:57 Lecture06.ipynb
> -rw-rw-r-- 1 bryan bryan 374K Aug 29 18:57 Lecture07.html
> -rw-rw-r-- 1 bryan bryan 316K Aug 29 18:57 Lecture07.ipynb
> -rw-rw-r-- 1 bryan bryan 227K Aug 29 18:56 Lecture08.html
> -rw-rw-r-- 1 bryan bryan  21K Aug 29 18:57 Lecture08.ipynb
> -rw-rw-r-- 1 bryan bryan 204K Aug 29 18:56 Lecture09.html
> -rw-rw-r-- 1 bryan bryan  14K Aug 29 18:57 Lecture09.ipynb
> -rw-rw-r-- 1 bryan bryan 228K Aug 29 18:57 Lecture10.html
> -rw-rw-r-- 1 bryan bryan  18K Aug 29 18:56 Lecture10.ipynb
> -rw-rw-r-- 1 bryan bryan 201K Aug 29 18:56 Lecture11.html
> -rw-rw-r-- 1 bryan bryan  12K Aug 29 18:57 Lecture11.ipynb
> -rw-rw-r-- 1 bryan bryan 208K Aug 29 18:56 Lecture12.html
> -rw-rw-r-- 1 bryan bryan  15K Aug 29 18:57 Lecture12.ipynb
> -rw-rw-r-- 1 bryan bryan 232K Aug 29 18:57 Lecture13.html
> -rw-rw-r-- 1 bryan bryan  26K Aug 29 18:57 Lecture13.ipynb
> -rw-rw-r-- 1 bryan bryan 220K Aug 29 18:57 Lecture14.html
> -rw-rw-r-- 1 bryan bryan  15K Aug 29 18:56 Lecture14.ipynb
> -rw-rw-r-- 1 bryan bryan 217K Aug 29 18:57 Lecture15.html
> -rw-rw-r-- 1 bryan bryan  22K Aug 29 18:57 Lecture15.ipynb
> -rw-rw-r-- 1 bryan bryan 221K Aug 29 18:57 Lecture16.html
> -rw-rw-r-- 1 bryan bryan  18K Aug 29 18:56 Lecture16.ipynb
> -rw-rw-r-- 1 bryan bryan 231K Aug 29 18:57 Lecture17.html
> -rw-rw-r-- 1 bryan bryan  26K Aug 29 18:57 Lecture17.ipynb
> -rw-rw-r-- 1 bryan bryan 236K Aug 29 18:56 Lecture18.html
> -rw-rw-r-- 1 bryan bryan  33K Aug 29 18:57 Lecture18.ipynb
> -rw-rw-r-- 1 bryan bryan 209K Aug 29 18:56 Lecture19.html
> -rw-rw-r-- 1 bryan bryan  33K Aug 29 18:57 Lecture19.ipynb
> -rw-rw-r-- 1 bryan bryan 218K Aug 29 18:57 Lecture20.html
> -rw-rw-r-- 1 bryan bryan  29K Aug 29 18:56 Lecture20.ipynb
> -rw-rw-r-- 1 bryan bryan 198K Aug 29 18:57 Lecture21.html
> -rw-rw-r-- 1 bryan bryan 9.4K Aug 29 18:56 Lecture21.ipynb
> -rw-rw-r-- 1 bryan bryan 198K Aug 29 18:57 Lecture22.html
> -rw-rw-r-- 1 bryan bryan 8.4K Aug 29 18:57 Lecture22.ipynb
> -rw-rw-r-- 1 bryan bryan 197K Aug 29 18:57 Lecture25.html
> -rw-rw-r-- 1 bryan bryan 9.7K Aug 29 18:57 Lecture25.ipynb
> -rw-r----- 1 bryan bryan  47K Aug 29 18:57 lexprocess.svg
> -rw-rw-r-- 1 bryan bryan  46K Sep 14 10:52 lex.yy.c
> -rw-rw-r-- 1 bryan bryan 1.8K Aug 29 18:57 lower.txt
> -rw-r----- 1 bryan bryan  15K Aug 29 18:57 midtermParse1.svg
> -rw-r----- 1 bryan bryan  25K Aug 29 18:56 midtermParse2.svg
> -rw-rw-r-- 1 bryan bryan 212K Aug 29 18:57 Midterm Review.html
> -rw-rw-r-- 1 bryan bryan 214K Aug 29 18:57 MidtermReview.html
> -rw-rw-r-- 1 bryan bryan  19K Aug 29 18:57 MidtermReview.ipynb
> -rw-r--r-- 1 bryan bryan 152K Aug 29 18:56 moneyDFA.jpg
> -rw-r--r-- 1 bryan bryan  36K Aug 29 18:56 NFA1.jpg
> -rw-r--r-- 1 bryan bryan  36K Aug 29 18:56 NFA2.jpg
> -rw-rw-r-- 1 bryan bryan   39 Aug 29 18:56 output.txt
> -rw-r----- 1 bryan bryan 221K Aug 29 18:57 parsetable.jpg
> -rw-r----- 1 bryan bryan  61K Aug 29 18:57 parsetree1.jpg
> -rw-rw-r-- 1 bryan bryan   45 Sep 14 11:06 pascal_example
> -rw-r----- 1 bryan bryan 3.3K Aug 29 18:57 turing.tsv
> -rw-rw-r-- 1 bryan bryan 3.3K Aug 29 18:57 turing.txt
> -rw-rw-r-- 1 bryan bryan 1.1K Aug 29 18:56 Untitled1.ipynb
> -rw-rw-r-- 1 bryan bryan 3.7K Aug 29 18:57 Untitled2.ipynb
> -rw-rw-r-- 1 bryan bryan 3.1K Aug 29 18:57 Untitled.ipynb
> -rw-rw-r-- 1 bryan bryan  917 Aug 29 18:56 words.txt

In [2]:
head -n1 part1.tsv
Atlanta United	Mercedes-Benz Stadium
In [3]:
head -n1 part2.csv
71000,2017,FieldTurf
In [4]:
paste <(cut -f2 part1.tsv) <(cut -f2 part2.csv -d,)
Mercedes-Benz Stadium	2017
Toyota Park	2006
RFK Stadium :(	1961
StubHub Center	2003
Stade Saputo	2008
Yankee Stadium	2009
	
	
	

/dev/null¶

  • Unix has a special device that allows streams to be redirected to it but doesn't save any of the redirected text
  • By redirecting to /dev/null you are throwing away that stream
    • Can be very useful to ignore errors, but many commands have a quiet option built in
In [5]:
gcc no_file 2>/dev/null

xargs¶

  • Theoretically, you could pass the rm command a long list of directories to delete
    • When this list of arguments becomes arbitarilaly too long, rm may break
    • It is better to call rm on each of the directories in turn
  • xargs allows us to process a string, determine what the arguments are and how to split them up, and how many times to call a command
    • Very useful for calling a command on the output of find
In [16]:
echo 1 2 3 4 | xargs ls
ls: cannot access 1: No such file or directory
ls: cannot access 2: No such file or directory
ls: cannot access 3: No such file or directory
ls: cannot access 4: No such file or directory

In [13]:
ls *.html | xargs file
Lecture00.html: HTML document, ASCII text, with very long lines
Lecture01.html: HTML document, ASCII text, with very long lines
Lecture02.html: HTML document, ASCII text, with very long lines
Lecture03.html: HTML document, ASCII text, with very long lines
Lecture04.html: HTML document, ASCII text, with very long lines
Lecture05.html: HTML document, ASCII text, with very long lines
In [18]:
ls *.png | xargs -I{} convert {} {}.jpg
In [24]:
rm *.jpg
ls *.png > pngs
more pngs
xargs -IFILE convert FILE FILE.jpg < pngs
ls *.jpg
fb_messenger.png
fb_verify.png
registers.png
fb_messenger.png.jpg  fb_verify.png.jpg  registers.png.jpg

If-Then-Else¶

  • The if block must end with fi
  • The then keyword is required in bash
    • For both elif and if
    • Must be on a different line or follow on the same line after a semicolon
      if CONDITIONAL; then
      #CODE
      elif CONDITIONAL; then
      #CODE
      else
      #CODE
      fi
      

If-Then-Else¶

  • The if block must end with fi
  • The then keyword is required in bash
    • For both elif and if
    • Must be on a different line or follow on the same line after a semicolon
      if CONDITIONAL
      then
      #CODE
      elif CONDITIONAL
      then
      #CODE
      else
      #CODE
      fi
      

Conditional Expression in Bash¶

  • Binary expressions in bash are evaluated
    • Using the test command
    • Using the [ command (an alias of test)
    • Using the [[ syntax
  • Results are stored as a return code
    • Not normally invoked on its own
  • Whitespace is very important

[ and test vs [[¶

  • [ and test are commands
  • [[ is part of bash syntax
    • Allows for easier composition of conditionals using && and ||
    • Parentheses don't have to be escaped
    • Can do pattern matching and regular expressions as a conditional

Conditional Operators¶

  • Bash has three types of conditional operators
    • numeric operators
    • string operators
    • file operators
  • You can always negate an comparison by using ! in front of it

Conditionals on Numbers¶

  • Equal: -eq
  • Not Equal: -ne
  • Greater Than: -gt
  • Greater Than or Equal: -ge
  • Less Than: -lt
  • Less Than or Equal: -le
In [25]:
if [ 1 -eq 7 ]; then
echo "What math are you doing?"
else
echo "One is not equal to 7"
fi
One is not equal to 7
In [30]:
if [ 1 -ne 7 ]; then
echo "One is not equal to 7" 
else
echo "What math are you doing?"
fi
One is not equal to 7
In [35]:
if [ ! 1 -eq 7 ]; then
echo "What math are you doing?"
else
echo "One is not equal to 7"
fi
What math are you doing?
In [32]:
a=1
b=2
if [ $a -lt $b ]; then
echo "$a is smaller than $b"
else
echo "$b is smallter than $a"
fi
1 is smaller than 2

Conditionals on Strings¶

  • Equal: =
  • Not Equal: !=
  • Is Empty: -z
  • Is Not Empty: -n
In [36]:
string1="A string"
string2="Another string"
string3=
if [[ $string1 = $string1 ]]; then
echo "The strings are the same"
fi
The strings are the same
In [40]:
if [[ -z $string3 ]]; then
echo "The string is empty"
fi
The string is empty
In [38]:
if [[ -n $string2 ]]; then
echo "The string is not empty"
fi
The string is not empty

Conditionals on Files¶

  • There are about 20 different tests that can be performed on a file
  • Some common ones are:
    • Existence: -e
    • Is a file: -f
    • Is a directory: -d
    • Is readable/writable/executable: -r/-w/-x
    • Isn't empty: -s
In [43]:
more a_missing_file
Lets make a file
In [42]:
if [[ ! -e 'a_missing_file' ]]; then
echo "Lets make a file" > a_missing_file
fi
In [44]:
touch an_empty_file
if [[ -e 'an_empty_file' ]]; then
echo "An empty file exists"
fi
if [[ -s 'an_empty_file' ]]; then
echo "The file isn't empty"
fi
An empty file exists
In [45]:
if [ -f . ]; then
echo "This directory isn't a file...something is messed up"
else
echo "All is right in the world"
fi
All is right in the world

Switch Statements¶

  • Switch statements start with the keyword case and end with the keyword esac
  • Each clause is a pattern to match the expression against
    • The pattern in a clause ends with a right parentheses )
    • A clause must end with two semicolons (;;)
In [46]:
expression="This is a String"

case $expression in
    0)
        echo "The variable is 0"
        ;;
    *ing)
        echo "The variable ends in ing"
        ;;
    *String)
        echo "The variable ends in String"
        ;;
    *)
        echo "This is the default"
        ;;
esac
The variable ends in ing

For Loops¶

  • Bash has traditionally used a foreach style loop ( similar to Python)
  • Can loop over any type of array
    • Can also loop over files
  • Both loops have the general syntax of
    for EXPRESSION(S); do
    # CODE_GOES_HERE
    done
    

Foreach Style Loop¶

  • The foreach style loop uses the setup of
    for variable in list; do
    
  • list can be
    • a space seperated list
    • an expanded array
    • a shell-style regular expression (globbing)
    • the output of a command
In [47]:
for x in 1 2 3; do
    echo $x;
done
1
2
3
In [48]:
my_array=(1 2 3)
for y in ${my_array[@]}; do
    echo $y
done
1
2
3
In [49]:
for f in *.html; do
    wc -l $f
done
12045 Lecture00.html
14121 Lecture01.html
13010 Lecture02.html
13173 Lecture03.html
14242 Lecture04.html
13017 Lecture05.html
In [50]:
for f in $(ls); do
    if [[ $f == *.html ]]; then
        wc $f
    fi
done
 12045  28379 263529 Lecture00.html
 14121  41179 374471 Lecture01.html
 13010  36181 320828 Lecture02.html
 13173  31135 293564 Lecture03.html
 14242  32770 309209 Lecture04.html
 13017  31235 293615 Lecture05.html

C-Style Loop¶

  • Support for the C-style loop is widespread in bash, but not all shell scripts
  • The syntax for the C-style loop is:
    for (( START ; END ; CHANGE)); do
    
  • The variable isn't prefixed with the dollar sign ($) inside the loop definition
In [51]:
for ((x = 1; x < 4; x++)); do
    echo $x
done
1
2
3
In [52]:
for ((x = 1; x < 4; x += 2)); do
    echo $x
done
1
3

seq Command¶

  • There are many other ways to do a c-style loop while using the traditional syntax
  • One option is the seq command, which returns a list of numbers
  • The syntax of the seq command is
    seq START INCREASE? END
    
In [54]:
for i in $(seq 1 3); do
    echo $i
done
1
2
3
In [55]:
for i in $(seq 0 2 10); do
    echo $i
done
0
2
4
6
8
10

Brace Expansion¶

  • Another feature of bash that is often, but not exclusively used, with loops is brace expansion
  • Bash will expand anything in braces into a list
  • Braces can take two forms:
    {A_LIST,OF,OPTIONS}
    
    or
{START..END}
In [58]:
echo Lecture0{0,1,2,3,4,5}.html | xargs ls -lh | cut -f6,7,8  -d' '
Aug 31 10:59
Sep  5
Sep 14 10:34
Sep 14 10:34
Sep 19 10:47
Sep 19 10:47
In [59]:
for i in {0..5}; do
    ls -lh Lecture0$i.html | cut -f6,7,8 -d' '
done
Aug 31 10:59
Sep  5
Sep 14 10:34
Sep 14 10:34
Sep 19 10:47
Sep 19 10:47

While Loops¶

  • While loops also use the do expression after the condition
  • The syntax for a while loop is
    while CONDITION; do
      #CODE_HERE
    done
    
In [60]:
string='Some Characters'
while [[ -n $string ]]; do
    echo ${string:0:1}
    string=${string:1}
done
S
o
m
e

C
h
a
r
a
c
t
e
r
s

Until Loops¶

  • The until loop is almost identical to the while loop, but continues until the statement is True
  • The until is still places at the top of the loop and checked before entering it
  • The syntax of until is
    until CONDITIONAL; do
      #CODE GOES HERE
    done
    
In [61]:
string='Some Characters'
until [[ -z $string ]]; do
    echo ${string:0:1}
    string=${string:1}
done
S
o
m
e

C
h
a
r
a
c
t
e
r
s