Special handling (a.k.a Drive letters)
This is how Windows handles filesystems. It explicitly shows which drive you’re working on.
Virtual Filesystem
This is a more subtle approach, used by Linux, where you specify in the beginning (booting) which filesystem will be used for which part of the virtual filesystem.
[myname@l3_ ~]$
* tells you:
* who you are
* which computer you’re on
* which directory you’re in
* can be configured
* variable ''%%$PS1%%''
* default: ''%%echo $PS1%%''
> Ways to get help when you’re stuck:
Most of the time a command doesn’t act as expected, it shows an error message. From this point you have multiple approaches:
* Think about why the program failed - maybe you (un-)intentionally tried to force the program to do something it’s not intended for?
* Just copy that message into your favorite search engine and don’t forget to remove the parts that are specific to your environment (e.g. directory and user names).
* Most programs supports a ''%%-h%%''/''%%--help%%'' flag
* ''%%man
gcc FizzBuzz.c -o FizzBuzz
./FizzBuzz
false
echo $?
The examples show compiling a program, executing the result, trying to load a module on our cluster and checking if the previous command succeeded.
* ''%%gcc%%'' is a program that is in a directory specified by the ''%%$PATH%%'' variable and will be found without specifying its exact location.
* ''%%./FizzBuzz%%'' is a newly compiled executable, which is not found by looking at ''%%$PATH%%'', so we explicitly add ''%%./%%'', to show that we want to execute it from the current directory
* ''%%module load non-existent-module%%'' fails, as the module command can’t find ''%%non-existent-module%%''. Whenever a command fails, its ''%%return value%%'' is set to a value other than zero. The manual for some commands has a map from return-value to error-description to aid the user debugging.
* ''%%echo $?%%'' is a command that prints the return value of the previous command.
=== History ===
> Your shell keeps a log of all the commands you executed.
* the ''%%history%%'' command is used to access this history
* for fast reuse of commands try the ''%%
COMMAND -j 2 -a -b -c
COMMAND -j 2 -abc
== Ordering parameters ==
One thing to look out for is the order of parameters. Most of the time no specific order is required, but you should look out for things like copying the target over the source file. Also watch out to keep parameters and their arguments together.
COMMAND # OK
COMMAND # PROBABLY WRONG
COMMAND -j 2 --color auto # OK
COMMAND -j auto --color 2 # PROBABLY WRONG
==== Escapes & Quotes ====
Whenever a parameter has to contain a character that is either unprintable or reserved for the shell, you can use:
- Backslash escape:
* Escapes a character, that would have a special meaning
* Can be used inside of quotes
- Double Quotes:
* Similar to escaping all whitespace characters
- Single Quotes:
* Additionally prevents expansion of variables
COMMAND This\ is\ a\ single\ parameter
COMMAND "This is a single parameter"
COMMAND 'This is a single parameter'
==== Aliases ====
You can define aliases in your shell. These are usually used to shorten names for commands which are used often with a fixed set of parameters or where you have to be careful to get things right. These aliases are accessable as if they were commands.
alias ll='ls -alh'
alias rm='rm -i'
alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -'
After that, you can use the aliases synonymously.
ll # Same as 'ls -alh'
rm # Same as 'rm -i'
myProject # Same as 'cd $ProjectDir; testSuite; compile && testSuite; cd -'
==== Patterns ====
Patterns are an easy way of defining multiple arguments, which are mostly the same. The pattern will match anything in it’s place.
The other concept is a expansion. In this case only defined patterns will be matched.
* the most important patterns are:
* **?** — matches one character
* ***** — matches any character sequence
* the most important expansions are:
* **A{1,9}Z** — expands to A1Z A9Z
* **A{1..9}Z** — expands to A1Z A2Z … A9Z
You can try these commands and see what they do. These are all totally safe, even if you modify the arguments.
ls file.???
ls *.*
echo {{A..Z},{a..z}}
echo {{A,B},{X,Y}}
echo {A..Z},{a..z}
echo {A,B}{X,Y}
==== Regular Expressions ====
Often you need to specify some string, but patterns and expansions aren’t enough, to cover all possibilities. In these cases you can use a regular expression also known as regex. These regexes are used by editors for search and replace, the ''%%egrep%%'' command for filtering through files and inside many scripts to validate parameters.
.+ # Match any character, once or more
\. # match a dot
(A|a)p{2}le # apple, Apple
^[^aeiouAEIOU]+$ # any line of only non-vowels
For a detailed explanation see [[https://en.wikipedia.org/wiki/Regular_expression|Wikipedia]], [[https://www.regular-expressions.info/|Regular-expressions.info]] or try [[https://regex101.com/|regex101]].
If you want to challenge yourself, try [[https://regexcrossword.com/|Regex Crossword]]!
==== Control Flow ====
In the shell language there are a few ways to organize the execution path. The most important ones are:
- chaining of commands
- looping constructs
- conditionals (if/case)
=== Chaining Commands ===
The simplest mechanism for control flow is to chain commands together in a simple ''%%if COMMAND then NEXTCOMMAND else ERRORCOMMAND%%''. Since this would be cumbersome to write, most shells provide simple syntax for this: ''%%COMMAND && NEXTCOMMAND || ERRORCOMMAND%%'' and if a command should be run without relying on the return value of its predecessor it’s written: ''%%COMMAND; NEXTCOMMAND%%''. And if you only want to execute further commands in one case (but not the other), you don’t even have to specify both branches.
false ; echo "Should I be Printed?"
false && echo "Should I be Printed?"
false || echo "Should I be Printed?"
Should I be Printed?
Should I be Printed?
=== Loops ===
The other way to execute commands conditionally are loops. You can loop over files, numeric arguments, until a either the loop condition is false or a break is encountered.
for i in *
do
mv $i{,.bak}
done
while true
do
echo "Annoying Hello World"
sleep 3
done
for i in *; do mv $i{,.bak}; done
while true; do echo "Annoying Hello World"; sleep 3; done
=== Conditionals ===
''%%If%%'' is similar to the previous chaining of commands, except that it is more verbose and nicer to read if you have many commands to execute one branch of the decision. For more conditions the ''%%elif%%'' (else if) statement can be used. If you use a lot of ''%%elif%%''s and you only check one variable with them, you should consider using a ''%%case%%'' statement.
if [ $VARIABLE1 ]
then
COMMAND1
elif [ $VARIABLE2 ]
COMMAND2
else
COMMAND3
fi
''%%Case%%'' statements are for querying all states of a single variable and making a decision based on that. It can match some simple expansions, which do NOT follow the general syntax of bash expansions. Also it processes alternative matches when seperated with ''%%|%%'' (pipe character).
case $VARIABLE in
[0-9] | [1-2][0-9])
COMMAND1
;;
*)
COMMAND2
;;
esac
==== Streams ====
=== Redirects ===
> Write **output** to a **file** or **file-descriptor**
^Command^Redirect ^Append ^Description ^
|program|''%%> std.log%%''|''%%>> std.log%%''|redirect ''%%stdout%%'' to a file |
|program|''%%2> err.log%%''|''%%2>> err.log%%''|redirect ''%%stderr%%'' to a file |
|program|''%%2>&1%%'' | |redirect ''%%stderr%%'' to ''%%stdout%%''|
=== Pipes ===
> Write **output** into the **input**-stream of another process
^Command^Pipe ^Description ^
|program|''%%| grep -i foo%%'' |pipe ''%%stdout%%'' into ''%%grep%%''|
|program|''%%| tee file1 file2%%''|overwrite files and ''%%stdout%%'' |
|program|''%%| tee -a file%%'' |append to files and ''%%stdout%%'' |
===== Environment Variables =====
==== Setting, getting and unsetting ====
=== Set ===
LANG=en_US.UTF-8 bash
export LANG=en_US.UTF-8
=== Get ===
env
echo ${LANG}
echo $PWD
=== Unset ===
unset LANG
env -u LANG
==== Use cases ====
> Some variables that could affect you are:
$EDITOR # the default editor for the CLI
$PAGER # utility to read long streams
$PATH # program paths, in priority order
> if you’re aiming for programming, these could be more interesting:
$LIBRARY_PATH # libraries to link by the compiler
$LD_LIBRARY_PATH # libraries to link at runtime
$CC # sometimes used to set default C compiler
$CFLAGS # default flags for compiling C
export PATH="./:$HOME/bin/:$PATH"
# only works with root privileges
chown user file
chown -R user:group dirs files
=== chmod ===
> Change the mode of files and directories by:
chmod -R u=rwx,g+w,o-rwx dirs files
chmod 640 files
chmod 750 dirs
chmod 750 executables
* there are three bits for:
* read (4)
* write (2)
* execute (1)
* three times for:
* user
* group
* other
==== Shebang ====
A little test program, which we mark as executable and hand it over to the corresponding interpreter:
cat << EOF > test.sh
echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
bash test.sh
> Don’t we have an OS, capable of executing everything it recognizes as an executable?
> Yes, we do!
cat << EOF > test.sh
#!/bin/bash
echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
./test.sh
==== Functions (more like procedures) ====
Programming in bash would be cumbersome without functions, so here we go:
allNumbersFromTo () {
echo "1 2 3"
}
> This isn’t good, as were only getting a fixed amount of numbers. Let’s try a recursive approach:
allNumbersFromTo () {
num=$1
max=$2
echo "${num}"
if [ $num -lt $max ]; then
allNumbersFromTo "$(($num + 1))" $max
fi
}
allNumbersFromTo () {
min=$1
max=$2
for num in $(seq $min $max)
do
echo "${num}"
done
}
allNumbersFromTo 1 10
===== .bashrc =====
==== .bashrc ====
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
alias sq='squeue -u $USER'
alias rm='rm -i'
export PATH="./:$HOME/bin:$PATH"