type Name = String
type Children = [FSObject]
type Path = [FSObject]
data LinkType = Soft | Hard
> This is the code describing a trivial Linux-ish filesystem in a mathematical notation:
data FSObject = File Name
| Dir Name Children
| Link Name Path LinkType
> Missing:
* Special files
* device
* fifo pipe
* socket
* Attributes
* Advanced FS features
* data integrity
* device managment
* subvolume support
==== FHS intro ====
{{.:linux_directories.jpg}}
==== FHS ====
> The **Filesystem Hierarchy Standard** is a guideline for the structuring of local filesystems.
It aims for better **interoperability** between different Systems, by making it easy to figure out the **location** of any **kind of file**.
^Directory ^Purpose ^
|**/** |root of the hierarchy |
|**/bin/** |basic binary files |
|**/boot/** |files for boot support |
|**/dev/** |virtual files, for direct access to devices |
|**/etc/** |system-wide confidurations - historical, misleading name |
|**/home/** |users’ home direcories are located here |
|**/lib/** |library files for ''%%/bin/%%'' ''%%/sbin%%'' |
|**/media/**|removable media mounts |
|**/mnt/** |temporary mounts |
|**/opt/** |optional applications |
|**/proc/** |system-internals accessable as files |
|**/root/** |home directory of root |
|**/run/** |variable data should be removed while booting |
|**/sbin/** |like ''%%/bin/%%'', but require special permissions |
|**/srv/** |data served to other systems |
|**/sys/** |statistics about devices |
|**/tmp/** |temporary files, can be removed while booting |
|**/usr/** |most higher level applications are inside, looks a lot like ''%%/%%''|
|**/var/** |files that change often (cache/db/log/mail/…) |
==== Corner cases of FHS ====
> Some systems don’t comply with the FHS, for various reasons.
=== VSC3 ===
* As the OS is basically a CentOS, the ''%%/lib/%%'' and ''%%/lib64/%%'' are just symlinks
=== Distributions bringing new ideas to the table ===
> [[http://gobolinux.org/|GoboLinux]]:
* installed software packages get their own directory
* filesystem itself is a package store
> [[http://nixos.org/|NixOS]]:
* solves the problem of user installable packages
* dependency management extended in a way, that aims to make trust a non-issue
====== Shell ======
==== Login ====
> The default login screen on most distributions looks something like this:
<<< Welcome to ${DISTRO} - tty{1..6} >>>
${MACHINE_NAME} login:
password:
username
$ ssh username@remote
password:
[myname@l3_ ~]$
* tells you:
* who you are
* which computer you’re on
* which directory you’re in
* can be configured
* variable ''%%$PS1%%''
* default: ''%%echo $PS1%%''
> Ways to get help when you’re stuck:
* ''%%man%%''
* manual for specified topic
* find man-pages: ''%%apropos%%''
* ''%%whatis%%''
* ''%%info%%''
* most commands support a ''%%-h%%''/''%%--help%%'' flag
* colleagues are often helpful
* the internet is often helpful
==== Execution ====
To execute a program, we call it:
nano file.txt
./a.out -flag=value
gcc -I/home/lv12345/myuser/include
module load non-existent-module
* Every command that is executed will provide a return-value on exit.
* {{.:tango_ok.png?0x30}} A value of ''%%0%%'' means success
* {{.:tango_cancel.png?0x30}} any other value means failure
=== Program Arguments ===
> The default way to apply arguments to a program is to write a space separated list of arguments after the program when calling it.
> Patterns and expansions are defining multiple arguments with little overhead:
* the most important patterns are:
* **?** — matches one character
* ***** — matches any character sequence
* the most important expansions are:
* **A{1,9}Z** — expands to A1Z A9Z
* **A{1..9}Z** — expands to A1Z A2Z … A9Z
==== Control Flow ====
=== Chaining Commands ===
false ; echo "Should I be Printed?"
false && echo "Should I be Printed?"
false || echo "Should I be Printed?"
Should I be Printed?
Should I be Printed?
=== Loops ===
for i in *
do
mv $i{,.bak}
done
while true
do
echo "Annoying Hello World"
sleep 3
done
> Or as one-liners:
for i in *; do mv $i{,.bak}; done
while true; do echo "Annoying Hello World"; sleep 3; done
==== Aliases ====
Whenever you have a command, which you:
- run often and it
* has a long list of parameters you always use
* is dangerous
- which is an aggregate of many other commands
* but you don’t want to write a script
you can define an alias for it:
alias ll='ls -alh'
alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -'
==== History ====
{{.:tango_search.svg?0x100}}
> Your shell keeps a log of all the commands you executed.
* the ''%%history%%'' command is used to access this history
* for fast reuse of commands try the ''%%
LANG=de_AT.UTF-8 firefox
env LANG=de_AT.UTF-8 firefox
export LANG=de_AT.UTF-8
=== Get ===
env
echo ${LANG}
=== Unset ===
unset LANG
env -u LANG
==== Use cases ====
> Some variables that could affect you are:
$EDITOR # the default editor for the CLI
$PAGER # utility to read long streams
$PATH # program paths, in priority order
> if you’re aiming for programming, these could be more interresting:
$LIBRARY_PATH # libraries to link by the compiler
$LD_LIBRARY_PATH # libraries to link at runtime
$CC # sometimes used to set default C compiler
$CFLAGS # default flags for compiling C
====== Core Utilities ======
==== Looking around ====
> **looking around** is done by the **''%%ls%%''** (list) command
$ ls # shows files and directories
$ ls -a # includes hidden ones
$ ls -l # detailed view
testdir test
. .. testdir test
drwxr-xr-x 1 myuser p12345 0 Apr 13 11:55 testdir
-rw-r--r-- 1 myuser p12345 4 Apr 13 11:55 test
{{.:linux_directories.jpg?0x400}}
^command ^ result^
|**''%%ls /%%''** | ''%%bin dev etc home sbin usr%%''|
|**''%%ls /home%%''** | ''%%user1 user2%%''|
==== Moving around ====
> **moving around** is done by the **''%%cd%%''** (change directory) command
> **finding** the current **position** in the filesystem is done by the **''%%pwd%%''** (print working directory) command
$ cd /bin # go to an absolute directory
$ cd [~] # go home
$ cd - # go to previous directory
$ pwd # /bin
$ pwd # /home/user1
$ pwd # /bin
{{.:linux_directories.jpg?0x400}}
==== Copying & moving files around ====
> **moving files around** is done by the **''%%mv%%''** (move) command
> **copying** is done by the **''%%cp%%''** (copy) command
$ mv old new # rename old to new
$ mv old dir/ # move old into dir
$ mv file1 file2 # overwrite file2 with file1
# (BEWARE)
$ cp -i input input.bak # input to input.bak
$ cp -i input backup/ # input into backup
$ cp -a dir1/ dir2 # exact copy of dir1
{{.:screenshot_mv.png?0x350}} {{.:screenshot_cp.png?0x350}}
==== Finding stuff ====
> To look at everything
- in your home directory
- and nested up to three levels deep inside it
- that ends in ''%%.txt%%''
- or starts with ''%%log_%%''
- and is an ordinary file
- concatenated as one stream:
find \
~ \
-maxdepth 3 \
-iname "*.txt" \
-or -iname "log_*" \
-type f \
-exec cat '{}' \; | less
find \
~ \
-maxdepth 3 \
-iname "*.txt" \
-or -iname "log_*" \
-type f \
-exec cat '{}' + | less
==== Creating and deleting ====
=== directories ===
> **creating directories** is done by the **''%%mkdir%%''** (make directory) command:
> **deleting directories** is done by the **''%%rmdir%%''** (remove directory) command:
$ mkdir newdir # create directory newdir
$ mkdir -p new/dir # create dir inside of new
$ rmdir emptydir # removes emptydir
$ rmdir -v emptydir # removes emptydir (verbose)
=== files ===
> **creating files** is done by the **''%%touch%%''** or **''%%echo%%''** commands:
> **deleting files** is done by the **''%%rm%%''** (remove) command:
$ touch file # create empty file
$ echo "abc" > file # create/overwrite file
$ rm file # removes file
$ rm -ri dir # removes dir recursively
==== Contents of files ====
> **viewing** is done by the **''%%less%%''** command:
> **concatenating** is done by the **''%%cat%%''** command:
$ less file.txt # exit with 'q'
$ less -R file.txt # keep colors
$ cat file1 file2 | less
$ cat file
$ cat -A printable
$ cat -n numbered
$ echo "VSC is great" > file
$ cat file
VSC is great
$ echo "VSC is awesome" >> file
$ cat file
VSC is great
VSC is awesome
$ cat file | grep awesome
VSC is awesome
$ grep awesome file
VSC is awesome
==== Space accounting ====
> viewing **used space** is done by the **''%%du%%''** (disk usage) command:
> viewing **free space** is done by the **''%%df%%''** (disk free) command:
$ du -h file1 file2 # human readable output
$ du -s dir # summarize
$ df -h # human readable output
$ df -t nfs # only list filesystems of a type
{{.:screenshot_df.png?0x440}}
==== Recap ====
$ mv space.log space.log.bak
$ df -h | grep "lv12345\|lv54321" > space.log
$ cat space.log
nfs05.ib.cluster:/e/lv12345 200G 185G 16G 93% /home/lv12345
nfs04.ib.cluster:/e/lv54321 1000G 979G 22G 98% /home/lv54321
> we do this often, let’s wrap it up!
==== Recap++ ====
$ mv space.log space.log.bak
$ df -h | grep "lv12345\|lv54321" > space.log
$ cat space.log
nfs05.ib.cluster:/e/lv12345 200G 185G 16G 93% /home/lv12345
nfs04.ib.cluster:/e/lv54321 1000G 979G 22G 98% /home/lv54321
> we do this often, let’s wrap it up!
$ echo '#!/bin/bash' > spacelog.sh
$ echo 'mv space.log space.log.bak' >> spacelog.sh
$ echo 'df -h | grep "lv12345\|lv54321" > space.log' >> spacelog.sh
$ echo 'cat space.log' >> spacelog.sh
$ chmod +x spacelog.sh
$ ./spacelog.sh
==== Sed and awk ====
> **sed** (stream editor) and **awk** are powerful tools when working with the command line
$ mycommand | sed "..."
$ mycommand | awk '{...}'
> Using sed and awk in action
^program ^ command ^description ^
|sed | ''%%s/old/new/%%'' |replace ''%%old%%'' with ''%%new%%'' |
|sed | ''%%/from/,/to/ s/old/new/%%'' |replace ''%%old%%'' with ''%%new%%'', between ''%%from%%'' and ''%%to%%'' |
|awk | ''%%'print $5, $3'%%'' |print columns 5 and 3 of every line |
Example script:
#!/bin/bash
mv space.log space.log.bak
df -h | grep "lv12345\|lv54321" > space.log
cat space.log
#!/bin/bash
mv space.log space.log.bak
df -h | grep "lv12345\|lv54321" > space.log
cat space.log | sed "s|/home/lv12345|ProjectA|" \
| awk '{print $6, "free:", $4}' \
| column -t
====== Scripting ======
==== Ownership and Permissions ====
> Just to ensure, that you are able to run your scripts
=== chown ===
> Change the owner of files and directories by:
chown -R user:group dirs files
=== chmod ===
> Change the mode of files and directories by:
chmod -R u=rw,g+w,o-rwx dirs files
chmod -R 644 dirs files
==== Shebang ====
A little test program, which we mark as executable and hand it over to the corresponding interpreter:
cat << EOF > test.sh
echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
bash test.sh
> Don’t we have an OS, capable of executing everything it recognises as a executable?
> Yes, we do!
cat << EOF > test.sh
#!/bin/bash
echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
./test.sh
==== Functions (more like Procedures) ====
Programming in bash would be cumbersome without functions, so here we go:
printAllNumbers () {
echo "1 2 3"
echo "done"
}
=== Arguments ===
> This isn’t good, as were only getting a fixed amount of numbers. let’s try a recursive approach
#!/bin/bash
printAllNumbersFrom () {
num=$1
echo "${num}"
printAllNumbersFrom "$(($num + 1))"
}
printAllNumbersFrom $2
====== Editors ======
==== General ====
* Many different editors
* Unique (dis-)advantages
* Different look and feel
* Editors should provide us with
- Simple text editing
- Copy and paste
- Search and replace
- Saving changes
- Wide availability
=== Two editors that satisfy our needs: ===
* nano
* vim
=== Common starting point ===
nano filename
vim filename
==== Nano ====
{{.:screenshot_nano.png}}
==== Nano explained ====
> This editor is focused on being easy to use, but still providing every feature a user might need.
=== Interface ===
> The interface consists of four parts, namely from top to bottom:
* Title bar
* Text area
* Command line
* Key bindings
=== Usage ===
> Nothing special, key-bindings visible while editing
^Feature ^ Usage ^
|Navigation | Arrow keys |
|Actual editing| Typing text, as usual |
|Cut/Paste line| ''%%