Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revisionBoth sides next revision
pandoc:linux-wochen-wien:linux-101:linux-101 [2018/01/31 11:11] – Pandoc Auto-commit pandocpandoc:linux-wochen-wien:linux-101:linux-101 [2020/10/20 08:09] – Pandoc Auto-commit pandoc
Line 1: Line 1:
 +====== History ======
 +
 +==== Unix — Linux’ predecessor ====
 +
 +> Unix is from the ’70s:
 +
 +
 +  * terminals are monochrome
 +  * computers are big and expensive
 +  * many people use the same computer
 +  * networking becomes important
 +
 +> Unix is free to modify:
 +
 +
 +  * universities and companies have modified unix
 +  * released as BSD/Solaris/HP-UX/IRIX
 +  * fertile environment
 +  * rise of similar, non-collaborating features
 +
 +
 +> Unix becomes the GCD of computing.
 +
 +
 +==== Hacker culture ====
 +
 +> Freedom at stake:
 +
 +
 +  * the way of sharing programs is changing
 +  * best case: bsd-style license
 +  * usually: corporate licenses
 +    * exclude freedom of sharing code with anyone
 +
 +> Sharing is caring:
 +
 +
 +  * write code
 +  * give away/sell product
 +    * include **sourcecode**
 +    * include the freedom/restriction to do share code under the same license
 +
 +
 +> People want to save their idea of moral responsibility by formalising it as a licenses
 +
 +
 +==== GNU ====
 +
 +{{..:gnu.svg?0x200}}
 +
 +> In 1984 RMS starts the **GNU** project, with the goal to provide a **free Unix**-compatible OS
 +
 +> Steps to success:
 +
 +
 +
 +  * Write a license (GPL)
 +    * respect the freedom of the users
 +    * endorse ‘sharing is caring’
 +    * make the license transitive
 +  * Write an Editor (emacs)
 +  * Write a Compiler (gcc)
 +  * Write common utilities (coreutils)
 +  * Write everything else
 +  * Write a kernel (hurd)
 +  * Rewrite the kernel (hurd)
 +  * One more time (hurd)
 +
 +
 +> Even though this project gets mocked for hurd, it is essential for Linux’ success!
 +
 +
 +==== Linux ====
 +
 +{{..:gnu_tux.svg?0x220}}
 +
 +> In 1992 Linus Torvalds writes his own little **terminal emulator**, to be able to remotely access his university account.
 +
 +> Later this code evolves into the **Linux kernel**, that together with many **GNU utilities** becomes **GNU/Linux** — a.k.a Linux
 +
 +
 +^Segment            ^             Linux variant              Market share ([[https://en.wikipedia.org/wiki/Usage_share_of_operating_systems#Market_share_by_category|Source]])^
 +|Desktop            |      Debian / Fedora / Arch / …                                                                                                                      ~2%|
 +|Smartphone / Tablet|                Android                |                                                                                                               ~70%|
 +|Server              Debian / RHEL / Gentoo / … / CUSTOM  |                                                                                                               ~65%|
 +|**Supercomputer**  |          **CentOS** / CUSTOM          |                                                                                                               ~99%|
 +|Mainframe          |              RHEL / SLES              |                                                                                                               ~30%|
 +|Embedded                        Android / …              |                                                                                                               ~30%|
 +
 +
 +==== Linux on Supercomputers ====
 +
 +{{..:os_supercomputers.svg}}
 +
 +====== Prerequisites ======
 +
 +==== Assumptions ====
 +
 +> You are familiar with either:
 +
 +
 +  * **computers** in general
 +  * **Windows**
 +  * **macOS**
 +
 +> When something breaks, you:
 +
 +
 +  * ask for **assistance** //or//
 +  * **read** up on the subject and **fix it** //and//
 +  * **don’t** go on **using** it until it’s completly **dead**
 +
 +
 +==== What do you know about Linux? ====
 +
 +> by a show of hands:
 +
 +{{..:tux.svg?0x500}}
 +
 +
 +  * Nothing
 +  * I know my way around when it comes to:
 +    * files and directories
 +    * the shell basics
 +    * most core utilities
 +    * variables
 +    * scripts
 +    * streams
 +  * All of the above and I want to take over
 +
 +====== Filesystems ======
 +
 +==== Filesystem 101 ====
 +
 +
 +  - Everything starts at the root
 +    * the root is a directory
 +    * ''%%/%%'' denotes the root directory
 +  - the filesystem has different kinds of objects
 +    - files
 +    - directories
 +      * containers for multiple objects
 +    - links to objects, which either
 +      * add a second name for the same object
 +      * point to a position in the filesystem
 +  - objects can be referenced by their path
 +    * absolute: ''%%/dir1/dir2/object%%''
 +    * relative: ''%%dir2/object%%''
 +  - special objects in directories:
 +    * ''%%.%%'' — is a reference to the directory itself
 +    * ''%%..%%'' — is a reference to the parent directory
 +  - the system may consist of multiple filesystems
 +    * filesystems may be mounted at any (empty) directory
 +
 +
 +==== The Filesystem represented as an ADT ====
 +
 +> These are the helper types:
 +
 +<code>
 +type Name     = String
 +type Children = [FSObject]
 +type Path     = [FSObject]
 +data LinkType = Soft | Hard
 +</code>
 +
 +> This is the code describing a trivial Linux-ish filesystem in a mathematical notation:
 +
 +<code>
 +data FSObject = File Name
 +              | Dir  Name Children
 +              | Link Name Path LinkType
 +</code>
 +
 +
 +> Missing:
 +
 +  * Special files
 +    * device
 +    * fifo pipe
 +    * socket
 +  * Attributes
 +  * Advanced FS features
 +    * data integrity
 +    * device managment
 +    * subvolume support
 +
 +
 +
 +==== FHS intro ====
 +
 +{{..:linux_directories.jpg}}
 +
 +==== FHS ====
 +
 +> The **Filesystem Hierarchy Standard** is a guideline for the structuring of local filesystems.
 +
 +It aims for better **interoperability** between different Systems, by making it easy to figure out the **location** of any **kind of file**.
 +
 +^Directory  ^Purpose                                                              ^
 +|**/**      |root of the hierarchy                                                |
 +|**/bin/**  |basic binary files                                                   |
 +|**/boot/** |files for boot support                                               |
 +|**/dev/**  |virtual files, for direct access to devices                          |
 +|**/etc/**  |system-wide confidurations - historical, misleading name             |
 +|**/home/** |users’ home direcories are located here                              |
 +|**/lib/**  |library files for ''%%/bin/%%'' ''%%/sbin%%''                        |
 +|**/media/**|removable media mounts                                               |
 +|**/mnt/**  |temporary mounts                                                     |
 +|**/opt/**  |optional applications                                                |
 +|**/proc/** |system-internals accessable as files                                 |
 +|**/root/** |home directory of root                                               |
 +|**/run/**  |variable data should be removed while booting                        |
 +|**/sbin/** |like ''%%/bin/%%'', but require special permissions                  |
 +|**/srv/**  |data served to other systems                                         |
 +|**/sys/**  |statistics about devices                                             |
 +|**/tmp/**  |temporary files, can be removed while booting                        |
 +|**/usr/**  |most higher level applications are inside, looks a lot like ''%%/%%''|
 +|**/var/**  |files that change often (cache/db/log/mail/…)                        |
 +
 +
 +
 +==== Corner cases of FHS ====
 +
 +> Some systems don’t comply with the FHS, for various reasons.
 +
 +=== VSC3 ===
 +
 +  * As the OS is basically a CentOS, the ''%%/lib/%%'' and ''%%/lib64/%%'' are just symlinks
 +
 +=== Distributions bringing new ideas to the table ===
 +
 +> [[http://gobolinux.org/|GoboLinux]]:
 +
 +  * installed software packages get their own directory
 +  * filesystem itself is a package store
 +
 +> [[http://nixos.org/|NixOS]]:
 +
 +  * solves the problem of user installable packages
 +  * dependency management extended in a way, that aims to make trust a non-issue
 +
 +
 +====== Shell ======
 +
 +==== Login ====
 +
 +> The default login screen on most distributions looks something like this:
 +
 +<code>
 +<<< Welcome to ${DISTRO} - tty{1..6} >>>
 +
 +${MACHINE_NAME} login:
 +password:
 +</code>
 +<code>
 +
 +
 +username
 +<blank while typing>
 +</code>
 +
 +<HTML>
 +<div class="incremental">
 +<div>
 +<hr style="clear: both;">
 +</HTML>
 +> remote login looks similar:
 +
 +<code bash>
 +$ ssh username@remote
 +password:
 +</code>
 +<HTML>
 +</div>
 +<div>
 +<hr style="clear: both;">
 +</HTML>
 +{{..:tango_locked.svg?0x100}} {{..:tango_unlocked.svg?0x100}}
 +<HTML>
 +</div>
 +</HTML>
 +> If the login was not successful, just try again.
 +
 +> Beware, that systems may be configured to lock your account if too many attempts were made.
 +
 +<HTML>
 +</div>
 +</HTML>
 +==== Prompt ====
 +
 +> This is how the prompt looks by default:
 +
 +{{..:tango_terminal.svg?0x100}}
 +
 +<code bash>
 +[myname@l3_ ~]$ 
 +</code>
 +
 +
 +  * tells you:
 +    * who you are
 +    * which computer you’re on
 +    * which directory you’re in
 +  * can be configured
 +    * variable ''%%$PS1%%''
 +    * default: ''%%echo $PS1%%''
 +
 +> Ways to get help when you’re stuck:
 +
 +
 +  * ''%%man%%''
 +    * manual for specified topic
 +    * find man-pages: ''%%apropos%%''
 +  * ''%%whatis%%''
 +  * ''%%info%%''
 +  * most commands support a ''%%-h%%''/''%%--help%%'' flag
 +  * colleagues are often helpful
 +  * the internet is often helpful
 +
 +
 +==== Execution ====
 +
 +To execute a program, we call it:
 +
 +<code python>
 +nano file.txt
 +./a.out -flag=value
 +gcc -I/home/lv12345/myuser/include
 +module load non-existent-module
 +</code>
 +
 +
 +  * Every command that is executed will provide a return-value on exit.
 +    * {{..:tango_ok.png?0x30}} A value of ''%%0%%'' means success
 +    * {{..:tango_cancel.png?0x30}} any other value means failure
 +
 +
 +=== Program Arguments ===
 +
 +> The default way to apply arguments to a program is to write a space separated list of arguments after the program when calling it.
 +
 +> Patterns and expansions are defining multiple arguments with little overhead:
 +
 +
 +
 +  * the most important patterns are:
 +    * **?** — matches one character
 +    * ***** — matches any character sequence
 +  * the most important expansions are:
 +    * **A{1,9}Z** — expands to A1Z A9Z
 +    * **A{1..9}Z** — expands to A1Z A2Z … A9Z
 +
 +
 +==== Control Flow ====
 +
 +=== Chaining Commands ===
 +
 +<code bash>
 +false ;  echo "Should I be Printed?"
 +false && echo "Should I be Printed?"
 +false || echo "Should I be Printed?"
 +</code>
 +<code bash>
 +Should I be Printed?
 +
 +Should I be Printed?
 +</code>
 +
 +=== Loops ===
 +
 +<code bash>
 +for i in *
 +do
 +  mv $i{,.bak}
 +done
 +</code>
 +<code bash>
 +while true
 +do
 +  echo "Annoying Hello World"
 +  sleep 3
 +done
 +</code>
 +
 +> Or as one-liners:
 +
 +<code bash>
 +for i in *; do mv $i{,.bak}; done
 +while true; do echo "Annoying Hello World"; sleep 3; done
 +</code>
 +
 +==== Aliases ====
 +
 +Whenever you have a command, which you:
 +
 +
 +  - run often and it
 +    * has a long list of parameters you always use
 +    * is dangerous
 +  - which is an aggregate of many other commands
 +    * but you don’t want to write a script
 +
 +you can define an alias for it:
 +
 +<code bash>
 +alias ll='ls -alh'
 +alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -'
 +</code>
 +
 +==== History ====
 +
 +{{..:tango_search.svg?0x100}}
 +
 +> Your shell keeps a log of all the commands you executed.
 +
 +
 +  * the ''%%history%%'' command is used to access this history
 +  * for fast reuse of commands try the ''%%<CTRL>-R%%'' keys
 +
 +==== Streams ====
 +
 +=== Redirects ===
 +
 +> Write **output** to a **file** or **file-descriptor**
 +
 +^Command^Redirect          ^Append             ^Description                              ^
 +|program|''%%>  std.log%%''|''%%>>  std.log%%''|redirect ''%%stdout%%'' to a file        |
 +|program|''%%2> err.log%%''|''%%2>> err.log%%''|redirect ''%%stderr%%'' to a file        |
 +|program|''%%2>&1%%''      |                   |redirect ''%%stderr%%'' to ''%%stdout%%''|
 +
 +
 +=== Pipes ===
 +
 +> Write **output** into the **input**-stream of another process
 +
 +^Command^Pipe                     ^Description                          ^
 +|program|''%%| grep -i foo%%''    |pipe ''%%stdout%%'' into ''%%grep%%''|
 +|program|''%%| tee file1 file2%%''|overwrite files and ''%%stdout%%''   |
 +|program|''%%| tee -a file%%''    |append to files and ''%%stdout%%''   |
 +
 +
 +====== Environment Variables ======
 +
 +==== Setting, getting and unsetting ====
 +
 +=== Set ===
 +
 +<code python>
 +LANG=de_AT.UTF-8 firefox
 +env LANG=de_AT.UTF-8 firefox
 +</code>
 +<code python>
 +export LANG=de_AT.UTF-8
 +</code>
 +
 +=== Get ===
 +
 +<code python>
 +env
 +echo ${LANG}
 +</code>
 +
 +=== Unset ===
 +
 +<code python>
 +unset LANG
 +env -u LANG
 +</code>
 +
 +==== Use cases ====
 +
 +> Some variables that could affect you are:
 +
 +<code python>
 +$EDITOR            # the default editor for the CLI
 +$PAGER             # utility to read long streams
 +$PATH              # program paths, in priority order
 +</code>
 +> if you’re aiming for programming, these could be more interresting:
 +
 +<code python>
 +$LIBRARY_PATH      # libraries to link by the compiler
 +$LD_LIBRARY_PATH   # libraries to link at runtime
 +$CC                # sometimes used to set default C compiler
 +$CFLAGS            # default flags for compiling C
 +</code>
 +
 +====== Core Utilities ======
 +
 +==== Looking around ====
 +
 +> **looking around** is done by the **''%%ls%%''** (list) command
 +
 +<code>
 +$ ls      # shows files and directories
 +
 +$ ls -a   # includes hidden ones
 +
 +$ ls -l   # detailed view
 +  
 +</code>
 +<code>
 +testdir  test
 +
 +.  ..  testdir  test
 +
 +drwxr-xr-x 1 myuser p12345 0 Apr 13 11:55 testdir
 +-rw-r--r-- 1 myuser p12345 4 Apr 13 11:55 test
 +</code>
 +
 +{{..:linux_directories.jpg?0x400}}
 +
 +^command                                                result^
 +|**''%%ls /%%''**      |  ''%%bin  dev  etc  home  sbin  usr%%''|
 +|**''%%ls /home%%''**  |                    ''%%user1  user2%%''|
 +
 +
 +==== Moving around ====
 +
 +> **moving around** is done by the **''%%cd%%''** (change directory) command
 +
 +> **finding** the current **position** in the filesystem is done by the **''%%pwd%%''** (print working directory) command
 +
 +
 +<code>
 +$ cd /bin   # go to an absolute directory
 +
 +$ cd [~]    # go home
 +
 +$ cd -      # go to previous directory
 +</code>
 +<code>
 +$ pwd   # /bin
 +
 +$ pwd   # /home/user1
 +
 +$ pwd   # /bin
 +</code>
 +
 +{{..:linux_directories.jpg?0x400}}
 +
 +==== Copying & moving files around ====
 +
 +> **moving files around** is done by the **''%%mv%%''** (move) command
 +
 +> **copying** is done by the **''%%cp%%''** (copy) command
 +
 +
 +<code>
 +$ mv old   new     # rename old to new
 +
 +$ mv old   dir/    # move old into dir
 +
 +$ mv file1 file2   # overwrite file2 with file1
 +                   # (BEWARE)
 +</code>
 +<code>
 +$ cp -i input input.bak   # input to input.bak
 +
 +$ cp -i input backup/     # input into backup
 +
 +$ cp -a dir1/ dir2        # exact copy of dir1
 +  
 +</code>
 +
 +{{..:screenshot_mv.png?0x350}} {{..:screenshot_cp.png?0x350}}
 +
 +==== Finding stuff ====
 +
 +> To look at everything
 +
 +
 +  - in your home directory
 +  - and nested up to three levels deep inside it
 +  - that ends in ''%%.txt%%''
 +  - or starts with ''%%log_%%''
 +  - and is an ordinary file
 +  - concatenated as one stream:
 +
 +<code bash>
 +find \
 +  ~ \
 +  -maxdepth 3 \
 +  -iname "*.txt" \
 +  -or -iname "log_*" \
 +  -type f \
 +  -exec cat '{}' \; | less
 +</code>
 +<code bash>
 +find \
 +  ~ \
 +  -maxdepth 3 \
 +  -iname "*.txt" \
 +  -or -iname "log_*" \
 +  -type f \
 +  -exec cat '{}' +  | less
 +</code>
 +
 +==== Creating and deleting ====
 +
 +=== directories ===
 +
 +> **creating directories** is done by the **''%%mkdir%%''** (make directory) command:
 +
 +> **deleting directories** is done by the **''%%rmdir%%''** (remove directory) command:
 +
 +
 +<code bash>
 +$ mkdir newdir       # create directory newdir
 +
 +$ mkdir -p new/dir   # create dir inside of new
 +</code>
 +<code>
 +$ rmdir emptydir      # removes emptydir
 +
 +$ rmdir -v emptydir   # removes emptydir (verbose)
 +</code>
 +
 +=== files ===
 +
 +> **creating files** is done by the **''%%touch%%''** or **''%%echo%%''** commands:
 +
 +> **deleting files** is done by the **''%%rm%%''** (remove) command:
 +
 +
 +<code bash>
 +$ touch file          # create empty file
 +
 +$ echo "abc" > file   # create/overwrite file
 +</code>
 +<code>
 +$ rm file      # removes file
 +
 +$ rm -ri dir   # removes dir recursively
 +</code>
 +
 +==== Contents of files ====
 +
 +> **viewing** is done by the **''%%less%%''** command:
 +
 +> **concatenating** is done by the **''%%cat%%''** command:
 +
 +
 +<code>
 +$ less file.txt      # exit with 'q'
 +
 +$ less -R file.txt   # keep colors
 +
 +$ cat file1 file2 | less
 +</code>
 +<code>
 +$ cat file
 +
 +$ cat -A printable
 +
 +$ cat -n numbered
 +</code>
 +
 +<code>
 +$ echo "VSC is great" > file
 +$ cat file
 +VSC is great
 +</code>
 +<code>
 +$ echo "VSC is awesome" >> file
 +$ cat file
 +VSC is great
 +VSC is awesome
 +</code>
 +<code>
 +$ cat file | grep awesome
 +VSC is awesome
 +</code>
 +<code>
 +$ grep awesome file
 +VSC is awesome
 +</code>
 +
 +==== Space accounting ====
 +
 +> viewing **used space** is done by the **''%%du%%''** (disk usage) command:
 +
 +> viewing **free space** is done by the **''%%df%%''** (disk free) command:
 +
 +
 +<code>
 +$ du -h file1 file2   # human readable output
 +
 +$ du -s dir           # summarize
 +</code>
 +<code>
 +$ df -h       # human readable output
 +
 +$ df -t nfs   # only list filesystems of a type
 +</code>
 +
 +{{..:screenshot_df.png?0x440}}
 +
 +==== Recap ====
 +
 +<code>
 +$ mv space.log space.log.bak
 +</code>
 +<code>
 +$ df -h | grep "lv12345\|lv54321" > space.log
 +</code>
 +<code>
 +$ cat space.log
 +</code>
 +<code>
 +nfs05.ib.cluster:/e/lv12345    200G  185G   16G  93% /home/lv12345
 +nfs04.ib.cluster:/e/lv54321   1000G  979G   22G  98% /home/lv54321
 +</code>
 +> we do this often, let’s wrap it up!
 +
 +
 +==== Recap++ ====
 +
 +<code>
 +$ mv space.log space.log.bak
 +</code>
 +<code>
 +$ df -h | grep "lv12345\|lv54321" > space.log
 +</code>
 +<code>
 +$ cat space.log
 +</code>
 +<code>
 +nfs05.ib.cluster:/e/lv12345    200G  185G   16G  93% /home/lv12345
 +nfs04.ib.cluster:/e/lv54321   1000G  979G   22G  98% /home/lv54321
 +</code>
 +> we do this often, let’s wrap it up!
 +
 +
 +<code>
 +$ echo '#!/bin/bash' > spacelog.sh
 +
 +$ echo 'mv space.log space.log.bak' >> spacelog.sh
 +
 +$ echo 'df -h | grep "lv12345\|lv54321" > space.log' >> spacelog.sh
 +
 +$ echo 'cat space.log' >> spacelog.sh
 +</code>
 +<code>
 +$ chmod +x spacelog.sh
 +</code>
 +<code>
 +$ ./spacelog.sh
 +</code>
 +
 +==== Sed and awk ====
 +
 +> **sed** (stream editor) and **awk** are powerful tools when working with the command line
 +
 +<code>
 +$ mycommand | sed "..."
 +</code>
 +<code>
 +$ mycommand | awk '{...}'
 +</code>
 +
 +> Using sed and awk in action
 +
 +^program  ^             command              ^description                                                                ^
 +|sed      |        ''%%s/old/new/%%''        |replace ''%%old%%'' with ''%%new%%''                                       |
 +|sed      |  ''%%/from/,/to/ s/old/new/%%''  |replace ''%%old%%'' with ''%%new%%'', between ''%%from%%'' and ''%%to%%''  |
 +|awk      |      ''%%'print $5, $3'%%''      |print columns 5 and 3 of every line                                        |
 +
 +Example script:
 +
 +
 +<code bash>
 +#!/bin/bash
 +
 +mv space.log space.log.bak
 +df -h | grep "lv12345\|lv54321" > space.log
 +cat space.log
 +
 +  
 +</code>
 +<code bash>
 +#!/bin/bash
 +
 +mv space.log space.log.bak
 +df -h | grep "lv12345\|lv54321" > space.log
 +cat space.log | sed "s|/home/lv12345|ProjectA|" \
 +              | awk '{print $6, "free:", $4}' \
 +              | column -t
 +</code>
 +
 +====== Scripting ======
 +
 +==== Ownership and Permissions ====
 +
 +> Just to ensure, that you are able to run your scripts
 +
 +=== chown ===
 +
 +> Change the owner of files and directories by:
 +
 +<code bash>
 +chown -R user:group dirs files
 +</code>
 +=== chmod ===
 +
 +> Change the mode of files and directories by:
 +
 +<code bash>
 +chmod -R u=rw,g+w,o-rwx dirs files
 +chmod -R 644            dirs files
 +</code>
 +==== Shebang ====
 +
 +A little test program, which we mark as executable and hand it over to the corresponding interpreter:
 +
 +<code bash>
 +cat << EOF > test.sh
 +
 +echo "${LANG}"
 +echo "${PATH}"
 +EOF
 +</code>
 +<code bash>
 +chmod +x test.sh
 +</code>
 +<code bash>
 +bash test.sh
 +</code>
 +
 +> Don’t we have an OS, capable of executing everything it recognises as a executable?
 +
 +> Yes, we do!
 +
 +<code bash>
 +cat << EOF > test.sh
 +#!/bin/bash
 +echo "${LANG}"
 +echo "${PATH}"
 +EOF
 +</code>
 +<code bash>
 +chmod +x test.sh
 +</code>
 +<code bash>
 +./test.sh
 +</code>
 +
 +
 +==== Functions (more like Procedures) ====
 +
 +Programming in bash would be cumbersome without functions, so here we go:
 +
 +<code bash>
 +printAllNumbers () {
 +echo "1 2 3"
 +echo "done"
 +}
 +</code>
 +=== Arguments ===
 +
 +> This isn’t good, as were only getting a fixed amount of numbers. let’s try a recursive approach
 +
 +<code bash>
 +#!/bin/bash
 +
 +printAllNumbersFrom () {
 +num=$1
 +echo "${num}"
 +printAllNumbersFrom "$(($num + 1))"
 +}
 +
 +printAllNumbersFrom $2
 +</code>
 +
 +====== Editors ======
 +
 +==== General ====
 +
 +
 +  * Many different editors
 +    * Unique (dis-)advantages
 +    * Different look and feel
 +  * Editors should provide us with
 +    - Simple text editing
 +    - Copy and paste
 +    - Search and replace
 +    - Saving changes
 +    - Wide availability
 +
 +=== Two editors that satisfy our needs: ===
 +
 +
 +  * nano
 +  * vim
 +
 +=== Common starting point ===
 +
 +<code bash>
 +nano filename
 +</code>
 +<code bash>
 +vim filename
 +</code>
 +
 +
 +==== Nano ====
 +
 +{{..:screenshot_nano.png}}
 +
 +==== Nano explained ====
 +
 +> This editor is focused on being easy to use, but still providing every feature a user might need.
 +
 +=== Interface ===
 +
 +> The interface consists of four parts, namely from top to bottom:
 +
 +
 +  * Title bar
 +  * Text area
 +  * Command line
 +  * Key bindings
 +
 +=== Usage ===
 +
 +> Nothing special, key-bindings visible while editing
 +
 +^Feature                       Usage                 ^
 +|Navigation    |              Arrow keys               |
 +|Actual editing|         Typing text, as usual         |
 +|Cut/Paste line|  ''%%<CTRL>%%''+k / ''%%<CTRL>%%''+u  |
 +|…                explained in key bindings field    |
 +
 +
 +
 +=== Short ===
 +
 +> Use this editor if you are new to the command line.\\
 +> It is straight forward, but can be extended on the way.
 +
 +  * Auto-indentation
 +  * Syntax highlighting
 +  * Multi-buffer
 +
 +
 +{{..:screenshot_nano.png?0x300}}
 +
 +==== Vi(m) ====
 +
 +{{..:screenshot_vim.png}}
 +
 +==== Vi(m) explained ====
 +
 +> This editor is focused on productivity and efficiency, providing everything a user might need.
 +
 +=== Interface ===
 +
 +> The simple interface consists of two parts:
 +
 +
 +
 +  * Text area
 +  * Command line
 +
 +> Since this editor is very easy to extend, after setting up a few plugins, it will probably look quite different!
 +
 +
 +=== Usage ===
 +
 +> This is a multimode editor, you’ll have to switch modes whenever you change what you want to do.
 +
 +^Feature                      Usage                    ^
 +|Navigation|              hjkl / Arrow keys              |
 +|Writing    change to input mode, then write as usual  |
 +|Commands  |     exit current mode, press ''%%:%%''      |
 +|…                   explained on next slide           |
 +
 +
 +
 +=== Short ===
 +
 +> Use this editor if you like a challenge.\\
 +> It is fast and very nice — but you’ll sometimes get hurt on the way.
 +
 +  * Auto-indentation, Syntax highlighting, Multi-buffer – just like nano
 +  * File/Project Management
 +  * Use a plugin manager
 +
 +
 +{{..:screenshot_vim.png?0x300}}
 +
 +==== Vi(m) modes and keys ====
 +
 +  * any mode:
 +    * back to the **default mode**: **''%%<ESC>%%''**
 +  * command mode (followed by ''%%<RETURN>%%''):
 +    * **save** current file: **''%%w [filename]%%''**
 +    * **quit** the editor
 +      * after saving: **''%%q%%''**
 +      * without confirmation: **''%%q!%%''**
 +    * **help**: **''%%h [topic]%%''**, e.g. **''%%h tutorial%%''**
 +    * **search and replace**: **''%%%s/old/new/gc%%''**
 +  * //default mode//:
 +    * enter **input mode**: **''%%i%%''**
 +    * enter **command mode**: **''%%:%%''** //(colon)//
 +    * **mark**
 +      * **character**-wise: **''%%v%%''**
 +      * **line**-wise: **''%%<SHIFT>-v%%''**
 +    * **delete**
 +      * **character**-wise: **''%%x%%''**
 +      * **line**-wise: **''%%dd%%''**
 +      * **marked content**: **''%%d%%''**
 +    * **search**: **''%%/abc%%''**
 +
 +
 +====== QED ======
 +
 +==== Fin ====
 +
 +{{..:tux_banner.png?0x600}}
 +
 +
 +  * Questions?
 +  * Need help installing Linux?
 +  * Contact: <Email>
 +
 +<HTML>
 +<!--
 +
 +> Public domain:
 +
 +- [Ubuntu Tango Icons](https://wiki.ubuntuusers.de/Wiki/Icons/Tango/)
 +
 +> Mixed (CC, public domain, ...)
 +
 +- [Wikipedia Tango Icons](https://commons.wikimedia.org/wiki/Tango_icons)
 +
 +
 +# Timers
 +## Systemd Units
 +## Systemd Timers
 +## Putting it together
 +
 +
 +
 +
 +## Background
 +### What is GNU/Linux {.incremental}
 +
 +![](pictures/gnu.svg){height=200px style="float: left;" .incremental}
 +<div class="columns">
 +> - It's a collection of Software, including:
 +>     - **Linux** itself, as the `kernel`
 +>     - basic tools, mostly from the **GNU** project
 +>     - additional, specialised software for most tasks
 +> - the **OS** that powers the internet, the mobile world and more
 +> - **opensource** and **free software** effort to retain essential freedoms of the user
 +</div>
 +
 +### Reasons for Linux {.incremental}
 +
 +![](pictures/tux.svg){height=200px style="float: right;".incremental}
 +<div class="columns">
 +> - Wide Adoption
 +> - Stable platform
 +> - Anyone can try and use Linux
 +> - Easy adaption to custom environments
 +</div>
 +
 +### Basic Usage {.incremental}
 +
 +![](https://upload.wikimedia.org/wikipedia/commons/6/66/Utilities-terminal.svg){height=200px style="float: left;" .incremental}
 +<div class="columns">
 +> - using and understanding the shell
 +> - moving through the directory structure
 +> - moving files and directories
 +> - editing files
 +> - writing and executing scripts
 +> - environment and variables
 +</div>
 +
 +
 +
 +## CLI Basics
 +<div class="columns">
 +### Shell
 +
 +> The shell used on the VSC is called **bash**. It's the most common shell today, preinstalled by most Linux and macOS versions.
 +
 +> - It's behaviour is controlled by the file **.bashrc**
 +>     - loaded at login
 +>     - can be customized
 +>     - is a shell script
 +> - Has some usability features
 +>     - most of the time commands can be auto-completed by pressing `TAB`
 +>     - can recall history with `CTRL-R` or `history`
 +> - Provides a full programming language/environment
 +>     - can do native processing of Strings, Integers and Arrays
 +>     - can execute any program
 +>     - is turing-complete, but not considered a general-purpose language
 +
 +### Environment
 +
 +> The shell and some default tools are configured through **variables**.
 +
 +> - list variables with `env`
 +> - modify them with `export X=Y`
 +> - get the content by prepending it with a dollar sign `$A`
 +> - useful in scripts
 +
 +<div class="incremental">
 +Some variables that could affect you are:
 +
 +```
 +$EDITOR # the default editor for the CLI
 +$PAGER  # utility to read long streams
 +$PATH   # program paths, in priority order
 +```
 +</div>
 +</div>
 +
 +
 +
 +## Executing Commands
 +
 +<div class="columns">
 +<div class="incremental">
 +To execute a program, we call it:
 +
 +```{.bash}
 +nano file.txt
 +./a.out -flag=value
 +gcc -I/home/lv12345/myuser/include
 +module load non-existent-module
 +```
 +</div>
 +
 +> - Every command that is executed will provide a return-value on exit.
 +>     - ![](https://media-cdn.ubuntu-de.org/wiki/attachments/21/07/dialog-ok.png){height=30px} A value of `0` means success 
 +>     - ![](https://media-cdn.ubuntu-de.org/wiki/attachments/39/07/dialog-cancel.png){height=30px} any other value means failure 
 +</div>
 +
 +### Program Arguments {.incremental}
 +
 +<div class="incremental">
 +> The default way to apply arguments to a program is to write a space separated list of arguments after the program when calling it.
 +
 +> Patterns and expansions are defining multiple arguments with little overhead:
 +</div>
 +<div class="columns">
 +> - the most important patterns are:
 +>     - **?** --- matches one character
 +>     - **\*** --- matches any character sequence
 +> - the most important expansions are:
 +>     - **A{1,2}Z**  --- expands to A1Z A2Z
 +>     - **1{a..z}A**     --- expands to 1aA 1bA ... 1zA
 +</div>
 +
 +### Chaining Commands {.incremental}
 +
 +<div class="columns incremental">
 +``` {.bash .numberLines}
 +false ;  echo "Should I be Printed?"
 +false && echo "Should I be Printed?"
 +false || echo "Should I be Printed?"
 +```
 +
 +``` {.bash .numberLines}
 +Should I be Printed?
 +
 +Should I be Printed?
 +```
 +</div>
 +
 +<!--
 +
 +- - -
 +
 +### Using Patterns and Expansions {.incremental}
 +
 +<div class="columns">
 +> - the most important patterns are:
 +>     - **?** --- matches one character
 +>     - **\*** --- matches any character sequence
 +> - the most important expansions are:
 +>     - **A{1,2}Z**  --- expands to A1Z A2Z
 +>     - **1{a..z}A**     --- expands to 1aA 1bA ... 1zA
 +</div>
 +-->
 +<!--
 +## Loops
 +
 +> look like this in a shell:
 +
 +<div class="columns incremental">
 +```{.bash}
 +for i in *
 +do
 +  mv $i{,.bak}
 +done
 +```
 +
 +```{.bash}
 +while true
 +do
 +  echo "Annoying Hello World"
 +  sleep 3
 +done
 +```
 +</div>
 +
 +#### Generating parameters in bash {.incremental}
 +
 +<div class="columns incremental">
 +```{.bash}
 +i=1
 +while [ $i -lt 10 ]
 +do
 +  j=1
 +  while [ $j -le $i ]
 +  do
 +    echo "$i $j" # Your command comes here
 +    let j+=1
 +  done
 +  let i+=1
 +done
 +unset i j
 +```
 +
 +```{.bash}
 +1 1
 +2 1
 +2 2
 +3 1
 +3 2
 +3 3
 +...
 +9 6
 +9 7
 +9 8
 +9 9
 +```
 +
 +</div>
 +
 +
 +## Getting familiar with the CLI
 +
 +<div class="columns">
 +
 +> This is how the prompt looks by default:
 +
 +``` {.bash}
 +[myname@l3_ ~]$ 
 +```
 +
 +![](src/Screenshot_vsc3_login.png){height=190px .incremental}
 +![](src/Screenshot_vsc3_login_prompt.png){height=190px .incremental}
 +![](src/Screenshot_vsc3_login_prompt2.png){height=190px .incremental}
 +</div>
 +
 +<div style="float: left">
 +
 +#### Prompt {.incremental}
 +
 +> - it's "promting" you to enter a command
 +> - tells you:
 +>     - who you are
 +>     - which computer you're on
 +>     - which directory you're in
 +> - can be configured to show more information
 +>     - setting is in a variable called `PS1`
 +>     - default value of `$PS1`: `[\u@\h \w]\$`
 +>     - can add date, time and the output of custom commands
 +> - see `man bash` for details about your shell
 +
 +</div>
 +
 +<div style="float: right" class="incremental">
 +#### In case of confusion
 +
 +> there are many ways of getting further:
 +
 +> - the `man` command
 +>     - opens the manual for a specified topic
 +>     - especially nice to find man-pages: `apropos`
 +> - the `whatis` command
 +> - the `info` command
 +>     - is more complex and more like a browser
 +> - most commands support a `-h`/`--help` flag
 +> - colleagues are often helpful
 +> - the internet is often helpful
 +</div>
 +
 +<div style="float: left" class="incremental">
 +![`apropos help`](src/Screenshot_vsc3_apropos.png){width=270px .incremental}
 +</div>
 +
 +## Shell - More than a prompt
 +
 +### Simple Input/Output
 +
 +<div class="columns incremental">
 +Any program running in the shell can by default read from the keyboard and write to the screen.
 +
 +We try this with the `cat` command, which is short for concatenate.
 +
 +```{.bash}
 +cat
 +some text   # Input
 +some text   # Output    
 +<CTRL>-d
 +```
 +</div>
 +
 +### Piping Input/Output
 +
 +<div class="columns incremental">
 +Since echoing input back to the user isn't particularly useful, we'll be echoing back filtered output.
 +
 +We add a pipe, which redirects the output of the command before it to the command following it, and filter for any line containing `VSC`.
 +
 +```{.bash}
 +cat | grep VSC
 +1  2  3   # Input, no Output
 +1 VSC 3   # Input
 +1 VSC 3   # Output
 +<CTRL>-d
 +```
 +</div>
 +
 +### Redirecting Input/Output
 +
 +<div class="columns incremental">
 +Now we can write the resulting stream into a file called VSC.txt.
 +
 +This is done by adding a greater-sign between the end of the command and the filename of the resulting document.
 +
 +```{.bash}
 +cat | grep VSC > VSC.txt
 +1  2  3   # Will not be in VSC.txt
 +1 VSC 3   # Will be in VSC.txt
 +<CTRL>-d
 +```
 +<!--
 +```{.bash}
 +cat VSC.txt
 +1 VSC 3
 +```
 +-->
 +<!--
 +</div>
 +
 +## Cool Tools
 +
 +### Less is More
 +
 +<div class="columns">
 +Since piping output is awesome we want to have a program, that can handle all the output of the other programs and present it in screen-sized parts.
 +
 +Hence the program `less`, which is just `more`.
 +
 +```{.bash}
 +find ~ -iname "*.txt" -type f -exec cat '{}' \; | less
 +```
 +</div>
 +
 +### Permissions
 +
 +<div class="columns">
 +If some file can't be accessed/modified, there is a chance that you haven't got the rights to do so.
 +
 +For these cases there is the tool `chmod`, which changes the mode of a file.
 +
 +```{.bash}
 +chmod a=w file
 +chmod u+x file
 +chmod g-w file
 +chmod o=r file
 +```
 +</div>
 +
 +### Aliasing commands
 +
 +<div class="columns">
 +you should define an `alias`, whenever there is a command which:
 +
 +- is run with parameters which don't change between runs
 +- has some hard to remember name
 +- needs a non-default safeguard 
 +
 +```{.bash}
 +alias f='find ~ -type f -iname'
 +alias rm='rm -i'
 +alias def-env='cd ~/projectA && \
 +               module purge && \
 +               module load x/1 y/2 z/3'
 +```
 +</div>
 +
 +## Filesystem 101
 +
 +<div class="columns">
 +> 1. Everything starts at the root
 +>     - the root is a directory
 +>     - `/` denotes the root directory
 +> 2. the filesystem has different kinds of objects
 +>     1. files
 +>     2. directories
 +>         - containers for multiple objects
 +>     3. links to objects, which either
 +>         - add a second name for the same object
 +>         - point to a position in the filesystem
 +> 3. objects can be referenced by their path
 +>     - absolute: `/dir1/dir2/object`
 +>     - relative: `dir2/object`
 +> 4. special objects in directories:
 +>     - `.` --- is a reference to the directory itself
 +>     - `..` --- is a reference to the parent directory
 +</div>
 +
 +##### The Filesystem represented as an ADT {.incremental}
 +<div class="incremental columns">
 +
 +> These are the helper types:
 +
 +```{.haskell}
 +type Name     = String
 +type Children = [FSObject]
 +type Path     = [FSObject]
 +data LinkType = Soft | Hard
 +```
 +
 +> This is the code describing the trivial Linux-ish filesystem in a mathematical notation:
 +
 +```{.haskell}
 +data FSObject = File Name
 +              | Dir  Name Children
 +              | Link Name Path LinkType
 +```
 +</div>
 +
 +- - -
 +
 +### Getting and Manipulating our position in the FS
 +
 +<div class="columns">
 +> - `'ls'`: short for `'list'`
 +> - `'tree'`: tree view of files and directories
 +> - `'pwd'`: short for `'print work directory'`
 +> - `'cd'`: short for `'change directory'`
 +</div>
 +
 +<div class="incremental">
 +> Example for **looking around**:
 +
 +<div class="columns incremental">
 +```
 +$ ls          # shows files and directories
 +
 +$ ls -a       # includes hidden ones
 +
 +$ ls -l       # detailed view
 +```
 +
 +```
 +testdir  testfile
 +
 +.  ..  testdir  testfile
 +
 +drwxr-xr-x 1 myuser p12345 0 Apr 13 11:55 testdir
 +-rw-r--r-- 1 myuser p12345 4 Apr 13 11:55 testfile
 +```
 +</div>
 +</div>
 +
 +<div class="incremental">
 +> Example for **moving around**, starting at the home directory:
 +</div>
 +<div class="columns incremental">
 +```
 +$ pwd
 +
 +$ cd ..; pwd
 +
 +$ cd /global; pwd
 +
 +$ cd; pwd
 +
 +$ cd -; pwd
 +```
 +
 +```
 +/home/lv12345/myname
 +
 +/home/lv12345
 +
 +/global
 +
 +/home/lv12345/myname
 +
 +/global
 +```
 +</div>
 +
 +- - -
 +
 +### Moving files and directories
 +
 +<div class="columns">
 +> - `'mv'`: short for `'move'`
 +> - `'cp'`: short for `'copy'`
 +</div>
 +<div class="incremental">
 +> Example:
 +
 +<div class="columns">
 +```
 +$ cp file.orig file.copy
 +$ mv file.orig file.copy
 +
 +$ cp file.orig dir
 +$ mv file.orig dir
 +
 +$ cp dir1 dir2    # ERROR
 +$ cp -r dir1 dir2 # WORKS
 +$ mv dir1 dir2    # WORKS
 +```
 +
 +> - copying or moving directories can result in unexpected behaviour:
 +>     - if dir2 is an existing directory, dir1 will become a subdirectory of dir2
 +>     - if dir2 is non-existent, dir1 will become dir2
 +
 +</div>
 +
 +<div class="columns">
 +##### Transfer to/from Cluster
 +
 +> - `'scp'`: short for `'secure copy'`
 +>     - arguments can be either local or remote files and direcotries
 +> - `'rsync'`: short for `'remote syncronization'`
 +>     - similar to regular `mv`/`cp`/`scp`, except:
 +>         - pedantic about trailing slashes
 +>         - can merge directories
 +>         - can delete/overwrite changed files
 +</div>
 +
 +`rsync` seems easy to use, but the advanced options could remove data from both sides of your transfer! It's a **syncronazation** utility, no simple move/copy tool
 +</div>
 +
 +<!--
 +- - -
 +
 +##### SCP
 +
 +```
 +$ cp file.orig file.new
 +
 +$ scp user@host:dir/file.orig file.new
 +
 +$ scp file.orig user@host:dir/file.new
 +```
 +
 +- this is easy to use and preferable for little, one time transfers
 +
 +
 +[ID1]: src/tux.svg "Hover Text 1" {height=150px style="margin: 10px;"}
 +[ID2]: src/tux.svg "Hover Text 2"
 +
 +- ![Test][ID1]
 +- ![Test](src/tux.svg "Hover Text 1"){height=150px style="margin: 10px;"}
 +- [<font color=#cc3300>&#x27A0;</font>][ID2]
 +
 +
 +
 +
 +
 +- - -
 +
 +##### RSYNC
 +
 +```
 +$ rsync dir1 dir2        # Local copy
 +$ rsync -e ssh dir1 dir2 # Remote copy
 +
 +# These two have the same result:
 +$ rsync dir1  dir2
 +$ rsync dir1/ dir2/dir1
 +
 +# This doesn't:
 +$ rsync dir1/ dir2
 +```
 +
 +- there are many useful options, like:
 +    - `-a` $\rightarrow$ Archive mode, usually you want this
 +    - `-v` $\rightarrow$ Verbose mode
 +    - `-p` $\rightarrow$ Preserve permissions
 +    - `...` $\rightarrow$ RTFM or don't use those options   
 +- the `-e ssh` option tells rsync to connect via ssh to the remote machine
 +    - in this case the remote location is written like this:
 +         - user@host:remoteDir
 +- trailing slashes have an effect on where your files will go
 +
 +-->
 +<!--
 +## Copyleft
 +
 +Third party content and its licensing information can be found at:
 +
 +- [Levitating Gnu](https://www.gnu.org/graphics/meditate.html):
 +    - María del Pilar Saenz: [GFDL](https://www.gnu.org/copyleft/fdl.html)
 +- [Tux](https://pixabay.com/en/tux-penguin-animal-cute-linux-158547/):
 +    - [CC0 Public Domain](https://creativecommons.org/publicdomain/zero/1.0/deed.en)
 +- [Tux and Gnu](https://commons.wikimedia.org/wiki/File:GNU_and_Tux.svg):
 +    - Aurelio A. Heckert: [Free Art License](http://artlibre.org/licence/lal/en/)
 +    - Larry Ewing, Simon Budig and Anja Gerwinski: probably [CC-BY](https://creativecommons.org/licenses/by/4.0/)
 +- [Tango Icons](https://commons.wikimedia.org/wiki/Tango_icons ):
 +    - Public Domain
 +
 +## Copyright
 +
 +The not explicitly licensed or non freely-redistributable licensed stuff:
 +
 +- [the Good, the Bad & the Ugly](http://cinetropolis.net/wp-content/uploads/2013/10/the-good-the-bad-and-the-ugly-t-anderson-banner.jpg):
 +    - [Dunno]()
 +
 +## TestStuff
 +😁 $\rightarrow$ 😄  
 +😁 $\Rightarrow$ 😄  
 +😁 $\longrightarrow$ 😄  
 +😁 $\Longrightarrow$ 😄
 +
 +https://apps.timwhitlock.info/emoji/tables/unicode#emoji-modal
 +
 +[//]:#(Das ist ein Kommentar, dass nirgends aufscheint)
 +
 +[TuxAndGnu]: https://upload.wikimedia.org/wikipedia/commons/5/53/GNU_and_Tux.svg "Tux and Gnu"
 +[VSC]: https://vsc.ac.at "Vienna Scientific Cluster"
 +
 +## ToDo
 +
 +- nano/vi
 +- df -h .
 +- du -sh ./*
 +- cat | sort | uniq | tr | awk | sed 
 +- example 
 +
 +## table
 +
 +head1 | head2 | head3
 +:-----|:-----:|-----:
 +p1 | e11 | e12
 +p1 | e11 | e12e12
 +p1 | e12e12e12e12e12 | e12
 +
 +head1 | head2 | head3
 +:-----|:-----:|-----:
 +p1 | e11 | e12
 +p1 | e11 | e12e12
 +p1 | e12e12e12e12e12 | e12
 +
 +
 +
 +-->
 +</HTML>
  
  • pandoc/linux-wochen-wien/linux-101/linux-101.txt
  • Last modified: 2020/10/20 09:13
  • by pandoc