Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
pandoc:introduction-to-vsc:03_linux_primer:linuxprimer [2017/10/18 11:42] – Pandoc Auto-commit pandocpandoc:introduction-to-vsc:03_linux_primer:linuxprimer [2020/10/20 09:13] (current) – Pandoc Auto-commit pandoc
Line 1: Line 1:
 +
 ====== Linux Primer ====== ====== Linux Primer ======
  
   * Article written by Balázs Lengyel (VSC Team) <html><br></html>(last update 2017-10-11 by bl).   * Article written by Balázs Lengyel (VSC Team) <html><br></html>(last update 2017-10-11 by bl).
 +
 +==== Note ====
 +
 +Please use the ''%%Table of Contents%%'' or your browsers ''%%quick search%%'' to find what you’re looking for, as this document is auto-generated from a presentation and context may not always be recognizable without the corresponding talk.
 +
 +
 +
 +
 +
 +
 +----
 +
  
  
Line 7: Line 20:
  
 ==== Filesystem 101 ==== ==== Filesystem 101 ====
 +
 +The job of filesystems is to keep data in a structured way. Every filesystem has a filesystem root, directories and files. The filesystem root is a special object which acts as an entry-point for using that filesystem. All other objects (directories and files) are structured below the root.
 +
 +Two main concepts emerged for using more than one filesystem on a single machine:
 +
 +<HTML><ol style="list-style-type: decimal;"></HTML>
 +<HTML><li></HTML><HTML><p></HTML>Special handling (a.k.a Drive letters)<HTML></p></HTML>
 +<HTML><p></HTML>This is how Windows handles filesystems. It explicitly shows which drive you’re working on.<HTML></p></HTML><HTML></li></HTML>
 +<HTML><li></HTML><HTML><p></HTML>Virtual Filesystem<HTML></p></HTML>
 +<HTML><p></HTML>This is a more subtle approach, used by Linux, where you specify in the beginning (booting) which filesystem will be used for which part of the virtual filesystem.<HTML></p></HTML><HTML></li></HTML><HTML></ol></HTML>
 +
 +Linux’ virtual filesystem tree represents all the files and directories that are reachable from the system. The nice part is that you can work on a Linux machine and not care about whether your file is on the network or on a local filesystem. The main difference for users is the performance delivered by different filesystems.
 +
 +This is how the (virtual) filesystem looks on Linux:
 +
  
  
   - Everything starts at the root   - Everything starts at the root
     * the root is a directory     * the root is a directory
-    * "''%%/%%''denotes the root directory+    * ''%%/%%''” denotes the root directory
   - the filesystem has different kinds of objects   - the filesystem has different kinds of objects
     - files     - files
Line 23: Line 51:
     * relative: ''%%dir2/object%%''     * relative: ''%%dir2/object%%''
   - special objects in directories:   - special objects in directories:
-    * ''%%.%%'' --- is a reference to the directory itself +    * ''%%.%%''” is a reference to the directory itself 
-    * ''%%..%%'' --- is a reference to the parent directory+    * ''%%..%%''” is a reference to the parent directory
   - the system may consist of multiple filesystems   - the system may consist of multiple filesystems
     * filesystems may be mounted at any (empty) directory     * filesystems may be mounted at any (empty) directory
Line 30: Line 58:
  
 ==== Further concepts ==== ==== Further concepts ====
 +
 +Next to basic storage and organization of data filesystems have different properties and functionality. Most filesystems provide a way to store and access attributes, different kinds of special files and some filesystems provide various advanced features:
  
   * Attributes   * Attributes
Line 47: Line 77:
   * Advanced FS features   * Advanced FS features
     * data integrity     * data integrity
-    * device managment +    * device management 
-    * subvolume support+    * sub-volume support 
  
  
Line 54: Line 85:
 ==== Filesystem tree ==== ==== Filesystem tree ====
  
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:linux_directories.jpg}}+{{.:linux_directories.jpg}}
  
-<HTML> 
-<!-- 
-### Mountpoints 
- 
-> Linux presents only one tree for file access - but every filesystems has its own tree! 
- 
-#### Attaching a filesystem to the current tree: {.incremental} 
- 
-> - takes a filesystem 
->     - **`FS`** usually a device file or a network address 
-> - takes an empty directory in the current tree 
->     - usually `/mnt/XXX` or `/mnt` 
-> - makes the root of the new filesystem available in place of the empty directory 
->     - **`FS`** `-> /mnt/` 
- 
-<div class=incremental> 
-> This process is called mounting. 
-</div> 
---> 
-</HTML> 
 ==== Special filesystems used on VSC ==== ==== Special filesystems used on VSC ====
  
Line 91: Line 102:
   * uses RAM instead of other media   * uses RAM instead of other media
   * lost at shutdown   * lost at shutdown
 +
 +
 +The home directory of the user is located on an NFS filesystem, which ensures that all parts of the cluster have a consistent view of files.
 +
 +The filesystem behind the ''%%$SCRATCH%%'' variable is located on a tmpfs filesystem, which is a double-edged sword. On one hand it’s fast, but since it uses RAM as a storage device it does limit the amount of memory available for programs. Also you can only use data stored in a tmpfs only on the host itself.
  
  
Line 99: Line 115:
 > This is how the prompt looks by default: > This is how the prompt looks by default:
  
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:tango_terminal.svg?0x100}+<HTML> 
 +<!-- ![](pictures/tango_terminal.svg){height=100px style="float: left; padding-left: 2em;" .slidy}--> 
 +</HTML>
 <code bash> <code bash>
 [myname@l3_ ~]$ [myname@l3_ ~]$
Line 108: Line 125:
   * tells you:   * tells you:
     * who you are     * who you are
-    * which computer you're on +    * which computer youre on 
-    * which directory you're in+    * which directory youre in
   * can be configured   * can be configured
     * variable ''%%$PS1%%''     * variable ''%%$PS1%%''
     * default: ''%%echo $PS1%%''     * default: ''%%echo $PS1%%''
  
-> Ways to get help when you're stuck:+> Ways to get help when youre stuck
 + 
 + 
 +Most of the time a command doesn’t act as expected, it shows an error message. From this point you have multiple approaches:
  
 +  * Think about why the program failed - maybe you (un-)intentionally tried to force the program to do something it’s not intended for?
 +  * Just copy that message into your favorite search engine and don’t forget to remove the parts that are specific to your environment (e.g. directory and user names).
 +  * Most programs supports a ''%%-h%%''/''%%--help%%'' flag
 +  * ''%%man <COMMAND>%%'' will be available for most programs too, if not ''%%man -K <KEYWORD>%%'' will search all man-pages that contain the keyword. (FYI: press ‘q’ to quit)
 +  * An alternative to ''%%man%%'' is ''%%info <COMMAND>%%'', which is like a browser from the ’80s
 +  * Ask colleagues for help
  
-  * ''%%man%%'' 
-    * manual for specified topic 
-    * find man-pages: ''%%apropos%%'' 
-  * ''%%whatis%%'' 
-  * ''%%info%%'' 
-  * most commands support a ''%%-h%%''/''%%--help%%'' flag 
-  * colleagues are often helpful 
-  * the internet is often helpful 
  
  
Line 134: Line 152:
 gcc FizzBuzz.c -o FizzBuzz gcc FizzBuzz.c -o FizzBuzz
 ./FizzBuzz ./FizzBuzz
-module load non-existent-module+false
 echo $? echo $?
 </code> </code>
  
-  * Every command that is executed will provide return-value on exit+ 
-    {{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:tango_ok.png?0x30}} A value of ''%%0%%'' means success +The examples show compiling a program, executing the result, trying to load a module on our cluster and checking if the previous command succeeded. 
-    * {{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:tango_cancel.png?0x30}} any other value means failure+ 
 +  * ''%%gcc%%'' is a program that is in a directory specified by the ''%%$PATH%%'' variable and will be found without specifying its exact location. 
 +  * ''%%./FizzBuzz%%'' is newly compiled executable, which is not found by looking at ''%%$PATH%%'', so we explicitly add ''%%./%%'', to show that we want to execute it from the current directory 
 +  ''%%module load non-existent-module%%'' fails, as the module command can’t find ''%%non-existent-module%%''Whenever a command fails, its ''%%return value%%'' is set to a value other than zero. The manual for some commands has a map from return-value to error-description to aid the user debugging. 
 +  * ''%%echo $?%%'' is a command that prints the return value of the previous command.
  
  
 === History === === History ===
  
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:tango_search.svg?0x100}}+ 
  
 > Your shell keeps a log of all the commands you executed. > Your shell keeps a log of all the commands you executed.
Line 156: Line 178:
 ==== Parameters ==== ==== Parameters ====
  
-For most commands you can combine multiple single-character arguments. This doesn't change the meaning of the parameters, but is limited to single-character arguments which dont take parameters.+> The default way to apply parameters to a program is to write a space separated list of parameters after the program when calling it. 
 + 
 +These parameters are either 
 + 
 +  - Single-character 
 +  - Multi-character 
 +  - Strings 
 + 
 +where some parameters also take additional arguments. 
 + 
 +== Combining parameters == 
 + 
 +For most commands you can combine multiple single-character parameters. This doesnt change the meaning of the parameters, but is limited to single-character parameters which don’t take extra arguments.
  
 <code bash> <code bash>
-# works: +COMMAND -j 2 -a -b -c 
-COMMAND -a -b -c -d -e +COMMAND -j 2 -abc 
-COMMAND -abcde+</code> 
 +== Ordering parameters ==
  
-# works partially: +One thing to look out for is the order of parameters. Most of the time no specific order is required, but you should look out for things like copying the target over the source file. Also watch out to keep parameters and their arguments together.
-COMMAND -j 2 -a -b +
-COMMAND -j 2 -ab+
  
-doesn't work: +<code bash> 
-COMMAND --long-parameter --flag-parameter=<FLAGVALUE>+COMMAND <SRC> <DEST>        OK 
 +COMMAND <DEST> <SRC>        # PROBABLY WRONG 
 +COMMAND -j 2 --color auto   # OK 
 +COMMAND -j auto --color 2   # PROBABLY WRONG
 </code> </code>
-One thing to look out for is the order of parameters. Most of the time no specific order is required, but you should look out for things like copying the target over the source file.+==== Escapes & Quotes ==== 
 + 
 +Whenever a parameter has to contain a character that is either unprintable or reserved for the shell, you can use: 
 + 
 +  - Backslash escape: 
 +    * Escapes a character, that would have a special meaning 
 +    * Can be used inside of quotes 
 +  - Double Quotes: 
 +    * Similar to escaping all whitespace characters 
 +  - Single Quotes: 
 +    * Additionally prevents expansion of variables
  
 <code bash> <code bash>
-# order does matter here: +COMMAND This\ is\ a\ single\ parameter 
-COMMAND <SRC> <DEST>+COMMAND "This is a single parameter" 
 +COMMAND 'This is a single parameter'
 </code> </code>
  
-==== Aliases ==== 
  
-Whenever you have a command 
  
  
-  - run often and it +==== Aliases ====
-    * has a long list of parameters you always use +
-    * is dangerous +
-  - which is an aggregate of many other commands +
-    * but you don't want to write a script+
  
-you can define an alias for it:+You can define aliases in your shell. These are usually used to shorten names for commands which are used often with a fixed set of parameters or where you have to be careful to get things right. These aliases are accessable as if they were commands.
  
-<code bash>+ 
 + 
 +<code>
 alias ll='ls -alh' alias ll='ls -alh'
-alias rm ='rm -i'+alias rm='rm -i'
 alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -' alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -'
 +</code>
 +
 +
 +After that, you can use the aliases synonymously.
 +
 +<code python>
 +ll        # Same as 'ls -alh'
 +rm        # Same as 'rm -i'
 +myProject # Same as 'cd $ProjectDir; testSuite; compile && testSuite; cd -'
 </code> </code>
  
 ==== Patterns ==== ==== Patterns ====
  
-Patterns and expansions define multiple arguments with little overhead:+ 
 +Patterns are an easy way of defining multiple arguments, which are mostly the same. The pattern will match anything in it’s place. 
 + 
 +The other concept is a expansion. In this case only defined patterns will be matched. 
  
  
   * the most important patterns are:   * the most important patterns are:
-    * **?** --- matches one character +    * **?** — matches one character 
-    * ***** --- matches any character sequence+    * ***** — matches any character sequence
   * the most important expansions are:   * the most important expansions are:
-    * **A{1,9}Z** --- expands to A1Z A9Z +    * **A{1,9}Z** — expands to A1Z A9Z 
-    * **A{1..9}Z** --- expands to A1Z A2Z ... A9Z+    * **A{1..9}Z** — expands to A1Z A2Z … A9Z 
 + 
 + 
 +You can try these commands and see what they doThese are all totally safe, even if you modify the arguments.
  
  
Line 225: Line 284:
 ==== Regular Expressions ==== ==== Regular Expressions ====
  
-> Used in many programs, to define matching rules for filters or other text transfomations:+Often you need to specify some string, but patterns and expansions aren’t enough, to cover all possibilities. In these cases you can use a regular expression also known as regex. These regexes are used by editors for search and replace, the ''%%egrep%%'' command for filtering through files and inside many scripts to validate parameters.
  
 <code python> <code python>
-.+ +.+                 # Match any character, once or more 
-\. +\.                 # match a dot 
-(A|a)p{2}le +(A|a)p{2}le        # apple, Apple 
-^[^aeiouAEIOU]+$ +^[^aeiouAEIOU]+$   # any line of only non-vowels
-</code> +
-<code> +
-Match any character, once or more +
-match a dot +
-apple, Apple +
-any line of only non-vowels+
 </code> </code>
  
-> Example usage of regex: 
  
-<code bash> +For a detailed explanation see [[https://en.wikipedia.org/wiki/Regular_expression|Wikipedia]], [[https://www.regular-expressions.info/|Regular-expressions.info]] or try [[https://regex101.com/|regex101]].
-egrep -i '(cluster|vsc)' <FILE> +
-</code> +
-For a detailed explanation see [[https://en.wikipedia.org/wiki/Regular_expression|Wikipedia]], or try [[https://regex101.com/|regex101]].+
  
 If you want to challenge yourself, try [[https://regexcrossword.com/|Regex Crossword]]! If you want to challenge yourself, try [[https://regexcrossword.com/|Regex Crossword]]!
- 
  
 ==== Control Flow ==== ==== Control Flow ====
 +
 +In the shell language there are a few ways to organize the execution path. The most important ones are:
 +
 +  - chaining of commands
 +  - looping constructs
 +  - conditionals (if/case)
 +
  
 === Chaining Commands === === Chaining Commands ===
 +
 +The simplest mechanism for control flow is to chain commands together in a simple ''%%if COMMAND then NEXTCOMMAND else ERRORCOMMAND%%''. Since this would be cumbersome to write, most shells provide simple syntax for this: ''%%COMMAND && NEXTCOMMAND || ERRORCOMMAND%%'' and if a command should be run without relying on the return value of its predecessor it’s written: ''%%COMMAND; NEXTCOMMAND%%''. And if you only want to execute further commands in one case (but not the other), you don’t even have to specify both branches.
 +
  
 <code bash> <code bash>
Line 266: Line 324:
  
 === Loops === === Loops ===
 +
 +The other way to execute commands conditionally are loops. You can loop over files, numeric arguments, until a either the loop condition is false or a break is encountered.
 +
  
 <code bash> <code bash>
Line 284: Line 345:
 for i in *; do mv $i{,.bak}; done for i in *; do mv $i{,.bak}; done
 while true; do echo "Annoying Hello World"; sleep 3; done while true; do echo "Annoying Hello World"; sleep 3; done
 +</code>
 +
 +
 +=== Conditionals ===
 +
 +
 +
 +''%%If%%'' is similar to the previous chaining of commands, except that it is more verbose and nicer to read if you have many commands to execute one branch of the decision. For more conditions the ''%%elif%%'' (else if) statement can be used. If you use a lot of ''%%elif%%''s and you only check one variable with them, you should consider using a ''%%case%%'' statement.
 +
 +
 +<code bash>
 +if [ $VARIABLE1 ]
 +then
 +  COMMAND1
 +elif [ $VARIABLE2 ]
 +  COMMAND2
 +else
 +  COMMAND3
 +fi
 +</code>
 +''%%Case%%'' statements are for querying all states of a single variable and making a decision based on that. It can match some simple expansions, which do NOT follow the general syntax of bash expansions. Also it processes alternative matches when seperated with ''%%|%%'' (pipe character).
 +
 +
 +<code bash>
 +case $VARIABLE in
 +[0-9] | [1-2][0-9])
 +  COMMAND1
 +  ;;
 +*)
 +  COMMAND2
 +  ;;
 +esac
 </code> </code>
  
Line 343: Line 436:
 $PATH              # program paths, in priority order $PATH              # program paths, in priority order
 </code> </code>
-> if you're aiming for programming, these could be more interesting:+> if youre aiming for programming, these could be more interesting:
  
 <code python> <code python>
Line 365: Line 458:
 </div> </div>
 </div> </div>
-</HTML> 
-===== Basic command line programs ===== 
- 
-==== Looking around ==== 
- 
-> Looking through files is done with the ''%%ls%%'' command under Linux, which is an abreviation for ''%%list%%''. 
- 
-By default (meaning without parameters) the ''%%ls%%'' command shows the content in the current directory. It does hide elements starting with a dot though. These hidden elements can be shown by adding the ''%%-a%%'' parameter, which stands for ''%%all%%''. If you want detailed information about the elements, add the ''%%-l%%'' parameter for ''%%long listing%%''. You can also specify a file/directory which you want to examine, by just appending the path to the command. 
- 
-<code> 
-$ ls      # shows files and directories 
-testdir  test 
- 
-$ ls -a   # includes hidden ones 
-.  ..  testdir  test 
- 
-$ ls -l   # detailed view 
-drwxr-xr-x 1 myuser p12345 0 Apr 13  2017 testdir 
--rw-r--r-- 1 myuser p12345 4 Apr 13 11:55 test 
- 
-$ ls /tmp/ 
-allinea-USERNAME 
-ssh-6E553lWZCM 
-ssh-C6a754pJ1d 
-systemd-private-a4214393983d448fbdc689791806519c-ntpd.service-LrAgBP 
-tmp7CJZRA 
-yum_save_tx.2017-04-03.12-07.VCUowf.yumtx 
- 
-$ ls -alh ~ 
-drwxr-xr-x  5 myuser mygroup   45 Jan 30  2017 .allinea 
--rw-------  1 myuser mygroup  21K May  3 17:14 .bash_history 
--rw-r-----  1 myuser mygroup  231 Dec  2  2016 .bashrc 
-drwx------  1 myuser mygroup   76 Aug 22  2017 Simulation 
-drwx------  2 myuser mygroup   76 Dec 12  2016 .ssh 
-</code> 
- 
-==== Moving around ==== 
- 
-> Moving through directories is done with the ''%%cd%%'' command, which stands for ''%%change directory%%''. 
- 
-The ''%%cd%%'' command can be called without any arguments, in which case it just switches to the home directory. Otherwise it takes a absolute (starting with a dash) or relative path as an argument and switches to that directory. The argument ''%%-%%'' (just a single dash) will cause cd to switch to the previous directory. This can be used to alternate between two directories without typing their path's every time. 
- 
-<code> 
-$ cd /bin   # go to an absolute directory 
- 
-$ cd [~]    # go home 
- 
-$ cd -      # go to previous directory 
-</code> 
- 
-==== Copying & moving files around ==== 
- 
-> Copying and moving files and directories is done with the ''%%cp%%'' and ''%%mv%%'' commands, which stand for ''%%copy%%'' and ''%%move%%'' respectively. 
- 
-Both commands take at least two parameters, which correspond to the ''%%<SOURCE>%%'' and ''%%<DESTINATION>%%'' files or directories. For ''%%cp%%'' to work with directories as a source, it needs the ''%%-r%%'' (recursive) or ''%%-a%%'' (archive) flag. 
- 
-> Beware of the pitfalls! you can overwrite data and therefor lose it, without getting any confirmation prompt! If in doubt use ''%%-i%%'' (interactive) flag. 
- 
-<code> 
-$ mv old   new     # rename old to new 
- 
-$ mv old   dir/    # move old into dir 
- 
-$ mv file1 file2   # overwrite file2 with file1 
-                   # (BEWARE) 
-</code> 
-<code> 
-$ cp -i input input.bak   # input to input.bak 
- 
-$ cp -i input backup/     # input into backup 
- 
-$ cp -a dir1/ dir2        # exact copy of dir1 
- 
-</code> 
- 
-==== Creating and deleting ==== 
- 
-=== directories === 
- 
-> creating directories is done with the ''%%mkdir%%'' command, which stands for ''%%make directory%%''. 
- 
-The ''%%mkdir%%'' command takes an optional ''%%-p%%'' (parents) parameter and a path. When optional parameter is given, it will create all the ancestors of the specified directory aswell. Otherwise the command fails if the directory either exists already or its parent doesn't exist either. 
- 
-> deleting directories is done with the ''%%rmdir%%'' command, which stands for ''%%remove directory%%''. 
- 
-This command removes the specified directory, but only if it's already empty. it can also take an optional ''%%-p%%'' parameter, in which case it removes the specified directory and all the ancestors you included. 
- 
- 
-==== Finding stuff ==== 
- 
-> To look at everything 
- 
- 
-  - in your home directory 
-  - and nested up to three levels deep inside it 
-  - that ends in ''%%.txt%%'' 
-  - or starts with ''%%log_%%'' 
-  - and is an ordinary file 
-  - concatenated as one stream: 
- 
-<code bash> 
-find \ 
-  ~ \ 
-  -maxdepth 3 \ 
-  -iname "*.txt" \ 
-  -or -iname "log_*" \ 
-  -type f \ 
-  -exec cat '{}' \; | less 
-</code> 
-<code bash> 
-find \ 
-  ~ \ 
-  -maxdepth 3 \ 
-  -iname "*.txt" \ 
-  -or -iname "log_*" \ 
-  -type f \ 
-  -exec cat '{}' +  | less 
-</code> 
- 
-==== Contents of files ==== 
- 
-> **viewing** is done by the **''%%less%%''** command: 
- 
-> **concatenating** is done by the **''%%cat%%''** command: 
- 
- 
-<code> 
-$ less file.txt      # exit with 'q' 
- 
-$ less -R file.txt   # keep colors 
- 
-$ cat file1 file2 | less 
-</code> 
-<code> 
-$ cat file 
- 
-$ cat -A printable 
- 
-$ cat -n numbered 
-</code> 
- 
-<code> 
-$ echo "VSC is great" > file 
-$ cat file 
-VSC is great 
-</code> 
-<code> 
-$ echo "VSC is awesome" >> file 
-$ cat file 
-VSC is great 
-VSC is awesome 
-</code> 
-<code> 
-$ cat file | grep awesome 
-VSC is awesome 
-</code> 
-<code> 
-$ grep awesome file 
-VSC is awesome 
-</code> 
- 
-==== Space accounting ==== 
- 
-> viewing **used space** is done by the **''%%du%%''** (disk usage) command: 
- 
-> viewing **free space** is done by the **''%%df%%''** (disk free) command: 
- 
- 
-<code> 
-$ du -h file1 file2   # human readable output 
- 
-$ du -s dir           # summarize 
-</code> 
-<code> 
-$ df -h       # human readable output 
- 
-$ df -t nfs   # only list filesystems of a type 
-</code> 
- 
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:screenshot_df.png?0x440}} 
- 
-==== Recap ==== 
- 
-<code> 
-$ mv space.log space.log.bak 
-</code> 
-<code> 
-$ df -h | grep "lv12345\|lv54321" > space.log 
-</code> 
-<code> 
-$ cat space.log 
-</code> 
-<code> 
-nfs05.ib.cluster:/e/lv12345    200G  185G   16G  93% /home/lv12345 
-nfs04.ib.cluster:/e/lv54321   1000G  979G   22G  98% /home/lv54321 
-</code> 
-> we do this often, let's wrap it up! 
- 
- 
-==== Recap++ ==== 
- 
-<code> 
-$ mv space.log space.log.bak 
-</code> 
-<code> 
-$ df -h | grep "lv12345\|lv54321" > space.log 
-</code> 
-<code> 
-$ cat space.log 
-</code> 
-<code> 
-nfs05.ib.cluster:/e/lv12345    200G  185G   16G  93% /home/lv12345 
-nfs04.ib.cluster:/e/lv54321   1000G  979G   22G  98% /home/lv54321 
-</code> 
-> we do this often, let's wrap it up! 
- 
- 
-<code> 
-$ echo '#!/bin/bash' > spacelog.sh 
- 
-$ echo 'mv space.log space.log.bak' >> spacelog.sh 
- 
-$ echo 'df -h | grep "lv12345\|lv54321" > space.log' >> spacelog.sh 
- 
-$ echo 'cat space.log' >> spacelog.sh 
-</code> 
-<code> 
-$ chmod +x spacelog.sh 
-</code> 
-<code> 
-$ ./spacelog.sh 
-</code> 
- 
-==== Sed and awk ==== 
- 
-> **sed** (stream editor) and **awk** are powerful tools when working with the command line 
- 
-<code> 
-$ mycommand | sed "..." 
-</code> 
-<code> 
-$ mycommand | awk '{...}' 
-</code> 
- 
-> Using sed and awk in action 
- 
-^program  ^             command              ^description                                                                ^ 
-|sed      |        ''%%s/old/new/%%''        |replace ''%%old%%'' with ''%%new%%''                                       | 
-|sed      |  ''%%/from/,/to/ s/old/new/%%''  |replace ''%%old%%'' with ''%%new%%'', between ''%%from%%'' and ''%%to%%''  | 
-|awk      |      ''%%'print $5 $3'%%''       |print columns 5 and 3 of every line                                        | 
- 
-Example script: 
- 
- 
-<code bash> 
-#!/bin/bash 
- 
-mv space.log space.log.bak 
-df -h | grep "lv12345\|lv54321" > space.log 
-cat space.log 
- 
- 
-</code> 
-<code bash> 
-#!/bin/bash 
- 
-mv space.log space.log.bak 
-df -h | grep "lv12345\|lv54321" > space.log 
-cat space.log | sed "s|/home/lv12345|ProjectA|" \ 
-              | awk '{print $6, "free:", $4}' \ 
-              | column -t 
-</code> 
- 
-<HTML> 
 <!-- <!--
-### Sort and uniq +## Pause Please?<br><br> &#8594; Get Coffee<br&#8594; Get Cookies {.slidy}
- +
-**sort** and **uniq** (unique) are used to sort and uniquify adjacent lines+
 --> -->
 </HTML> </HTML>
Line 656: Line 473:
  
 <code bash> <code bash>
 +# only works with root privileges
 +
 +chown user file
 chown -R user:group dirs files chown -R user:group dirs files
-# only works with root privilages 
 </code> </code>
 === chmod === === chmod ===
Line 669: Line 488:
 chmod 750                executables chmod 750                executables
 </code> </code>
 +
 +  * there are three bits for:
 +    * read (4)
 +    * write (2)
 +    * execute (1)
 +  * three times for:
 +    * user
 +    * group
 +    * other
 +
 +
 ==== Shebang ==== ==== Shebang ====
  
Line 687: Line 517:
 </code> </code>
  
-> Don't we have an OS, capable of executing everything it recognises as an executable?+> Dont we have an OS, capable of executing everything it recognizes as an executable?
  
 > Yes, we do! > Yes, we do!
Line 715: Line 545:
 } }
 </code> </code>
-> This isn't good, as were only getting a fixed amount of numbers. Let's try a recursive approach:+> This isnt good, as were only getting a fixed amount of numbers. Lets try a recursive approach:
  
 <code bash> <code bash>
Line 741: Line 571:
 allNumbersFromTo 1 10 allNumbersFromTo 1 10
 </code> </code>
- 
-===== Editors ===== 
- 
-==== General ==== 
- 
- 
-  * Many different editors 
-    * Unique (dis-)advantages 
-    * Different look and feel 
-  * Editors should provide us with 
-    - Simple text editing 
-    - Copy and paste 
-    - Search and replace 
-    - Saving changes 
-    - Wide availability 
- 
-=== Two editors that satisfy our needs: === 
- 
- 
-  * nano 
-  * vim 
- 
-=== Common starting point === 
- 
-<code bash> 
-nano filename 
-</code> 
-<code bash> 
-vim filename 
-</code> 
- 
- 
-==== Nano ==== 
- 
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:screenshot_nano.png}} 
- 
-==== Nano explained ==== 
- 
-> This editor is focused on being easy to use, but still providing every feature a user might need. 
- 
-=== Interface === 
- 
-> The interface consists of four parts, namely from top to bottom: 
- 
- 
-  * Title bar 
-  * Text area 
-  * Command line 
-  * Key bindings 
- 
-=== Usage === 
- 
-> Nothing special, key-bindings visible while editing 
- 
-^Feature                       Usage                 ^ 
-|Navigation    |              Arrow keys               | 
-|Actual editing|         Typing text, as usual         | 
-|Cut/Paste line|  ''%%<CTRL>%%''+k / ''%%<CTRL>%%''+u  | 
-|...              explained in key bindings field    | 
- 
- 
- 
-=== Short === 
- 
-> Use this editor if you are new to the command line.\\ 
-> It is straight forward, but can be extended on the way. 
- 
-  * Auto-indentation 
-  * Syntax highlighting 
-  * Multi-buffer 
- 
- 
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:screenshot_nano.png?0x300}} 
- 
-==== Vi(m) ==== 
- 
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:screenshot_vim.png}} 
- 
-==== Vi(m) explained ==== 
- 
-> This editor is focused on productivity and efficiency, providing everything a user might need. 
- 
-=== Interface === 
- 
-> The simple interface consists of two parts: 
- 
- 
- 
-  * Text area 
-  * Command line 
- 
-> Since this editor is very easy to extend, after setting up a few plugins, it will probably look quite different! 
- 
- 
-=== Usage === 
- 
-> This is a multimode editor, you'll have to switch modes whenever you change what you want to do. 
- 
-^Feature                      Usage                    ^ 
-|Navigation|                 Arrow keys                  | 
-|Writing    change to input mode, then write as usual  | 
-|Commands  |     exit current mode, press ''%%:%%''      | 
-|...                 explained on next slide           | 
- 
- 
- 
-=== Short === 
- 
-> Use this editor if you like a challenge.\\ 
-> It is fast and very nice --- but you'll sometimes get hurt on the way. 
- 
-  * Auto-indentation, Syntax highlighting, Multi-buffer -- just like nano 
-  * File/Project Management 
-  * Use a plugin manager 
- 
- 
-{{pandoc:introduction-to-vsc:03_linux_primer:linuxprimer:screenshot_vim.png?0x300}} 
- 
-==== Vi(m) modes and keys ==== 
- 
-  * any mode: 
-    * back to the **default mode**: **''%%<ESC>%%''** 
-  * command mode (followed by ''%%<RETURN>%%''): 
-    * **save** current file: **''%%w [filename]%%''** 
-    * **quit** the editor 
-      * after saving: **''%%q%%''** 
-      * without confirmation: **''%%q!%%''** 
-    * **help**: **''%%h [topic]%%''**, e.g. **''%%h tutorial%%''** 
-    * **search and replace**: **''%%%s/old/new/gc%%''** 
-  * //default mode//: 
-    * enter **input mode**: **''%%i%%''** 
-    * enter **command mode**: **''%%:%%''** //(colon)// 
-    * **mark** 
-      * **character**-wise: **''%%v%%''** 
-      * **line**-wise: **''%%<SHIFT>-v%%''** 
-    * **delete** 
-      * **character**-wise: **''%%x%%''** 
-      * **line**-wise: **''%%dd%%''** 
-      * **marked content**: **''%%d%%''** 
-    * **search**: **''%%/abc%%''** 
- 
- 
-[[http://www.ks.uiuc.edu/Training/Tutorials/Reference/virefcard.pdf|VI Reference]] 
  
 ===== .bashrc ===== ===== .bashrc =====
Line 903: Line 590:
 export PATH="./:$HOME/bin:$PATH" export PATH="./:$HOME/bin:$PATH"
 </code> </code>
 +
 +
 <HTML> <HTML>
 <!-- <!--
  • pandoc/introduction-to-vsc/03_linux_primer/linuxprimer.1508326936.txt.gz
  • Last modified: 2017/10/18 11:42
  • by pandoc