1 / 41

Linux for Researchers

Linux for Researchers. Dr Andrew Richards, Oxford e-Research Centre & Oxford Supercomputing Centre. Topics. What does the operating system do? Brief History of UNIX and Linux What is Linux ? Unix Philosophy Comparison to Windows Practical Exercises Further Information.

rimona
Télécharger la présentation

Linux for Researchers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Linux for Researchers Dr Andrew Richards, Oxford e-Research Centre & Oxford Supercomputing Centre

  2. Topics • What does the operating system do? • Brief History of UNIX and Linux • What is Linux ? • Unix Philosophy • Comparison to Windows • Practical Exercises • Further Information

  3. What does the operating system do? • Manage the computer resources • Interact with the user • Run applications

  4. Operating Systems?

  5. Operating Systems

  6. Brief history of UNIX and Linux • A multi-user, multi-tasking operating system • 1960s • MIT, AT&T, General Electric develop Multics (Multiplexing Information and Computing Services) • 1970s • Unics (Uniplexed Information and Computing Service) – renamed to Unix • 1972 • rewritten in C – resulted in more portable software to different hardware • 1980 • AT&T licenses UNIX System III – consolidated differences into UNIX System V release 1 • 1983 • AT&T commercialise UNIX System V • Academia continue to develop BSD Unix as an alternative • BSD Unix main contributor of TCP/IP network code to the mainstream Unix kernel • 1990 • Open Software Foundation released OSF/1 – standard Unix implementation (Mach and BSD) • 1991 • Linus Torvalds starts Linux (originally Freax). Mixture of BSD and SYS-V

  7. What is Linux ? • Linux is an Operating System • Software loaded into your computer's memory when you switch it on • Software which manages the hardware • Linux is “Unix-like” • Unix is an operating system originally developed in the 1970s to run large, central computers with many users • Runs everything from phones to supercomputers today • Features and Benefits • Stability • Security • Performance • Price • Flexibility & Open Source nature

  8. Unix Philosophy • Unix is based around commands which • Carry out one (or very few) simple tasks • Carry tasks out very efficiently • Carry tasks out very quietly • Commands tend to be typed in a “shell”, the layer between you and the core Linux “kernel” • More complex tasks are carried out by joining simple commands together:

  9. Example • grep – search for a pattern in a file • sort - sort into order • uniq – only show one copy of identical things grep OSC *txt | sort | uniq > output.txt • The above command could search thousands of files for lines containing “OSC” and save one copy of each to a new file (in seconds). What would a Windows user do?

  10. More Unix Philosophy • Treat everything as files and whenever possible, make files plain text and readable by anyone with anything – don't hide or obscure stuff unless it needs to be hidden • Hierarchical File System • Top level = / • /bin • /sbin • /dev • /etc • /tmp • … • Assume that the user knows what they are doing • Comes with it's own perils!

  11. Comparison to Windows • Windows is just another Operating system • Windows is not just Desktop, Server and ‘HPC’ variants • Not Free, Not Open Source • Mostly interacted with through a GUI but there is a ‘shell’ -> ‘CMD’ or PowerShell • Things to consider: • Installation • Driver Support • User Experience • Development platform? Where will you develop and use your code? • Read Wikipedia • http://en.wikipedia.org/wiki/Comparison_of_Windows_and_Linux

  12. Practical Exercises (Information) • Logging into a remote system • From a Unix machine (including Mac OS ) – use SSH or GSISSH to access grid resources • From a Windows machine – use PuTTY (and Xming for X11 support) • http://www.chiark.greenend.org.uk/~sgtatham/putty/ • http://www.straightrunning.com/XmingNotes/ • http://sourceforge.net/projects/xming/ (*** Use to Download ***) • Conventions • Commands are typed in at a “prompt” which is represented by: • prompt> • Parts of a command you should replace (e.g. fill-in-your-name) are in itallics.

  13. Remote Login • ssh prompt> ssh <username>@<hostname> prompt> ssh andy@linux.university.ac.uk prompt> ssh –X andy@linux.university.ac.uk

  14. Practical Exercise: Where am I ? • A Linux system is arranged in a tree like structure with files (leaves) and directories (branches). • When you do something with files and directories on Linux (making a new file for example), if you do not specify where in the directory “tree” the file should be created, it defaults to your current “location”. • This will show you your current location • And with the ‘ls’ command will show the contents of this directory prompt> pwd prompt> ls

  15. Practical Exercise: Create a Directory • prompt> mkdirtraining • This will create a new directory called training in your current location • prompt> ls • Now lets move from our home area to the training directory • prompt> pwd • prompt> cd training • prompt> pwd • prompt> ls • No content should be listed as this is a new directory with no content!

  16. Practical Exercise: Moving around the filesystem prompt> pwd To move up one level prompt> cd .. prompt> pwd You can also jump to other parts of the filesystem prompt> cd /usr/local prompt> pwd prompt> ls Now keep trying to go up one directory level until you can't go any higher – you have reached the “root” of the file system – what is it called? prompt> pwd

  17. Practical Exercise: Moving to your home directory prompt> pwd To go back to your home directory, there are two short cuts: prompt> cd Or prompt> cd ~

  18. Practical Exercise: More moving around • Pushd and Popd • prompt > pwd • prompt > pushd /tmp • prompt > pwd • prompt > popd • prompt > pwd

  19. Practical Exercise: Question What do you think will happen if you type prompt> cd ~training

  20. Practical Exercise: Permissions (1) • All users have their own identity on a Linux system and in addition they belong to one or more groups. • Linux (and Unix) attaches an ownership and set of permissions to every file (including directories) which define what you can and can't do to that file depending on who you are and which groups you are in • How do we find out what permissions are in place? • prompt> ls –l • The ‘-l’ option is an example of ‘flag’ in Unix

  21. Practical Exercise: Permissions (2) • Lets look at the permissions on a specific directory • prompt> ls –l /bin • The permissions of the files are listed in the first column of the output • 1st character is the type of file • Characters 2-4 are the read/write/execute permissions for the owner of the file • Characters 5-7 are the read/write/execute permissions for the group the file belongs to • Characters 8-10 are the read/write/execute permissions for everyone else.

  22. Practical Exercise: Permissions (3) • First character is “-” so it's a normal file (d means a directory) • Characters 2-4 are “rwx” so the user can read and write and execute (run) this file as a program • 5-7 are “r-w” so any other user in the group “root” can read and run the file but not change it • 8-10 is also “r-w” so any user in any group can read and run the file but not change it • Permissions can be changed with the commands chmod • prompt> chmod g-w filename • Most common problem on Unix systems is the error message ‘Permissions Denied’

  23. Practical Exercise: HELP ! Help this is getting complicated !!! All Unix / Linux commands come with their own manual prompt> man ls prompt> man chmod The ‘man’ pages will give further information on ‘flags’ available and often give examples of how to use Note: - Press ‘q’ to quit a man page - typing man –k something will search for commands that relate to something

  24. Practical Exercise: Wildcards (1) • As you saw from the /bin directory, some areas contain lots of files. • Many Linux commands support the use of Wildcards • prompt> ls /bin • And then try • prompt> ls /bin/a* • “*” is a substitute for “anything from zero to N characters of any type”. So everything starting with a “a” is listed. • Note: If a directory starts with an “a” then all its files are listed even if they don’t start with an “a”

  25. Practical Exercise: Wildcards (2) • Now try • prompt> ls /bin/A* • Is the result the same of different? • Now try • prompt> ls /bin/*a* • Differences ?

  26. Practical Exercise: Wildcards (3) • Try the following: • prompt> ls /bin/*a • prompt> ls /bin/[a-c]* • prompt> ls /bin/[a-c]*f* • prompt> ls /bin/*[0-9]* • What is happening in each of the different cases?

  27. Practical Exercise: Wildcards (4) • As well as “*” there is another wild card, “?”. • This means “anything which is is one character long”. • Try the following: • prompt> ls /bin/?a* • prompt> ls /bin/*a? • prompt> ls /bin/??a* • Can you explain what happens in each case ? • Make sure you understand the difference between “/bin/*a*” and “/bin/?a?”

  28. Practical Exercise: More ls • ls and wildcards • prompt > ls • For Directories ? • prompt > ls –d */ • prompt > ls –l | grep ^d

  29. Practical Exercise: Keyboard shortcuts • Three shortcuts which will speed things up: • The up and down arrow keys – these enable you to go back and forth through your history of previous commands • The left and right arrow keys, allow you to move along a line so you can change things • Tab – this auto-completes command names and file names

  30. Practical Exercise: Creating a file with a text editor • Linux tries to use plain text format files as much as possible • This keeps files simple and readable by many different applications • prompt> cd ~/training • prompt> ls • prompt> nano myfile.txt • Type some text into the window that has appeared. • “The Oxford Supercomputing Centre is a complete High Performance Computing Service” • Now save the file (Ctrl-O) and exit (Ctrl-X). • prompt> ls • You should have a file called myfile.txt

  31. Practical Exercise: Looking inside Files (1) You can use a text editor to look inside a file without changing anything of course but there are quicker (and it turns out, more useful) ways of looking inside text files. To see what's inside a text file: prompt> cat myfile.txt (cat is short for concatenate). The problem with “cat” is that big files rapidly scroll off the top of the terminal screen. prompt> less myfile.txt A more powerful command is called “more” but that has been supplanted by an even more powerful command named “less”: Remember!! For more information try ‘man less’

  32. Practical Exercise: Looking inside Files (2) There are even faster ways to look for words in text files. Use “grep” to look for the word “OSC” on myfile.txt: prompt> grepsuper myfile.txt Note: this will search for ‘super’ exactly as it is written. Case sensitive! Prompt> grep –i super myfile.txt This will do the same search but ignore the case of the word

  33. Practical Exercise: Copying, Moving and Deleting Files & Directories(1) Warning: Linux does not warn about deletions To copy a file, use “cp” and to move or rename it, use “mv”. prompt> cp myfile.txt myjunk.txt prompt> ls prompt> mv myjunk.txt myold.txt prompt> ls To delete a file we use the command “rm”. prompt> rm myold.txt prompt> ls Q. What would ‘rm *’ do ?

  34. Practical Exercise: Copying, Moving and Deleting Files & Directories(2) Now make a new directory called “temporary” and then try to remove it using: rm rmdir rm is for removing files, rmdir for empty directories.

  35. Practical Exercise: Making a Shell Script • Linux is made up from lots of very small commands which perform very specific jobs. • It is often the case that you need to group lots of commands together into a “recipe” which you use over and over again. • Such recipes are held in text files and are known as “scripts”. • There are different flavours of scripts depending on which language they are written in and because we are making a script written with BASH-shell commands, this will be a “shell script”.

  36. Practical Exercise: An Example Script Our script will do two things: 1. List the contents of /etc 2. Put the output into a new file called listing.txt Use “nano” to enter the following script into a file called listing.sh: # A comment. Other than #!, anything starting with # is ignored #List /etc and REDIRECT the output to a file called listing.txt ls /etc > listing.txt and save the script as “listing.sh”. There are two ways we can run this script. Method one involves sourcing the script (saying to the shell you're typing at, “please run this text file as a BASH script”. prompt> . ./listing.sh The “.” character on its own means “run this” file as a BASH script.

  37. Practical Exercise: An Example Script continued… The second way of doing this is a bit tidier. We place the strange looking line “#!/bin/bash” at the start of the script which forces the script to run as a BASH shell script every time. Use nano to change the first line of the script so it looks like this #!/bin/bash # A comment. Other than #!, anything starting with # is ignored #List /etc and REDIRECT the output to a file called listing.txt ls /etc > listing.txt Now try to run the file directly (so no “.” sign). prompt> ./listing.sh You should get ‘Permission Denied’

  38. Practical Exercise: Fixing the script to run • Use the chmod command to add executable permission to listing.sh executable and run it using the line above. Re-read the chmod man page • prompt> chmoda+x listing.sh • prompt> ./listing.sh • Now looking inside your new file, “listing.txt” to check the script has worked. Note how “>” redirected the output from a command into a file. It is also possible to redirect files into commands using “<”. This is known as “redirection”

  39. Practical Exercise: Other tools, finding tools, adding tools • which <command> • e.g. which man • whereis man • Package management • (prompt > cat /etc/issue ) • (prompt> uname –a ) • Apt (Debian) • apt-get install <package> • apt-cache search <string> • Yum (Redhat) • yum search <string> • yum install <package> • Others include yast on SuseLinux, up2date (older Redhatetc. )

  40. But its not all Command Line

  41. Further Information • History of UNIX • http://www.levenez.com/unix/ • File system hierarchy standard • http://www.pathname.com/fhs/pub/fhs-2.3.html • Vi cheat Sheet • http://www.lagmonster.org/docs/vi.html • X • Xming (use sourceforge version • FreeNX or NoMachine NX • http://www.nomachine.com/ • http://freenx.berlios.de/

More Related