Sunday, 21 August 2011

What is SFTP, and how do I use it to transfer files?

SFTP, or secure FTP, is a program that uses SSH to transfer files. Unlike standard FTP, it encrypts both commands and data, preventing passwords and sensitive information from being transmitted in the clear over the network. It is functionally similar to FTP, but because it uses a different protocol, you can't use a standard FTP client to talk to an SFTP server, nor can you connect to an FTP server with a client that supports only SFTP.
You can use SFTP with a graphical SFTP client or at the command line.

Graphical SFTP clients

Using graphical SFTP clients simplifies file transfers by allowing you to transmit files simply by dragging and dropping icons between windows. When you open the program, you will have to enter the name of the host (e.g., quarry.uits.indiana.edu) and your username and password. At Indiana University, students, faculty, and staff can download two SFTP clients (CyberDuck for Mac and WinSCP for Windows) from IUware.

Command line SFTP

You can use command line SFTP from your Unix account, or from your Mac OS X or Unix workstation. To start an SFTP session, at the command prompt, enter:
sftp username@host For example, to connect to your dvader account on the host sithlord.org, you would enter:
sftp dvader@sithlord.org Enter your password when prompted.
Some standard commands for command line SFTP include:
cd Change the directory on the remote computer
chmod Change the permissions of files on the remote computer
chown Change the owner of files on the remote computer
dir (or ls) List the files in the current directory on the remote computer
exit (or quit) Close the connection to the remote computer and exit SFTP
get Copy a file from the remote computer to the local computer
help (or ?) Get help on the use of SFTP commands
lcd Change the directory on the local computer
lls See a list of the files in the current directory on the local computer
lmkdir Create a directory on the local computer
ln (or symlink) Create a symbolic link for a file on the remote computer
lpwd Show the current directory (present working directory) on the local computer
lumask Change the local umask value
mkdir Create a directory on the remote computer
put Copy a file from the local computer to the remote computer
pwd Show the current directory (present working directory) on the remote computer
rename Rename a file on the remote host
rm Delete files from the remote computer
rmdir Remove a directory on the remote host (the directory usually has to be empty)
version Display the SFTP version
 !  In Unix, exit to the shell prompt, where you can enter commands. Enter exit to get back to SFTP. If you follow  !  with a command (e.g., !pwd), SFTP will execute the command without dropping you to the Unix prompt.

Talk to another user in unix

talk - talk to another user

talk person [ttyname ]

Talk is a visual communication program which copies lines from your terminal to that of another user. Options available:
person
If you wish to talk to someone on your own machine, then person is just the person's login name. If you wish to talk to a user on another host, then person is of the form `user@host'
ttyname
If you wish to talk to a user who is logged in more than once, the ttyname argument may be used to indicate the appropriate terminal name, where ttyname is of the form `ttyXX' or `pts/X'

When first called, talk contacts the talk daemon on the other user's machine, which sends the message
Message from TalkDaemon@his_machine...
talk: connection requested by your_name@your_machine.
talk: respond with: talk your_name@your_machine
to that user. At this point, he then replies by typing

talk your_name@your_machine
It doesn't matter from which machine the recipient replies, as long as his login name is the same. Once communication is established, the two parties may type simultaneously; their output will appear in separate windows. Typing control-L (^L) will cause the screen to be reprinted. The erase, kill line, and word erase characters (normally ^H, ^U, and ^W respectively) will behave normally. To exit, just type the interrupt character (normally ^C); talk then moves the cursor to the bottom of the screen and restores the terminal to its previous state.

As of netkit-ntalk 0.15 talk supports scrollback; use esc-p and esc-n to scroll your window, and ctrl-p and ctrl-n to scroll the other window. These keys are now opposite from the way they were in 0.16; while this will probably be confusing at first, the rationale is that the key combinations with escape are harder to type and should therefore be used to scroll one's own screen, since one needs to do that much less often.

If you do not want to receive talk requests, you may block them using the mesg(1) command. By default, talk requests are normally not blocked. Certain commands, in particular nroff(1), pine(1), and pr(1), may block messages temporarily in order to prevent messy output.

mesg [y|n]  

Mesg controls the access to your terminal by others. It's typically used to allow or disallow other users to write to your terminal

y
Allow write access to your terminal.
n
Disallow write access to your terminal.
If no option is given, mesg prints out the current access state of your terminal.

nroff - emulate nroff command with groff  


nroff [ -h ] [ -i ] [ -m name ] [ -n num ] [ -o list ] [ -p ] [ -r cn ] [ -S ] [ -t ] [ -T name ] [ -U ] [ -v ] [ file ... ]

The nroff script emulates the nroff command using groff. Only ascii, ascii8, latin1, utf8, nippon, and cp1047 are valid arguments for the -T option. If an invalid or no -T option is given, nroff checks the current locale to select a default output device. It first tries the locale program, then the environment variables LC_ALL, LC_CTYPE, and LANG, and finally the LESSCHARSET environment variable.
The -h and -c options are equivalent to grotty's options -h (using tabs in the output) and -c (using the old output scheme instead of SGR escape sequences). The -C, -i, -n, -m, -o, and -r options have the effect described in troff(1). In addition, nroff silently ignores the options -e, -q, and -s (which are not implemented in troff). Options -p (pic), -t (tbl), -S (safer), and -U (unsafe) are passed to groff. -v shows the version number.  

Saturday, 20 August 2011

'jobs' and 'process'

You run a "job" (or schedule it for later running) While its running, its a "process".
 
The ps command can show you the process ID number that the server assigned to your job which is useful for things like shutting it down early if you want to.

The lsof command is mostly useful to see what processes, files, directories, libraries, etc are being accessed by a job.

There is not much difference between jobs and processes. A job may consist of one ore more processes. Not all processes are jobs.

ps lists running processes with various statistics per process


lsof lists open files and the processes that opened them



ps
Typing ps alone would list the current running processes. Below is an example of the output that would be generated by the ps command.
PID   TTY   TIME   CMD
6874  pts/9   0:00     ksh
6877  pts/9   0:01     csh
418    pts/9   0:00     csh


ps -ef
Display full information about each of the processes currently running.
UID PID PPID C STIME TTY TIME CMD
hope 29197 18961 0 Sep27 ? 00:00:06 sshd: hope@pts/87
hope 32097 29197 0 Sep27 pts/87 00:00:00 -csh
hope 7209 32097 0 12:17 pts/87 00:00:00 ps -ef


% ps -gx
To find out all the process ID numbers of processes you are running, type:
        
Here is an example of output from this command:
          PID TT STAT  TIME COMMAND
        12030 p5 S     0:01 -csh (csh)
        12068 p5 T     1:46 emacs foo
        12788 p5 R     0:00 ps -gx

ps -l
Displays processes including those that are in a wait state, similar to the below example.
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 0 T 0 12308 29722 0 80 0 - 16136 finish pts/0 00:00:00 pico 0 R 0 12530 29722 0 80 0 - 15884 - pts/0 00:00:00 ps 4 S 0 29722 29581 0 80 0 - 16525 wait pts/0 00:00:00 bash
 
To find the job numbers of all the background jobs you are running, type:
% jobs

 

Pipes

To see who is on the system with you, type
% who 

One method to get a sorted list of names is to type,
% who > names.txt
% sort < names.txt 

This is a bit slow and you have to remember to remove the temporary file called names when you have finished. What you really want to do is connect the output of the who command directly to the input of the sort command. This is exactly what pipes do. The symbol for a pipe is the vertical bar |
For example, typing
% who | sort 
will give the same result as above, but quicker and cleaner.

To find out how many users are logged on, type
% who | wc -l

grep

grep is one of many standard UNIX utilities. It searches files for specified words or patterns.

% grep science science.txt 

As you can see, grep has printed out each line containg the word science.
Or has it ????
Try typing
% grep Science science.txt 

The grep command is case sensitive; it distinguishes between Science and science.

To ignore upper/lower case distinctions, use the -i option, i.e. type
% grep -i science science.txt 

To search for a phrase or pattern, you must enclose it in single quotes (the apostrophe symbol). For example to search for spinning top, type

% grep -i 'spinning top' science.txt 

Some of the other options of grep are:

-v display those lines that do NOT match
-n precede each matching line with the line number
-c print only the total count of matched lines


Try some of them and see the different results. Don't forget, you can use more than one option at a time. For example, the number of lines without the words science or Science is
% grep -ivc science science.txt

clear (clear screen),less,wc (word count)


Clears the screen.

Syntax
clear

Examples
clear

Clear the screen.

Additional information
Many newer Unix systems and Unix variants will also accept the cls command or have cls as an alias for clear.


less

The command less writes the contents of a file onto the screen a page at a time. Type
% less science.txt 

Press the [space-bar] if you want to see another page, and type [q] if you want to quit reading. As you can see, less is used in preference to cat for long files.


Simple searching using less

Using less, you can search though a text file for a keyword (pattern). For example, to search through science.txt for the word 'science', type

% less science.txt
then, still in less, type a forward slash [/] followed by the word to search
/science

As you can see, less finds and highlights the keyword. Type [n] to search for the next occurrence of the word.

wc (word count)

A handy little utility is the wc command, short for word count. To do a word count on science.txt, type

% wc -w science.txt

To find out how many lines the file has, type
% wc -l science.txt




Head And Tail

head is a program on Unix and Unix-like systems used to display the first few lines of a text file or piped data. The command syntax is:

head [options] <file_name>
By default, head will print the first 10 lines of its input to the standard output. The number of lines printed may be changed with a command line option.

The following example shows the first 20 lines of filename:
head -n 20 filename 
This displays the first 5 lines of all files starting with foo:
head -n 5 foo* 
 
Tail Command 
tail [options] <file_name>

By default, tail will print the last 10 lines of its input to the standard output. With command line options the number of lines printed and the printing units (lines, blocks or bytes) may be changed. The

following example shows the last 20 lines of filename:
tail -n 20 filename
This example shows the last 15 bytes of all files starting with foo:
tail -c 15 foo*
This example shows all lines of filename from the second line onwards:
tail -n +2 filename

File monitoring

tail has a special command line option -f (follow) that allows a file to be monitored. Instead of just displaying the last few lines and exiting, tail displays the lines and then monitors the file. As new lines are added to the file by another process, tail updates the display. This is particularly useful for monitoring log files. The following command will display the last 10 lines of messages and append new lines to the display as new lines are added to messages:

tail -f /var/adm/messages 
In cases when the user is following a log file that rotates then it  is advisable to use the -F option as it keeps folllowing the log even  when it is recreated, renamed, or removed as part of log rotation.

tail -F /var/adm/messages 
To interrupt tail while it is monitoring, break-in with Ctrl+C. This command can be run "in the background" with &,
 

Removing files and directories

To delete (remove) a file, use the rm command. As an example, we are going to create a copy of the science.txt file then delete it.

Inside your unixstuff directory, type
% cp science.txt tempfile.txt
% ls
% rm tempfile.txt
% ls

You can use the rmdir command to remove a directory (make sure it is empty first). Try to remove the backups directory. You will not be able to since UNIX will not let you remove a non-empty directory.

rmdir mydir
Removes the directory mydir 

rm -r directory
Remove a directory, even if files existed in that directory.
 
And for extra measures (since files rm'd cannot be recovered easily if at all),

rm -rfv /path/to/dir

The -f forces the deletion (if you're allowed to delete the files) and the -v shows verbose information (tells you what is being deleted).

If you're sudo'd and you're using rm, be sure to check what you're deleting and make sure the path is correct. 

Moving files

mv file1 file2 moves (or renames) file1 to file2

 To move a file from one place to another, use the mv command. This has the effect of moving rather than copying the file, so you end up with only one file rather than two. 

It can also be used to rename a file, by moving the file to the same directory, but giving it a different name. 

We are now going to move the file science.bak to your backup directory. 

First, change directories to your unixstuff directory (can you remember how?). Then, inside the unixstuff directory, type

% mv science.bak  backups/.

Type ls and ls backups to see if it has worked.

Changing Owners and Groups

While creating an account on Unix, it assigns a owner ID and a group ID to each user. All the permissions mentioned above are also assigned based on Owner and Groups.
Two commands are available to change the owner and the group of files:
  1. chown: The chown command stands for "change owner" and is used to change the owner of a file.

    Syntax

    chown [-R] newowner filenames
     
    -RChange the permission on files that are in the subdirectories of the directory that you are currently in.
    newownerThe alias/username of the new owner of the file.
    filenamesThe file that you are changing the rights to.

  2. chgrp: The chgrp command stands for "change group" and is used to change the group of a file.


    The chrgp command changes the group ownership of a file. The basic syntax is as follows:
    [amrood]$ chgrp group filelist


    The value of group can be the name of a group on the system or the group ID (GID) of a group on the system.
    Following example:
    [amrood]$ chgrp special testfile

permissions and chmod

To change file or directory permissions, you use the chmod (change mode) command. There are two ways to use chmod: symbolic mode and absolute mode.

Using chmod in Symbolic Mode:                             

The easiest way for a beginner to modify file or directory permissions is to use the symbolic mode. With symbolic permissions you can add, delete, or specify the permission set you want by using the operators in the following table.

Chmod operatorDescription
+Adds the designated permission(s) to a file or directory.
-Removes the designated permission(s) from a file or directory.
=Sets the designated permission(s).


Here's an example using testfile. Running ls -1 on testfile shows that the file's permissions are as follows:
[amrood]$ls -l testfile
-rwxrwxr--  1 amrood   users 1024  Nov 2 00:10  testfile



Then each example chmod command from the preceding table is run on testfile, followed by ls -l so you can see the permission changes:
[amrood]$chmod o+wx testfile
[amrood]$ls -l testfile
-rwxrwxrwx  1 amrood   users 1024  Nov 2 00:10  testfile
[amrood]$chmod u-x testfile
[amrood]$ls -l testfile
-rw-rwxrwx  1 amrood   users 1024  Nov 2 00:10  testfile
[amrood]$chmod g=r-x testfile
[amrood]$ls -l testfile
-rw-r-xrwx  1 amrood   users 1024  Nov 2 00:10  testfile


Here's how you could combine these commands on a single line:
[amrood]$chmod o+wx,u-x,g=r-x testfile
[amrood]$ls -l testfile
-rw-r-xrwx  1 amrood   users 1024  Nov 2 00:10  testfile



Using chmod with Absolute Permissions: (1,2,4)          

The second way to modify permissions with the chmod command is to use a number to specify each set of permissions for the file.
Each permission is assigned a value, as the following table shows, and the total of each set of permissions provides a number for that set.
NumberOctal Permission RepresentationRef
0No permission---
1Execute permission--x
2Write permission-w-
3Execute and write permission: 1 (execute) + 2 (write) = 3-wx
4Read permissionr--
5Read and execute permission: 4 (read) + 1 (execute) = 5r-x
6Read and write permission: 4 (read) + 2 (write) = 6rw-
7All permissions: 4 (read) + 2 (write) + 1 (execute) = 7rwx

Here's an example using testfile. Running ls -1 on testfile shows that the file's permissions are as follows:
[amrood]$ls -l testfile
-rwxrwxr--  1 amrood   users 1024  Nov 2 00:10  testfile

Then each example chmod command from the preceding table is run on testfile, followed by ls -l so you can see the permission changes:
[amrood]$ chmod 755 testfile
[amrood]$ls -l testfile
-rwxr-xr-x  1 amrood   users 1024  Nov 2 00:10  testfile
[amrood]$chmod 743 testfile
[amrood]$ls -l testfile
-rwxr---wx  1 amrood   users 1024  Nov 2 00:10  testfile
[amrood]$chmod 043 testfile
[amrood]$ls -l testfile
----r---wx  1 amrood   users 1024  Nov 2 00:10  testfile

look, modify or combine a file



cat command Allows you to look, modify or combine a file.
Syntax 

cat filename [-n] [-b] [-u] [-s] [-v] 

filenameThe name of the file or files that you wish to look at or perform tasks on.
-nPrecede each line output with its line number.
-bNumber the lines, as -n, but omit the line numbers from blank lines.
-uThe output is not buffered. (The default is buffered output.)
-scat is silent about non-existent files.
-vNon-printing characters (with the exception of tabs, new-lines and form-feeds) are printed visibly. ASCII control characters (octal 000 - 037) are printed as ^n, where n is the corresponding ASCII character in the range octal 100 - 137 (@, A, B, C, . . ., X, Y, Z, [, \, ], ^, and _); the DEL character (octal 0177) is printed ^?. Other non-printable characters are printed as M-x, where x is the ASCII character specified by the low order seven bits.
-eA $ character will be printed at the end of each line (prior to the new-line).
-tTabs will be printed as ^I's and formfeeds to be printed as ^L's.
*If the -v is used -e and -t will be ignored.

Examples
cat file1.txt file2.txt > file3.txt

Reads file1.txt and file2.txt and combines those files to make file3.txt.
 
Now type cat without specifing a file to read
% cat

Then type a few words on the keyboard and press the [Return] key. 
Finally hold the [Ctrl] key down and press [d] (written as ^D for short) to end the input.
What has happened? 

If you run the cat command without specifing a file to read, it reads the standard input (the keyboard), and on receiving the 'end of file' (^D), copies it to the standard output (the screen). 

In UNIX, we can redirect both the input and the output of commands. 

We use the > symbol to redirect the output of a command. For example, to create a file called list1 containing a list of fruit, type   

% cat > list1
Then type in the names of some fruit. Press [Return] after each one.
pear
banana
apple
^D {this means press [Ctrl] and [d] to stop}

What happens is the cat command reads the standard input (the keyboard) and the > redirects the output, which normally goes to the screen, into a file called list1
 
To read the contents of the file, type
% cat list1

The form >> appends standard output to a file. So to add more items to the file list1, type
% cat >> list1

Then type in the names of more fruit
peach
grape
orange
^D (Control D to stop)
To read the contents of the file, type
% cat list1

You should now have two files. One contains six fruit, the other contains four fruit.
We will now use the cat command to join (concatenate) list1 and list2 into a new file called biglist. Type
% cat list1 list2 > biglist

What this is doing is reading the contents of list1 and list2 in turn, then outputing the text to the file biglist
 
To read the contents of the new file, type
% cat biglist


We use the < symbol to redirect the input of a command.
The command sort alphabetically or numerically sorts a list. Type
% sort 

Then type in the names of some animals. Press [Return] after each one.
dog
cat
bird
ape
^D (control d to stop)

The output will be
ape
bird
cat
dog 

Using < you can redirect the input to come from a file rather than the keyboard. For example, to sort the list of fruit, type
% sort < biglist 

and the sorted list will be output to the screen. 

To output the sorted list to a file, type,
% sort < biglist > slist
Use cat to read the contents of the file slist



Copying a file to another Directory

To make a copy of a file in the current directory, enter:
cp prog.c prog.bak
This copies prog.c to prog.bak. If the prog.bak file does not already exist, the cp command creates it. If it does exist, the cp command replaces its contents with the contents of the prog.c file.

To copy a file in your current directory into another directory, enter:

cp jones /home/nick/clients

This copies the jones file to /home/nick/clients/jones.

To copy a file to a new file and preserve the modification date, time, and access control list associated with the source file, enter:
cp -p smith smith.jr
This copies the smith file to the smith.jr file. Instead of creating the file with the current date and time stamp, the system gives the smith.jr file the same date and time as the smith file. The smith.jr file also inherits the smith file's access control protection.

To copy all the files in a directory to a new directory, enter:
cp /home/janet/clients/* /home/nick/customers
This copies only the files in the clients directory to the customers directory.

To copy a directory, including all its files and subdirectories, to another directory, enter:
 cp -R /home/nick/clients /home/nick/customers
This copies the clients directory, including all its files, subdirectories, and the files in those subdirectories, to the customers/clients directory. Be careful about including a trailing slash in the source directory, however. If you run cp -R /home/nick/clients/ /home/nick/customers on a GNU-based system, it does the same thing as without the slash; however, if you run the same thing on a BSD-based system, it will copy all the contents of the "clients" directory over, instead of the "clients" directory itself.

To copy a specific set of files to another directory, enter:
cp jones lewis smith /home/nick/clients 
This copies the jones, lewis, and smith files in your current working directory to the /home/nick/clients directory.

To use pattern-matching characters to copy files, enter:
  cp programs/*.c .
This copies the files in the programs directory that end with .c to the current directory, signified by the single . (dot). You must type a space between the c and the final dot.
Copying a file to an existing file is done by opening the existing file in update mode which requires write access and results in the target file retaining the permissions it had originally.

Listing files and directories

ls (list)

When you first login, your current working directory is your home directory. Your home directory has the same name as your user-name, for example, ee91ab, and it is where your personal files and subdirectories are saved.

To find out what is in your home directory, type

          %  ls 

ls does not, in fact, cause all the files in your home directory to be listed, but only those ones whose name does not begin with a dot (.) Files beginning with a dot (.) are known as hidden files and usually contain important program configuration information. They are hidden because you should not change them unless you are very familiar with UNIX!!!

To list all files in your home directory including those whose names begin with a dot, type

% ls -a


% ls -l (l for long listing!)

Each file (and directory) has associated access rights, which may be found by typing ls -l. Also, ls -lg gives additional information as to which group owns the file (beng95 in the following example):
-rwxrw-r-- 1 ee51ab beng95 2450 Sept29 11:52 file1

mkdir (make directory)

We will now make a subdirectory in your home directory to hold the files you will be creating and using in the course of this tutorial.

To make a subdirectory called unixstuff in your current working directory type

% mkdir unixstuff 

To see the directory you have just created, type

% ls 


The syntax for the mkdir command is:

mkdir [options] directories

options:

-m mode Sets the access mode for the new directory.
-p If the parent directories don't exist, this command creates them.


Examples:

mkdir -m 444 tech
mkdir -p tech/net/faqs

cd (change directory)

The command cd directory means change the current working directory to 'directory'. The current working directory may be thought of as the directory you are in, i.e. your current position in the file-system tree.

To change to the directory you have just made, type

% cd unixstuff

The current directory (.)

In UNIX, (.) means the current directory, so typing

% cd . 

NOTE: there is a space between cd and the dot means stay where you are (the unixstuff directory).
This may not seem very useful at first, but using (.) as the name of the current directory will save a lot of typing, as we shall see later in the tutorial.

The parent directory (..)

(..) means the parent of the current directory, so typing
% cd .. 

will take you one directory up the hierarchy (back to your home directory). Try it now.

Note: typing cd with no argument always returns you to your home directory. This is very useful if you are lost in the file system.

pwd (print working directory)

Pathnames enable you to work out where you are in relation to the whole file-system. For example, to find out the absolute pathname of your home-directory, type cd to get back to your home-directory and then type

% pwd
The full pathname will look something like this -

/home/its/ug1/ee51vn



~ (your home directory)

Home directories can also be referred to by the tilde ~ character. It can be used to specify paths starting at your home directory. So typing

% ls ~/unixstuff

will list the contents of your unixstuff directory, no matter where you currently are in the file system.
What do you think

% ls  ~ would list? 

What do you think % ls  ~/.. would list? 

Unix directory structure

In Unix and Unix-like operating systems, the Unix directory structure is a convention for filesystem layout.
Several attempts exist to standardize the Unix filesystem layout, such as the "Filesystem Hierarchy Standard, however, targeted primarily at Linux. Also, as part of the "Linux Standards Base", a common filesystem layout is defined, to encourage interoperability among different Linux distributions.
What follows is a generalized overview of common locations of files in a Unix operating system:
  • / — the slash / character alone denotes the root of the virtual filesystem tree.
  • /bin — stands for "binaries" and contains certain fundamental utilities, such as ls or cp, needed by all users.
  • /sbin — stands for "system (or "superuser") binaries" and contains fundamental utilities, such as init, usually needed to start, maintain and recover the system.
  • /etc — contains configuration files and system databases.
  • /dev — stands for "devices". Contains file representations of peripheral devices.
  • /dev/null — also known as the "bit bucket" or "black hole", this pseudo device discards all contents written to it, and is typically used to pipe away unwanted data.
  • /dev/random — this pseudo device returns pseudorandom numbers (subject to the limitations of random number generators in computing) when read from. It uses system noise to generate random numbers and blocks if not enough entropy in the noise is available. Random is commonly used by programs such as SSH that absolutely need cryptographically-strong random data to generate an encryption key.
  • /dev/urandom — similar to /dev/random, except it always returns (less cryptographically-strong) pseudorandom numbers, even if there is not enough entropy in the system noise available.
  • /home — contains the home directories for the users.
  • /mnt — contains filesystem mount points.
  • /lib — contains system libraries.
  • /root — the home directory for the superuser root.
  • /tmp — a place for temporary files. Many Unices clear this directory upon start up.
  • /usr — originally the directory holding user home directories, its use has changed, and it now holds executables, libraries, and shared resources that are not system critical, like the X Window System, KDE, Perl, etc. (The name "Unix System Resources" is a post hoc backronym. However, on some Unix systems, some user accounts may still have a home directory that is a direct subdirectory of /usr, such as the default ast in Minix.
  • /usr/bin — this directory stores all binary programs distributed with the operating system not residing in /bin, /sbin or (rarely) /etc.
  • /usr/include — /usr/include stores the development headers used throughout the system.
  • /usr/lib — the required libraries for executables within /usr or elsewhere for that matter, live here.
  • /usr/local — resembles /usr, except that its subdirectories are used for additions not part of the operating system distribution, such as custom programs or files from a BSD Ports collection. Usually has subdirectories such as /usr/local/lib or /usr/local/bin.
  • /var — a short for "variable." A place for files that may change often.
  • /var/log — contains system log files.
  • /var/mail — the place where all the incoming mails are stored. The user can access his/her own mail only, unless he/she has admin rights.
  • /var/spool — spool directory; contains print jobs, mail spools and other queued tasks.
  • /var/tmp — A place for temporary files preserved between system reboots.
  • /proc — contains all processing data (Process information about a running operating system).
  • /opt — contains add-on software.
  • /media — default mount point for removable devices.
  • /srv — server data (data for services provided by system).
  • /boot — contains all the important files which are required for successful booting process.
  • /sys — contains information related to hardware.

What is UNIX?

UNIX is an operating system which was first developed in the 1960s, and has been under constant development ever since. By operating system, we mean the suite of programs which make the computer work. It is a stable, multi-user, multi-tasking system for servers, desktops and laptops. 


UNIX systems also have a graphical user interface (GUI) similar to Microsoft Windows which provides an easy to use environment. However, knowledge of UNIX is required for operations which aren't covered by a graphical program, or for when there is no windows interface available, for example, in a telnet session.

There are many different versions of UNIX, although they share common similarities. The most popular varieties of UNIX are Sun Solaris, GNU/Linux, and MacOS X.

The UNIX operating system is made up of three parts; the kernel, the shell and the programs. 
 
The kernel:The kernel is the hub of the operating system: it allocates time and memory to programs and handles the filestore and communications in response to system calls. 

The shell:The shell acts as an interface between the user and the kernel. When a user logs in, the login program checks the 
username and password, and then starts another program called the shell. The shell is a command line interpreter 
(CLI). It interprets the commands the user types in and arranges for them to be carried out. The commands are 
themselves programs: when they terminate, the shell gives the user another prompt.

Bourne shell (sh) :This is the original Unix shell written by Steve Bourne of Bell Labs. It is available on all UNIX systems. 

This shell does not have the interactive facilites provided by modern shells such as the C shell and Korn shell. You are advised to to use another shell which has these features.

The Bourne shell does provide an easy to use language with which you can write shell scripts.

C shell (csh)

This shell was written at the University of California, Berkeley. It provides a C-like language with which to write shell scripts - hence its name.

TC shell (tcsh)

This shell is available in the public domain. It provides all the features of the C shell together with emacs style editing of the command line.
  
Korn shell (ksh)
This shell was written by David Korn of Bell labs. It is now provided as the standard shell on Unix systems.
It provides all the features of the C and TC shells together with a shell programming language similar to that of the original Bourne shell.
It is the most efficient shell. Consider using this as your standard interactive shell.

Bourne Again SHell (bash)

This is a public domain shell written by the Free Software Foundation under their GNU initiative. Ultimately it is intended to be a full implementation of the IEEE Posix Shell and Tools specification. This shell is widely used within the academic commnity.
bash provides all the interactive features of the C shell (csh) and the Korn shell (ksh). Its programming language is compatible with the Bourne shell (sh).
If you use the Bourne shell (sh) for shell programming consider using bash as your complete shell environment.

 Files and processes :Everything in UNIX is either a file or a process.A process is an executing program identified by a unique PID (process identifier).A file is a collection of data. They are created by users using text editors, running compilers etc.
Examples of files:
  • a document (report, essay etc.)
  • the text of a program written in some high-level programming language
  • instructions comprehensible directly to the machine and incomprehensible to a casual user, for example, a collection of binary digits (an executable or binary file);
  • a directory, containing information about its contents, which may be a mixture of other directories (subdirectories) and ordinary files. 
 
Some Commercial Use of Unix