[§herlock]

This is the fourth chapter of our Linux tutorial series. We have listed all the tutorials under Linux Beginning Series, here for your convenience. In this chapter we are gonna learn about the Linux shell, the program that reads your command input and runs the specified commands. The shell environment is the most fundamental way to interact with the system.

What is in this Chapter?

• Keys for Command Line Editing
  
• Redirecting Input and Output
  
• Managing Jobs
  
• Command History
  
• Recording a Shell Session
  
• Customizing Your Shell
  

Sources:

• Intro Linux
  
• Linux Cookbook
  

---

The ‘$’ character preceding the cursor is called the shell prompt, it tells you that the system is ready and waiting for input. In some systems like Debian, the default shell prompt also includes the name of the current directory. A tilde character (‘~’) denotes your home directory, which is where you’ll find yourself when you log in.

For example, a typical user’s shell prompt might look like this (if you are in your home directory):

Code:

username@hostname:~$

Where, hostname is the name of your system. If you are in a directory other than your home directory the shell will show you the name of that directory:

Code:

root@debian:~/Desktop#

If your shell prompt shows a number sign (‘#’) instead of a ‘$’, this means that you’re logged in with the superuser, or root, account.

Beware: The root account has complete control over the system; one wrong keystroke and you might accidentally break it something awful. You need to have a different user account for yourself, and use that account for your regular use (see section Making a User Account). Every Linux system has at least one shell program, some may have several. We’ll cover bash, which is the standard shell on most Linux systems. (bash stands for “Bourne again shell” a pun on the name of Steve Bourne, who was author of the traditional Unix shell, the Bourne shell.)

1. Keys For Command Line Editing

The text that you type at a shell prompt is called the command line (it’s also called the input line). The following list describes the keystrokes used for typing command lines.

• ENTER KEY OR RET: Send the command line to bash for execution (in other words, it runs the command typed at the shell prompt). You can run command when the cursor is anywhere on the command line. You need not to be necessarily at the end to run a command.
  
• CTRL + A: Move the cursor to the beginning of the input line.
  
• CTRL + D: Same as DEL (this is the Emacs equivalent).
  
• CTRL + E: Move the cursor to the end of the input line.
  
• CTRL + K: Kill, or “cut,” all text on the input line, from the character the cursor is underneath to the end of the line.
  
• CTRL + L: Clear the terminal screen.
  
• CTRL + U: Kill the entire input line.
  
• CTRL + Y: Yank, or “paste,” the text that was last killed. Text is inserted at the point where the cursor is.
  
• CTRL + _: Undo the last thing typed on this command line. (Note that you need to press Shift button for the ‘_’ character)
  
• Left Arrow Button: Move the cursor to the left one character.
  
• Right Arrow Button: Move the cursor to the right one character.
  
• Up Arrow and Down Arrow Button: Cycle through the command history.
  

1.1 Passing Special Characters To Command

Some characters are reserved and have some special meaning to the shell just like keywords in a programming language. If you need to pass one of these characters to the Linux shell, you must quote it by enclosing the entire argument in single quotes (‘). This method is particularly use for if you want to pass these special characters as an argument to a command, like for filtering or for file-name.

To pass special characters as a string, give them as:

Code:

$'string'

Where string is the string of characters to be passed. Special backslash escape sequences for certain characters are commonly included in a string, as listed in the following table.

• \a Alert (rings the system bell).
  
• \b Backspace.
  
• \e Escape.
  
• \f Form feed.
  
• \n Newline.
  
• \r Carriage return.
  
• \t Horizontal tab.
  
• \v Vertical tab.
  
• \\ Backslash.
  
• \NNN Character whose ASCII code is NNN in octal (base 8).
  

Here is how to pass backslash as an argument with echo:

Code:

$ echo $'\\'

or a new line character:

Code:

$ echo Hey$'\n'There

To pass a form feed character followed by a pilcrow sign character (octal character code 266) type:

Code:

$ echo $'\f\266'

1.2 Autocompletion

You can use bash’s autocomplete feature to make your typing more easier. To use it, press the TAB button while writing a part of a command or a file name and the shell will complete the word for you. Completion is one of those things that, once you begin to use it, you will wonder how you ever managed to get by without. Completion works on both file names and command names. In normal condition it will automatically complete the command/file/directory name for you, but what if you have multiple folder starting with same characters.

For example on Desktop I have 4 directories:

vmware-tools-install-folder-with-a-really-long-name
vmware-tools-install-folder-with-a-really-long-name-again
vmware-tools-install-folder
vmware-tools-install-folder-again
vmware-tools

I want to go to vmware-tools-install-folder-with-a-really-long-name-again, in cases like these when you have multiple files starting with similar name, pressing TAB will give you a list of available files with that name. Like, when I type vmware and press TAB once, shell will automatically add ‘-tools’:



and then if I press TAB button again, it would do nothing, but if I will press TAB button twice, it will list all files have ‘vmware-tools’ string in it:



now I will add a hyphen (-) and press TAB , the shell will add folder automatically:

Code:

$ cd vmware-tools-install-folder

then when I put ‘-w’ and press TAB again it will add other part again:

Code:

$ cd vmware-tools-install-folder-with-a-really-long-name

Now, add another hyphen and press TAB we will get to the folder we wanted. You just need some practice to get familiar with this feature.

Another example: Suppose you want to specify, as an argument to the ls command, the ‘/usr/lib/emacs/20.4/i386−debian−linux−gnu/’ directory−−that’s a lot to type. So instead of typing out the whole directory name, you can type TAB to complete it for you. Notice how our first attempt, typing only the letter ‘e’ in ‘/e’, brings up a series of files−−while the second attempt, typing ’em’, further refines our search:

Code:

$ ls /usr/lib/e<TAB>
elm−me emacs emacsen−common entity−map expect5.30
$ ls /usr/lib/em<TAB>

At this point, the system beeps and the shell completes the word ’emacs’, since all options in this directory beginning with the letters ’em’ complete to at least that word. Press TAB to access this word and go on, and the shell completes the subdirectory ‘20.4’ since that is the only file or directory in the ’emacs’ subdirectory:

Code:

$ ls /usr/lib/emacs/<strong><TAB></strong>20.4/

Press TAB again to have the shell complete the only subdirectory in ‘20.4’:

Code:

$ ls /usr/lib/emacs/20.4/<strong><TAB></strong>i386−debian−linux−gnu/

1.3 Repeating the last command you typed

Press the up arrow button to put the last command you typed back on the input line. You can then run the command again, or you can edit the command first. By pressing the Up Arrow key more than once, you can go back to earlier commands you’ve typed; this is a function of your command history.

Additionally, you can use the bash reverse−incremental search feature, CTRL+R, to search, in reverse, through your command history. You’ll find this useful if you remember typing a command line with ‘foo’ in it recently, and you wish to repeat the command without having to retype it. Press CTRL+R followed by the text foo, and the last command you typed containing ‘foo‘ appears on the input line. Typing the string ‘cat’ will first search for (and display) the last input line containing a ‘c’, then ‘ca’, and finally ‘cat’, as you type the individual characters of the search string. Pressing CTRL+R again retrieves the next previous command line that has a match for the search string.

To put the last command you entered containing the string ‘grep’ back on the input line, type:

Code:

$ <CTRL+R>
(reverse−i−search)'': grep

To put the third−to−the−last command you entered containing the string grep back on the input line, type:

Code:

$ <CTRL+R>
(reverse−i−search)'': grep
<CTRL+R> <CTRL+R>

When a command is displayed on the input line, press Enter to run it. You can also edit the command line as usual.

1.4 Running Multiple Commands at a time

There are many ways to run multiple commands at a time in linux. Running multiple commands can be useful in cases when we have to run several non-interactive commands in sequence. We can either use semi-colon ( or double ampersand (&&) symbol to separate different commands, and both have different uses.

If you want to execute all commands regardless of whether the previous ones failed or not, separate them with semicolons:

Code:

$ clear; cd my_folder; rm -rf *.jpg; exit

If you want to execute each command only if the previous one succeeded, then combine them using the && operator:

Code:

$ clear && cd my_folder && rm -rf *.jpg && exit

Another way to run multiple commands is putting all commands in a script and executing that instead:

Code:

#! /bin/sh
cd /my_folder \
&& rm *.jar \
&& exit

The first line is called shebang line which we will see later. For now, remember that the shebang line is important for execution of bash scripts. The backslashes at the end of the line are there to prevent the shell from thinking that the next line is a new command; if you omit the backslashes, you would need to write the whole command in a single line.

Save that to a file, for example script, and make it executable:

Code:

chmod +x script

You can now execute that script like other programs on the machine. But if you don’t place it inside a directory listed in your PATH environment variable (for example /usr/local/bin, or on some Linux distributions ~/bin), then you will need to specify the path to that script.If it’s in the current directory, you execute it with:

Code:

./script

2. Redirecting Input and Output

The shell moves text in designated “streams.” The standard output is where the shell streams the text output of commands−−the screen on your terminal, by default. The standard input, typically the keyboard, is where you input data for commands. When a command reads the standard input, it usually keeps reading text until you type CTRL+D on a new line by itself.

When a command runs and exits with an error, the error message is usually output to your screen, but as a separate stream called the standard error. You redirect these streams−−to a file, or even another command−−with redirection. The following sections describe the shell redirection operators that you can use to redirect standard input and output.

2.1 Redirecting Output to Another Command’s Input

Piping is when you connect the standard output of one command to the standard input of another. You do this by specifying the two commands in order, separated by a vertical bar character, ‘|‘ (sometimes called a “pipe“). Commands built in this fashion are called pipelines. We will use piping in other parts of this section.

2.2 Redirecting Input to a File

Standard input, is the source of input data for command line programs. Standard input is by default any text entered from the keyboard. Standard input can be redirected through the use of arguments and redirection operators so that it becomes a source other than the keyboard . An argument, also called a command line argument, is a file name or other data that is provided as an input to a command.

A simple example is provided by the wc command, whose default behavior is to count the number of lines, words and characters in text. Typing this command in at the command line and pressing the ENTER key to start a new line allows any text subsequently entered at the keyboard to become the standard input for wc. When the command is then executed (by pressing CONTROL+D key on a new, blank line), wc counts the numbers of lines, words and characters in the typed-in text and reports the results on the display monitor (i.e., standard output), and the program is terminated.

In the event that one or more input files are provided as arguments to a program, standard input is automatically redirected to become those input files instead of the keyboard. Thus, in the following example the standard input becomes file1 and wc will print (UNIX terminology for write) the totals of lines, words and characters in file1 on the display monitor:

Code:

$ wc file1

Standard input can also be redirected to come from any text file in place of the keyboard by using the input redirection operator, which is represented by a leftward pointing angle bracket ( < ). Thus, the following would redirect the standard input for wc from the keyboard to the file named file1:

Code:

$ wc < file1

The result is the same as in the example in which file1 is used as an argument. However, the mechanism is different. This is consistent with the tenet of the Unix philosophy (which is also fundamental to Linux) that states that it is desirable that there exist more than one way to perform any task.

Instead of obtaining input from the keyboard or from a file, a program can use the output of another program as its input. This is accomplished through the use of a pipe, which is represented by the vertical bar character. For example, the head command (which by default reads the first ten lines of text) could be used to read the first ten lines of file1 and pipe the output to wc:

Code:

$ head file1 | wc

Thus, wc would report the totals for the numbers of lines (which would, of course, be ten), words and characters in the first ten lines of file1.

2.3 Redirecting Output to a File

Use the ‘>‘ operator to redirect standard output to a file. To use it, follow a command with ‘>‘ and the name of the file the output should be written to.

To redirect standard output of the command apropos shell bash to the file ‘commands‘, type:

Code:

$ apropos shell bash > commands

If you redirect standard output to an existing file, it will overwrite the file, unless you use the ‘>>‘ operator to append the standard output to the contents of the existing file.

To append the standard output of apropos shells to an existing file ‘commands‘, type:

Code:

$ apropos shells >> commands

Output from one command line program can be redirected to become the input of another command line program by using the pipe operator, which is represented by a vertical bar character. Thus, for example, the list of file names and types from the above examples could be redirected to the grep command, whose function is to search text for specified strings (i.e., sequences of characters). In the following example, grep is used to find any lines in that list which contain the string directory and to send those lines to standard output (which will be the display screen because grep‘s output is not redirected):

Code:

$ file * | grep directory

To carry the example further, the output of grep could easily be redirected from the display screen to a file, to a printer or to another program. For example, it could be redirected to the wc program, whose default behavior is to count the number of lines, words and characters in a file or other input.

The following pipeline of commands (i.e., two or more commands connected by a pipe or pipes) redirects grep‘s standard output to become the input for wc -l.

Code:

$ file * | grep directory | wc -l

The -l option tells wc to only count the number of lines and to ignore the counts of words and characters. The number of lines is the same as the number of directories in the current directory, because each directory is listed on a separate line (as is each file of every other type).

The output from wc -l can likewise be redirected from the display monitor, for example to a file named file4:

Code:

$ file * | grep directory | wc -l > file4

2.4 Redirecting Error to a File

To redirect the standard error stream to a file, use the ‘>‘ operator preceded by a ‘2‘. Follow a command with 2> and the name of the file the error stream should be written to.

To redirect the standard error of apropos shell bash to the file ‘errorfile‘, type:

Code:

$ apropos hello 2> errorfile

As with the standard output, use the ‘>>‘ operator instead of ‘>‘ to append the standard error to the contents of an existing file.

Code:

$ apropos hello 2>> errorfile

To redirect both standard output and standard error to the same file, use ‘&>‘ instead.

Code:

$ apropos shells &> commands

3. Managing Jobs/Processes

The processes you have running in a particular shell are called your jobs. You can have more than one job running from a shell at once, but only one job can be active at the terminal, reading standard input and writing standard output. This job is the foreground job, while any other jobs are said to be running in the background. The shell assigns each job a unique job number. Use the job number as an argument to specify the job to commands.

3.1 Stopping or Aborting a Job

Press CTRL+Z to suspend or stop the foreground job−−useful for when you want to do something else in the shell and return to the current job later. The job stops until you either bring it back to the foreground or make it run in the background. This effectively puts it on hold and returns control to the shell, but does not actually kill the job.

For example, if you are reading a document in info, typing CTRL+Z will suspend the info program and return you to a shell prompt where you can do something else . The shell outputs a line giving the job number (in brackets) of the suspended job, the text ‘Stopped’ to indicate that the job has stopped, and the command line itself.

If you are in console mode (not graphical mode) and have any stopped jobs when you log out, the shell will tell you this instead of logging you out:

Code:

$ logout
There are stopped jobs.

CTRL+C causes an interrupt signal to be sent to the program telling it to abort what it is doing and exit immediately. Some programs will hear this signal and do some emergency clean up work on themselves before exiting. Others will not respond to the signal and are subsequently just aborted.

Use kill to interrupt (“kill“) a background job, specifying the job number as an argument. To kill job number 2, type:

Code:

$ kill %2

3.2 Putting a process in background

New jobs always run in foreground unless you specify otherwise. To run a job in the background, end the input line with an ampersand (‘&‘). This is useful for running non−interactive programs that perform a lot of calculations.

To run the command apropos shell > shell−commands as a background job, type:

Code:

apropos shell > shell−commands &
[1] 7791

The shell outputs the job number (in this case, 1) and process ID (in this case, 7791), and then returns to a shell prompt. When the background job finishes, the shell will list the job number, the command, and the text ‘Done’, indicating that the job has completed successfully:

Code:

[1]+  Done                    apropos shell > shell−commands

To move a job from foreground to background, first suspend it and then use bg command. For example, start a command info apropos in foreground then suspend it and specify it to complete in background using bg command:

Code:

$ info apropos
<CTRL+Z>
[1]+  Stopped                 info apropos
$ bg
[1]+ info apropos &

If you have suspended multiple jobs, specify the job to be put in the background by giving its job number as an argument. To run job 4 in the background, type:

Code:

$ bg %4

3.3 Putting a job in foreground

Type fg to move a background job to the foreground. By default, fg works on the most recent background job.

Code:

$ fg

To move a specific job to the foreground when you have multiple jobs in the background, specify the job number as an option to fg. To bring job 3 to the foreground, type:

Code:

$ fg %3

3.4 Listing Jobs

To list the jobs running in the current shell, type jobs.

Code:

$ jobs
[1]+  Stopped                 info apropos
[2]-  Done                    apropos shell > shell-comands

This example shows two jobs info apropos and apropos shell > shell-comands. The ‘+‘ character next to a job number indicates that it’s the most recent job, and the ‘−‘ character indicates that it’s the job previous to the most recent job. If you have no current jobs, jobs returns nothing.

4. Command History

Command history is the sequential list of commands you have typed, in the current or previous shell sessions. The commands in this history list are called events. By default bash remembers the last 500 events, but you can change this. It is stored in your home directory in a file called ‘.bash_history’ you can view this file or edit it like any other text file. In this section we will see how to view your history and specify events from it on the command line.

4.1 Viewing your command history

Use history command to view your command history:

Code:

$ history
1 cd /media/cdrom
2 cp VMwareTools-10.0.0-2977863.tar.gz /root/Desktop
3 cd && cd Desktop
4 tar -xvf VMwareTools-10.0.0-2977863.tar.gz
....

This command shows the contents of your command history file, listing one command per line prefaced by its sequence number. Use an event number to specify that event in your history. If your history is a long one you may want to pipe the output to utility such as less in order to read it. It’s also common to search for a past command by piping the output to grep.

To search your history for the text ‘apropos‘, type:

Code:

$ history | grep apropos
137       apropos < keywords
165       apropos < a.txt
166       apropos browser
185       head a | apropos

This command will show the events from your history containing the text ‘apropos‘. (The last line of output is the command you just typed.)

4.2 Specifying a command from history

You can specify a past event from your history on the input line, in order to run it again. The simplest way to specify a history event is to use the up and down arrow keys at the shell prompt to browse your history. The up arrow key (@uparrow) takes you back through past events, and the down arrow key (@downarrow) moves you forward into recent history. When a history event is on the input line, you can edit it as normal, and press Enter to run it as a command; it will then become the newest event in your history.

For ex. To specify the second−to−the−last command in your history, type:

Code:

$ @uparrow @uparrow

To run a history event by its event number, enter an exclamation point (‘!‘, sometimes called “bang“) followed by the event number. To run event number 1:

Code:

$ !1

4.3 Recording a Shell session

Use script command to create a typescript, or “capture log,” of a shell session it writes a exact copy of your session to a file, including commands you type and their output. The first and last lines of the file show the beginning and ending time and date of the capture session.

To stop recording the typescript, type exit at a shell prompt. By default, typescripts are saved to a file called ‘typescript‘ in the current directory If you want to use any other name for the file you can specify it as an argument.

To create a typescript of a shell session and save it to the file log.001‘, type:

Code:

$ script log.001
Script started, output file is log.001
$ hostname
geekizer
$ apropos bash > bash.commands
$ exit
exit
Script done, output file is log.001

After it is done, if you open the log.001 file, you will see exact copy of your shell session between that interval.

NOTE: It’s possible, but usually not desirable, to run script from within another script session. This usually happens when you’ve forgotten that you are running it, and you run it again inside the current typescript, even multiple times−−as a result, you may end up with multiple sessions “nested” inside each other.

Thanks for reading our tutorial. In next tutorial, we will read about shell customization. Please share this tutorial with others.

[Ajax]

Awesome Guide

Thanks for that Bro!

Ajax

[Cyber Warrior]

Very nice tutorial Sherlock, useful informations for beginners. Thank you for your time writing this.

[pvnk]

Very nice guide. I must ask, do you write these up from scratch?

[DarkMuse]

I really appreciate the in-depth tutorial. Very useful so bookmarked. Cheers!

[§herlock]

Very nice guide. I must ask, do you write these up from scratch?

No, I have already mentioned sources I used.

[pvnk]

No, I have already mentioned sources I used.

I don't see them.

[Loki123]

I don't see them.

There is a link at the top of his post in the text. Quite hidden actually but it is there.