Introduction

Indeed during my college days I used to hate programming, it was difficult to understand the objective behind using programs and their purpose. Understanding how it was running on the system was another heck of a challenge. Demystifying essential components of an operating system is indeed a challenge unless you work on Linux.

Linux has been my first love, since the day I came across it. It helps you understand the fundamentals of the OS & it's working, which can immensely boost your productivity. Linux is also one of the most popular and most adopted Operating systems in the world right next to Microsoft Windows. 99% of the world's Super Computers are powered with Linux. Everything right from your Cellphone to Cloud-based systems runs on some form of Linux.

I planned to take free training on Linux in which I covered following aspects of the Linux OS. This training is intended for someone who is starting his journey with Linux or someone who already has some familiarity.

Made with

by Amaan

Day1 ( Introduction to Linux )

It was mostly an Introductory class of the workshop, however I had shared my most important advice.

Learning the Art of Doing one thing well

Pick one small thing that will help with my problem
Make Sure I understand it pretty well!
Celebrate & Repeat!

Agenda for Day1

Workshop Kickoff!
Trainer Introduction
The Art of Doing One Thing At a Time!
Some Mind-Blowing Facts about Linux! 🤯
Overview of the Linux Operating System
Linux Distributions
Logging in to Linux system
Kernel

Day2 ( Introduction to Linux File System )

After introducing the course, we covered the most important concept of the Linux operating system i.e the file-system.

Ironically, do you know? Everything in Linux is a file!

Agenda for Day2

Directory Hierarchy
Basics of File
UNIX file system
Inode Structure
Links in Unix

Day3 ( LFS: Exploring File & Directory Attributes )

We saw what attributes does file and directories carry, by understanding how to modify and query them.

Agenda for Day3

Displaying the contents of a file
File and Directory Attributes
File and Directory Permissions
Viewing File and Directory Permissions/Ownership's
Changing the Permissions/Ownership's
Modifying Default Permissions/Ownership's
File Descriptors (Important)

Day4 ( Working with Files and Directories )

We continued our file-system exploration journey by looking at some commands. Also most importantly the file descriptors!

File Descriptors are very important to understand, they are widely used in Shell scripts!

Agenda for Day4

Determining Where you are in the Directory Structure
- pwd (print name of current/working directory)
Viewing File Content ( Revisited )
- cat (concatenate files and print on the standard output)
Copying Files and Directories
- cp (copy files and directories)
Searching Files and Directories
- find (search for files in a directory hierarchy)
- locate (find files by name)
Creating and Removing Files and Directories
- touch (change file timestamps, create blank files)
- vi (Vi Improved, a programmer's text editor)
- nano (Nano's ANOther editor, an enhanced free Pico clone)
The Standard Files
File Descriptors ( 0, 1, 2 ) ( Revisited )
Input Redirection
Output Redirection
Combined Redirection
Pipes ( | )

cat, cp, sort, tee

Day5 ( Filters and Commands )

Filters make it easy to query data in files and make necessary changes. In this module, we covered some important filters used in Linux.

Agenda for Day5

find
Using find
Executing Commands with find ( -exec )
find Examples
locate
locate options / locate examples / updatedb
tr / wc
grep
Basic grep
grep Examples
grep Options ( -A, -B, -C, -l, -r, -i )
sed
sort command & examples
head and tail commands
tee command
/dev/tty and /dev/null special files
wall command

find command

sed command

grep command

Day6 ( Using vi editor )

vi editor is the most widely used editor on Linux operating system, however you will need to invest subsequent amount of time to make yourself familiar with it. In this module we covered some ground on vi editor.

Agenda for Day6

Introducing the vi editor
The vi editor and its modes
Editing with vi
Moving the Cursor
Editing, Copying and Moving Text
Pattern Searching
Repeating the Last Editor Command
Undoing Commands
Using the Shell from inside vi editor

Vi Editor

Day7 ( Process Control & User Management )

Process management and User administration is very important aspect on a multi-user operating system like Linux. We covered some important commands that help you work on these aspects.

Agenda for Day7

Performing basic Process Control
- System Processes Overview
- View a process ( ps )
- Search for a specific process ( pgrep )
- Send a signal to a process ( pkill )
- Terminating processes with the kill command ( kill )
User Administration / Management
- Describing the Fundamentals of User Administration
- Managing User Accounts – Addition, Modification, and password reset of User Accounts
- Managing Initialization Files ( .bashrc, .bash_profile, .bash_history )

User Management

Day8 ( Package Management & System Performance Monitoring )

When working on any OS you will tend to install some applications. Every OS has its own way of installing application and so do Linux. We will cover some ways of installing packages on a Cent-OS & Ubuntu Operating System.

Agenda for Day8

Package Management
- Package Management in Linux
- RPM, YUM, APT, DPKG
System Performance Monitoring
- Process Management and Performance Monitoring
- Viewing System process ( top , ps )
- Monitor System Performance Statistics
- Monitoring CPU, Memory, Swap space, Disk I/O
- Working with vmstat , iostat and SAR command

Ubuntu Package Managers

Redhat Package Manager - YUM

Redhat Package Manager - RPM

Day9 ( Network Management & Performing Remote Connections and File Transfers )

I like to call Linux as a network based operating system. It is very efficiently used as high performance network based system. In this module we will cover some ways of inspecting network on a Linux system & perform some remote connection transfers.

Agenda for Day9

Network Management
- Controlling and Monitoring Network Interfaces
- Checking Available and In-use network interfaces
- Checking Running ports and services
Performing Remote Connections and File Transfers
- Establish a Remote login session – Telnet, SSH, SFTP, etc
- Copy files or directories to and from another system
- Transfer files between systems

Cheatsheets

For the convenience of running commands I have provided cheatsheets for the following.

Made with

by Amaan

User Management

Add Users

Create A User (Called "ashley")

useradd --create-home ashley

Create A User In Groups "users" And "dev"

useradd --create-home --groups users dev ashley

Create A User With The Uid 1337

useradd --create-home --uid 1337 ashley

Create Or Change A User Password

passwd ashley

Add User To A Group ("coder")

usermod --append --groups coder ashley

Permissions

Set Default Permission Of Files To 644

echo "masc 022" >> /etc/profile

Set Default Permission Of Files To 664

echo "umask 002" >> /etc/profile

Change Ownership Of A File ("example.txt") To User ("ashley")

chown ashley:users example.txt

Give Read Permission To User, Group, And Others

chmod ugo+r example.txt

Give Write Permission To User And Group

chmod ug+w example.txt

Remove Write Permission To A File For Group Members

chmod g-w example.txt

Make A File Executable

chmod +x example.txt

su - ashley

Run A Command ("ls") As A Different User

su - ashley --command ls

Delete Users

Remove A User From A Group ("coder")

gpasswd --delete ashley coder

Delete A User From The System

userdel ashley

Delete A User And All User Data From The System

userdel --remove ashley

History

Show Which Users Are Currently Logged In

Show Login History

last

Package Managers of Ubuntu

apt, apt-get and dpkg are Package manager's of Ubuntu Linux.

To check the list of packages available in the repositories

  apt list | grep package-name
  apt-cache pkgnames | grep package-name
  apt-cache search package-name
  apt-file search/find package-name

To list the installed packages of the system

  apt list --installed | grep package-name
  dpk --get-selections

Also you can evaluate the /var/log/apt/history.log file for looking at the history of packages installed.

To get the list of files installed from a particular package

  apt-file list package-name | grep conf$
  dpkg-query -L package-name | grep filename

To update the packages available in the system

  apt-get update
  apt update

To upgrade the packages in the system

  apt-get upgrade
  apt upgrade

To hold a package for the upgrade/update

  apt-mark hold package-name
  echo "package-name    hold" | dpkg --set-selections

To get the details of the packages kept on hold

  apt-mark showhold
  dpkg --get-selections | grep hold

To unhold the package for the system update

  apt-mark unhold
  echo "package-name   install" | dpkg --set-selections

To list the packages installed automatically or manually

  apt-mark showauto
  apt-mark showmanual

To see the list of up-gradable packages in the system

  apt list --upgradable
  apt-get upgrade --dry-run

To remove the packages automatically from the system that are not needed anymore

  apt autoremove
  apt-get autoremove - Remove automatically all unused packages
  apt-get autoclean  - Erase old downloaded archive files
  apt-get clean      - Erase downloaded archive files

To see the list of sources of repositories configured

  apt edit-sources	-- sources file /etc/apt/sources.list.

This gives you information of all available package versions

  apt-cache policy <packageName>

To install a package in the system

  apt install package-name
  apt-get install package-name
  dpkg -i package-name.deb

To erase the package from the system

  apt remove package-name
  apt-get remove package-name  - Remove the package
  apt-get purge package-name   - Remove the package/configuration files

To go for the distribution update

  apt-get dist-upgrade
  do-release-upgrade

To check the package for broken dependencies

  apt-get check package-name

`apt-add-repository` is a script for adding apt sources.list entries

apt-add-repository <source-line> - The apt repository source line to add. This is one of:

a complete apt line in quotes,
a repo url and areas in quotes (areas defaults to'main')
a PPA shortcut.
a distro component

To get the changelog information of some package (so we can check for the vulnerability updates viz: CVE)

apt-get changelog package-name | grep -i cve
rpm -q --changelog package-name   - Incase of rpm based distro
aptitude changelog package-name   - Incase of the aptitude pkg manager.

Redhat Package Manager ( YUM )

yum is the package manager for Redhat based system.

Help display yum commands and options

yum help

Individual packages

List package names from repositories

yum list available

List all available packages

yum list installed

List all installed packages

yum list all

List installed and available packages

yum list kernel

List info about vsftpd package

yum info vsftpd

Display dependencies for a package (deplist)

yum deplist nfs-utils

Find packages that provide the queried file

Show package that contains top command

yum provides “*bin/top”

Show package containing README.top file

yum provides “*/README.top”

Search package names and descriptions for a term

Find packages with samba in name or description

yum search samba

Get information about available package updates

yum updateinfo security

Groups of packages

Display description and contents of a package group

yum groupinfo “Web Server”

Manage Yum Repositories

Display information about enabled yum repositories

yum repoinfo rhel-7-server-rpms

See info on rhel-7-server-rpms repo

yum repo-pkgs my-rpms list

List packages from my-rpms repo

yum repo-pkgs my-rpms install

Remove all packages from my-rpms repo

yum repo-pkgs my-rpms remove

Troubleshoot And Maintain Yum

List all yum install, update and erase actions

yum history list

Show details of yum transaction 3

yum history info 3

Undo the yum action from transaction 3

yum history undo 3

Clear out cached package data

yum clean packages

Clean out all packages and meta data from cache

yum clean all

Install, Remove And Upgrade Packages With Yum

Install the vsftpd package

yum install vsftpd

Update one or all packages on your system

yum update

Update the httpd package (if available)

yum update httpd

Apply security-related package updates

yum update --security

Reinstall the current version of a package

yum reinstall util-linux

Downgrade a package to an earlier version

yum downgrade abc

Install abc package from local directory

yum localinstall http://myrepo/abc-1-1.i686.rpm

Remove the vsftpd package and dependencies

yum remove vsftpd

autoremove Same as erase, plus removes additional unneeded packages *

yum autoremove httpd

Popular Options For Different Yum Commands

Option Description

-y                               - Assume yes if prompted
--assumeno                       - Assume no if prompted
-q                               - Produce no output
-v                               - Produce extra debugging output
--noplugins                      - Run command without loading any yum plugins
--disableplugin                  - Disable a particular plugin for single command
--enableplugin                   - Enable a plugin that is installed, but currently disabled
--changelog                      - Display changelog information of package
--enablerepo                     - Enable currently disabled repo for a single command (wildcards okay)

yum install docker --enablerepo=rhel-7-server-extras-rpm

--disablerepo                    - Disable currently enabled repo for a single command (wildcards okay)

yum list available --disablerepo=epel

--downloadonly                   - Download to /var/cache/yum/ arch / prod / repo / packages/, but don’t install

Download vsftpd package to cache
yum install --downloadonly vsftpd

More Yum-Related Commands (install the yum-utils package)

Command Description

find-repos-of-install               - Find which repository a package comes from
needs-restarting                    - Find processes that have been updated and need to restart
repoquery --requires --resolve bash - Show dependent packages
reposync                            - Synchronize yum repositories to a local directory
repotrack                           - Download a package and all its dependencies
show-installed                      - List installed RPM packages and statistics
verifytree                          - Check the local yum repository for consistency
yum-complete-transaction            - Try to complete yum transactions that didn’t finish
yumdb                               - Check or change the yum database
yumdownloader                       - Download a package from a repo to current directory

Redhat Package Manager ( rpm )

rpm is a powerful Package Manager for Red Hat, Suse and Fedora Linux. It can be used to build, install, query, verify, update, and remove/erase individual software packages. A Package consists of an archive of files, and package information, including name, version, and description:

Syntax Description Example(s)

Install the package

rpm -ivh {rpm-file}
rpm -ivh mozilla-mail-1.7.5-17.i586.rpm
rpm -ivh --test mozilla-mail-1.7.5-17.i586.rpm

Upgrade package

rpm -Uvh {rpm-file}
rpm -Uvh mozilla-mail-1.7.6-12.i586.rpm
rpm -Uvh --test mozilla-mail-1.7.6-12.i586.rpm

Erase/remove/ an installed package

rpm -ev {package}
rpm -ev mozilla-mail

Erase/remove/ an installed package without checking for dependencies

rpm -ev --nodeps {package}
rpm -ev --nodeps mozilla-mail

Display list all installed packages

rpm -qa
rpm -qa
rpm -qa | less

Display installed information along with package version and short description

rpm -qi {package}
rpm -qi mozilla-mail

Find out what package a file belongs to i.e. find what package owns the file

rpm -qf {/path/to/file}
rpm -qf /etc/passwd
rpm -qf /bin/bash

Display list of configuration file(s) for a package

rpm -qc {pacakge-name}
rpm -qc httpd

Display list of configuration files for a command

rpm -qcf {/path/to/file}
rpm -qcf /usr/X11R6/bin/xeyes

Display list of all recently installed RPMs

rpm -qa --last
rpm -qa --last
rpm -qa --last | less

Find out what dependencies a rpm file has

rpm -qpR {.rpm-file}
rpm -qR {package}
rpm -qpR mediawiki-1.4rc1-4.i586.rpm
rpm -qR bash

Vi Editor

File management

:e              reload file
:q              quit
:q!             quit without saving changes
:w              write file
:w {file}       write new file
:x              write file and exit

Movement

    k
  h   l         basic motion
    j

w               next start of word
W               next start of whitespace-delimited word
e               next end of word
E               next end of whitespace-delimited word
b               previous start of word
B               previous start of whitespace-delimited word
0               start of line
$               end of line
gg              go to first line in file
G               go to end of file
gk		move down one displayed line
gj		move up one displayed line

Insertion

To exit from insert mode use Esc or Ctrl-C. Enter insertion mode and:

a               append after the cursor
A               append at the end of the line
i               insert before the cursor
I               insert at the beginning of the line
o               create a new line under the cursor
O               create a new line above the cursor
R               enter insert mode but replace instead of inserting chars
:r {file}       insert from file

Editing

u               undo
yy              yank (copy) a line
y{motion}       yank text that {motion} moves over
p               paste after cursor
P               paste before cursor
<Del> or x      delete a character
dd              delete a line
d{motion}       delete text that {motion} moves over

Search and replace with the `:substitute` (aka `:s`) command

:s/foo/bar/	    replace the first match of 'foo' with 'bar' on the current line only
:s/foo/bar/g	replace all matches (`g` flag) of 'foo' with 'bar' on the current line only
:%s/foo/bar/g	replace all matches of 'foo' with 'bar' in the entire file (`:%s`)
:%s/foo/bar/gc	ask to manually confirm (`c` flag) each replacement

Preceding a motion or edition with a number repeats it 'n' times

Examples:

50k         moves 50 lines up
2dw         deletes 2 words
5yy         copies 5 lines
42G         go to line 42

Multiple windows

:e filename      - edit another file
:split filename  - split window and load another file
ctrl-w up arrow  - move cursor up a window
ctrl-w ctrl-w    - move cursor to another window (cycle)
ctrl-w_          - maximize current window
ctrl-w=          - make all equal size
10 ctrl-w+       - increase window size by 10 lines
:vsplit file     - vertical split
:sview file      - same as split, but readonly
:hide            - close current window
:only            - keep only this window open
:ls              - show current buffers
:b 2             - open buffer #2 in this window

Find Command

To find files by case-insensitive extension (ex: .jpg, .JPG, .jpG)

find . -iname "*.jpg"

To find directories

find . -type d

To find files

find . -type f

To find files by octal permission

find . -type f -perm 777

To find files with setuid bit set

find . -xdev \( -perm -4000 \) -type f -print0 | xargs -0 ls -l

To find files with extension `'.txt'` and remove them

find ./path/ -name '*.txt' -exec rm '{}' \;

To find files with extension `'.txt'` and look for a string into them

find ./path/ -name '*.txt' | xargs grep 'string'

To find files with size bigger than 5 Mebibyte and sort them by size

find . -size +5M -type f -print0 | xargs -0 ls -Ssh | sort -z

To find files bigger than 2 Megabyte and list them

find . -type f -size +200000000c -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

To find files modified more than 7 days ago and list file information

find . -type f -mtime +7d -ls

To find symlinks owned by a user and list file information

find . -type l -user <username-or-userid> -ls

To search for and delete empty directories

find . -type d -empty -exec rmdir {} \;

To search for directories named build at a max depth of 2 directories

find . -maxdepth 2 -name build -type d

To search all files who are not in `.git` directory

find . ! -iwholename '*.git*' -type f

To find all files that have the same node `(hard link) as MY_FILE_HERE`

find . -type f -samefile MY_FILE_HERE 2>/dev/null

To find all files in the current directory and modify their permissions

find . -type f -exec chmod 644 {} \;

sed command

To replace all occurrences of "day" with "night" and write to stdout

sed 's/day/night/g' <file>

To replace all occurrences of "day" with "night" within

sed -i 's/day/night/g' <file>

To replace all occurrences of "day" with "night" on stdin

echo 'It is daytime' | sed 's/day/night/g'

To remove leading spaces

sed -i -r 's/^\s+//g' <file>

To remove empty lines and print results to stdout

sed '/^$/d' <file>

To replace newlines in multiple lines

sed ':a;N;$!ba;s/\n//g' <file>

To insert a line before a matching pattern

sed '/Once upon a time/i\Chapter 1'

To add a line after a matching pattern

sed '/happily ever after/a\The end.'

grep command

To search a file for a pattern

grep <pattern> <file>

To perform a case-insensitive search (with line numbers)

grep -in <pattern> <file>

To recursively grep for `string <pattern> in <dir>`

grep -R <pattern> <dir>

Read search patterns from a file (one per line)

grep -f <pattern-file> <file>

Find lines NOT containing pattern

grep -v <pattern> <file>

To grep with regular expressions

grep "^00" <file>                                               # Match lines starting with 00
grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" <file> # Find IP add

To find all files that match `<pattern> in <dir>`

grep -rnw <dir> -e <pattern>

To exclude grep from your grepped output of ps

(Add [] to the first letter. Ex: sshd -> [s]shd)

ps aux | grep '[h]ttpd'

Colour in red {bash} and keep all other lines

ps aux | grep -E --color 'bash|$'

cp command

To copy a file

cp ~/Desktop/foo.txt ~/Downloads/foo.txt

To copy a directory

cp -r ~/Desktop/cruise_pics/ ~/Pictures/

To create a copy but ask to overwrite if the destination file already exists

cp -i ~/Desktop/foo.txt ~/Documents/foo.txt

To create a backup file with date

cp foo.txt{,."$(date +%Y%m%d-%H%M%S)"}

cat command

To display the contents of a file

cat <file>

To display file contents with line numbers

cat -n <file>

To display file contents with line numbers (blank lines excluded)

cat -b <file>

tee command

To tee stdout to

ls | tee <outfile>

To tee stdout and append to

ls | tee -a <outfile>

To tee stdout to the terminal, and also pipe it into another program for further processing

ls | tee /dev/tty | xargs printf "\033[1;34m%s\033[m\n"

sort command

To sort a file

sort <file>

To sort a file by keeping only unique

sort -u <file>

To sort a file and reverse the result

sort -r <file>

To sort a file randomly

sort -R <file>

Q&A

If you have any questions, reach out to me over Linkedin. Or alternatively you can paste your questions in this GoogleSheet. I will answer your questions as time permits.

Although, you may find answers from Google, but if want to understand with someone in-person then don't hesitate to contact me!

Made with

by Amaan

About your Trainer

Hi! I’m Amaan and I spend a lot of my time in front of computers, phones and tablets trying to get my work done as efficiently as possible. I have spent a lot of time using Linux. It really is an obsession of mine as I think we live in the golden age of Open-Source Software's and Tools.

I am always finding out how to do new things that make my work life a little easier and wanted a place to share what I have learned, for which I have created my blog. Feel free to connect with me for Contract & Freelance projects, Linux queries, Interview tips or Just say a 'hi' maybe :)

Linux Essentials Training