What is meaning of bashrc?

bashrc : Bourne Again Shell shell run control

It contains all configurations of the bash shell where it loads before terminal opens for the execution of the commands.

When started as an interactive login shell

Bash reads and executes /etc/profile (if it exists). (Often this file calls /etc/bash.bashrc.)

After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile in that order, and reads and executes the first one that exists and is readable.
When a login shell exits

Bash reads and executes ~/.bash_logout (if it exists).
When started as an interactive shell (but not a login shell)

Bash reads and executes ~/.bashrc (if it exists). This may be inhibited by using the --norc option. The --rcfile file option forces Bash to read and execute commands from file instead of ~/.bashrc.

Creating Linux users and gropus

Commands are typed and executed in a shell terminal. When a terminal is opened, a prompt is available which usually has the following format:

username@hostname$

Or:

root@hostname #

or simply as $ or #.

$ represents regular users and # represents the administrative user root. Root is the most privileged user in a Linux system.


Effected Files:

/etc/passwd,
 /etc/shadow,
/etc/group
/etc/gshadow


We can create groups in a way to group users before or after creating users.

Adding/modifying groups:

groupadd
groupmod
gpasswd
groupdel

Switches -a  : to add users
               -d : to delete users
               -m : to add multiple users

Eg :

#groups - displays group of user


Adding/modifying users:

Switches -a  : to add users
               -d : to delete users
               -m : to add multiple users


useradd
usermod
passwd
userdel
chage

Eg:

#useradd

1. Username: User login name used to login into system. It should be between 1 to 32 charcters long.
2. Password: User password (or x character) stored in /etc/shadow file in encrypted format.
3. User ID (UID): Every user must have a User ID (UID) User Identification Number. By default UID 0 is reserved for root user and UID’s ranging from 1-99 are reserved for other predefined accounts. Further UID’s ranging from 100-999 are reserved for system accounts and groups.
4. Group ID (GID): The primary Group ID (GID) Group Identification Number stored in /etc/group file.
5. User Info: This field is optional and allow you to define extra information about the user. For example, user full name. This field is filled by ‘finger’ command.
6. Home Directory: The absolute location of user’s home directory.
7. Shell: The absolute location of a user’s shell i.e. /bin/bash.

Linux releases latest 4.0 kernel

Linux releases latest 4.0 kernel.

Linux kernel is the essential part of any Linux operating system. It is responsible for resource allocation, low-level hardware interfaces, security, simple communications, basic file system management, and more. Written from scratch by Linus Torvalds (with help from various developers), Linux is a clone of the UNIX operating system. It is geared towards POSIX and Single UNIX Specification compliances. 

Installing/Upgrading kernel

DOWNLOAD kernel from https://www.kernel.org/

$ sudo dpkg -i linux-headers-3.18.3*.deb linux-image-3.18.3*.deb

$ sudo update-grub

$ sudo reboot

Linux - all Hadoop ports

Web UIs for the Common User

The default Hadoop ports are as follows:

  Daemon Default Port Configuration Parameter
HDFS Namenode 50070 dfs.http.address
Datanodes 50075 dfs.datanode.http.address
Secondarynamenode 50090 dfs.secondary.http.address
Backup/Checkpoint node? 50105 dfs.backup.http.address
MR Jobracker 50030 mapred.job.tracker.http.address
Tasktrackers 50060 mapred.task.tracker.http.ad


Under the Covers for the Developer and the System Administrator

Internally, Hadoop mostly uses Hadoop IPC to communicate amongst servers. (Part of the goal of the Apache Avro project is to replace Hadoop IPC with something that is easier to evolve and more language-agnostic; HADOOP-6170 is the relevant ticket.) Hadoop also uses HTTP (for the secondarynamenode communicating with the namenode and for the tasktrackers serving map outputs to the reducers) and a raw network socket protocol (for datanodes copying around data).

The following table presents the ports and protocols (including the relevant Java class) that Hadoop uses. This table does not include the HTTP ports mentioned above.

Daemon Default Port Configuration Parameter Protocol Used for
Namenode 8020 fs.default.name? IPC: ClientProtocol Filesystem metadata operations.
Datanode 50010 dfs.datanode.address Custom Hadoop Xceiver: DataNode and DFSClient DFS data transfer
Datanode 50020 dfs.datanode.ipc.address IPC: InterDatanodeProtocol, ClientDatanodeProtocol
ClientProtocol Block metadata operations and recovery
Backupnode 50100 dfs.backup.address Same as namenode HDFS Metadata Operations
Jobtracker Ill-defined.? mapred.job.tracker IPC: JobSubmissionProtocol, InterTrackerProtocol Job submission, task tracker heartbeats.
Tasktracker 127.0.0.1:0¤ mapred.task.tracker.report.address IPC: TaskUmbilicalProtocol Communicating with child jobs
? This is the port part of hdfs://host:8020/.
? Default is not well-defined. Common values are 8021, 9001, or 8012. See MAPREDUCE-566.
¤ Binds to an unused local port.

Linux distributions package management

dpkg, used originally by Debian and now by other systems like Ubuntu, uses the .deb format and was the first to have a widely known dependency resolution tool (APT).

RPM Package Manager, created by Red Hat and used by a number of other Linux distributions. RPM is the Linux Standard Base packaging format and the base of a number of additional tools, including apt4rpm; Red Hat's up2date; Mandriva's urpmi; openSUSE's ZYpp; PLD Linux's poldek; and YUM, which is used by Fedora, Red Hat Enterprise Linux, and Yellow Dog Linux.

tgz package system that combines the standard tar and gzip. Used by Slackware Linux and its closer derivates, there are a few higher-level tools that use the same tgz packaging format, including: slapt-get, slackpkg, zendo, netpkg, and swaret.

Pacman for Arch Linux, Frugalware and DeLi Linux uses pre-compiled binaries distributed in a compressed Tar archive.

Smart Package Manager, used by CCux Linux

ipkg, a dpkg-inspired, very lightweight system targeted at storage-constrained Linux systems such as embedded devices and handheld computers. It is used on, among others, HP's webOS.

opkg, fork of ipkg

pkgutils, used by CRUX Linux

PETget, used by Puppy Linux

Upkg, a package management and build system based on Mono and XML specifications, used by paldo GNU/Linux and previously by ExTiX Linux

PISI, used by Pardus
appbrowser, a special purpose tool in Tiny Core Linux for browsing and selecting applications from online repositories.

Conary, used by Foresight Linux

Equo, used by Sabayon Linux

Tazpkg - for Slitaz, uses LZMA compressed archives

linuxbrew - A fork of Homebrew for Linux. It does not require sudo.

eopkg, used by Evolve OS Linux

X Window System in linux - GNOME ,UNITY,Cinnamon ,MATE ,KDE ,Xfce

The X Window System (X11, X, and sometimes informally X-Windows) is a windowing system for bitmap displays, common on UNIX-like computer operating systems.

X provides the basic framework for a GUI environment: drawing and moving windows on the display device and interacting with a mouse and keyboard. X does not mandate the user interface — this is handled by individual programs. As such, the visual styling of X-based environments varies greatly; different programs may present radically different interfaces.

Different X Window System Environments in Linux :

GNOME 

GNOME 3 is an easy and elegant way to use your computer. It is designed to put you in control and bring freedom to everybody. GNOME 3 is developed by the GNOME community, a diverse, international group of contributors that is supported by an independent, non-profit foundation.

UNITY

Unity is a graphical shell for the GNOME desktop environment developed by Canonical Ltd. for its Ubuntu operating system. Unity debuted in the netbook edition of Ubuntu 10.10. It was initially designed to make more efficient use of space given the limited screen size of netbooks, including, for example, a vertical application switcher called the launcher, and a space-saving horizontal multipurpose top menu bar.

Unlike GNOME, KDE Software Compilation, Xfce, or LXDE, Unity is not a collection of applications but is designed to use existing programs.

Cinnamon 

Cinnamon is a desktop environment based on GTK+ 3. The project originally started as a fork of the GNOME Shell, i.e. a mere graphical shell. Cinnamon was initially developed by (and for) the Linux distribution Linux Mint. It tries to emulate the Windows 7 and older look and feel and is marketed to people who are uncomfortable with more modern Linux user interfaces such as Ubuntu Unity or GNOME 3.

MATE

The MATE Desktop Environment is the continuation of GNOME 2. It provides an intuitive and attractive desktop environment using traditional metaphors for Linux and other Unix-like operating systems.

KDE

The KDE® Community is an international technology team dedicated to creating a free and user-friendly computing experience, offering an advanced graphical desktop, a wide variety of applications for communication, work, education and entertainment and a platform to easily build new applications upon. We have a strong focus on finding innovative solutions to old and new problems, creating a vibrant atmosphere open for experimentation.

Xfce

Xfce is a lightweight desktop environment for UNIX-like operating systems. It aims to be fast and low on system resources, while still being visually appealing and user friendly.

Command Line Tools to Monitor Linux Performance

Top – Linux Process Monitoring

Linux Top command is a performance monitoring program which is used frequently by many system administrators to monitor Linux performance and it is available under many Linux/Unix like operating systems.

Lsof – List Open Files

Lsof command used in many Linux/Unix like system that is used to display list of all the open files and the processes. The open files included are disk files, network sockets, pipes, devices and processes.


VmStat – Virtual Memory Statistics

Linux VmStat command used to display statistics of virtual memory, kernerl threads, disks, system processes, I/O blocks, interrupts, CPU activity and much more.

Netstat – Network Statistics

Netstat is a command line tool for monitoring incoming and outgoing network packets statistics as well as interface statistics.

Tcpdump – Network Packet Analyzer

Tcpdump one of the most widely used command-line network packet analyzer or packets sniffer program that is used capture or filter TCP/IP packets that received or transferred on a specific interface over a network.

Htop – Linux Process Monitoring

Htop is a much advanced interactive and real time Linux process monitoring tool. This is much similar to Linux top command but it has some rich features like user friendly interface to manage process, shortcut keys, vertical and horizontal view of the processes and much more.

Iostat – Input/Output Statistics

IoStat is simple tool that will collect and show system input and output storage device statistics.


Iotop – Monitor Linux Disk I/O

Iotop is also much similar to top command and Htop program, but it has accounting function to monitor and display real time Disk I/O and processes.

Monit – Linux Process and Services Monitoring

Monit is a free open source and web based process supervision utility that automatically monitors and managers system processes, programs, files, directories, permissions, checksums and filesystems.


IPTraf – Real Time IP LAN Monitoring

IPTraf is an open source console-based real time network (IP LAN) monitoring utility for Linux.

Psacct or Acct – Monitor User Activity

psacct or acct tools are very useful for monitoring each users activity on the system. Both daemons runs in the background and keeps a close watch on the overall activity of each user on the system and also what resources are being consumed by them.

Suricata – Network Security Monitoring

Suricata is an high performance open source Network Security and Intrusion Detection and Prevention Monitoring System for Linux, FreeBSD and Windows.

 NetHogs – Monitor Per Process Network Bandwidth

NetHogs is an open source nice small program (similar to Linux top command) that keeps a tab on each process network activity on your system. It also keeps a track of real time network traffic bandwidth used by each program or application.

Monitorix – System and Network Monitoring

Monitorix is a free lightweight utility that is designed to run and monitor system and network resources as many as possible in Linux/Unix servers. It has a built in HTTP web server that regularly collects system and network information and display them in graphs.

Arpwatch – Ethernet Activity Monitor

Arpwatch is a kind of program that is designed to monitor Address Resolution (MAC and IP address changes) of Ethernet network traffic on a Linux network.

Collectl: All-in-One Performance Monitoring Tool
Collectl is a yet another powerful and feature rich command line based utility, that can be used to gather information about Linux system resources such as CPU usage, memory, network, inodes, processes, nfs, tcp, sockets and much more.

 VnStat PHP – Monitoring Network Bandwidth

VnStat PHP a web based frontend application for most popular networking tool called “vnstat“. VnStat PHP monitors a network traffic usage in nicely graphical mode.


Nagios – Network/Server Monitoring

Nagios is an leading open source powerful monitoring system that enables network/system administrators to identify and resolve server related problems before they affect major business processes.

iftop – Network Bandwidth Monitoring

iftop is another terminal-based free open source system monitoring utility that displays a frequently updated list of network bandwidth utilization (source and destination hosts) that passing through the network interface on your system. iftop is considered for network usage, what ‘top‘ does for CPU usage.

Nmon: Monitor Linux Performance

Nmon (stands for Nigel’s performance Monitor) tool, which is used to monitor all Linux resources such as CPU, Memory, Disk Usage, Network, Top processes, NFS, Kernel and much more.