Linux/Tools

alias
alias: manage command aliases

To list aliases: alias alias [ALIAS] alias -p [ALIAS]

To create alias: alias [ALIAS]="[COMMAND]"

To remove alias: unalias [ALIAS]

CentOS 5 Examples: alias cp='cp -i' alias l.='ls -d .* --color=tty' alias ll='ls -l --color=tty' alias ls='ls --color=tty' alias mv='mv -i' alias rm='rm -i' alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'

alias ls='ls -CF --color=tty'

To pass a parameter to alias, use a function instead: function foo { /path/to/command "$@" ;}

badblock
badblocks - search a device for bad blocks

Non Destructive read test: badblocks -s /dev/sdb
 * 1) -s show

Destructive write test: badblocks -s -w -t 0xff /dev/sdb
 * 1) -s show, -w write, -t test pattern

bc
bc - An arbitrary precision calculator language

Note: Default scale is 0, which means no decimal

echo "scale=2 ; 1/2" | bc echo '(1 + sqrt(5))/2' | bc -l echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bc   # This shows max FastE packet rate: echo 'obase=16; ibase=10; 64206' | bc   # Base conversion (decimal to hexadecimal) seq 100 | (tr '\n' +; echo 0) | bc      # Add a column of numbers. echo 'ibase=16; 2DEC' | bc              # Base conversion (hexadecimal to default decimal) echo 'obase=16; ibase=10; 64206' | bc   # Base conversion (decimal to hexadecimal) echo 'obase=10; ibase=16; 2DEC' | bc    # Base conversion (hex to dec)

Or just use python! echo "print 10 / 2.0" | python

cut
cut - remove sections from each line of files

echo "one,two,three" | cut -f 1 -d "," # one echo "one,two,three" | cut -b 1-2 # on echo "one,two,three" | rev | cut -d, -f 1 | rev  # three (last field)

other cool alternatives:

echo "one two three" | awk '{print $NF}' # three (last field) echo "one,two,three" | awk -F, '{print $NF}' # three (last field) echo "/string/to/cut.txt" | awk -F'/' '{for (i=1; i<NF; i++) printf("%s/", $i)}' awk -F, '{print $NF}' file

echo $LINE | grep -o '.*/'

basename /dev/sdb # sdb dirname /dev/sdb # /dev

dmesg
dmesg - print or control the kernel ring buffer

Display kernel ring: dmesg

Clear kernel ring: dmesg -c

Initial kernel ring at boot is saved to: /var/log/dmesg

Kernel ring is also logged to: /var/log/messages

du
du - estimate file space usage
 * Summarize disk usage of each FILE, recursively for directories.

du --si du -B M du --max-depth=1 du -c dir1 dir2 # combined total

---

Find large files

Print largest files and size in bytes find. -xdev -printf '%s %p\n' |sort -nr | head -20

find out top 10 largest file/directories is taking up the most space in a /var directory/file system: du -a /var | sort -n -r | head -n 10

more human readable output try: du -ks /var | sort -n -r | head -n 10

References:
 * Linux Find Large Files - http://www.cyberciti.biz/faq/find-large-files-linux/
 * How do I find the largest top 10 files and directories on a Linux / UNIX / BSD filesystem? - http://www.cyberciti.biz/faq/how-do-i-find-the-largest-filesdirectories-on-a-linuxunixbsd-filesystem/

less
less - opposite of more

cat /etc/passwd | less

Case insensitivity less -i [FILE] # --ignore-case

Case sensitivity can be toggled within the program by typing "-i". 

ln
ln - make links between files

Symbolic link: ln -s [TARGET] [ [LINK] ]

Hard link: ln [TARGET] [ [LINK] ]

Dereference links: readlink [link] # best option ls -l [link] | awk '{print $11}' # ok option file [link] | awk '{print $5} # includes some ugly quotes

To find all hard links to a file: find [BASEPATH] -xdev -samefile [LINK]

logger
logger - a shell command interface to the syslog(3) system log module

Log message to syslog

logger "message"

logger -t DNS-Made-Easy -s "Problem updating DNS record."

logger -t mail.info test

mkinitrd
mkinitrd - is a compat wrapper, which calls dracut to generate an initramfs

mkinitrd [image] [kernel-version] -f  # overwrite if image exists -v  # verbose

mkfs.vfat
Install: yum install dosfstools

vfat: mkfs -t vfat /dev/sdg1

fdisk /dev/sdb # n p 1 enter enter t 1 c w (part type: 0b or 0c) mkfs.vfat -F 32 /dev/sdb1   # or mkfs.msdos mkfs.vfat -F 32 /dev/sdb1 -n MOVIES   # or mkfs.msdos

partition type: b W95 FAT32         # BIOS INT 13 - Partitions up to 2047GB c W95 FAT32 (LBA)   # Extended-INT13 equivalent of 0b

"0x0b (FAT32 without LBA) uses the old BIOS INT 13 which means it can address a maximum of 7.8GB disk space"

modinfo
modinfo - program to show information about a Linux Kernel module

$ modinfo skge filename:      /lib/modules/2.6.18-92.1.18.el5/kernel/drivers/net/skge.ko version:        1.6 license:       GPL

mount
mount /dev/sdb1 /mnt

nice
Nice and ReNice

-20 high priority +19 low priority

A high nice value means a low priority for your process: you are going to be nice. A low or negative value means high priority: you are not very nice. The range of allowable niceness values is -20 to +19.

renice
 * Renice alters the scheduling priority of one or more running processes.

renice +1 987 -u daemon root -p 32
 * 1) change the priority of process ID's 987 and 32, and all processes owned by users daemon and root.

$ nice -n 5 ~/bin/longtask  # Lowers priority (raise nice) by 5 $ sudo renice -5 8829       # Sets nice value to -5 $ sudo renice 5 -u boggs    # Sets nice value of boggs's procs to 5

References:
 * Major Tom, This is Job Control: Acting Nice with the Linux System - http://www.linuxplanet.com/linuxplanet/tutorials/2116/1/
 * Linux Process Control - http://www.comptechdoc.org/os/linux/usersguide/linux_ugprocesses.html
 * Linux.com :: Influence scheduling priority with nice and renice - http://www.linux.com/archive/feed/58638

nohup
nohup - run a command immune to hangups, with output to a non-tty

nohup [APPLICATION] &

Make sure to add the '&' to put the process into the background, or you will have to do it manually with: [ctrl]+z jobs bg %1 To nohup a running process: [ctrl]+z jobs bg %1 disown %1

parted
GNU Parted - a partition manipulation program

Used by OpenELEC to create SD card partitions: (create_sdcard) DISK = /dev/sdh

parted -s /dev/sdh mklabel msdos
 * 1) writing new disklabel on $DISK (removing all partitions)...

parted -s "$DISK" unit cyl mkpart primary fat32 -- 0 16 parted -s "$DISK" unit cyl mkpart primary ext2 -- 16 -2
 * 1) creating partitions on $DISK...

parted -s "$DISK" set 1 boot on
 * 1) make partition active (bootable)

partprobe "$DISK"
 * 1) tell kernel we have a new partition table

mkfs.vfat "$PART1" -I -n System mkfs.ext4 "$PART2" -L Storage
 * 1) create filesystem
 * 2) creating filesystem on $PART1...
 * 1) creating filesystem on $PART2...

sync
 * 1) sync disk

printf
printf - format and print data

printf: printf "%'.2f" var printf "%'.2d" var printf "Total Rs.%'.2f" var printf "Total $.%'.2f" var printf "%'.2f\n" $x

References:
 * Unix / Linux: Bash Number Currency Formatting Thousands Grouping Separator - http://www.cyberciti.biz/faq/unix-linux-bash-number-formatting-in-with-thousand-separator/

scp
scp file user@host:/some/path/ scp user@host:/some/path/file /some/path scp -P 22 file user@host:/some/path/ scp -i myidentity file user@host:/some/path/

smartctl
SMART Tools

Smartmon Tools

Get Disk Information: smartctl -a /dev/sda

Turn on SMART: smartctl -s on -o on -S on /dev/hda
 * 1) -s on  # Enable/disable SMART on device (on/off)
 * 2) -o on  # Enable/disable automatic offline testing on device (on/off)
 * 3) -S on  # Enable/disable Attribute autosave on device (on/off)

How long has this disk (system) been powered on in total: smartctl -A /dev/sda | grep Power_On_Hours

smartctl -i /dev/hda smartctl -H /dev/hda smartctl -t short /dev/hda smartctl -l selftest /dev/hda

References:
 * Know when your drives are failing, with smartd | Linux Journal

ssh
ssh user@host ssh user@host some_command ssh -p 22 user@host scp -i myidentity user@host

---

Keep Alive:

ssh -o ServerAliveInterval=30 user@host
 * 1) send keep alive every 30 seconds

References:
 * An ssh keep-alive tip
 * Keeping SSH Sessions Alive
 * OpenSSH ClientAliveInterval
 * Slow SSH connect to N800 (ServerAliveInterval)
 * SSH connection stability

strace
strace - trace system calls and signals

List files opened by command: strace -e trace=open vmkload_mod -s iomemory-vsl > /dev/null

Summarise/profile system calls made by command: strace -c ls >/dev/null

List system calls made by command: strace -f -e open ls >/dev/null

tar
tar - GNU ‘tar’ saves many files together into a single tape or disk archive, and can restore individual files from the archive.

List tar contents: tar -tf file.tar

Build tar: tar -cf file.tar file1 file2 file3 tar -cf file.tar folder1 tar -cf file.tar -C folder1. # chdir, so don't include folder in tar cd folder1 ; tar -cf ../file.tar * # build tar of current folder, but don't include tar -cf file.tar *                 # make archive of all files, will have issues second run

Extract tar: tar -xf file.tar tar -xf file.tar file1           # only extract file1 tar -xf file.tar -C folder1      # chdir, then extract

Options: -h, --dereference	# dereference links: (copy actual file, not link) -A, --catenate, --concatenate	# append tar files to an archive -c, --create		#create a new archive -t, --list		#list the contents of an archive -u, --update		#only append files that are newer than the existing in archive -x, --extract, --get	#extract files from an archive -C, --directory DIR	# change to directory DIR

Excludes: tar --exclude='./folder' --exclude='./upload/folder2' somefolder -zcvf /backup/deploy.tgz tar -xvf deploy.tgz --exclude '.htaccess'
 * 1) archive
 * 1) extract

time
time: time a process

Time a process: time [process]

Example: $ time sleep 10 real   0m10.006s user   0m0.001s sys    0m0.003s

Capture time: $ ( time sleep 10 ) 2>&1 1>/dev/null | grep real | awk '{print $2}' 0m10.011s

Convert to Seconds: OUTPUT=$( ( time staf $IPPREFIX$1 ping ping ) 2>&1 ) FULLTIME=$( echo "$OUTPUT" | grep real | awk '{print $2}' | awk -F . '{ print $1 }' ) MTIME=$( echo $FULLTIME | awk -F 'm' '{print $1}' ) STIME=$( echo $FULLTIME | awk -F 'm' '{print $2}' ) TOTAL_SECONDS=$(( $MTIME * 60 + $STIME ))

touch
touch - change file timestamps

create file: touch [file]

set file time: touch -c -t 0304050607 [file]

touch -d "2004-02-27 14:19:13.489392193 +0530" [file] touch --date="2004-02-27 14:19:13.489392193 +0530" [file] touch --date "2004-02-27 0:0:0" [file] # 1 hour behind?

tr
tr - translate or delete characters

Convert to upper or lower case: ... | tr [:lower:] [:upper:] ... | tr [:upper:] [:lower:]

tree
tree - list contents of directories in a tree-like format.

Directory tree: tree

Shell script way: ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/  /' -e 's/-/|/'

Output as CSV file: find | sed -e's/"/\\"/g' -e's/^..//' -e's/\//","/g' -e's/^/"/' -e's/$/"/' > /tmp/listing.csv

http://www.centerkey.com/tree/tree.sh:
 * 1)  UNIX TREE                                          #
 * 2)  Version: 2.3                                       #
 * 3)  File: ~/apps/tree/tree.sh                          #
 * 4)  Displays Structure of Directory Hierarchy          #
 * 5)  This tiny script uses "ls", "grep", and "sed"      #
 * 6)  in a single command to show the nesting of         #
 * 7)  sub-directories.  The setup command for PATH       #
 * 8)  works with the Bash shell (the Mac OS X default).  #
 * 9)  Setup:                                             #
 * 10)     $ cd ~/apps/tree                                #
 * 11)     $ chmod u+x tree.sh                             #
 * 12)     $ ln -s ~/apps/tree/tree.sh ~/bin/tree          #
 * 13)     $ echo "PATH=~/bin:\${PATH}" >> ~/.profile      #
 * 14)  Usage:                                             #
 * 15)     $ tree [directory]                              #
 * 16)  Examples:                                          #
 * 17)     $ tree                                          #
 * 18)     $ tree /etc/opt                                 #
 * 19)     $ tree ..                                       #
 * 20)  Public Domain Software -- Free to Use as You Like  #
 * 21)  http://www.centerkey.com/tree  -  By Dem Pilafian  #
 * 1)  Examples:                                          #
 * 2)     $ tree                                          #
 * 3)     $ tree /etc/opt                                 #
 * 4)     $ tree ..                                       #
 * 5)  Public Domain Software -- Free to Use as You Like  #
 * 6)  http://www.centerkey.com/tree  -  By Dem Pilafian  #
 * 1)  http://www.centerkey.com/tree  -  By Dem Pilafian  #

echo if [ "$1" != "" ] #if parameter exists, use as base folder then cd "$1" fi pwd ls -R | grep ":$" | \ sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/' if [ `ls -F -1 | grep "/" | wc -l` = 0 ] # check if no folders then echo " -> no sub-directories" fi echo exit
 * 1) 1st sed: remove colons
 * 2) 2nd sed: replace higher level folder names with dashes
 * 3) 3rd sed: indent graph three spaces
 * 4) 4th sed: replace first dash with a vertical bar

ulimit
ulimit - Provides control over the resources available to the shell and to processes started by it, on systems that allow such control.

The default maximum open files (file descriptors) on a Redhat Linux system is 1024 per session. This is set here: /etc/security/limits.d/90-nproc.conf

See all of the current soft limits: ulimit -a ulimit -Sa

See all of the current hard limits: ulimit -Ha

To see the current session limit: ulimit -n

To see the system limit: cat /proc/sys/fs/file-max

To set the max to half the system limit: ulimit -n $(( `cat /proc/sys/fs/file-max` / 2 ))

Umlimited?

The system file descriptor limit is set in /proc/sys/fs/file-max. The following command will increase the limit to 65535: echo 65535 > /proc/sys/fs/file-max

You should then be able to increase the file descriptor limits using: ulimit -n unlimited

This did not work for me.

A bash forkbomb $ :{ :|:& };:

References:
 * Linux increase the maximum number of open files or file descriptors
 * Performance tuning
 * Linux Howtos: Tips and Tricks -> ulimit and sysctl