Linux


List waiting processes

ps -edalf | grep cv_wai

Recursively grep source files

grep -r --include="*.php" "something" ./

Cleanup old backups

find /bigdata/backups_day/*tgz -mtime +22 -exec rm {} \;

Global search replace

perl -p -i -e 's/oldstring/newstring/g' `find . | grep [.]php`

Find often running executables (useful to determine load average bottlenecks)

ps aux | awk '{print $11}' | sort | uniq -c | sort -nk1 | tail -n5

 Detect incomming connections (analyze potential DOS attacks)

netstat -an | grep :80 | awk '{print $5}' | cut -f1 -d":" | sort | uniq -c | sort -n

Find space eating folders

for i in T G M; do du -ah 2>&1 | grep "^[0-9.]*$i" | sort -nr -k 1; done | head -n 25

Test disk speed (TransIP 530 MB/s, Vultr 460 MB/s, Digital Ocean 307 MB/s)

dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync

Auto restart apache (crontab -e)

* * * * * if [[ `wget --tries=1 --timeout=30 -qO- http://???/quickcheck.php` != "OK" ]]; then /sbin/service httpd restart; fi

Log server load (requires iotop, usable as root crontab entry)

(echo `date` ; echo ; /usr/*bin/iotop -b -n 2 | sed -n '1,10 d;/^Total DISK READ/,// p' | sed '11,$d' ; echo ; top -b -n 1 | head -n 20 ; echo ) >> /var/log/toptop-`date +\%Y\%m`.log

Prevent a script from running simultaneously

(
  flock -x -n 200 || exit 1
  echo Doing stuff
  sleep 3
  echo Done doing stuff
) 200>/var/lock/.myscript.exclusivelock

SSH login without password (usually ssh-copy-id  will do all this for you)

  •  ssh-keygen -t rsa (generates .ssh/id_rsa and .ssh/id_rsa.pub)
  • add the .ssh/id_rsa.pub of the system you use to login to the .ssh/authorized_keys of the system you want to login to
  • do ssh user@system once to allow adding the connection to the .ssh/known_hosts of the system you use to login

Screen

  • screen (create new screen and activate it)
  • exit (exit shell inside screen session to exit screen session)
  • ctrl-a + d (detach, leave screen but keep it running)
  • ctrl-a + l (kill, force exit the current screen)
  • screen -r (re-attach, go back to the screen, will give list if there are multiple screens)
  • screen -ls (list the running screens)

Raid / Volume management / Filesystems

  • Global overview #1, global overview #2 (images)
  • df -h / df - i (show filesystem usage)
  • mount (show mounted filesystems)
  • vgdisplay -v (show physical volumes, volume groups and logical volumes)
  • pvscan; vgscan; lvscan (compact overview of  physical/groups/logical)
  • cat /proc/mdstat (show raid status and configuration overview)
  • LVM Administration with CLI Commands (Redhat)
  • HOWTO convert EXT2/3/4 partitions to Btrfs in CentOS 6 without losing data

Tips / Hyperlinks

Security

Run Windows on hosted Linux (Centos 6.x) with VMWare

Optimizing e-mail delivery

MySQL / MariaDB

  • Optimize all tables:  mysqlcheck -u root -p*** -o --all-databases
  • Howto: Clean a mysql InnoDB storage engine?
  • Determine size of MySQL / MariaDB databases:
    SELECT table_schema "Data Base Name",
    sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB",
    sum( data_free )/ 1024 / 1024 "Free Space in MB"
    FROM information_schema.TABLES
    GROUP BY table_schema ;

Mazimizing a partion after enlarging a virtual disk (assumes XFS and Parted 3.2)

Check setup (including disk size) with: lsblk
localhost bigstorage # parted
GNU Parted 3.2
Using /dev/vda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) select /dev/vdb
Using /dev/vdb
(parted) resizepart 1
Warning: Partition /dev/vdb1 is being used. Are you sure you want to continue?
Yes/No? Yes
End? [2500GB]? 100%
(parted) quit
Information: You may need to update /etc/fstab.
xfs_growfs /mnt/bigstorage/

Update from GNU Parted 3.1 to GNU Parted 3.2 (CentOS 7)

yum remove parted.x86_64
yum install gcc libuuid-devel device-mapper-devel ncurses-libs ncurses-devel readline-devel
cd
wget http://ftp.gnu.org/gnu/parted/parted-3.2.tar.xz
tar xvf parted-3.2.tar.xz
cd parted-3.2/
./configure
make
make install (if that fails, cp parted/parted /usr/sbin/parted)
To remove: make uninstall