Command Line Magic

Last year I published Vim Magic, where I gathered all the commands that I found particularly useful when using the Vim editor. I’ve kept updating it ever since whenever I’ve found new useful tricks. It’s been so far helpful to me as a reference and hopefully to somebody else out there too. This post is the same but for the command line in Linux/Unix.

This is not intended to be a tutorial with the basics for you to learn how to use the command line. Instead, it’s more like a reference for somebody who’s been using the terminal for some time now and wants to learn some new tricks or remember older ones.

Scripts

#!/bin/bash use this as the first line of a bash script, for the interpreter to know how to handle it

Files

for i in *.png ; do convert "$i" "${i%.*}.jpg" ; done convert all png files into jpg
find . -type f -exec grep -El "someWord|someOtherWord" {} \; find all files in the current directory (and above) that contain someWord or someOtherWord. An escaped semicolon is necessary to indicate the end of exec. The brackets are also necessary, since they contain the output of the find command, that will be fed into the grep command
find . -type f | xargs grep -El "something|appstore" exactly the same as the previous command, using xargs instead of -exec
find /path/to/files* -mtime +5 -exec rm {} \; delete files older than 5 days
find . -name '*.zip' -exec unzip {} \; unzip all the zip files in the current directory
find . -type d | egrep -o '.*src$' find src directories and none of their subdirectories
find . 2>/dev/null find without showing permission denied messages
wdiff one.txt two.txt | colordiff color the differences between one.txt and two.txt
diff -rq(w) dir1 dir2 compare two directory structures
SOME_FILE=$(<some-file.txt) load the contents of some-file.txt into the variable SOME_FILE
cat oldfile.txt | tr -d '\n' > newfile.txt create newfile.txt as a copy of oldfile where all newlines have been removed
convert file.{jpg,png} equivalent to convert file.jpg file.png

With xargs you can avoid loops (it passes the stdout as input to the specified command):

find . -name *.png | xargs rm -rf remove recursively all png files
git branch --merged | grep -v \* | xargs git branch -D remove all local git branches that have already been merged

Users and Groups

cat /etc/passwd | cut -d: -f1 list all users
cat /etc/group | cut -d: -f1 list all groups
chown user file change the owner of a file
chgrp group file change the group of a file
chown user:group file change the owner and group of a file

Networking and SSH

lsof -nP | grep 9092 show which process is blocking the port 9092
rsync -avz --remove-source-files -e ssh /local/dir remoteuser@remotehost:/remote/dir SSH copy files and remove source
ssh user@your-server.com -L 2000:25 SSH session and port tunneling
ssh -Y user@your-server.com start an xterm
sudo route delete default gw 10.0.2.2 eth0 remove the default gateway on eth0
sudo route add default gw 192.168.1.254 eth0 add the default gateway on eth0
wget -c -t 0 --timeout=60 --waitretry=60 --read-timeout=10 http://example.com/file.mkv download the file using the partially downloaded file (resume -c) retrying ad infinitum (-t 0), with 60s timeout and 60s wait between attempts, resetting the connection if there’s no activity in 10 seconds (--read-timeout=10)
wget -r -np -nH -R index.html http://hostname/aaa/bbb/ccc/ddd/ download all files and subfolders of the ddd directory, recursively (-r), without going to upper directories (-np), not saving to the hostname folder (-nH), exclude index.html files (-r index.html)

Random Magic

<ctrl>+r reverse search: in order to search for a command you previously used
cal print the current month
sudo apt-get install build-essential checkinstall in debian-based distros, install the essentials for building packages (make; make install)
openssl base64 -in input.txt -out encoded.b64 encode input.txt in base64 and save it to the file encoded.b64
sudo !! run the last command with sudo
` `+any command: it won’t go to the history
$( command ) it gives the output of command
service --status-all status of services
echo "hello world" | sed -e "s/e/o/g" | sed -e "s/lo/a/g" | sed -e "s/world/mundo/g" translate “hello world” to spanish
docker ps | awk '{print $8}' print only the names of the current containers

Remote SSH session without a password

ssh-keygen -t rsa -b 2048 # Generate an rsa key pair
ssh-copy-id youruser@yourserver # Authorize your key in the remote server
ssh youruser@yourserver # Login without entering a password

CMD Magic