Command Line Magic

Last year I published Vim Magic, where I gathered all the commands that I found particularly useful when using the Vim editor. I’ve kept updating it ever since whenever I’ve found new useful tricks. It’s been so far helpful to me as a reference and hopefully to somebody else out there too. This post is the same but for the command line in Linux/Unix.

This is not intended to be a tutorial with the basics for you to learn how to use the command line. Instead, it’s more like a reference for somebody who’s been using the terminal for some time now and wants to learn some new tricks or remember older ones.

Scripts

  • #!/bin/bash use this as the first line of a bash script, for the interpreter to know how to handle it

Files

  • for i in *.png ; do convert "$i" "${i%.*}.jpg" ; done convert all png files into jpg
  • find . -type f -exec grep -El "someWord|someOtherWord" {} \; find all files in the current directory (and above) that contain someWord or someOtherWord. An escaped semicolon is necessary to indicate the end of exec. The brackets are also necessary, since they contain the output of the find command, that will be fed into the grep command
  • find . -type f | xargs grep -El "something|appstore" exactly the same as the previous command, using xargs instead of -exec
  • find /path/to/files* -mtime +5 -exec rm {} \; delete files older than 5 days
  • find . -name '*.zip' -exec unzip {} \; unzip all the zip files in the current directory
  • find . -type d | egrep -o '.*src$' find src directories and none of their subdirectories
  • find . 2>/dev/null find without showing permission denied messages
  • wdiff one.txt two.txt | colordiff color the differences between one.txt and two.txt
  • diff -rq(w) dir1 dir2 compare two directory structures
  • SOME_FILE=$(<some-file.txt) load the contents of some-file.txt into the variable SOME_FILE
  • cat oldfile.txt | tr -d '\n' > newfile.txt create newfile.txt as a copy of oldfile where all newlines have been removed
  • convert file.{jpg,png} equivalent to convert file.jpg file.png

With xargs you can avoid loops (it passes the stdout as input to the specified command):

  • find . -name *.png | xargs rm -rf remove recursively all png files
  • git branch --merged | grep -v \* | xargs git branch -D remove all local git branches that have already been merged

Users and Groups

  • cat /etc/passwd | cut -d: -f1 list all users
  • cat /etc/group | cut -d: -f1 list all groups
  • chown user file change the owner of a file
  • chgrp group file change the group of a file
  • chown user:group file change the owner and group of a file

Networking and SSH

  • lsof -nP | grep 9092 show which process is blocking the port 9092
  • rsync -avz --remove-source-files -e ssh /local/dir remoteuser@remotehost:/remote/dir SSH copy files and remove source
  • ssh user@your-server.com -L 2000:25 SSH session and port tunneling
  • ssh -Y user@your-server.com start an xterm
  • sudo route delete default gw 10.0.2.2 eth0 remove the default gateway on eth0
  • sudo route add default gw 192.168.1.254 eth0 add the default gateway on eth0
  • wget -c -t 0 --timeout=60 --waitretry=60 --read-timeout=10 http://example.com/file.mkv download the file using the partially downloaded file (resume -c) retrying ad infinitum (-t 0), with 60s timeout and 60s wait between attempts, resetting the connection if there’s no activity in 10 seconds (--read-timeout=10)
  • wget -r -np -nH -R index.html http://hostname/aaa/bbb/ccc/ddd/ download all files and subfolders of the ddd directory, recursively (-r), without going to upper directories (-np), not saving to the hostname folder (-nH), exclude index.html files (-r index.html)

Random Magic

  • <ctrl>+r reverse search: in order to search for a command you previously used
  • cal print the current month
  • sudo apt-get install build-essential checkinstall in debian-based distros, install the essentials for building packages (make; make install)
  • openssl base64 -in input.txt -out encoded.b64 encode input.txt in base64 and save it to the file encoded.b64
  • sudo !! run the last command with sudo
  • +any command: it won’t go to the history
  • $( command ) it gives the output of command
  • service --status-all status of services
  • echo "hello world" | sed -e "s/e/o/g" | sed -e "s/lo/a/g" | sed -e "s/world/mundo/g" translate “hello world” to spanish
  • docker ps | awk '{print $8}' print only the names of the current containers

Remote SSH session without a password

  • ssh-keygen -t rsa -b 2048 # Generate an rsa key pair
  • ssh-copy-id youruser@yourserver # Authorize your key in the remote server
  • ssh youruser@yourserver # Login without entering a password