paste is one of those tools nobody uses [1]. It puts two file side by side, line by line.

One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.

$ paste <(ls *.rpm) <(ls *.rpm | xargs -r rpm -q --queryformat '%{name} \n' -p)

[1] See “J” in The ABCs of Unix

[PS: I meant to blog this in 2011, but apparently never committed the file…]

In Wheezy’s grep version [1], you can omit the “.” in

$ grep -r foobar .

and just write

$ grep -r foobar

[1] actually since 2.11

Ever wondered how much memory a program needed? Install the “time” package:

$ /usr/bin/time ls
[...]
0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 4000maxresident)k
0inputs+0outputs (0major+311minor)pagefaults 0swaps

Unfortunately the “time” bash built-in makes it necessary to use the full path.

Thanks to youam for the tip.

Update: aba notes that calling \time works as well. Thanks!

I need to look this up every time I need a backport (mostly PostgreSQL) at a customer site with limited networking:

$ lftp -c 'mget http://backports.debian.org/debian-backports/pool/main/p/postgresql-8.4/*_8.4.5-1~bpo50+1_amd64.deb'

Hopefully I can remember this in the future.

I’ve always thought about collecting random bits of useful/interesting/cool Unix features and the like. Before I let that rot indefinitely in a text file in my $HOME, I’ll post it in a series of blog posts. So here’s bit #1:

Everyone knows /dev/null, and most will know /dev/zero. But /dev/full was unknown to me until some time ago. This device will respond to any write request with ENOSPC, No space left on device. Handy if you want to test if your program catches “disk full” - just let it write there:

$ echo foo > /dev/full
bash: echo: write error: No space left on device

/etc/fstab files tend to be an unreadable mess of unaligned fields.

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/benz-root /               ext3    errors=remount-ro 0       1
/dev/sda1       /boot           ext3    defaults        0       2
/dev/mapper/benz-home /home           ext3    defaults        0       2
/dev/mapper/benz-swap_1 none            swap    sw              0       0
newton:/home		/nfs		nfs	defaults,soft,intr,users	0 0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

Let’s remove some whitespace in the third line:

#<filesystem> <mountpoint>   <type>  <options>       <dump>  <pass>

And then pipe everything from line 3 to the end through column -t:

# /etc/fstab: static file system information.
#
#<filesystem>            <mountpoint>   <type>       <options>                 <dump>  <pass>
proc                     /proc          proc         defaults                  0       0
/dev/mapper/benz-root    /              ext3         errors=remount-ro         0       1
/dev/sda1                /boot          ext3         defaults                  0       2
/dev/mapper/benz-home    /home          ext3         defaults                  0       2
/dev/mapper/benz-swap_1  none           swap         sw                        0       0
newton:/home             /nfs           nfs          defaults,soft,intr,users  0       0
/dev/scd0                /media/cdrom0  udf,iso9660  user,noauto               0       0

Thanks to SP8472 for bringing this to my attention.

I’ve always been annoyed about how hard it is to convert seconds-since-epoch to strings. I’ve always been using “date -d ‘1970-01-01 + 1234 sec’”, but as it turned out, that’s wrong because it uses the wrong timezone. Luckily, there’s a slick replacement:

$ date -d '@1234'
Do 1. Jan 01:20:34 CET 1970

The right version of the “long” version is:

$ date -d '1970-01-01 UTC + 1234 sec'
Do 1. Jan 01:20:34 CET 1970

Just discovered (thanks to XTaran): df -T – show file system type.

$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/benz-root
              ext3    6.6G  4.9G  1.4G  79% /
tmpfs        tmpfs    1.6G     0  1.6G   0% /lib/init/rw
udev         tmpfs    1.6G  228K  1.6G   1% /dev
tmpfs        tmpfs    1.6G     0  1.6G   0% /dev/shm
/dev/sda1     ext3    236M   21M  203M  10% /boot
/dev/mapper/benz-home
              ext3    135G   82G   53G  61% /home
newton:/home   nfs    459G  145G  291G  34% /nfs

Lockfiles are usually hard to get right, especially in sh scripts. The best bet so far was “lockfile” included with procmail, but its semantics are pretty weird. (Try to understand what exit code you want, with or without “-!”.) Not to mention that failing to clean up the lockfile will stop the next cron run, etc.

Since Lenny, util-linux ships “flock”. Now you simply say

$ flock /path/to/lockfile /path/to/command

and are done. If you want it non-blocking, add “-n”:

$ flock -n /path/to/lockfile /path/to/command

I should probably migrate all my cronjobs to use this.

Since coreutils 8.1 (in Squeeze, not Lenny), there is a command that simply prints out how many processors (cores, processing units) are available:

$ nproc
2

The use case is obvious:

$ make -j $(nproc)

On a side note, this is gnulib’s nproc module wrapped into a C program. If you didn’t know gnulib before (it had slipped my attention until recently), it is a library of portability functions and other useful things. Adding it to a project is simply done by calling gnulib-tool and tweaking a few lines in the automake/whatever build scripts.

PS: do not use nproc unconditionally in debian/rules. Parse DEB_BUILD_OPTIONS for “parallel” instead.

paste is one of those tools nobody uses [1]. It puts two file side by side, line by line.

One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.

$ paste <(ls *.rpm) <(ls *.rpm | xargs -r rpm -q --queryformat '%{name} \n' -p)

[1] See “J” in The ABCs of Unix

“stat” is “date +format” for files:

$ stat -c %s ~/me.jpg  # size
520073
$ stat -c %U ~/me.jpg  # owner
cbe

No more parsing of “ls” output or similar hacks.

It also shows detailed information about files.

$ stat ~/me.jpg
  File: „/home/cbe/me.jpg“
  Size: 520073          Blocks: 1024       IO Block: 4096   reguläre Datei
Device: fe03h/65027d    Inode: 12427268    Links: 1
Access: (0600/-rw-------)  Uid: ( 2062/     cbe)   Gid: ( 2062/     cbe)
Access: 2010-06-06 12:58:07.000000000 +0200
Modify: 2010-04-09 22:38:46.000000000 +0200
Change: 2010-04-26 14:18:00.000000000 +0200

It supports similar features for stat’ing filesystems.

$ for i in `seq 1 40` ; do ./something $i ; done
$ for i in $(seq 1 40) ; do ./something $i ; done

seq is nice for that, but the syntax feels a bit hard to type. If you don’t need sh compatibility (read: in bash/zsh), try:

$ for i in {1..40} ; do ./something $i ; done

PS: no spaces allowed

PS 2: another useful feature is prefixes/suffixes as in “touch {1..40}.txt”

Mutt’s configuration is sometime more a toolbox than something offering ready solutions, and “how to I use multiple accounts?” is one of the most FAQs. Here’s a condensed version of my setup.

In the most simple solution, just ‘c’hange folders to the IMAP server:

c imaps://imap.example.com <enter>
c imaps://imap.otherdomain.tld <enter>

That’s cumbersome to type, so let’s automate it:

# .mutt/muttrc
macro index <f2> '<change-folder>imaps://imap.example.com<enter>'
macro index <f3> '<change-folder>imaps://imap.otherdomain.tld<enter>'

That would be the basic setup.

The two accounts have settings associated with them, we put them in two files:

# .mutt/account.example
set from=me@example.com
set hostname="example.com"
set folder="imaps://imap.example.com/"
set postponed="=Drafts"
set spoolfile="imaps://imap.example.com/INBOX"
set signature="~/.mutt/signature.example"
set smtp_url="smtp://me@mail.example.com" smtp_pass="$my_pw_example"

# .mutt/account.otherdomain
set from=myself@otherdomain.tld
set hostname="otherdomain.tld"
set folder="imaps://imap.otherdomain.tld/"
set postponed="=Drafts"
set spoolfile="imaps://imap.otherdomain.tld/INBOX"
set signature="~/.mutt/signature.otherdomain"
set smtp_url="smtp://myself@mail.otherdomain.tld" smtp_pass="$my_pw_otherdomain"

Now all that’s left to do is two folder-hooks to load the files:

# .mutt/muttrc
folder-hook 'example.com' 'source ~/.mutt/account.example'
folder-hook 'otherdomain.tld' 'source ~/.mutt/account.otherdomain'
# switch to default account on startup
source ~/.mutt/account.example

A slight variation of the macros also uses the account files:

macro index <f2> '<sync-mailbox><enter-command>source ~/.mutt/account.example<enter><change-folder>!<enter>'
macro index <f3> '<sync-mailbox><enter-command>source ~/.mutt/account.otherdomain<enter><change-folder>!<enter>'

To save entering the password all the time, we use account-hooks:

account-hook example.org 'set imap_user=me imap_pass=pw1'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=pw2'

Putting passwords in configs isn’t something I like, so I pull them from the Gnome keyring:

set my_pw_example=`gnome-keyring-query get mutt_example`
set my_pw_otherdomain=`gnome-keyring-query get mutt_otherdomain`
account-hook example.org 'set imap_user=me imap_pass=$my_pw_example'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=$my_pw_otherdomain'

(I found gnome-keyring-query in the Gentoo Wiki.)

Martti Rahkila has more verbose article with similar ideas.

Update 2013-01-02: smtp_url and smtp_pass added