Below you will find pages that utilize the taxonomy term “Unix”
Cool Unix Features: paste
paste is one of those tools nobody uses [1]. It puts two file side by side, line by line.
One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.
$ paste <(ls *.rpm) <(ls *.rpm | xargs -r rpm -q --queryformat '%{name} \n' -p)
[1] See “J” in The ABCs of Unix
[PS: I meant to blog this in 2011, but apparently never committed the file…]
grep -r foobar
In Wheezy’s grep version [1], you can omit the “.” in
$ grep -r foobar .
and just write
$ grep -r foobar
[1] actually since 2.11
Cool Unix Features: /usr/bin/time
Ever wondered how much memory a program needed? Install the “time” package:
$ /usr/bin/time ls [...] 0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 4000maxresident)k 0inputs+0outputs (0major+311minor)pagefaults 0swaps
Unfortunately the “time” bash built-in makes it necessary to use the full path.
Thanks to youam for the tip.
Update: aba notes that calling \time works as well. Thanks!
I need to look this up every time
I need to look this up every time I need a backport (mostly PostgreSQL) at a customer site with limited networking:
$ lftp -c 'mget http://backports.debian.org/debian-backports/pool/main/p/postgresql-8.4/*_8.4.5-1~bpo50+1_amd64.deb'
Hopefully I can remember this in the future.
Cool Unix Features: /dev/full
I’ve always thought about collecting random bits of useful/interesting/cool Unix features and the like. Before I let that rot indefinitely in a text file in my $HOME, I’ll post it in a series of blog posts. So here’s bit #1:
Everyone knows /dev/null, and most will know /dev/zero. But /dev/full was unknown to me until some time ago. This device will respond to any write request with ENOSPC, No space left on device. Handy if you want to test if your program catches “disk full” - just let it write there:
$ echo foo > /dev/full
bash: echo: write error: No space left on device
Cool Unix Features: column -t
/etc/fstab files tend to be an unreadable mess of unaligned fields.
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/mapper/benz-root / ext3 errors=remount-ro 0 1
/dev/sda1 /boot ext3 defaults 0 2
/dev/mapper/benz-home /home ext3 defaults 0 2
/dev/mapper/benz-swap_1 none swap sw 0 0
newton:/home /nfs nfs defaults,soft,intr,users 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0
Let’s remove some whitespace in the third line:
#<filesystem> <mountpoint> <type> <options> <dump> <pass>
And then pipe everything from line 3 to the end through column -t:
# /etc/fstab: static file system information.
#
#<filesystem> <mountpoint> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/mapper/benz-root / ext3 errors=remount-ro 0 1
/dev/sda1 /boot ext3 defaults 0 2
/dev/mapper/benz-home /home ext3 defaults 0 2
/dev/mapper/benz-swap_1 none swap sw 0 0
newton:/home /nfs nfs defaults,soft,intr,users 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0
Thanks to SP8472 for bringing this to my attention.
Cool Unix Features: date -d @
I’ve always been annoyed about how hard it is to convert seconds-since-epoch to strings. I’ve always been using “date -d ‘1970-01-01 + 1234 sec’”, but as it turned out, that’s wrong because it uses the wrong timezone. Luckily, there’s a slick replacement:
$ date -d '@1234'
Do 1. Jan 01:20:34 CET 1970
The right version of the “long” version is:
$ date -d '1970-01-01 UTC + 1234 sec'
Do 1. Jan 01:20:34 CET 1970
Cool Unix Features: df -T
Just discovered (thanks to XTaran): df -T – show file system type.
$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/benz-root
ext3 6.6G 4.9G 1.4G 79% /
tmpfs tmpfs 1.6G 0 1.6G 0% /lib/init/rw
udev tmpfs 1.6G 228K 1.6G 1% /dev
tmpfs tmpfs 1.6G 0 1.6G 0% /dev/shm
/dev/sda1 ext3 236M 21M 203M 10% /boot
/dev/mapper/benz-home
ext3 135G 82G 53G 61% /home
newton:/home nfs 459G 145G 291G 34% /nfs
Cool Unix Features: flock
Lockfiles are usually hard to get right, especially in sh scripts. The best bet so far was “lockfile” included with procmail, but its semantics are pretty weird. (Try to understand what exit code you want, with or without “-!”.) Not to mention that failing to clean up the lockfile will stop the next cron run, etc.
Since Lenny, util-linux ships “flock”. Now you simply say
$ flock /path/to/lockfile /path/to/command
and are done. If you want it non-blocking, add “-n”:
$ flock -n /path/to/lockfile /path/to/command
I should probably migrate all my cronjobs to use this.
Cool Unix Features: nproc
Since coreutils 8.1 (in Squeeze, not Lenny), there is a command that simply prints out how many processors (cores, processing units) are available:
$ nproc
2
The use case is obvious:
$ make -j $(nproc)
On a side note, this is gnulib’s nproc module wrapped into a C program. If you didn’t know gnulib before (it had slipped my attention until recently), it is a library of portability functions and other useful things. Adding it to a project is simply done by calling gnulib-tool and tweaking a few lines in the automake/whatever build scripts.
PS: do not use nproc unconditionally in debian/rules. Parse DEB_BUILD_OPTIONS for “parallel” instead.
Cool Unix Features: paste
paste is one of those tools nobody uses [1]. It puts two file side by side, line by line.
One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.
$ paste <(ls *.rpm) <(ls *.rpm | xargs -r rpm -q --queryformat '%{name} \n' -p)
[1] See “J” in The ABCs of Unix
Cool Unix Features: stat
“stat” is “date +format” for files:
$ stat -c %s ~/me.jpg # size
520073
$ stat -c %U ~/me.jpg # owner
cbe
No more parsing of “ls” output or similar hacks.
It also shows detailed information about files.
$ stat ~/me.jpg
File: „/home/cbe/me.jpg“
Size: 520073 Blocks: 1024 IO Block: 4096 reguläre Datei
Device: fe03h/65027d Inode: 12427268 Links: 1
Access: (0600/-rw-------) Uid: ( 2062/ cbe) Gid: ( 2062/ cbe)
Access: 2010-06-06 12:58:07.000000000 +0200
Modify: 2010-04-09 22:38:46.000000000 +0200
Change: 2010-04-26 14:18:00.000000000 +0200
It supports similar features for stat’ing filesystems.
seq is nice, but ...
$ for i in `seq 1 40` ; do ./something $i ; done
$ for i in $(seq 1 40) ; do ./something $i ; done
seq is nice for that, but the syntax feels a bit hard to type. If you don’t need sh compatibility (read: in bash/zsh), try:
$ for i in {1..40} ; do ./something $i ; done
PS: no spaces allowed
PS 2: another useful feature is prefixes/suffixes as in “touch {1..40}.txt”
Using multiple IMAP accounts with Mutt
Mutt’s configuration is sometime more a toolbox than something offering ready solutions, and “how to I use multiple accounts?” is one of the most FAQs. Here’s a condensed version of my setup.
In the most simple solution, just ‘c’hange folders to the IMAP server:
c imaps://imap.example.com <enter>
c imaps://imap.otherdomain.tld <enter>
That’s cumbersome to type, so let’s automate it:
# .mutt/muttrc
macro index <f2> '<change-folder>imaps://imap.example.com<enter>'
macro index <f3> '<change-folder>imaps://imap.otherdomain.tld<enter>'
That would be the basic setup.
The two accounts have settings associated with them, we put them in two files:
# .mutt/account.example
set from=me@example.com
set hostname="example.com"
set folder="imaps://imap.example.com/"
set postponed="=Drafts"
set spoolfile="imaps://imap.example.com/INBOX"
set signature="~/.mutt/signature.example"
set smtp_url="smtp://me@mail.example.com" smtp_pass="$my_pw_example"
# .mutt/account.otherdomain
set from=myself@otherdomain.tld
set hostname="otherdomain.tld"
set folder="imaps://imap.otherdomain.tld/"
set postponed="=Drafts"
set spoolfile="imaps://imap.otherdomain.tld/INBOX"
set signature="~/.mutt/signature.otherdomain"
set smtp_url="smtp://myself@mail.otherdomain.tld" smtp_pass="$my_pw_otherdomain"
Now all that’s left to do is two folder-hooks to load the files:
# .mutt/muttrc
folder-hook 'example.com' 'source ~/.mutt/account.example'
folder-hook 'otherdomain.tld' 'source ~/.mutt/account.otherdomain'
# switch to default account on startup
source ~/.mutt/account.example
A slight variation of the macros also uses the account files:
macro index <f2> '<sync-mailbox><enter-command>source ~/.mutt/account.example<enter><change-folder>!<enter>'
macro index <f3> '<sync-mailbox><enter-command>source ~/.mutt/account.otherdomain<enter><change-folder>!<enter>'
To save entering the password all the time, we use account-hooks:
account-hook example.org 'set imap_user=me imap_pass=pw1'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=pw2'
Putting passwords in configs isn’t something I like, so I pull them from the Gnome keyring:
set my_pw_example=`gnome-keyring-query get mutt_example`
set my_pw_otherdomain=`gnome-keyring-query get mutt_otherdomain`
account-hook example.org 'set imap_user=me imap_pass=$my_pw_example'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=$my_pw_otherdomain'
(I found gnome-keyring-query in the Gentoo Wiki.)
Martti Rahkila has more verbose article with similar ideas.
Update 2013-01-02: smtp_url and smtp_pass added