Linux

Highlight text on the fly thanks to terminfo and sed.

A colleague today was looking for a way to pipe something to a command that highlights particular strings of text, keeping intact the stdin, similar to what you can see on some modern distros, ie Gentoo.

Grep is not the way as it only prints the lines which contain the pattern that we are looking for.

While there are a bunch of tools that can do that, like supercat, etc. I felt no need to install additional packages for something you can do with few lines of bash and knowing terminal capabilities.

Therefore I wrote this small script, put it in the PATH env variable. The script uses sed and makes use of the terminfo database and of the terminal capabilities to color the text, here it is in case you find it useful:

#!/bin/bash

red=$(tput bold;tput setaf 1)
normal=$(tput sgr0)

while read line; do
  echo $line | sed -e "s/\($1\)/$red\1$normal/"
done

If you don’t like red just read `man tput` and read this link about terminal codes.

Interested in doing anything to your PDFs beyond reading and annotating?Use sodapdf editor to make updates such as adjusting content, layout, margins, and more.

Example of usage:

It also accept regexps (in sed syntax, not perl syntax), for example:


$ sudo tail -f /var/log/system.log | highlight [Ss]potify

Setting up WordPress Multisite on Debian 5.0 (lenny)

This post explains how to install WordPress 3.x on a Debian lenny and to set up a multisite installation on your Debian server.

All the blogs will log into the same file /var/log/apache2/access.log; the first column will be the blog host and port (see the LogFormat configured below).

This chapter is based on information collected at http://codex.wordpress.org/Create_A_Network and in /usr/share/doc/wordpress/README.Debian.

All WordPress blogs will use the same shared PHP code in /usr/share/wordpress.

Assumptions

  • Main website will be http://domain.com
  • Blogs will be under http://${blogname}.domain.com

File uploaded into the main blog (www.domain.com) will be stored in /srv/www/wp-uploads/domain.com/%year/%month/

Files uploaded into the *.domain.com blogs will be stored in /usr/share/wordpress/wp-content/blogs.dir/${blog_id}/files/%year/%month/, where ${blog_id} is the numeric id of the blog, as managed by the Multisite feature.

Configuration of Debian lenny-backports repository and installation

WordPress 3.x is not in lenny so we have to use the backport repository.
Excute this as root:

echo "deb http://backports.debian.org/debian-backports lenny-backports main" >> /etc/apt/sources.list

cat < /etc/apt/preferences
Package: *
Pin: release a=lenny-backports
Pin-Priority: 200
EOF

apt-get update
apt-get -t lenny-backports install wordpress

Installation of non free swf plugin

As stated in /usr/share/doc/wordpress/README.Debian WordPress originally comes with a Flash-based tool that allows to upload files. However, that tool violates the Debian Policy, as stated in the
bug #591195. That is why this tool is not shipped with the Debian package anymore. If you want to enable this feature, you need to install the Flash file yourself with the following command:

   
wget -O /usr/share/wordpress/wp-includes/js/swfupload/swfupload.swf http://core.svn.wordpress.org/branches/3.0/wp-includes/js/swfupload/swfupload.swf

Configure your DNS server

Add the following records in your zone (I suppose you know how to deal with your DNS zone):

; wp multisite
$ORIGIN        IN A       ${IP}
www            IN A       ${IP}
*.domain.com.  IN CNAME   www

Apache configuration

Configure the vhost_combined log format in /etc/apache2/apache2.conf modifying the default one:

   
LogFormat "%{Host}i:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined

Configure apache in /etc/apache2/sites-available/wordpress-multisite :


UseCanonicalName    Off

ServerAlias *.domain.com domain.com
ServerName www.domain.com
DocumentRoot /srv/www/domain.com/

Options All
ServerAdmin you@domain.com

# Store uploads of www.domain.com in /srv/www/wp-uploads/$0
RewriteEngine On
RewriteRule ^/wp-uploads/(.*)$ /srv/www/wp-uploads/%{HTTP_HOST}/$1


        Options FollowSymLinks
        AllowOverride All


CustomLog /var/log/apache2/access.log vhost_combined

# this is needed when activating multisite, WP needs to to a 
# fopen("http://randomname.domain.com") to verify
# that apache is correctly configured
php_admin_flag allow_url_fopen on


Configure /etc/wordpress/htaccess:


RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
## uploaded files
RewriteRule ^files/(.+) wp-includes/ms-files.php?file=$1 [L]
## real files dealt directly
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
## other go through index.php
RewriteRule . index.php [L]

Enable the config executing this as root:

a2ensite wordpress-multisite
a2enmod rewrite && a2enmod vhost_alias
/etc/init.d/apache2 restart

Setting some permissions

# required to allow the main site to upload content
chown www-data /srv/
chown www-data /srv/www/

# required to allow subdomains site to upload content
chgrp www-data /usr/share/wordpress/wp-content/blogs.dir/
chmod g+w /usr/share/wordpress/wp-content/blogs.dir/

# required in order to be able to install/update themes/plugins 
# from the webgui (only on the main site)
chown -R www-data:www-data /usr/share/wordpress/wp-content/
chown -R www-data:www-data /usr/share/wordpress/wp-admin/ 

Creating main site

At this stage we can create the main site using this bash script:

bash /usr/share/doc/wordpress/examples/setup-mysql -n main domain.com

This script creates the DB & tables and it also creates a file called /etc/wordpress/config-domain.com.php.

You should end up with the following files:

  • /etc/wordpress/config-domain.com.php
  • /srv/www/domain.com symlink to -> /usr/share/wordpress
  • /srv/www/wp-uploads/domain.com/

Activating Network (aka multisite)

At this stage you should log into http://domain.com and follow the on screen instructions.
Once you are in the backend you can than click into the “Tools -> Network” link and enable the mulsite, this will create some tables in your db and it will ask you to copy&paste some lines into /etc/wordpress/config-domain.com.php:

define( 'SUBDOMAIN_INSTALL', true );
$base = '/';
define( 'DOMAIN_CURRENT_SITE', 'domain.com' );
define( 'PATH_CURRENT_SITE', '/' );
define( 'SITE_ID_CURRENT_SITE', 1 );
define( 'BLOG_ID_CURRENT_SITE', 1 );

define( 'AUTH_KEY', 'blabla' );
define( 'SECURE_AUTH_KEY', 'blabla' );
define( 'LOGGED_IN_KEY', 'blabla' );
define( 'NONCE_KEY', 'blabla' );
define( 'AUTH_SALT', 'blabla' );
define( 'SECURE_AUTH_SALT', 'blabla' );
define( 'LOGGED_IN_SALT', 'blabla' );
define( 'NONCE_SALT', 'blabla' );

define( 'MULTISITE', true );

# for debugging purposes only
# define('WP_DEBUG', true);

After that you’re ready to go 😀

Logging as super admin and going into the SuperAdmin webgui area you can create new subdomains (blogs), and users, when you can create blogs an email is sent to the owner, you can personalize templates, decide how much storage space you want to assign to every single blog, etc, etc.

At this point you should be able to also install/upgrade plugins and themes straight from the web gui.
Note that define('WP_CORE_UPDATE', false); in your config file will disable the auto update, your wordpress update cycle will be managed using Debian apt tool.

That’s it folks, any question please comment 🙂

SPQR

I’ve been programming Perl for a while now, but I’ve seen lot of people around me (even Perl enthusiasts) getting very excited about Python.

In Newbay it’s used mainly for automated and performance tests, but some times also for some system engineering automation.

I can understand them, Python seems to be a pretty cool language, very clear and rigorous when compared to Perl, which tend to be messy (depending on the coder).

Python seems also to have more stuff into the default distribution which comes pre-installed on every Linux/Unix box. You already have http and xml libraries without the necessity of installing stuff (with Perl for example you don’t have LWP or any XML dom library by default and you have to install them through CPAN or your packaging system).

Also, with Jython is easier to interface with Java based stuff like JMS, etc. In the pat I’ve used some awful hacks using Inline::Java , they were working but, you know, they looked really scary.

I still do like Perl though, for doing simple and quick hacks, one-liners, etc.And I will probably continue to use it.

I’m currently reading this Diving into Python book, which is free to download or view online.

My colleague Brano has always tried to evangelize me, with no success so far (because of my chronic laziness), but today he gave me a nice puzzle which took me a couple of hours to solve 😛

I’m sharing it with you just in case you fancy the idea to solve it too. Feel free to send me the result at
pallotron (at) freaknet (dot) org 🙂

It’s called roman.py, it contains two functions called roman2dec and dec2roman, they should convert integers into roman numbers and viceversa.

Brano gave it to me with the excuse that, because I’m Italian, I should be pretty used with romans numbers and it wouldn’t take too much to solve it 😛

I started in the afternoon while at work, it took a 1 hour and half to me to figure out the algo for the dec2roman function, I don’t consider myself a math genius so excuse me if I was slow 😛

The roman2dec was a bit of a challenge, but after 3 glasses of nice white wine (Corvo Glicine) I started seen the pattern and I’ve nailed it 🙂

In vino veritas! ahaha!

You have to write the code of the two functions so that the all unit tests pass (there are also negatives tests that you should take care of), here is the code. Just copy paste in a file called roman.py and execute it with python roman.py, enjoy!

I’ll publish my script in a week or so (just to give you the time to play with it, PLEASE DO NOT CHEAT 🙂 ).

import unittest

# your global variables can go here, too ;)

def roman2dec(roman):
    assert isinstance(roman, str), 'Expected string argument'
    # your code goes here

    return None

def dec2roman(dec):
    assert isinstance(dec, int), 'Expected integer argument'
    # your code goes here

    return None

class RomanTests(unittest.TestCase):

    def testSimple(self):
        self.assertEqual(roman2dec('XXIV'), 24)
        self.assertEqual(roman2dec('mcmxcix'), 1999)
        self.assertEqual(dec2roman(13), 'XIII')
        self.assertEqual(dec2roman(78), 'LXXVIII')

    def testInvalid(self):
        self.assertRaises(ValueError, roman2dec, 'IIII')
        self.assertRaises(ValueError, roman2dec, 'VX')
        self.assertRaises(ValueError, roman2dec, 'VV')
        self.assertRaises(ValueError, roman2dec, 'IC')
        self.assertRaises(ValueError, dec2roman, -1)

    def testExhaustive(self):
        for i in range(4000):
            self.assertEqual(roman2dec(dec2roman(i)), i)

    def testExhaustive2(self):
        letters = ('I', 'V', 'X', 'L', 'C', 'D', 'M')
        first_example = None
        count = 0
        for i in xrange(8*8*8*8*8):
            rom = ''
            while True:             # do { ... } while i>0
                rom += letters[i % 7]
                i /= 7
                if i<=0: break

            try:
                dec = roman2dec(rom)
                roman = dec2roman(dec)
                if rom <> roman:
                    if first_example is None:
                        first_example = '%s != %s (decimal %d)' % (rom, roman, dec)
                    count += 1
            except ValueError, e:
                pass

        self.assertEqual(count, 0, 'Accepted %d invalid roman numbers (e.g. %s)' % (count, first_example))

if __name__ == '__main__':
    unittest.main()

[Update]:

There much more invalid roman numbers that you have to check, the quiz has been updated above :P

Streammare musica da casa ovunque siate via ssh tunnel

A lavoro ho da poco migrato tutte le mie robe su un portatile nuovo. Mi sono accorto che l’80% dello spazio sul portatile era occupato da mp3 e musica varia, cosi’ ho deciso di non mettere mai piu’ musica nel suo hard disk, spreco di spazio inutile visto che viviamo nel mondo della connettivita’ ormai dappertutto.

Cosi’ mi sono detto, ho un bel mac mini a casa col wake-on-lan funzionante, zeppo di musica, questa ridondanza non ha senso, usiamo l’internet per streammare la musica dal computer di casa mentre sono a lavoro!

Mi sono documentato un po’ e ho visto che in Linux si puo’ installare un tool chiamato mt-daapd (http://www.fireflymediaserver.org), ora conosciuto come FireFly media server; questo tool implementa il protocollo DAAP (http://en.wikipedia.org/wiki/Digital_Audio_Access_Protocol), Digital Audio Access Protocol: e’ un protocollo proprietario usato da Apple nel suo iTunes per streammare audio/video in una rete locale. Questo vuol dire che i media player che parlano DAAP (iTunes ed Amarok per esempio) sono in grado di fare l’autodiscovery di host daap-enabled nella rete locale in cui vi trovate perche’ DAAP funziona in broadcast.

Per mettere su tutto questo ambaradam in Debian/Ubuntu basta installare il too:

$ sudo aptitude install mt-daapd libid3tag0 avahi-utils

Avahi-utils vi servira’ solo se volete ascoltare la musica fuori da casa, altrimenti mt-daapd e’ gia’ ok se il vostro intento e’ streammare musica da un altro host della vostra rete casalinga e niente piu’.

A questo punto configurate il daemon editando /etc/mt-daapd.conf. Io ho solo cambiato questa linea:

mp3_dir = /home/pallotron/Music/

Per puntare alla directory della mia home dove sta tutta la musica. Ma voi date un occhio al file per capire cosa altro potete cambiare a seconda delle vostre esigenze.

Avviate il daemon e fate in modo che parta al boot:

$ sudo  /etc/init.d/mt-daapd start $ sudo update-rc.d mt-daapd defaults

A questo punto il vostro server ascoltera’ sulla porta 3689:

root@pallomac:~# netstat -tunap | grep LISTEN | grep mt-daapd
tcp        0      0 0.0.0.0:3689            0.0.0.0:*               LISTEN      18584/mt-daapd

Questo tool ha una interfaccia web che potete usare per vari scopi: creare playlist, definire nuove directory dove trovare la musica, configurare il tool (password, etc). Basta puntare il proprio browser su http://localhost:6689/ o cambiare localhost con l’ip/hostname del vostro computer casalingo.

Ok, una volta che questo e’ fatto viene il bello, un requisito importante e’ che voi siate in grado di accedere al vostro host casalingo usando ssh, il vostro router casalingo deve essere configurato per redirigere la porta SSH (tcp/22) all’ip del vostro computer casalingo. Io a casa assegno due IP fissi ai mac address delle due interfaccie del computer (ethernet e wifi) tramite il DHCP server del router.

A questo punto dal vostro computer in ufficio o dovunque voi siate usate un bel ssh tunnel:

$ ssh vostrouser@ip_pubblico_di_casa -L :6689:localhost:3689

O se preferite potete configurare ~/.ssh/config per aggiungere qualcosa tipo:

Host IP_CASA
   Username VOSTROUSER
   LocalForward *:3689  localhost:3689

A questo punto potete connettere il vostro player preferito a localhost:3689 et voila’, potete vedere tutta la musica sul vostro computer a casa e iniziare ad ascoltare 🙂

Ho verificato che banshee ancora non supporta DAAP, mentre rhythmbox supporat DAAP ma senza autodiscovery, cioe’ inserendo a mano la URL del server… mentre amarok e iTunes supportano l’autodisvovery.

L’autodiscovery funzionera’ automaticamente se vi trovate a casa ma se vi trovate fuori dovete utilizzare questo comando Avahi (http://en.wikipedia.org/wiki/Avahi_(software)):

$ avahi-publish-address -v -s "my daap share" _daap._tcp. 6689 &

Avahi e’ una cosa che devo ancora approfondire meglio, potete verificare che il vostro computer sta esportando correttamente lo share usando il comando:

$ avahi-browse _daap._tcp

Dovreste vedere il nome del vostro share apparire.

Se intendete esportare la musica ai vostri colleghi vi avviso che probabilmente questo e’ contro le regole aziendali. Quindi meglio non usare AVAHI 😛

Buon ascolto!

Edit

Ho trovato un po’ di info a queste URL:

http://wiki.fireflymediaserver.org/SSH_Tunnel

http://wiki.fireflymediaserver.org/RemoteAccess

Pare sia meglio usare una porta diversa da 3689 nel comando ssh, perche’ pare banshee provi a bindare la 3689, io ho scelto 6689… Di conseguenza cambiare anche il comando avahi per usare la porta 6689… e pare anche che dovete utilizzare l’IP del vostro laptop a lavoro e non localhost, cosi’ i comandi diventano:

$ ssh vostrouser@ip_pubblico_di_casa -L *:6689:localhost:3689

oppure nel vostro ~/.ssh/config:

Host IP_CASA
   Username VOSTROUSER
   LocalForward *:6689  localhost:3689

E poi lanciare avahi cosi’:

$ avahi-publish-address -v -s "my daap share" _daap._tcp. 6689 &
Edit2
Potete aggiungere le opzioni -N e -f al comando ssh per evitare di mantenere una shell aperta tutto il tempo, ma poi vi dovete ricordare di killare il processo ssh connesso al vostro server casalingo:

$ ssh vostrouser@ip_pubblico_di_casa -N -g -L *:6689:localhost:3689

O2sms – mandare sms dalla console

Tutti noi abbiamo mandato almeno una volta un SMS dal sito del nostro operatore telefonico irlandese. Ok, e’ comodo, non si spendono soldi, ma ci si deve sorbire pubblicita’, e in piu’ l’interfaccia non e’ proprio immediata… girovagando per la rete ho trovato questo progetto chiamato 02sms.

Mi sento il Salvatore Aranzulla degli Irlandiani. Ricordate?

Questo non e’ altro che un progetto scritto in Perl che usa WWW::Mechanize per esporre una comoda interfaccia verso tutti gli operatori di telefonia irlandesi (O2, Vodafone, 3, meteor, etc). Oltre al modulo vero e proprio c’e’ anche una comoda utility command line per mandare sms dalla vostra console. Se avete una distribuzione debian e derivati potete installare cosi’:

# echo "deb http://apt.mackers.com/ unstable contrib" >> /etc/apt/sources.list
# curl http://www.mackers.com/about/gnupg.key.asc | apt-key add -
# apt-get update
# apt-get install o2sms

Dopodiche potete creare il file di configurazione nella vostra home:

pallotron@saridda ~ $ cat ~/.o2sms/config
username 086123456
password SECRET
carrier o2
alias mamma +353861234567
alias papa +353861234568
alias amici +353861234569 +353861234563

A quel punto potete usare il comando come se fosse il comando mail mandanto in pipe l’output di echo o cat o quello che volete voi:

pallotron@saridda ~ $ echo ciao | o2sms mamma
[ recipient : io (+353861234567) ]
[ logging in to 086123456@o2.ie ... ]
[ login successful ]
[ message sent to +353861234567, 247 remaining this month ]

Per gli amanti del mouse ci sono anche widget per gnome e kde. Varie funzioni sono disponibili per ulteriori info guardate il sito e la documentazione.

Spread the word.