• Chromaprint
• Acoustid Fingerprinter
• Acoustid Server
• Acoustid Index
• Chromaprint plug-in for GStreamer

Even though I still prefer the idea of Bazaar, the tools built around Git and especially GitHub make it worth switching for me. I started using Bazaar a long time ago because it was the only VCS that was able to work on Windows back then. I built several tools to make it more comfortable for me, but I get the feeling the situation is not improving. Maybe Git really is the VCS we should all use…

My firewall is configured to drop everything by default, so it needs to have rules with connections that it should be allowed. However, the default IPFW ban action looks like this:

actionban = ipfw add deny tcp from <ip> to <localhost> dst-port <port>

This inserts the rule at the end of the rule list, just before the default rule. This doesn’t work for me, because it’s after my own rule rule that allows traffic on that port from any address, so the newly added rule has no effect. I ended up modifying the IPFW action configuration file ($PREFIX/etc/fail2ban/action.d/ipfw.conf) to dynamically generate rule numbers that are before my own rules:

actionban = USEDNUMS=`ipfw list | perl -pe 's/(\d{5}) .*\n/$1|/' | perl -pe 's/\|$//'`
            NUM=`jot -w '%%05d' - <minnum> <maxnum> | grep -vE "($USEDNUMS)" | head -n1`
            ipfw add $NUM deny tcp from <ip> to <localhost> dst-port <port>

This only uses rule numbers between <minnum> and <maxnum>, selecting the first one that is available. I can set the variables in my jails.conf configuration like this:

[ssh-ipfw]

enabled  = true
filter   = sshd
action   = ipfw[localhost=any,minnum=01110,maxnum=01199]
logpath  = /var/log/auth.log
ignoreip = 168.192.0.1

The default unban action is also problematic. I was testing the above from a server that has some traffic on another port explicitly allowed. However, when canceling the SSH ban, Fail2ban deleted also my own rule because it was using only the IP for firewall rule matching. Changing it like this works for me, but it’s still not ideal:

actionunban = ipfw delete `ipfw list | grep -i 'deny tcp from  to ' | awk '{print $1;}'`

Ideally I would like to check also the port numbers, but I can’t find an easy way to resolve the service name to a port number.

The final version of the IPFW Fail2ban action configuration file that I’m using is here.

It has scripts for importing the database dumps as well as applying replication packets, so it can be used creating a MB database from scratch as well as updating an existing one. Instructions for setting up a new database can be found in the README file.

There are a few differences between these scripts and mb_server:

1. Depends only on Python and psycopg2. Should be easy to set it up on non-Linux platforms, but I’ve not tested that.
2. Doesn’t require additional space for extracting the database dump tarballs. It decompresses and imports the files on-the-fly.
3. Can be configured to import the MB database into any schema, not just “public”.
4. Can be configured to not replicate specific tables (e.g. direct search indexes, PUIDs).
5. Regrouped transactions. The MB replication packet consists of multiple transactions (each of which has multiple database operations). The original replication code in mb_server imports the transactions exactly as they were executed in the master database. My code applies the whole packet in a single transaction. This is a small disadvantage, but from an external point of view (external scripts, triggers), there is no practical difference.
6. New code, possibly buggy.

The code currently assumes that you know how to setup the MusicBrainz database, so it doesn’t do many checks. It it turns out it’s useful also to somebody else, I can make it more robust.

The problem is that eMusic download files are also XML files, so I can’t set the association just for “.emx” files. Another problem is that to open the files from Chrome, I can only use the default GNOME association. I can’t use any other application without going to Nautilus and looking up the executable. The correct way to fix this seems to be to add a new MIME type:

mkdir -p ~/.local/share/mime/packages
vim ~/.local/share/mime/packages/emusic.xml
update-mime-database ~/.local/share/mime/

Where the emusic.xml file contains the following:

<?xml version="1.0" encoding="UTF-8"?>
    <mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
        <mime-type type="application/x-emusic">
        <comment>eMusic Download File</comment>
        <glob pattern="*.emx"/>
    </mime-type>
</mime-info>

After running the commands, Nautilus identifies the .emx files as eMusic Download Files, so I can set to open them in the eMusic Download Manager, without touching the configuration for generic XML files.

First, you need the mingw32 package, which contains the compiler toolkit plus all the required WinAPI libraries. For cross-compiling CMake projects, you also need a file with paths to the MinGW toolkit:

set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_C_COMPILER i586-mingw32msvc-gcc)
set(CMAKE_CXX_COMPILER i586-mingw32msvc-g++)
set(CMAKE_FIND_ROOT_PATH /usr/i586-mingw32msvc /home/lukas/projects/win32build)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

I have a special directory for installing compiled binaries, which makes it easier to use previously compiled libraries, so that’s included in the file. To compile a Windows version of libmusicbrainz, which uses CMake, you can use a serie of command like this:

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=/home/lukas/projects/mingw.cmake -DCMAKE_INSTALL_PREFIX=/home/lukas/projects/win32build/
make
make install

To build libmusicbrainz I also needed neon, which uses autotools. Fortunately, cross-compiling autotools projects is usually also quite simple, but neon doesn’t support MinGW out of the box, so one patch is necessary. After applying the patch, it can be compiled like this:

./configure --host=i586-mingw32msvc --disable-debug --disable-webdav --prefix=/home/lukas/projects/win32build/
make
make install

mkdir -p ~/.quodlibet/plugins/events/
cd ~/.quodlibet/plugins/events/
wget http://dl.dropbox.com/u/5215054/mbtagsubmit.py

After you enable it and let it know your MusicBrainz username and password (Music → Plugins), it will watch the songs you listen and if any of them has a “musicbrainz_trackid” and at least one “genre” tag, it will use the MusicBrainz web service to submit them. The submission normally happens only every half an hour, but if you change many files in a short time, it will submit them in batches of 20 tracks (the maximum number allowed by the web service) every 2 minutes.

$ grep 'Start Here' -R ~/.local/share/tomboy/ -l
/home/lukas/.local/share/tomboy/Backup/4a47410a-4976-4cb6-8ddc-fd744710dba7.note
/home/lukas/.local/share/tomboy/7d41fff6-6cae-44bc-87b9-6486c809e7ee.note
$ gconftool-2 --set /apps/tomboy/start_note --type string 'note://tomboy/7d41fff6-6cae-44bc-87b9-6486c809e7ee'

1. It ignores timezones. All timestamps are treated as if they were somewhere in the US. For projects where committers live across the world (most open source projects) it’s therefore almost useless.
2. It only works for GitHub projects, obviously.

I’ve put together a little Bazaar plugin that can generate similar graphs for Bazaar branches. You can get the plugin here. It requires PyQt for image drawing, but many Bazaar users should already have PyQt installed. Once you install the plugin, you can generate a “Punchcard” by running a command like this in a branch directory:

bzr punchcard project-punchcard.png

For example, for QBzr you will get this graph:

You can see that the project is done by people who obviously have too much free time. There is activity pretty much any time. I guess only on weekend nights, people are tired and enjoy some sleep. :)

On the other hand, Picard was hacked by people with good sleep patterns over evenings and weekends:

MusicBrainz Server is in a similar situation, minus the good sleep patterns:

The last example is Bazaar. It’s mostly written by full-time employees and it shows:

Cheap Branches

One of the main reason people use DVCS is cheap branching. The most surprising thing for me was that Mercurial doesn’t offer many choices. It doesn’t have the concept of branches, like Bazaar or Git. All you have is an immutable repository that can contain multiple head revisions. By default, you are expected to clone the whole repository. As I mentioned earlier, I tend to work with many feature branches, so I prefer to have just one working tree.

One way to partially solve this problem is the bookmarks extension. It basically allows you to have dynamic tags that move to new revisions as you commit. The initial configuration looks like this:

[extensions]
bookmarks =
[bookmarks]
track.current = True

The next step is to go to your clone and set the main/trunk/master/etc. bookmark:

hg bookmark main

This is necessary, so that you easily know which revisions represents the project’s “mainline”. With the configuration I mentioned, the extension will track your current bookmark and update only that one. To set the current bookmark, you can use update:

hg up -r main

It’s important to always have the current bookmark set to main whenever you are fetching changes from the mainline. Otherwise you will have to manually fix the bookmark’s revision.

Creating new “feature branches” is easy then:

hg bookmark -r main my-feature
hg up -r my-feature

One important thing to remember is that after you use this, the repository will contain multiple heads. This means you can’t use plain hg push to push changes to remote repositories. You always have to specify the revision, for example:

hg push -r main https://hg.example.com/projects/trunk

Merging

When it comes to merging, Mercurial doesn’t make your life easier at all. The default assumption is that visual merging is the preferred way to go. Maybe I’m just too stupid, but I don’t get visual merge tools. I just don’t know how to use them. I prefer merging changes manually in my text editor, after I see the changes for the whole project. Mercurial would start up the merge tool for each file linearly, which means you don’t have a global picture of impact on the project, when you are supposed to resolve a merge. I just can’t work that way.

Fortunately, Mercurial does have an internal three-way merge algorithm, that can leave conflict markers in the merged files and let me do my job. You can configure it this way:

[ui]
merge = internal:merge

This actually uses the merge code from Bazaar, so I expected I’ll be done quickly. There is one problem though. Instead of using patience diff, like Bazaar, it uses its own diff algorithm. The result is that in Bazaar the algorithm works just fine, in Mercurial it produces horrible results. That’s actually pretty funny, considering the fact that I used ideas from Mercurial’s diff code to implement the C version of patience diff in Bazaar.

Next step, the Mercurial wiki suggests diff3 as a possible non-interactive merge tool. There are no examples how to configure it, but something like this technically works:

[ui]
merge = diff3
[merge-tools]
diff3.args = -m $local $base $other > $output

In my experience, this produces even worse results than internal:merge, so there is no reason to use it.

The last idea was to use the merge program from RCS (yes, that’s right, using an ancient VCS to fix issues in a “modern” VCS). I must say that installing RCS in 2010 feels weird, but as long as I can get my job done… Once it was installed, I used this configuration:

[ui]
merge = merge

I was surprised that the results were acceptably good. Not perfect, but I knew I can’t expect more from a three-way merge. Ideally I’d like Bazaar’s merge algorithm with patience diff, but I probably won’t find time to port it, so merge will stay as my default merge tool for some time.

That’s only the first part of the problem though. Now you have files with conflict markers and you need to resolve them. Based on my experience with other version control systems, I assumed conflicts would be listed in hg status, but I was wrong. They are nicely hidden here:

hg resolve -l | grep '^U'

After manually resolving the conflicts, you can use the following command to mark the files as resolved:

hg resolve -m path/to/file

To be continued…

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L]

If the server then gets any HTTP request, it will redirect the client to the same URL, but with the https scheme. This means that actual requests will be only ever served via HTTPS.