Lukáš Lalinský

AcoustID: Invalid API key?

Sun, 10 Feb 2013 00:00:00 -0000

Short version: I'm sorry, but today I had to block some applications from the AcoustID web service. If you were using AcoustID and now get an "invalid API key" error, please contact me.

The long version is that when doing some maintenance today, I've found a few applications that are using the service against the usage guidelines. I knew this would come and I would have to deal with it, I just didn't know when. When starting AcoustID, I have made it really easy to get new API keys, in hope to avoid copying them from other open source applications. I've also made it clear on the website, that the web service is intended for non-commercial use. If you want to use it for commercial purposes, you can contact me and I'm sure we will find a way to make it work, but the web service is by default intended for non-commercial users.

So I had to block a few applications and they should start receiving a HTTP 400 error, with the "invalid API key" error message. In order to avoid this in the future, please register your application on the website and make sure it has a valid e-mail address entered. That way I can contact you in case of a problem, rather than block the application.

AcoustID code moved to Bitbucket

Sat, 29 Dec 2012 00:00:00 -0000

I was not very happy when GitHub announced that it discontinues project downloads. I was using GitHub to host compiled packages for Chromaprint and the AcoustID fingerprinter, as well as release tarballs for all AcoustID projects, so I had to start looking for alternatives.

I could continue to use GitHub and host downloads elsewhere, but I like to have all project related files in a central location, so I've decided to move everything from GitHub to Bitbucket. You can now find all AcoustID source code and packages under the AcoustID account on Bitbucket.

I've not yet decided what to do with the GitHub repositories. They are still there for now, but should be considered out of date and not used.

Alternative MusicBrainz search server

Fri, 23 Nov 2012 00:00:00 -0000

If you are working with a local MusicBrainz database and want to search in it, there aren't that many choices. You can use the internal PostgreSQL full-text search or you can use the official MusicBrainz search server.

I've been playing with the idea of a customizable Solr-based solution for a long time. Creating a simple Solr index with the MusicBrainz data is not hard, but keeping it up to date is more complicated.

The mbslave package now includes a few scripts for creating and updating a Solr search index. See the README file for more information on how to set it up.

There are a few differences compared to the official MusicBrainz search server:

Everything is in a single index, so there is no need to tell it which entity you are interested in. If you want to restrict the search to a particular entity, use filter query (e.g. fq=kind:artist).
It's easy to extend the index to include more fields. There is no configuration file for this yet, but you can edit the field definition directly in the source code.
It doesn't store as much data as the MusicBrainz search server. The intended workflow is to use the database for displaying results.

I wrote it primarily for my own needs, but if it ends up being useful to people, I'll probably make it more configurable and move to a separate package as it does not necessarily depend on mbslave.

AcoustID: Faster fingerprint importing

Sun, 21 Oct 2012 00:00:00 -0000

Asynchronous importing of user submissions was always a big part of the AcoustID architecture. It makes things much easier to handle on the server side, allows database maintenance without turning the service to read-only mode (just delays the imports) and has many other benefits.

However, people often wanted to get back AcoustIDs for fingerprints they just submitted. It took about a minute to import the fingerprints, so the only solution was to wait a few minutes and then look up the submitted fingerprint to see if it's already imported.

This weekend I've done some changes in how the importing process works and one of the results is that fingerprints are imported within seconds, not minutes, after they are submitted. During normal operation, this means that submitted fingerprints will get imported practically immediately.

This change made it practical for the submission API call to wait a few seconds and collect the results of the import process. There is a new parameter wait which, which allows the client to set a timeout on how long is it willing to wait for the newly imported AcoustIDs. If the fingerprints are imported within the timeout, it will return the AcoustIDs and set the submission status to "imported". Otherwise, it will just return the submission ID and the status as "pending". These IDs can be used to look up the status later.

See the documentation on the submit and submission_status API calls for details.

Dropbox folder icon in GNOME 3

Sat, 20 Oct 2012 00:00:00 -0000

I upgraded to GNOME 3 just over a week ago and when installing the Feanza icon theme, I noticed that there is a nice Dropbox folder icon included. Nautilus allows you to set icon for any folder, but only lets you select a specific image file, which doesn't look particularly good when scaled.

GNOME icon themes have icons in multiple sizes for each icon, so rather then specific image files, it's better to use generic icon names. I didn't find any GUI to assign an icon name to a folder, but fortunately there is a way to do it using the gvfs-set-attribute tool:

gvfs-set-attribute ~/Dropbox metadata::custom-icon-name folder-dropbox

Twisted Trial and Jenkins

Wed, 17 Oct 2012 00:00:00 -0000

It's not completely obvious how to configure a Twisted-based job that uses Trial for running tests in Jenkins, so hopefully this post will save somebody a little time in the future.

Jenkins needs JUnit-style XML file to parse test results. You can get that output from Trial, if you pass the results thought subunit with junitxml.

Debian-based distributions have those two modules packaged, so you can install them with apt-get.

apt-get install python-subunit python-junitxml

Elsewhere you can use pip or easy_install.

pip install python-subunit junitxml

Then in your Jenkins configuration, you can use the following command and let Jenkins know to collect the test results from results.xml.

trial --reporter=subunit MYPACKAGE | subunit2junitxml >results.xml

Extending the SQLAlchemy SQL expression language

Tue, 25 Sep 2012 00:00:00 -0000

The SQL expression language from SQLAlchemy is already very flexible and allows you to build almost any standard SQL query, but sometimes you just need to use a SQL extension that isn't supported by SQLAlchemy.

Suppose I have a very simple music datababse and I want a list of all artists with their latest album name. Oracle has special aggregate functions FIRST/LAST, which together with the KEEP clause can help with queries like this, but unfortunately it isn't supported by the Oracle dialect in SQLAlchemy. The SQL query would look like this:

SELECT
    artist,
    MIN(name) KEEP (DENSE_RANK LAST ORDER BY release_date)
FROM album
GROUP BY artist

Recently, I've learned that it's possible to extend the SQL compiler and add support for custom clauses that SQLAlchemy doesn't understand natively. In my specific case of the KEEP clause, I need this code to make it work:

import itertools
from sqlalchemy.util import to_list
from sqlalchemy.sql.expression import ColumnElement, ClauseList
from sqlalchemy.ext.compiler import compiles


class Keep(ColumnElement):

    def __init__(self, func, order_by, first=True):
        self.func = func
        self.order_by = ClauseList(*to_list(order_by))
        self.first = first

    @property
    def type(self):
        self.func.type

    @property
    def _from_objects(self):
        return list(itertools.chain(
            *[c._from_objects for c in (self.func, self.order_by)]
        ))


def keep_first(func, order_by):
    return Keep(func, order_by)


def keep_last(func, order_by):
    return Keep(func, order_by, first=False)


@compiles(Keep)
def compile_keep(keep, compiler, **kwargs):
    return "%s KEEP (DENSE_RANK %s ORDER BY %s)" % (
        compiler.process(keep.func),
        "FIRST" if keep.first else "LAST",
        compiler.process(keep.order_by)
    )

With this, I could simply use keep_first() and keep_last() as any other SQL expression functions. For example, the above SQL query would have been written like this:

session.query(Album.artist,
              keep_last(sql.func.min(Album.name), Album.release_date)).\
    group_by(Album.artist)

The main advantage of having it all written as a SQL expression is that I don't need to know the actual column names, which get more and more complicated once you have some joins and subqueries in the SQL expression.

Offline for the next four weeks

Fri, 07 Sep 2012 00:00:00 -0000

I'm starting my hiking trip though Spain tomorrow, so I'll be online very sporadically during the next four weeks and I'll probably not answer emails.

TagLib 1.8 released

Thu, 06 Sep 2012 00:00:00 -0000

I'm sorry this took longer than expected, but I've just released the final version of TagLib 1.8. There are a few changes compared to the beta version, see the change log below.

Download:

Source Code Tarball

If you find any problems, please report them at the GitHub bug tracker.

Changes in 1.8 final:

Added support for OWNE ID3 frames.
Changed key validation in the new PropertyMap API.
ID3v1::Tag::setStringHandler will no londer delete the previous handler, the caller is responsible for this.
File objects will also no longer delete the passed IOStream objects. It should be done in the caller code after the File object is no longer used.
Added ID3v2::Tag::setLatin1StringHandler for custom handling of latin1-encoded text in ID3v2 frames.
Fixed validation of ID3v2 frame IDs (IDs with '0' were ignored).

Changes in 1.8 beta:

New API for accessing tags by name.
New abstract I/O stream layer to allow custom I/O handlers.
Support for writing ID3v2.3 tags.
Support for various module file formats (MOD, S3M, IT, XM).
Support for MP4 and ASF is now enabled by default.
Started using atomic int operations for reference counting.
Added methods for checking if WMA and MP4 files are DRM-protected.
Added taglib\_free to the C bindings.
New method to allow removing pictures from FLAC files.
Support for reading audio properties from ALAC and Musepack SV8 files.
Added replay-gain information to Musepack audio properties.
Support for APEv2 binary tags.
Many AudioProperties subclasses now provide information about the total number of samples.
Various small bug fixes.

Blog moved to Jekyll

Wed, 05 Sep 2012 23:33:00 -0000

Just finished moving this blog from WordPress to static files generated using Jekyll, with Disqus for comments. Please let me know if you spot a broken link or some other problem.