fhtagn.net

An educational python implementation of a bitcoin-like blockchain

julien@fhtagn.net (Julien Rebetez) — Sun, 17 Dec 2017 00:00:00 +0000

I’ve been loosely following the cryptocurrency stories over the past few years. When someone posted a link to the original Bitcoin paper on Hacker News a few days ago, I was amazed at how concise and elegant the paper was.

After reading it, I decided to do a toy blockchain implementation in Python to understand the inner workings better.

The result is a jupyter notebook that implements most of the ideas found in the Bitcoin paper. It’s pretty far from being a full implementation of the whole Bitcoin protocol, but that wasn’t the goal.

I posted it to Hacker News and the reception was pretty positive. I even made it to the top \o/

Kaggle Ultrasound Nerve Segmentation (25th with 0.70679 final ranking)

julien@fhtagn.net (Julien Rebetez) — Fri, 19 Aug 2016 00:00:00 +0000

1. Data
2. Model and results
3. Dreaming nerves
4. Conclusion

I participated in the Ultrasound Nerve Segmentation Kaggle challenge and it looks like my final rank is 25th (I was 48th on the public leaderboard).

The github repo with my code is available here. I should warn you that since this was a 3 months competition, the code is not optimized for reuse and probably quite messy with a lot of copy-paste.

1. Data

The goal of the competition was to label the area containing a collection of nerves called the Brachial plexus in ultrasound images. So the model is given an grayscale image as input and should output a binary image (= a binary mask). Some images (about 60% of the training set) do not contain the Brachial plexus area and should therefore have an empty mask. It basically looks like this :

(left is the image, right is the image and the label area overlayed in red)

You can have also a look at this script on kaggle which has a nice animation of some of the images.

The images have a size of 580x420 pixels. There are 5635 training images and 5508 test images (of which 20% were used for public ranking and 80% for the final ranking).

Contradictory labelling in the training data

As many people found out during the competition, the training dataset contains many contradictory examples. By contradictory examples, I mean two images that are very similar but with one image having a non-empty mask and the other one having an empty mask.

Here is an example. The top image has no mask while the one at the bottom has one.

There are a huge number of similar examples in the training set and this puts a (somewhat low) upper bound on the best result you can achieve, regardless of the model.

I believe the problem is partly due to the fact that the images are actually video frames. So the human expert who is putting the label does so using the video. But the challenge was set up in a way that we had to labellize individual images and they shuffled the frames to force us to NOT rely on the temporal information. I wonder why this choice was made since having videos instead of images could have been very interesting (CNN-RNN !).

2. Model and results

This is an image segmentation problem. Like many people in the challenge, I decided to use some deep learning as it works so well on images. I tried multiple architectures, but they are all based on fully-connected CNN or U-net.

Before talking about model, here is the evolution of my score on the public LB. As you can see, I had a pretty solid start and then spent much time trying to get really small incremental improvements (the red line indicates my best).

The subnotes.txt file in my repo contains some notes about each submission so I could keep track of what was helpful or not.

Pre-processing

Image scaling

The images have a size of 580x420. This is quite big and when I first tried to train a CNN (using a GTX 960) on those images, I quickly ran into memory and performance problems.

Also, since I wanted to do a fully-connected network, it’s much easier if the size of the image is a power of two, so you can pool and unpool without having to fiddle with padding.

I quickly decided to rescale my images to 128x128. I don’t have much data to back that claim, but this helped me train my networks much faster without loosing predictive power. This also mean I stretched the images (the scale factor isn’t the same for both axes), but this didn’t seem to hinder performances much.

Removing contradictory images

Based on what I highlighted above about contradictory labelling in the training set, I decided to remove the training images without a mask that were close to another training image with a mask.

I computed the similary between two images by computing a signature for each image as follow : I divided the image in small 20x20 blocks and then compute a histogram of intensity in each block. All the histograms for the image are then concatenated into a big vector. The disimilary is the the cosine distance between the two signature vectors of two images. This is what is done in the 13_clean/0_filter_incoherent_images.ipynb notebook.

This results in a distance matrix for all the training set images, which is then thresholded to decide which images should be removed. In the end, I kept 3906 training images (out of the 5500).

I also tried propagating masks instead of removing images, but this didn’t seem to help.

Architecture

My best single-model submission is from this notebook. It scored 0.69167 on the public LB nad 0.69234 on the private.

This is a multiple objective model as you can see in the following diagram :

The left part is a traditional CNN. After all the convolutions + pooling, there is a small dense network that has a single output (called outbin) which is the probability that the image has a non-empty mask.

The right part is similar to the U-net where we upsample and merge at each level.

The real output is the outmap one (the top green box) which contains the predicted mask.

In addition, I added two auxiliary outputs (outmap4 and outmap5) which are trained to predict very coarse (4x4 and 8x8) masks. This is only used for training and it looked like it helped a bit regularize the training.

I also found that using Maxout activations seemed to help the training a bit. I am not sure if I could have achieved the same perf by just increasing the number of epochs or by fine-tuning the learning rate.

This model has 95000 parameters. I found difficult to train models with more parameters without overfitting and I got really good results with 50000 parameter models as well.

I used some very simple data augmentation (small rotation, zoom, shear and translation). From some posts on the forum, it looks like doing some horizontal flips helped a lot, but I (wrongly apparently) thought that was a bad idea because the masks are not symmetric and so I didn’t explore that.

My best submission (0.70679 private LB)

This was submission 54 which scored 0.69237 on the public LB. This is an ensemble of the 4 models which got the best public LB scores. The ensembling is done in this notebook.

Post-processing

Sometimes, the predicted mask has some “holes” or incoherent shape. I tried three different post-processing approaches :

Morphological close to remove small holes. This helped a bit
Fitting an ellipse to the mask and filling the ellipse (since the mask are often somewhat ellipse-like). This didn't help and even sometimes decreased the performance.
Using a PCA decomposition of the training masks and reconstruct the predicted mask using a limited number of principal components. This seemed to help more consistently (although just a little bit).

PCA cleaning

The idea of using PCA to post-process came from the Eigenface concept. Basically, you consider your image as a big vector and you then do PCA on the training masks to learn “eigenmasks”.

To clean a predicted mask using this method, you project the cleaned mask on the subspace found by PCA and you then reconstruct using only the 20 (in my case) first principal components. This forces the reconstructed mask to have a shape similar to the training masks.

Here are two extreme examples (predicted mask is left, PCA-cleaned is right) :

Loss and mask percentage

I tried using dice coefficient as a loss, but I always got better results by using binary cross-entropy.

An empty submission on the public leaderboard gets a score of 0.51. This means 51% of the images have no mask. So 49% of the images should have mask and I tried to optimize the threshold on the binary and mask output to get close to 49% predicted masks. But in the end, my best submissions have around 35%-38% of predicted masks. I guess this is because lowering the thresholds to get higher number of predicted masks leads to too many false positive.

3. Dreaming nerves

Once you have a trained CNN, you can make the network “dream” to visualize what the networks wants to see to maximize some output or some intermediate layer. This gives a pretty picture that’s also useful for debugging.

In the image below, you can see some example input images that maximizes the activation of specific filters (each row has 4 filters of a given depth).

The fourth row is the innermost convolutional layer and you can clearly see that the network reacts to interleaved nerve structure.

4. Conclusion

This was an interesting competition. I learnt a lot by playing with various architectures and trying various pre/post-processing techniques.

One problem was the dataset labelling quality. Although it is a good challenge (and something common in everyday applications) to deal with contradictory or inconsistent labels, I think it hurts the competition because people spend time trying to predict the labelling mistakes instead of trying to build a better model of the problem.

Kinfu for Google Tango devkit

julien@fhtagn.net (Julien Rebetez) — Mon, 30 May 2016 00:00:00 +0000

I have been playing with Google’s Project Tango devkit recently.

Although Tango (or at least the current tablet) is more oriented towards scanning house-scale areas, I have been experimenting with smaller scale scanning to get a 3D model of humans. Tango uses optical features and its IMU to determine camera orientation. Although this works quite well in most cases, it does suffer from drift and that’s a bit of a problem to get good precision point clouds.

For example, that is the raw tango point cloud for a scan. You can clearly see the drift in the front view (left image).

To solve this, I implemented (inspired by the open source PCL implementation) the kinect fusion algorithm from Newcombe et al. It’s currently a Python/C/OpenCL implementation, so it’s not running on the tablet.

Here is the resulting point cloud after kinect fusion. As you can see, the drift has been mostly corrected, although there are still some problems on the arms.

We can also mesh the kinect fusion volume :

If you’re interested in the inner workings, here is a video of log plots to verify everything is working correctly. You can see the input color map, the input depth map, the raycasted depth from the kinfu volume, the raycasted color and normal maps. I still have an issue with my ray skipping algorithm that creates some holes in the raycast, but this should be fixed soon.

Exposing aligned Eigen::Quaternionf with boost::python

julien@fhtagn.net (Julien Rebetez) — Thu, 16 Apr 2015 00:00:00 +0000

Introduction

This post is about how to expose C++ struct/classes that have alignment requirements using boost::python.

In the general case, the easiest solution is to use a shared_ptr<Type> as the holder (second argument to bp::class_) and make sure the class has an operatore new that enforces alignment.

But sometimes, you want to expose a class that requires alignment by value, without using a shared_ptr<Type> (typically because you don’t want to write tons of wrappers for your existing C++ codebase). That is what we do here with Eigen::Quaternion.

If you are not interested in the analysis, go directly to the solution near the end.

Exposing Eigen::Quaternion

I was trying to expose an Eigen::Quaternion using boost::python

bp::class_<Quaternionf>("Quaternion", bp::init<float, float, float, float>())
  .def(bp::init<Matrix3f>())
  .add_property("w", get_prop_const(&Quaternionf::w))
  .add_property("x", get_prop_const(&Quaternionf::x))
  .add_property("y", get_prop_const(&Quaternionf::y))
  .add_property("z", get_prop_const(&Quaternionf::z))
  .def("matrix", &Quaternionf::matrix)
  .def("rotvec", &quaternion_to_rotvec);

Now, I also want to be able to export vector<Quaternionf>. So I want to expose the Quaternion by value and do not want to use a shared_ptr<Quaternion> as a Holder.

The problem is, Eigen Quaternion need to be aligned to 16 bytes.

With the code above, there is a high probability that you’ll trigger an assertion like the following (meaning Eigen detects that the Quaternion coefficients are not aligned) :

Assertion failed: ((reinterpret_cast<size_t>(array) & 0xf) == 0 && "this assertion is explained here: " "ht
tp://eigen.tuxfamily.org/dox-devel/group__TopicUnalignedArrayAssert.html" " **** READ THIS WEB PAGE !!! ****"), function plain_array, file /Users/julien/shapetwin/dev/shapy/libs/_install/_all/include/eigen3/Eigen/s
rc/Core/DenseStorage.h, line 86.
Process 93520 stopped

The full stacktrace is here.

Problem analysis

You can see frame #15 is boost/python/object/make_instance.hpp:71 :

template <class T, class Holder>
struct make_instance
    : make_instance_impl<T, Holder, make_instance<T,Holder> >
{
    template <class U>
    static inline PyTypeObject* get_class_object(U&)
    {
        return converter::registered<T>::converters.get_class_object();
    }
    
    static inline Holder* construct(void* storage, PyObject* instance, reference_wrapper<T const> x)
    {
->        return new (storage) Holder(instance, x);
    }
};

The -> indicate the line. Ok so the problem is that boost is trying to create a new Quaternion in a memory area (storage) that is not guaranteed to be aligned. To fix this, we have to find how to tell boost to allocate storage for our Quaternion using Eigen’s aligned_allocator.

Ok, so the make_instance::construct method is called (frame #16) in make_instance.hpp:45 :

template <class T, class Holder, class Derived>
struct make_instance_impl
{
    typedef objects::instance<Holder> instance_t;
      
    template <class Arg>
    static inline PyObject* execute(Arg& x)
    {
        BOOST_MPL_ASSERT((mpl::or_<is_class<T>, is_union<T> >));

        PyTypeObject* type = Derived::get_class_object(x);

        if (type == 0)
            return python::detail::none();

        PyObject* raw_result = type->tp_alloc(
            type, objects::additional_instance_size<Holder>::value);
        
        if (raw_result != 0)
        {
            python::detail::decref_guard protect(raw_result);
          
            instance_t* instance = (instance_t*)raw_result;
          
            // construct the new C++ object and install the pointer
            // in the Python object.
      ->    Derived::construct(&instance->storage, 
                (PyObject*)instance, x)->install(raw_result);
            
            // Note the position of the internally-stored Holder,
            // for the sake of destruction
            Py_SIZE(instance) = offsetof(instance_t, storage);

            // Release ownership of the python object
            protect.cancel();
        }
        return raw_result;
    }
};

And we see it passes &instance->storage as the storage and instance is raw_result which is allocated with type->tp_alloc. We can see the size is determined by objects::additional_instance_size<Holder>::value.

In our case, we find the type of Holder from the second template param to make_instance_impl :

frame #16: 0x00000001029d81c1 volumit_human.so`_object* boost::python::objects::make_instance_impl<
    Eigen::Quaternion<float, 0>, 
    boost::python::objects::value_holder<Eigen::Quaternion<float, 0> >, 
    boost::python::objects::make_instance<
        Eigen::Quaternion<float, 0>, 
        boost::python::objects::value_holder<Eigen::Quaternion<float, 0>>
    >
>::execute<boost::reference_wrapper<Eigen::Quaternion<float, 0> const> const>(x=0x00007fff5fbfd618) + 145 at make_instance.hpp:45

So our Holder is

boost::python::objects::value_holder<Eigen::Quaternion<float, 0>>

Now, if we look at boost/python/object/instance.hpp:40, where additional_instance_size is defined :

template <class Data>
struct additional_instance_size
{
    typedef instance<Data> instance_data;
    typedef instance<char> instance_char;
    BOOST_STATIC_CONSTANT(
        std::size_t, value = sizeof(instance_data)
                           - BOOST_PYTHON_OFFSETOF(instance_char,storage));
};

This computes the additional size by the particular instance (the substraction of a char instance is to get only the additional size without all the common members - dict, weakrefs, objects.

offsetof(type, var) returns the offset in bytes of the variable inside the struct of the given type.

With a bit more digging, we see that additional_instance_size uses the instance class which is defined just above in the same file :

// Each extension instance will be one of these
template <class Data = char>
struct instance
{
    PyObject_VAR_HEAD
    PyObject* dict;
    PyObject* weakrefs; 
    instance_holder* objects;

    typedef typename type_with_alignment<
        ::boost::alignment_of<Data>::value
    >::type align_t;
        
    union
    {
        align_t align;
        char bytes[sizeof(Data)];
    } storage;
};

This, in turn, figures out the type alignment using boost.typetraits and specifically alignment_of.

The union type is a way to force the alignment. The C++ standard requires the alignment of a struct to be at least the lowest common multiple of the alignments of all member of the struct. See the gcc doc about alignment.

So it looks like we can just specialize alignment_of for our type.

But it is already correct. Adding the following log

LOG(INFO) << "Alignment of Quaternion : " << boost::alignment_of<Quaternionf>::value;
LOG(INFO) << "Alignment of QuaternionHolder : " << boost::alignment_of<QuaternionfHolder>::value;

prints

0414 18:06:27.344784 2072736528 human_mod.cc:1072] Alignment of Quaternion : 16
I0414 18:06:27.344817 2072736528 human_mod.cc:1073] Alignment of QuaternionHolder : 16

If we specialize make_instance to LOG when a Quaternion is created, as follow :

namespace boost { namespace python { namespace objects {

//template <class T, class Holder>
template<>
struct make_instance<Quaternionf, QuaternionfHolder>
  : make_instance_impl<Quaternionf, QuaternionfHolder, make_instance<Quaternionf,QuaternionfHolder> >
{
    template <class U>
    static inline PyTypeObject* get_class_object(U&)
    {
        return converter::registered<Quaternionf>::converters.get_class_object();
    }

    static inline QuaternionfHolder* construct(void* storage, PyObject* instance, reference_wrapper<Quaternionf const> x)
    {
        LOG(INFO) << "Into make_instance";
        LOG(INFO) << "storage : " << storage;
        LOG(INFO) << "&x : " << x.get_pointer();
        LOG(INFO) << "&x alignment (0 = aligned): " << (reinterpret_cast<size_t>(x.get_pointer()) & 0xf);
        QuaternionfHolder* new_holder = new (storage) QuaternionfHolder(instance, x);
        LOG(INFO) << "&new_holder : " << new_holder;
        return new_holder;
    }
};

}}} // namespace boost::python::objects

We get the following log

I0416 11:49:48.403352 2072736528 quaternionf_specializations.h:103] Into make_instance
I0416 11:49:48.403367 2072736528 quaternionf_specializations.h:104] storage : 0x10593a698
I0416 11:49:48.403373 2072736528 quaternionf_specializations.h:105] &x : 0x1060d10e0
I0416 11:49:48.403378 2072736528 quaternionf_specializations.h:106] &x alignment (0 = aligned): 0

Now, I think that’s the problem. Our storage at 0x106239698 is not 16-bytes aligned. A 16-bytes aligned address has its 4 last bits (the last hex digit) at 0.

First idea - using an aligned python allocation function

CPython PyTypeObject allows to specify a custom allocation function, tp_alloc.

Ok so the real problem is the call to

PyObject* raw_result = type->tp_alloc(
    type, objects::additional_instance_size<Holder>::value);

In make_instance_impl.hpp. This means the boost::python::objects instance holding the storage where our QuaternionHolder (and therefore Quaternion) is newed is allocated using Python allocators. But we actually need this to be allocated using an aligned allocator, because it is holding our Quaternion instance and as explained in the Eigen docs, a structure holding an aligned type should be aligned itself.

Remark: even if we suceeded in doing this, we would need to use EIGEN_MAKE_ALIGNED_OPERATOR_NEW in the structure holding our quaternion)

Ok so we could specify the usage of a custom tp_alloc like this :

#define GC_UNTRACKED                    _PyGC_REFS_UNTRACKED

static PyObject*
aligned_PyObject_malloc(Py_ssize_t size) {
    void* memptr;
    int ret = posix_memalign(&memptr, 16, size);
    if (ret != 0) {
        return PyErr_NoMemory();
    }
    return (PyObject*)memptr;
}


// https://github.com/python/cpython/blob/2.7/Modules/gcmodule.c

// https://github.com/python/cpython/blob/2.7/Objects/typeobject.c
PyObject *
aligned_PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)
{
    PyObject *obj;
    const size_t size = _PyObject_VAR_SIZE(type, nitems+1);
    /* note that we need to add one, for the sentinel */

    // TODO: use posix_aligned_malloc
    if (PyType_IS_GC(type)) {
      // This would involve rewriting _PyObject_GC_malloc with posix_memalign.
      // The problem is that _PyObject_GC_malloc access some gc-internal
      // statis variables that we cannot access here. So we do not support
      // objects with special GC requirements (boost class type doesn't
      // have PyType_IS_GC anyway)
      // https://docs.python.org/2/c-api/gcsupport.html#c.PyObject_GC_NewVar
        //obj = aligned_PyObject_GC_malloc(size);
        LOG(ERROR) << "Aligned allocator does not support GC objects";
        return PyErr_NoMemory();
    } else
        obj = aligned_PyObject_malloc(size);

    if (obj == NULL)
        return PyErr_NoMemory();

    memset(obj, '\0', size);

    if (type->tp_flags & Py_TPFLAGS_HEAPTYPE)
        Py_INCREF(type);

    if (type->tp_itemsize == 0)
        (void)PyObject_INIT(obj, type);
    else
        (void) PyObject_INIT_VAR((PyVarObject *)obj, type, nitems);

    if (PyType_IS_GC(type))
        _PyObject_GC_TRACK(obj);
    return obj;
}
  
template<>
struct make_instance<Quaternionf, QuaternionfHolder>
    : make_instance_impl<Quaternionf, QuaternionfHolder, make_instance<Quaternionf,QuaternionfHolder> >
{
    template <class U>
    static inline PyTypeObject* get_class_object(U&)
    {
        //return converter::registered<Quaternionf>::converters.get_class_object();
        PyTypeObject* type = converter::registered<Quaternionf>::converters.get_class_object();
        type->tp_alloc = aligned_PyType_GenericAlloc;
        return type;
    }

    static inline QuaternionfHolder* construct(void* storage, PyObject* instance, reference_wrapper<Quaternionf const> x)
    {
      LOG(INFO) << "Into make_instance";
      LOG(INFO) << "storage : " << storage;
      LOG(INFO) << "&x : " << x.get_pointer();
      LOG(INFO) << "&x alignment (0 = aligned): " << (reinterpret_cast<size_t>(x.get_pointer()) & 0xf);

      // From the specialized make_instance_impl above, we are guaranteed to
      // be able to align our storage
      //void* aligned_storage = reinterpret_cast<void*>(
          //(reinterpret_cast<size_t>(storage) & ~(size_t(15))) + 16);
      QuaternionfHolder* new_holder = new (storage) QuaternionfHolder(instance, x);
      LOG(INFO) << "&new_holder : " << &new_holder;
      return new_holder;
      //return new (storage) QuaternionfHolder(instance, x);
    }
};

The problem is, this only works for python type that are not PyType_IS_GC. But boost::class are managed by the GC cyclic collector (in boost class.cpp, tp_is_gc is a function that returns true). So we would have to define our own _PyObject_GC_malloc but we cannot do that because it relies on some static variables in python gcmodule.

There is some discussions on allowing aligned allocators in Python and for Python 3.4, we could use PyMem_SetAllocator but there seem to be no solution for 2.7

Solution - Specialize boost::python::make_instance

We can specialize some of the classes boost/python/object/make_instance.hpp . We will make sure we get a storage area large enough so that we can align our Quaternion instance. To do so, we’ll request the size of our Quaternion struct plus 16 bytes. This means that we’ll use a bit more memory than what is required.

We need to do 3 modifications :

Force instance<QuaternionfHolder> to allocate 16 bytes more than the size of QuaternionfHolder. This will allow us to align our QuaternionfHolder

union
{
  align_t align;
  char bytes[sizeof(Data) + 16];
} storage;

Have make_instance_impl correctly set the size of the python object

Holder* holder = Derived::construct(
  &instance->storage, (PyObject*)instance, x);
holder->install(raw_result);
   
// Note the position of the internally-stored Holder,
// for the sake of destruction
// Since the holder not necessarily allocated at the start of
// storage (to respect alignment), we have to add the holder
// offset relative to storage
size_t holder_offset = reinterpret_cast<size_t>(holder)
                        - reinterpret_cast<size_t>(&instance->storage)
                        + offsetof(instance_t, storage);
Py_SIZE(instance) = holder_offset;

Have make_instance::construct new our QuaternionfHolder in an aligned memory block.

static inline QuaternionfHolder* construct(void* storage, PyObject* instance, reference_wrapper<Quaternionf const> x)
{
  // From the specialized make_instance_impl above, we are guaranteed to
  // be able to align our storage
  void* aligned_storage = reinterpret_cast<void*>(
    (reinterpret_cast<size_t>(storage) & ~(size_t(15))) + 16);
  QuaternionfHolder* new_holder = new (aligned_storage) 
    QuaternionfHolder(instance, x);
  return new_holder;
}

Here is the full code :

typedef bp::objects::value_holder<Eigen::Quaternionf> QuaternionfHolder;

namespace boost { namespace python { namespace objects {

using namespace Eigen;

//template <class Data = char>
template<>
struct instance<QuaternionfHolder>
{
  typedef QuaternionfHolder Data;
    PyObject_VAR_HEAD
    PyObject* dict;
    PyObject* weakrefs;
    instance_holder* objects;

    typedef typename type_with_alignment<
        ::boost::alignment_of<Data>::value
    >::type align_t;

    union
    {
        align_t align;
        char bytes[sizeof(Data) + 16];
    } storage;
};


// Adapted from boost/python/object/make_instance.hpp

//template <class T, class Holder, class Derived>
template<class Derived>
struct make_instance_impl<Quaternionf, QuaternionfHolder, Derived>
{
    typedef Quaternionf T;
    typedef QuaternionfHolder Holder;

    typedef objects::instance<Holder> instance_t;

    template <class Arg>
    static inline PyObject* execute(Arg& x)
    {
        BOOST_MPL_ASSERT((mpl::or_<is_class<T>, is_union<T> >));

        PyTypeObject* type = Derived::get_class_object(x);

        if (type == 0)
            return python::detail::none();

        PyObject* raw_result = type->tp_alloc(
            type, objects::additional_instance_size<Holder>::value);

        if (raw_result != 0)
        {
            python::detail::decref_guard protect(raw_result);

            instance_t* instance = (instance_t*)raw_result;

            // construct the new C++ object and install the pointer
            // in the Python object.
            //Derived::construct(&instance->storage, (PyObject*)instance, x)->install(raw_result);
            Holder* holder = Derived::construct(
                &instance->storage, (PyObject*)instance, x);
            holder->install(raw_result);

            // Note the position of the internally-stored Holder,
            // for the sake of destruction
            // Since the holder not necessarily allocated at the start of
            // storage (to respect alignment), we have to add the holder
            // offset relative to storage
            size_t holder_offset = reinterpret_cast<size_t>(holder)
                                 - reinterpret_cast<size_t>(&instance->storage)
                                 + offsetof(instance_t, storage);
            Py_SIZE(instance) = holder_offset;

            // Release ownership of the python object
            protect.cancel();
        }
        return raw_result;
    }
};


//template <class T, class Holder>
template<>
struct make_instance<Quaternionf, QuaternionfHolder>
    : make_instance_impl<Quaternionf, QuaternionfHolder, make_instance<Quaternionf,QuaternionfHolder> >
{
    template <class U>
    static inline PyTypeObject* get_class_object(U&)
    {
        return converter::registered<Quaternionf>::converters.get_class_object();
    }

    static inline QuaternionfHolder* construct(void* storage, PyObject* instance, reference_wrapper<Quaternionf const> x)
    {
      LOG(INFO) << "Into make_instance";
      LOG(INFO) << "storage : " << storage;
      LOG(INFO) << "&x : " << x.get_pointer();
      LOG(INFO) << "&x alignment (0 = aligned): " << (reinterpret_cast<size_t>(x.get_pointer()) & 0xf);

      // From the specialized make_instance_impl above, we are guaranteed to
      // be able to align our storage
      void* aligned_storage = reinterpret_cast<void*>(
          (reinterpret_cast<size_t>(storage) & ~(size_t(15))) + 16);
      QuaternionfHolder* new_holder = new (aligned_storage) QuaternionfHolder(instance, x);
      LOG(INFO) << "&new_holder : " << new_holder;
      return new_holder;
      //return new (storage) QuaternionfHolder(instance, x);
    }
};


}}} // namespace boost::python::objects

Alternative solution - disable Quaternion alignment

This require changes in the code, but one option is to use Quaternion<float, Eigen::DontAlign> instead of Quaternionf. This disable alignment and should solve the problem.

References

boost_python_headers.hpp

python gcmalloc.c

D-Link DWA-131 USB key wifi adapter with Ubuntu 13.04

julien@fhtagn.net (Julien Rebetez) — Fri, 30 Aug 2013 00:00:00 +0000

Some quick notes on how to get the DWA-131 rev B working under Ubuntu 13.04. First, this webpage explains how to check what revision you’ve got. The rest of the explanations (upgrading to 3.8.10 to use the rtl8192cu driver included in the kernel) worked, but only for some networks. The device would just fail to connect to other networks.

Instead, I installed a precompiled version of the chipset (RTL8192CU) drivers, found here. Then, add three lines to /etc/modprobe.d/blacklist.conf :

# Blacklist native RealTek 8188CUs drivers
blacklist rtl8192cu
blacklist rtl8192c_common
blacklist rtlwifi

One line to /etc/modules to load the module :

8192cu

And one line to /etc/rc.local to force the driver to recognize the DWA-131 (‘2001 330D’ is the USB id of the DWA-131) :

echo "2001 330D" | tee /sys/bus/usb/drivers/rtl8192cu/new_id

myzivi.ch : swiss civilian service search

julien@fhtagn.net (Julien Rebetez) — Fri, 15 Feb 2013 00:00:00 +0000

Background / the problem

In Switzerland, we still have mandatory military service. For those who don’t want to serve in the military, there is a replacement civilian service where you work for non-profits. After having completed he military boot camp, I couldn’t stand the idleness of the swiss army anymore and decided to do civilian service instead.

Non-profits that are interested in having civilian service men working for them can announce themselves to the administration. There is an official search engine that allows people to search for those civilian service jobs. The thing is, this official page is kind of clunky : it’s slow, doesn’t always return the same results and it doesn’t allow geographical filtering. Since I wanted to find a job that’s close to my home, I really wanted to be able to visualize the offers on a map.

Solution

So I created http://myzivi.ch. It’s a website that shows a map of the civilian jobs offer from non-profits.

The data is scraped from the official page using scrapy. I use OpenStreetMap’s Nominatim to do the geocoding.

The web frontend is a Django app that uses Backbone.js and django-tastypie to communicate between them. The Google Maps javascript API is used to display the map and MarkerClustererPlus does client-side clustering of the job offers. I’m thinking about doing (partial) server-side clustering to avoid having to send all the data to the client beforehand.

I’ve also used Sentry on the production server to monitor crawling/scraping errors.

That’s a lot of technological hype for a simple tool that I hope will be useful for others.

The sources are available on github and contributions are welcome !

Some notes on OpenCV findFundamentalMat

julien@fhtagn.net (Julien Rebetez) — Thu, 27 Sep 2012 00:00:00 +0000

These are some notes about OpenCV’s findFundamentalMat function. It all started while trying to write unittests that rely on the fundamental matrix. I created a set of points matches projected from an artificial shape (so they are near perfect matches) and realized that depending on the number of points, I was getting very different results from findFundamentalMat (in term of error).

I wrote a quick script to test findFundamentalMat function. I also wanted to test if it handled 32 and 64 bit floats/double correctly (because initially, I thought the bad results were due to some float/double conversion bug between python and C++).

import cv2
import numpy as np
import numpy.linalg as la

# 15 Points
p1 = [[245.77, 248.66, 263.39, 251.11, 249.88, 287.57, 240.93, 206.84,
       224.95, 231.47, 257.55, 264.71, 227.67, 246.01, 244.15],
      [169.57, 105.82, 182.45, 146.54, 197.99, 137.11, 113.15, 171.57,
       170.34, 170.21, 124.43, 176.16, 127.05, 113.26, 154.41]]

p2 = [[267.07, 252.92, 254.22, 284.33, 236.82, 220.04, 255.9, 259.09,
       241.73, 258.62, 277.83, 219.62, 262.91, 250.93, 287.09],
      [172.34, 105.02, 190.25, 145.1, 190.63, 135.11, 114.37, 165.9,
       164.27, 168.3, 112.98, 170.46, 129.03, 114.5, 155.89]]

x1_32 = np.array(p1, dtype='f4')
x1_64 = np.array(p1, dtype='f8')
x2_32 = np.array(p2, dtype='f4')
x2_64 = np.array(p2, dtype='f8')

def compute_F_error(F):
    errs = []
    for i, p in enumerate(zip(x1_32.T, x2_32.T)):
        e = np.r_[p[1], 1].T.dot(F_32).dot(np.r_[p[0], 1])
        # normalize error by the norm of F since F is defined up to scale
        e /= la.norm(F)
        errs.append(e)
    return np.mean(errs)

np.set_printoptions(suppress=True, precision=6)

print '-- 15 points (RANSAC)'
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

print '-- 12 points (LMEDS)'
x1_32 = x1_32[:,:12]
x2_32 = x2_32[:,:12]
x1_64 = x1_64[:,:12]
x2_64 = x2_64[:,:12]
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

print '-- 7 points'
x1_32 = x1_32[:,:7]
x2_32 = x2_32[:,:7]
x1_64 = x1_64[:,:7]
x2_64 = x2_64[:,:7]
F_32, status = cv2.findFundamentalMat(x1_32.T, x2_32.T)
print "F 32 : ", F_32/F_32[2,2]
print "Avg error : ", compute_F_error(F_32/F_32[2,2])
print "status : ", np.ravel(status)
F_64, status = cv2.findFundamentalMat(x1_64.T, x2_64.T)
print "F 64 : ", F_64/F_64[2,2]
print "Avg error : ", compute_F_error(F_64/F_64[2,2])
print "status : ", np.ravel(status)

This program gives the following output. Keep in mind that the fundamental matrix is defined only up to scale. So even if the matrices look very different, they are, in fact, all valid fundamental matrices.

~|=> python cvf.py
-- 15 points (RANSAC)
F 32 :  [[-0.        0.000013 -0.002001]
 [ 0.000013  0.       -0.004486]
 [-0.001997 -0.002184  1.      ]]
Avg error :  3.52578346368e-06
status :  [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
F 64 :  [[-0.        0.000013 -0.002001]
 [ 0.000013  0.       -0.004486]
 [-0.001997 -0.002184  1.      ]]
Avg error :  3.52578346367e-06
status :  [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
-- 12 points (LMEDS)
F 32 :  [[-0.00002  -0.000007  0.007063]
 [ 0.000093 -0.000031 -0.022699]
 [-0.00931   0.011469  1.      ]]
Avg error :  -0.00957514987137
status :  [1 0 1 0 1 0 0 1 1 1 1 0]
F 64 :  [[-0.        0.000013 -0.001986]
 [ 0.000013  0.       -0.004512]
 [-0.002002 -0.002178  1.      ]]
Avg error :  -0.00957874125736
status :  [1 1 0 1 0 0 1 0 0 1 1 1]
-- 7 points
F 32 :  [[ 0.000023 -0.000003 -0.005589]
 [-0.000007 -0.000006  0.002998]
 [-0.0045    0.001153  1.      ]]
Avg error :  -7.55324825546e-15
status :  [1 1 1 1 1 1 1]
F 64 :  [[ 0.000023 -0.000003 -0.005589]
 [-0.000007 -0.000006  0.002998]
 [-0.0045    0.001153  1.      ]]
Avg error :  -7.55324825538e-15
status :  [1 1 1 1 1 1 1]

As you can see, the average error is ok for the 15 and 7 points case, but is relatively high (with some outliers) for the 12 points case.

If you look at findFundamentalMat’s documentation, it says it uses RANSAC if the number of points is greater or equal to 7. But this is kind of misleading (OpenCV bug) because actually, OpenCV does the following :

Use the 7-point algorithm if N == 7
Use LMeDS if 7 < N < 15
Use RANSAC otherwise

So our test with 12 points uses LMeDS, which, according to a paper[1], is not very good when the number of outliers is small (figure 3 of the paper, you can see RANSAC is better until the outliers percentage growth over 50%).

Since this script tests with near-perfect matches, we don’t benefit at all from LMeDS’s robustness to high outliers percentage.

Conclusion : float/double conversion from python to C++ works fine, but be choice of the fundamental matrix estimation algorithm matters.

[1]: Robust Estimation of the Fundamental Matrix and Stereo Correspondences, by Nuno Gracias , José Santos-Victor

NDK : loading resources and assets from native code

julien@fhtagn.net (Julien Rebetez) — Fri, 27 Aug 2010 00:00:00 +0000

One of the first problem I ran into when I started developing Zoob with the Android NDK is to load resources (mainly textures to be used by OpenGL) from the APK. And here is a solution to this problem.

This sample code consists of a very simple Android Java App that basically delegate all the rendering to native code (using NDK). Additionally (and I think that’s the interesting part), I’ve added libzip and libpng to the NDK code so you can load you application APK from the C/C++ code and load resources from there (example include loading a PNG file using libpng and libzip).

The code is probably not the most efficient nor bugfree, but it works.

The archive file contains a whole ndk “project” (to put under NDK_HOME/apps).
In the project/jni directory, there are 3 modules. libpng and libzip are stripped-down version of the original libs (retaining only c sources basically). The “moob” project contain the main C++ code used for rendering and loading texture. Just dive in it, it’s pretty small and the function names should be understandable.

To load the APK and read resources from it, the APK path is sent to the nativeInit method, which then load the APK using libzip. libzip provides fread like functionnality for reading compressed files so that’s pretty easy then.

There is an eclipse Java/C++ project included in the code. You might have to tweak the configuration to work with your NDK_ROOT configuration. This code was tested on the emulator and on a HTC Magic.

The zip file is available here.

I first posted this on anddev.org.

Modèles MD5 - Chargement & Animation

julien@fhtagn.net (Julien Rebetez) — Tue, 05 Oct 2004 00:00:00 +0000

Introduction

Le sujet de cette documentation est le format de modèles MD5 version 10(et non l'algorithme de vérification d'intégrité), crée pour le moteur de Doom 3.

Ce format utilise l'animation par 'joints'(='bones'), ce qui le rend plus flexible (mais également un peu plus compliqué) que le MD3(Quake 3).

Les 'joints' forment la partie centrale de l'animation d'un modèle. C'est eux qui définissent la position de la peau (les vertices du modèle qui seront ensuite rendus). Chaque 'vertice' du modèle est influencé par plusieurs poids, qui définissent l'influence qu'un 'joint' a sur tel ou tel 'vertice'.

Une hiérarchie de 'bones' est également définie, c'est à dire que chaque 'joint' a un parent et sa position est liée à celle de son parent. (Si vous bougez votre bras, votre poignet va également bouger)

Il faut également savoir que les MD5 utilisent les quaternions. Nous ne discuterons pas en détail des quaternions ici. Cette page contient une FAQ sur les quaternions.

Le format MD5 utilise des accolades pour délimiter les différentes section et des espaces pour séparer les informations.

Le code source accompagnant ce tutorial est disponible ici. Il contient une Makefile linux, utilise SDL et SDL_image et devrait donc normalement compiler sans problème sur les autres plateformes.

Du format .md5mesh

Survol

Le fichier .md5mesh contient des listes de 'vertices', de triangles, de poids, de 'bones'. Nous l'examinerons en détail ici.
La première ligne nous donne des informations sur la version du format. Nous somme uniquement intéressés par la version 10, la version 6, celle utilisé dans l'alpha de Doom 3, étant trè;s différente.
La deuxiè;me ligne donne, apparement, des informations sur différentes choses, comme les noms des 'bones' qui peuvent être détachés du mesh ou qui éjecte des douilles. Nous n'en tiendrons pas compte.
La ligne 'numJoints' nous donne le nombre de 'joints' que contient le modè;le.
Enfin, le nombre de mesh(list de 'vertices', triangles et poids) que contient le modèle.

MD5Version 10
commandline (...) numJoints 72
numMeshes 3

Ensuite commencent les informations sur les 'joints'. Chaque 'joints' a un parent (c'est le premier nombre. Un -1 indique que le 'joint' n'a pas de parent). Les 3 nombres suivants entre parenthèses donnent la position du 'joint' (en coordonnées objets). Enfin, les 3 derniers nombres entre parenthèses nous donnent les composantes x,y,z du quaternion. La partie r (parfois nommé w) du quaternion est calculée par la formule :

float t = 1.0f-(x*x)-(y*y)-(z*z);
r = (t < 0.0f) ? 0.0f : -(float)sqrt(t);

Voilà à quoi ressemble une ligne d'information sur les 'joints' :

joints {
"origin" -1 ( 0 0 0 ) ( -0.5 -0.5 -0.5 )//

Ensuite, il y a autant de sections 'mesh' que le 'numMeshes' du début. C'est là que sont décrit les meshes:
Tout d'abord, le 'shader' du mesh, c'est à dire les textures qu'il utilise (dans doom3, beaucoup de textures sont utilisés pour notamment le bump-mapping. Elle ont toutes le même nom, avec des préfixes (_d, _local, ...) différents). Nous en déduirons plus tard le nom de la texture a utiliser.
Ensuite, le nombre de 'vertices' du mesh est indiqué, puis chaque 'vertice' est décrit :
Le premier nombre donne simplement le numéro du vertice.
Ensuite, entre parenthèses, vienent les coordonnées de textures du 'vertice'.
L'avant-dernier nombre indique l'index dans le tableau des poids du poids qui influence ce vertice.
Enfin, le dernier nombre donne le nombre de poids qui ont une influence sur ce vertice(ces poids sont obligatoirement consécutifs dans le tableau des poids).

mesh { // meshes: tounge shader "models/monsters/guardian/tongue" numverts 34 vert 0 ( 0 0.7843043804 ) 0 2

Viennent ensuite des informations sur les triangles:
Tout d'abord, le nombre de triangles (numtris).
Puis, chaque triangle est décrit individuellement :
Le premier nombre est le numéro du triangle.
Les trois nombres suivants sont les indices des 'vertices' qui composent la face.

numtris 58
tri 0 2 1 0

Enfin, des informations sur les poids:
Tout d'abord, le nombre de poids, puis chaque poids individuellement. Ensuite, l'index du bone duquel dépend le poids.
Puis, le bias, qui définit la 'force' d'un poids.
Enfin, les 3 nombres entre parenthèses donnent la position du poid.(qui sera mixé avec les positions des autres poids pour donner la position d'un vertice)

numweights 83
weight 0 57 0.499899447 ( -0.0000004443 6.0386610031 2.2817306519 )

Structures utilisées

Nous discuterons ici les différentes structures c/c++ utilisées pour représenter les données du .md5mesh.
Tout d'abord, il y a deux structures qui ne sont pas directement en rapport avec le MD5, 'Face' et 'Vertice' :

struct Face
{
	unsigned int pIndex[3];
	Plane plane;
	bool visible;
};

pIndex est un tableau qui donne les index des trois vertice qui composent la face.
Plane n'est pas utilisé pour l'instant.
visible n'est pas utilisé non plus pour l'instant.

struct Vertice
{
	Vector3 vPosition;
	float s, t;
	Vector3 sTangent, tTangent;
	Vector3 vNormal;
	Vector3 vTangentSpaceLight;
};

vPosition est la position du 'vertice'.
s et t sont les coordonnées de textures.
Le reste n'est d'aucune utilité pour l'instant.

struct _MD5Joint
{
	std::string sName;
	int iNumber;
	int iParent;
	Vector3 vPosition;
	Quaternion qOrientation;
	
	_MD5Joint* pParent;
	_MD5Joint** pChildrens;
	int iNumChildrens;
};

sName est le nom du 'joint'.
iNumber est le numéro du 'joint'.
iParent est le numéro du 'joint' parent.
vPosition est la position du 'joint'.
qOrientation est le quaternion représentant l'orientation(rotation) du 'joint'.
pParent est un pointeur sur le 'joint' parent.
pChildrens est un tableau de iNumChildrens pointeurs sur les 'joints' enfants(qui ont pour parent ce 'joint'-ci.

struct _MD5Mesh
{
	int iNumVerts;
	_MD5Vert* pVerts;
	
	int iNumWeights;
	_MD5Weight* pWeights;
	
	int iNumTris;
	_MD5Triangle* pTriangles;
	
	unsigned int* pIndexes;
	
	Material* pMaterial;
	std::string sColor;
	std::string sNormal;
};

pVerts est un tableau de iNumVerts structures _MD5Vert qui stockent des informations sur les 'vertices' du modè;le.
pWeights est un tableau de iNumWeights structures _MD5Weight qui stockent des informations sur les poids du modè;le.
pTriangles est un tableau de iNumTris structures _MD5Triangle qui stockent des informations sur les triangles du modè;le.
pIndexes est une copie des index pIndex des différent triangles, simplement pour pouvoir le passer en argument à glDrawElements.
Material est un pointeur sur le material du mesh.
sColor et sNormal sont les noms des textures de couleur et de normales.

struct _MD5Vert
{
	int iWeightIndex;
	int iNumWeights;
	float pTexCoords[2];
};

iWeightIndex est l'index dans le tableau des poids du premier poid dont dépend le 'vertice'.
iNumWeights est le nombre de poids qui ont une influence sur le 'vertice'. Ils sont consécutifs au premier (donné par iWeighIndex).

struct _MD5Weight
{
	int iBoneIndex;
	float fBias;
	Vector3 vWeights;
};

iBoneIndex est l'index dans le tableau des 'bones' du 'bone' duquel dépend ce poids.
fBias donne la force du poids. (un poids avec plus de force 'attire' les 'vertices' vers lui)
vWeights donne la position du poids.

Du format .md5anim

Survol

Le fichier .md5anim contient des informations sur une animation disponible pour un modè;le. Le MD5 utilisant des 'bones', les seules (ou presque) informations nécessaires pour une animation sont celles concernant les mouvements (translation et rotation) des 'bone' pendant ladite animation.
C'est pourquoi le fichier .md5anim contient les informations sur les composantes des positions et rotations qui changent.

MD5Version 10 commandline (...) numFrames 201 numJoints 72 frameRate 24 numAnimatedComponents 197

Comme dans le fichier .md5mesh, on a une information sur la version et une 'commandline' qui ne nous sera pas d'une grande utilité.
Suivent des choses plus intéressantes, à savoir le nombre d'images ('frames') que contient notre animation (numFrames), le nombre de 'bones' que contient notre animation (numJoints), le nombre d'image par seconde de l'animation (frameRate) et enfin, le nombre de composantes qui changent chaque frame (numAnimatedComponents, qui sera expliqué plus tard).
Notez que le nombre ainsi que la hiérarchie de l'animation et du fichier .md5mesh sont les mêmes et qu'à priori, ceci est valable pour tout les modèles de Doom3. Nous ignorerons donc les informations sur le nombre de joints et la hiérarchie du fichier .md5anim et utiliseront uniquement celles du fichier .md5mesh.

hierarchy {
     "origin"        -1 0 0  //
     "Body"  0 63 0  // origin ( Tx Ty Tz Qx Qy Qz )

La seule utilitée de la section 'hierarchy' et de nous informer sur quelles composantes de chaque 'joint' sont modifiées pendant l'animation. Cela est indiqué par le deuxième nombre, qui doit être utilisé comme un 'flag'. On peut les 'décrypter' avec la 'formule' suivante :
si 'flags & 1' est vrai, la position sur les X change
si 'flags & 2' est vrai, la position sur les Y change
si 'flags & 4' est vrai, la position sur les Z change
si 'flags & 8' est vrai, la composante X du quaternion de rotation change
si 'flags & 16' est vrai, la composante Y du quaternion de rotation change
si 'flags & 32' est vrai, la composante Z du quaternion de rotation change

Nous ignorerons donc le reste de la section 'hierarchy' pour les raisons ci-dessus.
La section 'bounds' donne les coordonnées min et max des boà®tes englobantes (Bounding Box) du modèle pendant l'animation. Cela n'entre pas en ligne de compte dans ce tutorial.
Nous sautons donc directement à la section 'baseframe'.

baseframe {
    ( 0 0 0 ) ( -0.5 -0.5 -0.5 )

Cette section contient numJoints lignes et donne la position (3 premiers nombres) et les composantes x,y,z du quaternion représentant la rotation de chaque joint à la première image de l'animation.
Ces positions/rotations servent de base pour toutes les autres informations du fichier .md5anim .

frame 0 { 
    111.1 141.4 -11.5 -0.0 0.9 0.3

Suivent ensuite numAnimatedComponents nombres qui représentent les composantes qui sont modifiées par l'animation pour chaque joint(une ligne par 'joint', contenant un nombre variable d'élément, cf ci-dessus).

Structure

struct _MD5Anim
{
	std::string sName;
	int iNumFrames;
	int iNumJoints;
	int iFrameRate;
	int iNumAnimatedComponents;
	float** pFrames;
	float** pBaseFrame; 
	struct _MD5JointInfos
	{
		int iParent;
		int iFlags;
		int iStartIndex;
	};
	_MD5JointInfos* pJointInfos;
};

sName est le nom de l'animation.
iNumFrames est le nombre d'images de l'animation.
iFrameRate est le nombre d'images par seconde de l'animation.
iNumAnimatedComponents est le nombre de composants animés dans l'animation.
pFrames est un tableau de [iNumFrames][iNumAnimatedComponents] float représentant les valeures modifiées (cf flags ci-dessus).
pBaseFrame est un tableau de [iNumJoints][6], stockant, dans l'ordre, Tx, Ty, Tz, Qx, Qy, Qz a la frame de base de l'animation.
_MD5JointInfos est une structure regroupant des informations sur les 'joints', dont le champ iFlags, donnant les 'flags' du 'joint'.
pJointInfos est une invocation pour le grand Cthulhu.

De l'utilisation de ces données

Les deux fonctions les plus intéressantes et qui s'occupent réellement de gérer ces structures sont void MD5Mesh::_SkinMesh et void MD5Mesh::_BuildBone. Intéressons-nous d'abord à cette dernière :

void MD5Mesh::_BuildBone (int iFrame, _MD5Joint* pJoint,const Quaternion& q, const Vector3& v)
{
	Vector3 animatedPosition;
	animatedPosition.x = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][0];
	animatedPosition.y = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][1];
	animatedPosition.z = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][2];
	
	Quaternion animatedOrientation;
	animatedOrientation.x = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][3];
	animatedOrientation.y = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][4];
	animatedOrientation.z = pAnimations[iCurrentAnimation].pBaseFrame[pJoint->iNumber][5];
	
	
	int flags = pAnimations[iCurrentAnimation].pJointInfos[pJoint->iNumber].iFlags;
	int n=0;
	int sIndex = pAnimations[iCurrentAnimation].pJointInfos[pJoint->iNumber].iStartIndex;

	if (flags & 1) //Tx est anime
	{
		animatedPosition.x = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}
	if (flags & 2) //Ty est anime
	{
		animatedPosition.y = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}
	if (flags & 4) //Tz est anime
	{
		animatedPosition.z = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}
	if (flags & 8) //Qx est anime
	{
		animatedOrientation.x = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}
	if (flags & 16) //Qy est anime
	{
		animatedOrientation.y = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}
	if (flags & 32) //Qz est anime
	{
		animatedOrientation.z = pAnimations[iCurrentAnimation].pFrames[iFrame][sIndex+n];
		n++;
	}

	animatedOrientation.ComputeR();
	if (pJoint->iParent < 0) //pas de parent
	{
		pJoint->vPosition = animatedPosition;
		pJoint->qOrientation = animatedOrientation;
	}
	else //parent
	{
		pJoint->vPosition = q.Rotate(animatedPosition);
		pJoint->vPosition += v;
		pJoint->qOrientation = q*animatedOrientation;
	}

	for (int i = 0; i < pJoint->iNumChildrens; i++)
	{
		_BuildBone(iFrame,pJoint->pChildrens[i], pJoint->qOrientation, pJoint->vPosition);
	}
}

Cette fonction sera appelée une fois pour chaque 'bone' racine. (qui n'a pas de parent) Pour commencer, on prend les positions/rotation de l'image de base (tout est relatif à cette image).
Ensuite, on test ce qu'on doit animer. sIndex nous donne l'index de base pour ce 'bone' dans le tableau des AnimatedComponents.
On doit ensuite multiplier la rotation et additioner la position du 'bone' parent (histoire que la hiérachie serve à quelque chose). Puis, on lance cette même fonction sur tout les 'bones' enfants.

Vector3 pos;
for (int i=0; i<iNumMeshes; i++) 
{
	for (int j=0; j<pMeshes[i].iNumVerts; j++)
	{
		pVerticesListTab[i][j].vPosition.Null();
		//assignation des texcoords.
		//FIXME: On pourrait éviter à§a et ne le faire qu'une seule fois nan ?
		pVerticesListTab[i][j].s = pMeshes[i].pVerts[j].pTexCoords[0];
		pVerticesListTab[i][j].t = pMeshes[i].pVerts[j].pTexCoords[1];

		for (int k=0; k<pMeshes[i].pVerts[j].iNumWeights; k++) 
		{
			int baseIndex = pMeshes[i].pVerts[j].iWeightIndex;
			int boneIndex = pMeshes[i].pWeights[baseIndex+k].iBoneIndex;
		
			pos = pJoints[boneIndex].qOrientation.Rotate(pMeshes[i].pWeights
                        [baseIndex+k].vWeights);
			pVerticesListTab[i][j].vPosition += (pos + pJoints[boneIndex].vPosition)*
                        pMeshes[i].pWeights[baseIndex+k].fBias;
		}
	}
}

Ici, pour chaque vertice, on fait la somme des poids qui ont une influence sur le vertice en question.
Il s'agit simplement de multiplier la position de chaque poids par son 'bias' et de faire la somme pour tout les poids qui ont une influence sur le vertice.