Convolutional Kernel Networks
What is CKN?
CKN (Convolutional Kernel Networks) is a software package corresponding to the
NIPS paper ``Convolutional Kernel Networks''.
In this paper, we are exploring a new type of convolutional neural network that has the following properties:
- learning is unsupervised, even though it could probably be used to initialize backpropagation (not tested yet).
- we control what the non-linearities of the network are really doing: the network tries to approximate the kernel map of a reproducing kernel.
At the moment, we have not put any engineering or implementation effort in the
software package since our goal was to understand whether such a new
formulation could be useful or not. This means that the current 100%
implementation is rather slow, but we are planning to make it faster in the
future, by changing both the learning algorithm (SGD instead of L-BFGS), and
implementation details (C++, GPUs).
We have also did not have the time to put a significant engineering effort in
the experimental design. Implementing and conducting all experiments of the
paper took us about two/three weeks (by using a small cluster).
At the moment, we do not use any arbitrary data augmentation or data
pre-processing, which implicitly increase the number of hyper-parameters and
increase the risk of (unintentionally) overfitting on the test set, making
results harder to reproduce. We are nevertheless planning to have a look
at such techniques for the future.
Version 1.0 and later are open-source under licence GPLv3.
19/11/2014: CKN v1.0 is released.