Multiple kernel learning methods

Mklaren - Simultaneous multiple kernel learning and low-rank approximation

Uniform - trivial combination of kernels

Align - Independent centered alignment

The algorithms based on centered aligmnent proposed in

  1. Cortes, M. Mohri, and A. Rostamizadeh, “Algorithms for Learning Kernels Based on Centered Alignment,” J. Mach. Learn. Res., vol. 13, pp. 795-828, Mar. 2012.

Given \(p\) kernel matrices \(\mathbf{K}_1, \mathbf{K}_2, ..., \mathbf{K}_p\), centered kernel alignment learns a linear combination of kernels resulting in a combined kernel matrix.

\[\mathbf{K}_{c\mu} = \sum_{q=1}^p \mu_q \mathbf{K}_{cq}\]

where \(\mathbf{K}_{cq}\) is the centered kernel matrix.

\[\mathbf{K}_{cq} = (\mathbf{I} - \dfrac{\mathbf{11}^T}{n})\mathbf{K}_q (\mathbf{I} - \dfrac{\mathbf{11}^T}{n})\]

The following methods perform a constrained optimization over \(\mathbf{\mu} = (\mu_1, \mu_2, ... \mu_p)\) maximizing the centered alignment:

\[A = \dfrac{<\mathbf{K}_{c\mu}, \mathbf{y}\mathbf{y}^T>_F} {n <\mathbf{K}_{c\mu}, \mathbf{K}_{c\mu}>_F}\]

where \(\mathbf{y}\mathbf{y}^T\) is the ideal kernel based on target vector \(\mathbf{y}\) and \(<\cdot, \cdot>_F\) is a matrix inner product.

class mklaren.mkl.align.Align(d=2)
Variables:
  • Kappa – (numpy.ndarray) Combined kernel matrix.
  • mu – (numpy.ndarray) Kernel weights.
__call__(i, j)

Access portions of the combined kernel matrix at indices i, j.

Parameters:
  • i – (int) or (numpy.ndarray) Index/indices of data points(s).
  • j – (int) or (numpy.ndarray) Index/indices of data points(s).
Returns:

(numpy.ndarray) Value of the kernel matrix for i, j.
__getitem__(item)

Access portions of the kernel matrix generated by kernel.

Parameters:item – (tuple) pair of: indices or list of indices or (numpy.ndarray) or (slice) to address portions of the kernel matrix.
Returns:(numpy.ndarray) Value of the kernel matrix for item.
fit(Ks, y, holdout=None)

Learn weights for kernel matrices or Kinterfaces.

Parameters:
  • Ks – (list) of (numpy.ndarray) or of (Kinterface) to be aligned.
  • y – (numpy.ndarray) Class labels \(y_i \in {-1, 1}\) or regression targets.
  • holdout – (list) List of indices to exlude from alignment.
class mklaren.mkl.align.AlignLowRank(d=2)

Use the align method using low-rank kernels. Useful for computing alignment of low-rank representations.

__call__(i, j)

Access portions of the combined kernel matrix at indices i, j.

Parameters:
  • i – (int) or (numpy.ndarray) Index/indices of data points(s).
  • j – (int) or (numpy.ndarray) Index/indices of data points(s).
Returns:

(numpy.ndarray) Value of the kernel matrix for i, j.
__getitem__(item)

Access portions of the kernel matrix generated by kernel.

Parameters:item – (tuple) pair of: indices or list of indices or (numpy.ndarray) or (slice) to address portions of the kernel matrix.
Returns:(numpy.ndarray) Value of the kernel matrix for item.
fit(Gs, y, holdout=None)

Learn weights for low-rank representations of kernel matrices.

Parameters:
  • Gs – (list) of (numpy.ndarray) to be aligned.
  • y – (numpy.ndarray) Class labels \(y_i \in {-1, 1}\).
  • holdout – (list) List of indices to exlude from alignment.

Alignf - Optimizing the linear combination of kernels