README.tex.md

Kernel PCA

Kernel PCA is an extension of PCA that allows for the separability of nonlinear data by making use of kernels. The basic idea behind it is to project the linearly inseparable data onto a higher dimensional space where it becomes linearly separable.

Kernel PCA can be summarized as a 4 step process [1]:

1. Construct the kernel matrix $K$ from the training dataset

$$K_{i,j} = \kappa(\mathbf{x_i, x_j})$$

2. If the projected dataset $\left{\phi (\mathbf{x}_i) \right}$ doesn’t have zero mean use the Gram matrix $\stackrel{\sim}{K}$ to substitute the kernel matrix $K$.

$$\stackrel{\sim}{K} = K - \mathbf{1_N} K - K \mathbf{1_N} + \mathbf{1_N} K \mathbf{1_N}$$

3. Use $K_{a_k} = \lambda_k N_{a_{k}}$ to solve for the vector $a_i$.

4. Compute the kernel principal components $y_k\left(x\right)$

$$y_k(\mathbf{x})= \phi \left(\mathbf{x}\right)^T \mathbf{v}_k = \sum_{i=1}^N a_{ki} \kappa(\mathbf{x_i, x_j})$$

[1] Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models

Resources

• Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models
• Kernel tricks and nonlinear dimensionality reduction via RBF kernel PCA
• PCA and kernel PCA explained
• What are the advantages of kernel PCA over standard PCA?