In this project, we have considered extensions of basic linear independent
component analysis (ICA) and blind source separation (BSS) to nonlinear
In the following, we first briefly describe in time order the most important
results of the ICA and Bayes research groups of our research centre on
nonlinear ICA and BSS. After that, we provide links to publications and useful
web sites on nonlinear ICA and BSS. A list of these publications and web sites
appears at the end of this page.
Extended summaries of our results on nonlinear ICA and BSS can be found in the
progress reports of our Bayes and ICA groups, which are available on the home
pages ,  of these groups.
Software on our Bayesian methods applicable to nonlinear BSS problems
as well as more our publications related to nonlinear BSS and ICA can be found
on the home pages of our Bayes group .
Researchers who have during the last years been active with respect to
nonlinear ICA and BSS in the Adaptive Informatics Research Centre at the
Department of Information and Computer Science, Aalto University Scool
of Science and Technology (formerly Helsinki University of Technology),
located in Espoo, Finland are
The most important early results of our ICA group deal with the existence
and uniqueness of the solutions of the nonlinear ICA problem. They have been
reported in the seminal theoretical paper , where A. Hyvärinen and
P. Pajunen have shown that the nonlinear ICA problem is highly
non-unique and ill-posed without additional regularization. More uniqueness
results can be found in the reviews  and .
Our other early results on nonlinear ICA and BSS include a method based on the self-organizing map and a maximum likelihood based approach. They have been reported in Chapter 17 of the book  and the maximum likelihood method also in . However, these methods are applicable to small-scale problems only because their computational load explodes with the dimensionality of the problem, and their accuracy is rather limited.
During the last years, we have successfully applied variational Bayesian learning
(called in our earlier papers Bayesian ensemble learning) to nonlinear
BSS and factor analysis. This methodology can be used to the selection,
construction, and unsupervised (blind) learning of suitable latent variable
models for nonlinear BSS. Basic methods for nonlinear factor analysis
and ICA have been introduced in . The simpler nonlinear factor analysis (NFA)
method in  first finds a nonlinear PCA (principal component analysis) solution.
A nonlinear BSS solution can then be obtained by applying standard linear
ICA as a post-processing step to the subspace provided by NFA. The NIFA (nonlinear
independent factor analysis) method is an extension of the NFA method which finds
the nonlinearly mixed source signals from the NFA solution by continuing the learning
process using a mixture-of-Gaussians model for the sources . The paper 
extends the method to blind identification of a nonlinear dynamic model,
and our results on this line of research have been reviewed in  and .
The figure above shows 10 sources signals extracted from 30-dimensional real-world pulp data using the NFA method. This data set is clearly nonlinear, because it can approximated using the NFA method with much less components than using standard principal component analysis. See the paper  and Chapter 17 of our ICA book  for a more detailed description of these results.
Each subfigure of this figure shows the original time series (components) of the
30-dimensional pulp data on the top of each subfigure and below them their reconstruction
based on the 10 source signals depicted in the previous figure.
The variational Bayesian methods - can be applied to considerably larger scale problems than our early methods, but even their computational load grows quite large especially for the nonlinear dynamic model introduced in . To alleviate this problem, we have developed variational Bayesian methods based on standardized building blocks . In these methods, all the computations can be carried out locally, resulting in linear computational complexity. A basic paper on the application this approach to the nonlinear BSS problem is , and the building block approach is described in detail in the long journal paper .
A recent general review on nonlinear blind source separation and independent
component analysis is the book chapter . It is an updated version of the
earlier review paper , which contains more references to older works.
These two reviews  and  discuss the theoretical foundation of nonlinear
ICA and BSS, especially their uniqueness issues. They also deal with the simpler
problem of separating post-nonlinear mixtures and our variational Bayesian
approach to nonlinear BSS. Both reviews contain an extensive list of
references on nonlinear BSS and ICA. A recent short book on nonlinear blind
source separation is . It reviews the main methods, including in particular
the MISEP method developed by the author L. Almeida.
Our book Independent Component Analysis  has become a standard reference in the field. It provides the necessary mathematical background to the reader, and deals extensively basic standard linear ICA as well as many of its extensions and a few applications. The handbook in  provides on a more advanced level for the researchers working in its fields an up-to-date reviews of currently used ICA and BSS methods as well as their most important extensions and applications.
 NEW! C. Jutten, M. Babaie-Zadeh, and J. Karhunen,
"Nonlinear mixtures", Chapter 14, pp. 549-592, in P. Comon and C. Jutten (Eds.),
Handbook of Blind Source Separation, Independent Component
Analysis and Applications, Academic Press, 2010.
- A new review of nonlinear ICA and BSS, containing theoretical results, methods, and applications, with many references.
 C. Jutten and J. Karhunen,
Advances in blind source separation (BSS) and independent component analysis
(ICA) for nonlinear mixtures.
International Journal of Neural Systems, Vol. 14, No. 5, 2004, pp.
- Invited general review article on nonlinear blind source separation and independent component analysis containing many references.
 L. Almeida,
Nonlinear Source Separation. Synthesis Lectures on Signal Processing,
Morgan&Claypool Publishers, 2005, 114 pages.
- New concise book which reviews the main nonlinear blind separation methods, including the MISEP method developed by the author.
 A. Hyvärinen and P. Pajunen,
Nonlinear independent component
analysis: existence and uniqueness results.
Neural Networks , Vol. 12, No. 3, 1999, pp. 429-439.
- Seminal paper on the existence and uniqueness of the solutions of the nonlinear ICA problem.
 H. Lappalainen and A. Honkela,
Bayesian nonlinear independent component analysis by multilayer perceptrons.
In M. Girolami (Ed.), Advances in Independent Component Analysis,
pp. 93-121, Springer-Verlag, 2000.
- Basic paper on our first variational Bayesian (ensemble) learning method for nonlinear blind source separation.
 H. Valpola, E. Oja, A. Ilin, A. Honkela, and J. Karhunen,
Nonlinear blind source separation by
variational Bayesian learning. IEICE Transactions (Japan),
Vol. E86-A, No. 3, March 2003, pp. 532-541.
- Invited paper which concisely reviews our static and dynamic nonlinear blind source separation methods based on variational Bayesian learning.
 H. Valpola and J. Karhunen,
An unsupervised ensemble learning method for nonlinear
dynamic state-space models. Neural Computation, Vol. 14, No. 11,
2002, pp. 2647-2692.
- Long and thorough paper which extends variational Bayesian learning to blind estimation of a nonlinear dynamic model for the source signals.
 J. Karhunen, Nonlinear ICA, in
S. Roberts and R. Everson (Eds.) Independent Component Analysis: Principles
and Practice, Cambridge Univ. Press, 2001, Chapter 4, pp. 113-134.
- A short review which discusses somewhat in more detail a maximum likelihood method and a variational Bayesian method. Chapter 17 of  is a longer version of this book chapter.
 H. Valpola, T. Östman, and J. Karhunen,
Nonlinear independent factor analysis by hierarchical
models, in Proc. of the 4th Int. Symp. on Independent Component
Analysis and Blind Signal Separation (ICA2003), Nara, Japan, April 2003,
- A basic paper on the application of the Bayes building blocks approach to nonlinear blind source separation.
 T. Raiko, H. Valpola, M. Harva, and J. Karhunen,
Building blocks for variational Bayesian learning
of latent variable models, Journal of Machine
Learning Research, Vol. 8, January 2007, pp. 155-201.
- A long journal paper where the Bayes building blocks approach is introduced and applied to the construction and variational Bayesian learning of several example structures.
 A. Honkela, H. Valpola, A. Ilin, and J. Karhunen,
Blind Separation of Nonlinear Mixtures by Variational
Bayesian Learning, Digital Signal Processing, Special issue on Bayesian
Source Separation, Vol. 17, Issue 5, September 2007, pp. 914-934.
- A review paper on our Bayesian methods and results on nonlinear BSS.
 Home page of the Bayes group in the Adaptive Informatics Research Centre of Aalto Univ. School of Science and Technology (formerly Helsinki University of Technology), Espoo, Finland. Publications and software related to nonlinear ICA and BSS.
 Home page of the ICA group in the Adaptive Informatics Research Centre of Aalto Univ. School of Science and Technology (formerly Helsinki University of Technology), Espoo, Finland. Useful information on many aspects of ICA and BSS and links.
 Home page of the book A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis, Wiley 2001. Chapter 17 deals with nonlinear ICA and BSS.