IMPLEMENTING DECISIONS IN BINARY DECISION TREES USING INDEPENDENT COMPONENT ANALYSIS

Petteri Pajunen

There are various ways to implement decisions in bina­ ry decision trees. Most approaches can be interpreted as greedy methods optimizing some local goodness cri­ teria at each node. Often it is required that either target values (regression trees) or class labels (classi­ fication trees) are available. In this paper linear ICA is applied to implement the decisions in a binary tree. The rationale is that ICA can be used to find directions where the data has ``structure''. The linear transforma­ tion defined by ICA can then be interpreted as a change of variables, where the new variable captures the struc­ ture, e.g. has smallest entropy. The linear decisions are then made by tresholding the variable. An experiment is presented which shows that the proposed method can find reasonable representations of real­world data in an unsupervised manner, i.e. without using class labels.