Objectives:
The objectives of this section are:
to impart on the reader the need for an efficient construct
to present you with a high level algorithm for creating a decision tree
to inform you of the problems that arise with the algorithm and how they are addressed.
Outcomes:
By the time you have completed this section you will be able to:
compare decision trees and decide whether or not they are efficient
explain from a high level Hunts’ algorithms and the difficulties encountered
Efficient Decision Tree Construction
Pacemaker
At this point you should have a basic understanding of what classification is and its usefulness in our society. The Decision Tree Classifiers was introduced in the previous section and you should be able to identify a decision tree and its various parts. You should also be able to pick out based on Occam’s Razor which decision tree is more efficient. Which one of these decision trees do you think is the most efficient?
If you picked the second one you are correct.
By looking at the two decision trees that have already been created it is easy to choose which one we believe will be more efficient but in reality this process is not that simple.
Need for Efficiency
If it took the model 10 seconds to build each tree and there are 2 million possible trees how long would you have to wait to see all possible trees?
Do you really want to have to look at all the 2 million trees and figure out by comparing and contrasting their depth which one is better?
And why would you want to construct all these decision trees from top to bottom just to discard them at the end.
Luckily for us there are algorithms that have been develop to help determine the most efficient decision tree;
I must add that we can never find the best tree in a fixed amount of time but we can find a semi-optimal tree that works.
Efficient Construction Application
As mentioned above finding the best tree in a fixed amount of time is near impossible but there are atttribute selection measures discussed in the intermediate level that help us find a semi-optimal tree. Click the link below to download an application that will allow you to build a decision tree for a particular dataset based on these measures. For instructions on how to use the Decision Tree Builder Application consult Help Section 4.
