Decision Trees

Building a Decision Tree

Building a decision tree is like buying a car. Try and imagine a situation in which I take you to a car dealer and ask you to pick out your dream car from a row of 10 cars. What is the first question that you ask the dealer? Do you ask him about the color or do you ask him about the car maker. Hopefully you would ask him about the car maker and this would help narrow down which car you would buy.

This is the same case when it comes to building a decision tree. Each time we build a decision tree we have to ask ourselves the question, what attribute (characteristic) should we evaluate first to help us get closer to classifying the data in the least amount of steps.
Would you really want to sit in a store and go through ever single attribute of the car, would you first ask about the color and the speaker size, iPod jack, seat color or would you start with important crucial questions like the make, year, model, gas mileage and so on? 

 

Occam's Razor

Occam’s Razor for cars: Imagine you were the car dealer, would you prefer the customer who asks 4 questions and then buys a car or the customer who asks 400 and still has a hard time deciding on which car to purchase. Occam’s Razor is a theorem that car dealers would like because it would weed out indecisive buyers who ask millions of questions. Basically it says that the least number of questions you have to ask in order to buy the car the better. In the field of computer science Occam’s Razor states that the shorter the solution is preferred over the more complicated one. In data mining this means that the shorter the decision tree the better, the smaller the tree, the more efficient it is.