Statistics 4240 - Data Mining

Thanks for a good semester! Have a happy holiday season and a good 2005! D.

Statistics 588 - Data Mining
Got it. We use a single threshold as the weak learner. That's why it works a big problems. Thank you.

Does "scaling" mean increasing the number of instance or increasing the number of features? Chunking is for the large number of instances. Xiaoxu

"Scaling" in general can refer to either. D.

David, I am using a package called SVMFu. It uses chunking to make SVM scale better. As our experiments show, logistical regression and SVM do not scale as well as boosting. You have mentioned this in class. I cannot remember how you rank these three exactly. Could you talk a little bit about their performance in large data set? Xiaoxu

I'm not quite sure what you mean when you say "logistical regression and SVM do not scale as well as boosting." Boosting repeatedly learns a base learner so its scalability (in either dimension) is certainly worse than that of its base learner. If you use a very simple base learner, you might be able to make boosting work on big problems.

Both logistic regression and SVM's have similar scaleability properties since they involve very similar optimization problems. D.

David,

How to choose the size of chunks in SVM? I have about 8500 instances. Each one has 586 features. Is any formula can calculate the size of the chunck? I tried 200, 500,1000. It keeps saying "things could get slow". Thanks a lot.

What is a chunk? Is this specific to some software? D.

David,

do you think it is relevant to the class to have a brief introduction in multidimensional scaling & min. description length?

Certainly relevant but we have probably run out of time to cover it. D.