Statistics 4240 - Data Mining

David,
Can the Bayesian Binary Regression Software on your website classify more than 2 classes? Is there a example for this? The example online calssifies 2 classes. Thank you.

There exists a multiclass version of the software but we have not yet publicly released it. If you email me I can send you the URL. D.

I'm trying to put together the prediction results for the homework assignment but its getting too complicated because of the different formats. Please download this spreadsheet, enter your predictions in the yellow cells (i.e., overwrite the numbers that are there), save the spreadhseet, and send it back to me. Thanks, D.

David,

Can you talk about ROC (receiver operating charateristic) curve and its implementation in class, or refer some materials to read? Thanks. K. P.

Sure. Here's a decent web page describing ROC curves. D.

when I use predict(train_model,se.fit=T), a list of train_model$se.fit are calculated, one for one y, what does se.fit mean?

What R function produced the train_model object? D.

Just linear:

train_model <- lm(newAccounts~.,data=train_data,se.fit=T)

The se's here are just stanard errors for the predictions. Do help(lm.predict) and it will tell you a little more. D.

If y is not standardized, will the best lambda be a different value?

Yes, it will impact the best lambda

Why would the lambda be different?

Because the scale of y affects the scale of the beta's. If you
are trying to predict degrees centigrade as against degrees fahrenheit,
the regression coefficients will change and hence lambda will change.

It seems to me that it is a little bit easier to find the best lambda in this problem if we standardize y. I am not quite sure. Thanks.

I don't think it will make much difference to predictive performance
in the this problem. D.

Maybe you can try other values of lambda and find the best one. :)

Yes, if the one standard error rule is not behaving sensibly, that's what I would do.

After feeding the normalizied feature values into ridge algorithm, we predict the values y'= (y-mean(y in the training set)/sqrt(variance(y in the training set))). Should we get y from y' by the reverse calculation? Thanks.

Exactly right - invert the standardization to get the predictions on the right scale. Note you don't have to standardize y but if you do you need to this inversion. D.

dear David,
for the bank data set, it seems that ridge regression doesn't help a lot. If we choose the model within one standard error of the best, lambda=0. I am not sure whether I did sth. wrong. --B.J.

The one standard error rule is just heuristic so it could screw up. You could just take value of lambda that gives the best average performance instead. D.

David,
For ridge regression in HW2, shoud we center the predictors in the first 200 branches only and center the rest 33 later before prediction? or should we take the mean of all? Tks ---B.J.

Just standardize the 233 all together. Note you need to subtract the mean *and* divide by the standard deviation. D.

but according to the book p.60, it seems that we should centered the Xs by subtracting mean only. so that we can estimate beta-null by y-bar. ---B.J.

Centering is enough to give a direct estimate of beta-null but you really need to standardize the predictors so that they are all on the same scale. D.

David,

We usually use the maximum likelihood to pick the most possible lable. If the label is a subset of a set of enumerate values and we don't know how many elements in that subset, what should we do?
For example, given age, salary, gender and education level, to predict if this person is {happy, rich, smart}. A person could be both rich and happy. But we do not know exactly how many labels we should attach to him/her. We cannot pick the most likely two because we don't know there are exactly two.
I read the "Bayesian logistic regression software" webpage and notice that some thresholds are used. Could you explain a bit more about this? Or could you give us some information?

I think you are describing a multivariate regression problem. Up to now we have looked only at univariate regression models where there is one variable that plays the role of the response variable. In your scenario there are three binary response variables, happy (yes or no), rich (yes or no), and smart (yes or no). So you could build three separate binary classifiers (e.g. using the BBR software or support vector machines or whatever). Even better would be to build a multivariate (aka multilabel) classifer. Neural networks are sometimes used for multilabel problems - Tuesday's class will talk about neural networks. D.

Here are the R slides I presented last night. Here is an R tutorial that Aynur found. D.

David,

Should predictors standardized in both cross validation model and ridge regression?
Should binary variables standardized also? Thanks.

Standardizing the predictors is certainly something you should do for ridge regression. For regular regression you don't have to do it. Yes, you should also standardize the binary predictors. D.

In R, how to enumerate a data.frame column by column? The pseudo code is like

foreach col ( data.frame ) {
process col;
store processed col elsewhere
}

There is probably some cooler way to do this but here is what I do:

numColumns <- dim(myDataFrame)[2]

for (i in 1:numColumns) {

myColumn <- myDataFrame[,i]

do stuff to myColumn

}

Also related, how to find out the index of the last row/element of a data.frame/vector?

See the "dim" command above. D.

I'm back. I will hold a little R session after class on Tuesday. This is of course completely optional. D.

deaR pRofessor, can we vote for the R cLass once more pLease?...
NeBa

OK. Please send me email if you are interested in a R class. Scheduling this would be tricky - would anyone stay if we did it after regular class on Tuesday? D.

David,
Can you post the schedule of the final project?

Here is project schedule: 1-page proposal to me by November 1. Project due November 23rd. Project reviews due November 26th (each person reviews three projects). "Best" Projects presented in class November 30th and December 7th.

Hi Prof.

can association rule mining yields this kind of rule for an instance?

(A or B) and not C => D

eh.

I don't know the association rule literature well enough to point at a specific paper, but I bet such an algorithm exists. You could certainly find A and not C => D and B and not C => D using the standard algorithm and then postprocess to get your rule. D.