Statistics 4240 - Data Mining

Happy holidays to all of you and thanks for fun semester! Regards, David.

Can you tell us what you were looking for on the second part of #10 of the final?

Lasso estimates correspond to the posterior mode under a Laplace prior. Ridge estimates correspond to the posterior mode under a Gaussian prior.

The final exam is Tuesday evening at 7:10pm in the usual room.

On the practice final, I am confused about what you mean by "model-based clustering". Did we skip over this in class? Is it covered in the book?

Its synonymous with mixture modeling. Take a look at the descriptive modeling slides.

A couple of people have asked me to post k-means R code. Here is
Yair's very nice code:


euc <- function(y, x, n, k)
return(sqrt(apply((x - matrix(rep(y,n), nrow=n, ncol=k, byrow=T))^2, 1, sum)))

kmeansyg <- function(x, n_clus, nstart=1)
{
n <- dim(x)[1]
k <- dim(x)[2]

for (i in 1:nstart)
{
### random starting points – make them unique points in the data
repeat
{
starts <- ceiling(runif(n_clus)*n)
if (length(unique(starts)) == n_clus) { break }
}
centers <- x[ceiling(runif(n_clus)*n),]

### initialize clusters so they're unequal, so the while() loop will start
cluster <- rep(0, n)
cluster_old <- rep(1, n)

while (sum(cluster - cluster_old == 0) != n)
{
### save old cluster for while() comparison, and assign new clusters
cluster_old <- cluster
cluster <- apply(apply(centers, 1, euc, x=x, n=n, k=k), 1, which.min)

### compute cluster means
for (j in 1:n_clus)
centers[j,] <- apply(x[cluster == j,], 2, mean)
}

withinss <- 0
for (j in 1:n_clus)
withinss <- (withinss + sum(dist(x[cluster == j,])))/2
withinss <- withinss / length(table(cluster))
test <- list(cluster=cluster, centers=centers, withinss=withinss)

if (i == 1)
answer <- test
if (test$withinss < answer$withinss)
answer <- test
}
return(answer)
}

What is the solution to the last part of the first question on the sample final? How can we apply naive Bayes to regression problems?

Naive Bayes is a generative probabilistic preditive model. The key assumption
is that the predictor variables are conditionally independent given the response
variable. None of this is specific to classification, right?

If we are analyzing the accuracy of a multi class classifier such as linear discriminant analysis, would it make sense to construct an ROC curve of the specificity and sensitivity for each class? For example, treat the false positive rate as the number of observations that the classifier determined to be in the class divided by the number of actual observations that are not in the class.

You certainly can use ROC on each class individually. There are a number
of suggestions for multi-class ROC extensions - see for example
http://portal.acm.org/citation.cfm?id=1279009.1279176&coll=GUIDE&dl=GUIDE