Statistics 4240 - Data Mining

Dear Dr.Madigan,

You had mentioned that if we have any ideas for the 5-6PM hour we should mention that.I was wondering regarding the same that the lecture you gave on Graphical Markov models was very interesting, if you can take time to give some examples of that or some related material in that hour, it might be very useful.

Thanks.

Good idea! D.

Dear Professor,

Can we get a sample from previous exams or back of the book so that we know what type of questions might be asked in the exam.Since we have to concentrate basically on what is taught in class but the books deals with a lot more.So if we get a sample it would be easier to know hot to prepare from the book.

Thanks a lot.

I'm still a little unsure about the utility of a final exam. I'm interested in people's opinions on this. I could base the grade entirely on the project and skip the final exam. Do you think this is a good idea or not? D.

Professor Madigan,
Are we having the 5PM PDL Class on Tuesday the 28th?
Thanks.

We are going to talk about projects today at 5pm. D.

Project proposals (about a page) are due at the end of this month. You can hand me something in class on Tuesday or send it via email anytime between now and the end of October 31. I will talk about projects at 5pm on Tuesday. D.

You don't list 5.8 as a section to skip but you didn't cover it in class. Should we skip it?

We're skipping it for now but we will come back to it later. D.

Perl again at 5pm today. This will probably be the last session. D.

Dear professor:
In last week's perl session, you showed us code for reading through an input file line by line. In tomorrow's class, could you please show what code you would use to:
a) read through a text file line by line checking for the occurrence of certain two words, either juxtaposed or possibly separated by some number of other words,
b) return a count for the number of occurences, identified by file name
c) go to the next text file and repeat.
d) If possible, see how to do this for multiple two-word occurrences.

I have been playing around with pattern matching but it's not clear how to do the above.

By the way, thank you for the perl sessions. E. Henry


For the multiple files, just put the filenames in an array and loop through them using a for loop. For the regular expression part, try this:

# pretend this is the document you read in from the file:
$document = "the fox the abc abc abc abc abc fox the abc fox";

# suppose you want to find "the" and "fox" with between 0 and three words between them:
$document =~ /the (\S+? ){0,3}fox/g;

# for the example above this returns 2 hits. I'll go over this later in class. D.

Thanks for your comments about the book. The book basically fills in the details that I skip over in class; these details are, for the most part, marthematical - that's life! However, the math required is mostly very basic calculus (a function achieves a maximum or minumum when its derivative is zero) and some matrix manipulations. There's a nice web page here on basic matrix algebra.


We will be skipping over parts of the book that I think are purely technical. Here's a web page where I keep a list of the sections that we are skipping - I'll be updating this as we go along.


In general we will focus more on concepts than details - for the final exam I will only consider stuff I covered specifically in class. D.

Yes, I too had a similar experience with the book. Perhaps it would be really helpful if we could get some more relevant material that would help prepare a base necessary to understand thoroughly the material in the book. Or perhaps some pointers as to how to go about studying, the sequence to follow.

Thank you very much.

Prof. Madigan,

I had a similar experience with the material from the book and I am glad to see that I am not the only one.
I am reading a lot of supplementary mterial as well though from Hand, Manilla book and Jiawei han's book along with some other papers.

thanks

Prof. Madigan,

I recently started reading the Hastie book and, to be honest, I was disheartened by the way the material is presented. After spending some hours on chapter 3, I didn't go further than the intuitive facts we covered in class, mostly thanks to some examples and figures. Other than that, the description seems to me almost as a juxtaposition of propositions and formulas from linear algebra and advanced statistics, which are condidered to be known (or even obvious) and are presented without the need to be justified or put into context. Should I start worrying about my lack of solid background on what the book pressupposes, or will the course continue to focus on the concepts rather on the underlying theory ? Thank you

Dear Professor,
In the class of 7th Oct, we had reviewd Graphical markov models, Could you post some pointers to that for additional material like a tutorial or something.
Thanks

Here's a tutorial on graphical models. Here's a tutorial specifically about learning graphical models from data.