Wednesday, November 24, 2004
David, Is BMR parameter-sensitive? Thank you.
Xiaoxu
Do you mean sensitive to the hyperparameter? In general, yes, which is why its a good idea to select it via cross-validation. D.
Xiaoxu
Do you mean sensitive to the hyperparameter? In general, yes, which is why its a good idea to select it via cross-validation. D.
Tuesday, November 23, 2004
I mean model average. bic.logit or glib both did not report intercept. I think I need those to calculate the prediction on the test data set. BTW, why I keep getting this message 'X matrix is not full rank' in s-plus when running glim? Do you have any idea? Tks!!!
I probably can't help very much since I haven't use these for a long time.
To force an intercept you can add a column of ones to your X matrix (that is,
add a new feature which always has the value one). "X matrix not of full rank"
is more of a problem since BIC evaluates the likelihood at the maximum
likelihood estimator (MLE). If the MLE does not exist, I don't what BIC
will give you and for sure, if X is not of full rank, the MLE might not exist.
What is the dimensionality of X? ie how many rows and columns? D.
I probably can't help very much since I haven't use these for a long time.
To force an intercept you can add a column of ones to your X matrix (that is,
add a new feature which always has the value one). "X matrix not of full rank"
is more of a problem since BIC evaluates the likelihood at the maximum
likelihood estimator (MLE). If the MLE does not exist, I don't what BIC
will give you and for sure, if X is not of full rank, the MLE might not exist.
What is the dimensionality of X? ie how many rows and columns? D.
Monday, November 22, 2004
David, How do you do prediction using BMA software? I found that the parameter reported is without the intercept. Do you have a better solution than fitting the model one by one? Tks!
Can you be more specific - what BMA software? Do you mean BBR? D.
Can you be more specific - what BMA software? Do you mean BBR? D.
Dear David,
I just checked the course webpage and found the target due date is Nov. 24 which is Wednesday. I thought it is Tuesday that is the due date. Am I wrong? Thanks.
What's on the webpage is correct. D.
I just checked the course webpage and found the target due date is Nov. 24 which is Wednesday. I thought it is Tuesday that is the due date. Am I wrong? Thanks.
What's on the webpage is correct. D.
Saturday, November 20, 2004
Dear David, dose BMA take care of missing values like CART?
BMA combined with Markov chain Monte Carlo can handle missing
values in a variety of ways. There are a number of papers out there that
do this but not much software that I am aware of. D.
BMA combined with Markov chain Monte Carlo can handle missing
values in a variety of ways. There are a number of papers out there that
do this but not much software that I am aware of. D.
Thursday, November 18, 2004
David,
In the results of BBR or SVMlight, does precision = #instances classified correctly / #instances. Is there any instance left out and rejected by all classies? Thank you.
You can see the definition of precision on page 4 of the slides on measuring the performance of a binary classifier. D.
In the results of BBR or SVMlight, does precision = #instances classified correctly / #instances. Is there any instance left out and rejected by all classies? Thank you.
You can see the definition of precision on page 4 of the slides on measuring the performance of a binary classifier. D.
Tuesday, November 16, 2004
Does anyone know the recipe to compute big value combination, say combin(3000000,20000)?
Thousand Thanks!
You should be able to do it on the log scale. There are quite a few routines out there for calculating log factorials (eg lfactorial in R). D.
Thousand Thanks!
You should be able to do it on the log scale. There are quite a few routines out there for calculating log factorials (eg lfactorial in R). D.
Monday, November 15, 2004
David,
For the Bayesian Logistic Regression Software, how to choose RIDGE or LASSO? If I use Gaussian prior, does it mean that I use RIDGE algorithm? If Laplace, then LASSO?
This is correct. As to which one is best, that is dataset-specific. However, Laplace/LASSO gives sparse solutions (i.e., many of the parameter estimates are zero) and that can be a decisive advantage sometimes.
For the BMR, I cannot figure out how to run it with correct parameters. Could you give me a simple example? Thank you.
bbrtrain -p 1 mydata.dat mymodel.mod
For the Bayesian Logistic Regression Software, how to choose RIDGE or LASSO? If I use Gaussian prior, does it mean that I use RIDGE algorithm? If Laplace, then LASSO?
This is correct. As to which one is best, that is dataset-specific. However, Laplace/LASSO gives sparse solutions (i.e., many of the parameter estimates are zero) and that can be a decisive advantage sometimes.
For the BMR, I cannot figure out how to run it with correct parameters. Could you give me a simple example? Thank you.
bbrtrain -p 1 mydata.dat mymodel.mod
will train a model with the laplace prior and the default hyperparameter.
If you want to select the hyperparameter by CV do, for example,
bbrtrain -p 1 -C 10 -S 0.01,0.1,1,10,100 mydata.dat mymodel.mod
The -C 10 does 10-fold CV. The -S tries those values for the hyperparameter.
D.
Xiaoxu
Saturday, November 13, 2004
David,
My dataset is in the format of text (tab delimited). Is there any way to automatically convert it into SVMlight format for BBR?
KY
You need to write a script in, for example, perl, to do that. If you need help let me know. D.
My dataset is in the format of text (tab delimited). Is there any way to automatically convert it into SVMlight format for BBR?
KY
You need to write a script in, for example, perl, to do that. If you need help let me know. D.
Thursday, November 11, 2004
David, the Bayesian Logistic Regression Software in your website, is it BMA for logistic?
Nope, it does not do Bayesian Model Averaging. Check out the
BMA website for BMA software. D.
Nope, it does not do Bayesian Model Averaging. Check out the
BMA website for BMA software. D.
Wednesday, November 10, 2004
Statistics 588 - Data MiningDavid, Would you please post your R codes for HW2?
