Monday, November 10, 2008
Friday Nov 14th, 2008
11:00am - 12:00pm
Interschool Lab, 7th Floor Schapiro CEPSR
Dr. John Hershey
Speech and Language Algorithms department
IBM T. J. Watson research center
http://domino.research.ibm.com/comm/research_people.nsf/pages/hershey.index.html
"Factorial Hidden Markov Models, Super-Human Speech Separation, and
Word Confusability"
ABSTRACT
In modeling complex phenomena such as speech, it is natural to
consider interactions between different models. In this talk we
consider two different types of interaction that share a common
computational structure. One stems from the problem of single-channel
speech separation, in which two or more speech signals have to be
extracted and recognized from an acoustic mixture. Another arises in
the problem of estimating word confusability, the probability of
mistaking one spoken word for another. Both of these problems, it
turns out, can be reduced to inference in factorial hidden Markov
models.
In the word confusability problem, commonly used divergence measures
such as the Kullback-Liebler divergence and Bhattacharyya distance are
point of departure. Since these divergence measures are intractable
for hidden Markov models, we introduce variational bounds to reduce
the computation to closed-form divergences between components. In the
case of Bhattacharyya divergence the resulting approximation has the
same structure as a two-dimensional factorial HMM. We apply the
resulting forward algorithm to a simple word confusability task,
discuss other uses for HMM divergences and future directions for this
work.
In the speech separation problem, we have models of the grammar and
acoustics of different speakers, which are combined according to an
acoustic interaction model. Unfortunately, the time complexity of
exact inference in the resulting factorial hidden Markov model is
exponential in the size of the grammar and the number of sources. To
surmount this problem we propose a linear-time loopy belief
propagation algorithm. We show that it achieves super-human speech
separation in the PASCAL speech-separation challenge, in a fraction of
the time required for exact factorial HMM inference.
BIOGRAPHY
John Hershey received the BS degree (1992) in cognitive science from
the University of California, Los Angeles and the PhD degree (2004) in
cognitive science from the University of California, San Diego, where
he was a founding member of the Machine Perception Laboratory. His
doctoral thesis explored the use of generative graphical models for
speech enhancement, face tracking, and combinations of the two. During
his time at UCSD, he interned at Mitsubishi Electric Research Labs in
Cambridge, Massachusetts in 2001, and in the Machine Learning and
Applied Statistics Group at Microsoft Research in Seattle in 2003. In
2004, he spent a year as a visiting researcher in the Speech Group at
Microsoft Research. Since 2005, he has been at IBM T. J. Watson
Research Center in New York, where he is a research staff member in
the Speech Algorithms and Engines group.
_______________________________________________
11:00am - 12:00pm
Interschool Lab, 7th Floor Schapiro CEPSR
Dr. John Hershey
Speech and Language Algorithms department
IBM T. J. Watson research center
http://domino.research.ibm.com/comm/research_people.nsf/pages/hershey.index.html
"Factorial Hidden Markov Models, Super-Human Speech Separation, and
Word Confusability"
ABSTRACT
In modeling complex phenomena such as speech, it is natural to
consider interactions between different models. In this talk we
consider two different types of interaction that share a common
computational structure. One stems from the problem of single-channel
speech separation, in which two or more speech signals have to be
extracted and recognized from an acoustic mixture. Another arises in
the problem of estimating word confusability, the probability of
mistaking one spoken word for another. Both of these problems, it
turns out, can be reduced to inference in factorial hidden Markov
models.
In the word confusability problem, commonly used divergence measures
such as the Kullback-Liebler divergence and Bhattacharyya distance are
point of departure. Since these divergence measures are intractable
for hidden Markov models, we introduce variational bounds to reduce
the computation to closed-form divergences between components. In the
case of Bhattacharyya divergence the resulting approximation has the
same structure as a two-dimensional factorial HMM. We apply the
resulting forward algorithm to a simple word confusability task,
discuss other uses for HMM divergences and future directions for this
work.
In the speech separation problem, we have models of the grammar and
acoustics of different speakers, which are combined according to an
acoustic interaction model. Unfortunately, the time complexity of
exact inference in the resulting factorial hidden Markov model is
exponential in the size of the grammar and the number of sources. To
surmount this problem we propose a linear-time loopy belief
propagation algorithm. We show that it achieves super-human speech
separation in the PASCAL speech-separation challenge, in a fraction of
the time required for exact factorial HMM inference.
BIOGRAPHY
John Hershey received the BS degree (1992) in cognitive science from
the University of California, Los Angeles and the PhD degree (2004) in
cognitive science from the University of California, San Diego, where
he was a founding member of the Machine Perception Laboratory. His
doctoral thesis explored the use of generative graphical models for
speech enhancement, face tracking, and combinations of the two. During
his time at UCSD, he interned at Mitsubishi Electric Research Labs in
Cambridge, Massachusetts in 2001, and in the Machine Learning and
Applied Statistics Group at Microsoft Research in Seattle in 2003. In
2004, he spent a year as a visiting researcher in the Speech Group at
Microsoft Research. Since 2005, he has been at IBM T. J. Watson
Research Center in New York, where he is a research staff member in
the Speech Algorithms and Engines group.
_______________________________________________
