HLT@INESC-ID - User contributions [en]

File:Logo-msc0708-12.jpg

2007-04-19T14:21:17Z

Dcaseiro:

WFST - Weighted Finite State Transducers Applied to Spoken Language Processing

2007-02-12T17:48:09Z

Dcaseiro: /* Demos */

== Team ==

Project Leader: [[Diamantino Caseiro]]

* [[Isabel Trancoso]]
* [[Rui Amaral]]
* [[Luís Caldas de Oliveira|Luís Oliveira]]
* [[Maria do Céu Viana|M. Céu Viana]]
* Pedro Carvalho

== Summary ==

Finite-State models (FSM) and, in particular, Weighted Finite-State transducers (WFST) have proven quite successful in many fields of written and spoken language processing. This includes in particular machine translation, large vocabulary continuous speech recognition and speech synthesis.
An interesting feature of FSMs is that they can be automatically built or "learned" from training data using corpusbased techniques. Compared to more traditional knowledgebased approaches, these techniques are attractive for their potential of much lower development costs. Another interesting property of FSMs is their feasibility for implementing or approximating knowledge-based techniques. Different knowledge sources can hence be represented via FSMs, thus allowing the integration of apriori knowledge with inductive techniques in a natural and formally elegant way. This makes the FSM framework an adequate one for language processing.

The main goal of this project is the application of this framework to speech recognition and synthesis, which will constitute the themes of the two major tasks.

The group has already acquired some experience in modeling the various components of recognition systems using WFSTs, having developed specialized algorithms for transducer composition for on-the-fly lexicon and language model integration. Preliminary experiments with the explicit integration of phonological rules have also produced encouraging results, but much remains to be done in order to be able to achieve higher recognition rates, specially in what concerns spontaneous speech, not only in terms of pronunciation modeling, but also of language modeling.

The group's experience with the application of WFSTs to different modules of text-to-speech (TTS) synthesis is much more recent. We are currently exploring the potential of WFST application, namely in terms of grapheme-to-phone conversion and variable duration segment selection in synthesis by concatenation. We plan to continue this preliminary work and extend it to other modules.

A third task will deal with more exploratory themes on which the group currently has no experience, having as a goal the integration of different knowledge sources. Potential themes are the use of transducers for topic indexation, or for integrating translation models to help speech recognition of documents which have originally been written in one language and are being dictated by a human translator in another language.

The long term goal of this project is a unified framework for spoken language processing, that would encompass not only speech recognition and synthesis but rather speech understanding and speech generation from concept, being able to include all the components of a speech-to-speech translation system. This goal is far too ambitious, but we believe that significant progress towards it can be made by learning how to represent and integrate new knowledge sources.

== Demos ==

A demonstration of tightly integrated [http://www.l2f.inesc-id.pt/projects/wfst/demo/trans-pc.smi speech-to-text] translation is available.
The translation module is implemented as a single WFST that is used as the language model in the speech recognizer. This architecture produces sentences in the target language directly from source language speech.

A demonstration of [http://www.l2f.inesc-id.pt/projects/wfst/demo-bnews/2006_10_19-19_59_01_en.smi large vocabulary translation] is also available.
The output of the WFST-based speech recognition module was translated using
a WFST-based machine translation module trained in the European Parliament domain.

[[category:Research]]
[[category:Projects]]
[[category:National Projects]]

WFST - Weighted Finite State Transducers Applied to Spoken Language Processing

2007-02-12T17:47:44Z

Dcaseiro: /* Demos */

== Team ==

Project Leader: [[Diamantino Caseiro]]

* [[Isabel Trancoso]]
* [[Rui Amaral]]
* [[Luís Caldas de Oliveira|Luís Oliveira]]
* [[Maria do Céu Viana|M. Céu Viana]]
* Pedro Carvalho

== Summary ==

Finite-State models (FSM) and, in particular, Weighted Finite-State transducers (WFST) have proven quite successful in many fields of written and spoken language processing. This includes in particular machine translation, large vocabulary continuous speech recognition and speech synthesis.
An interesting feature of FSMs is that they can be automatically built or "learned" from training data using corpusbased techniques. Compared to more traditional knowledgebased approaches, these techniques are attractive for their potential of much lower development costs. Another interesting property of FSMs is their feasibility for implementing or approximating knowledge-based techniques. Different knowledge sources can hence be represented via FSMs, thus allowing the integration of apriori knowledge with inductive techniques in a natural and formally elegant way. This makes the FSM framework an adequate one for language processing.

The main goal of this project is the application of this framework to speech recognition and synthesis, which will constitute the themes of the two major tasks.

The group has already acquired some experience in modeling the various components of recognition systems using WFSTs, having developed specialized algorithms for transducer composition for on-the-fly lexicon and language model integration. Preliminary experiments with the explicit integration of phonological rules have also produced encouraging results, but much remains to be done in order to be able to achieve higher recognition rates, specially in what concerns spontaneous speech, not only in terms of pronunciation modeling, but also of language modeling.

The group's experience with the application of WFSTs to different modules of text-to-speech (TTS) synthesis is much more recent. We are currently exploring the potential of WFST application, namely in terms of grapheme-to-phone conversion and variable duration segment selection in synthesis by concatenation. We plan to continue this preliminary work and extend it to other modules.

A third task will deal with more exploratory themes on which the group currently has no experience, having as a goal the integration of different knowledge sources. Potential themes are the use of transducers for topic indexation, or for integrating translation models to help speech recognition of documents which have originally been written in one language and are being dictated by a human translator in another language.

The long term goal of this project is a unified framework for spoken language processing, that would encompass not only speech recognition and synthesis but rather speech understanding and speech generation from concept, being able to include all the components of a speech-to-speech translation system. This goal is far too ambitious, but we believe that significant progress towards it can be made by learning how to represent and integrate new knowledge sources.

== Demos ==

A demonstration of tightly integrated [http://www.l2f.inesc-id.pt/projects/wfst/demo/trans-pc.smi speech-to-text] translation is available.
The translation module is implemented as a single WFST that is used as the language model in the speech recognizer. This architecture produces sentences in the target language directly from source language speech.

A demonstration of [http://www.l2f.inesc-id.pt/projects/wfst/demo/bnews/2006_10_19-19_59_01_en.smi large vocabulary translation] is also available.
The output of the WFST-based speech recognition module was translated using
a WFST-based machine translation module trained in the European Parliament domain.

[[category:Research]]
[[category:Projects]]
[[category:National Projects]]

WFST - Weighted Finite State Transducers Applied to Spoken Language Processing

2007-02-12T17:40:23Z

Dcaseiro: /* Demos */

== Team ==

Project Leader: [[Diamantino Caseiro]]

* [[Isabel Trancoso]]
* [[Rui Amaral]]
* [[Luís Caldas de Oliveira|Luís Oliveira]]
* [[Maria do Céu Viana|M. Céu Viana]]
* Pedro Carvalho

== Summary ==

Finite-State models (FSM) and, in particular, Weighted Finite-State transducers (WFST) have proven quite successful in many fields of written and spoken language processing. This includes in particular machine translation, large vocabulary continuous speech recognition and speech synthesis.
An interesting feature of FSMs is that they can be automatically built or "learned" from training data using corpusbased techniques. Compared to more traditional knowledgebased approaches, these techniques are attractive for their potential of much lower development costs. Another interesting property of FSMs is their feasibility for implementing or approximating knowledge-based techniques. Different knowledge sources can hence be represented via FSMs, thus allowing the integration of apriori knowledge with inductive techniques in a natural and formally elegant way. This makes the FSM framework an adequate one for language processing.

The main goal of this project is the application of this framework to speech recognition and synthesis, which will constitute the themes of the two major tasks.

The group has already acquired some experience in modeling the various components of recognition systems using WFSTs, having developed specialized algorithms for transducer composition for on-the-fly lexicon and language model integration. Preliminary experiments with the explicit integration of phonological rules have also produced encouraging results, but much remains to be done in order to be able to achieve higher recognition rates, specially in what concerns spontaneous speech, not only in terms of pronunciation modeling, but also of language modeling.

The group's experience with the application of WFSTs to different modules of text-to-speech (TTS) synthesis is much more recent. We are currently exploring the potential of WFST application, namely in terms of grapheme-to-phone conversion and variable duration segment selection in synthesis by concatenation. We plan to continue this preliminary work and extend it to other modules.

A third task will deal with more exploratory themes on which the group currently has no experience, having as a goal the integration of different knowledge sources. Potential themes are the use of transducers for topic indexation, or for integrating translation models to help speech recognition of documents which have originally been written in one language and are being dictated by a human translator in another language.

The long term goal of this project is a unified framework for spoken language processing, that would encompass not only speech recognition and synthesis but rather speech understanding and speech generation from concept, being able to include all the components of a speech-to-speech translation system. This goal is far too ambitious, but we believe that significant progress towards it can be made by learning how to represent and integrate new knowledge sources.

== Demos ==

A demonstration of tightly integrated [http://www.l2f.inesc-id.pt/projects/wfst/demo/trans-pc.smi speech-to-text] translation is available.
The translation module is implemented as a single WFST that is used as the language model in the speech recognizer. This architecture produces sentences in the target language directly from source language speech.

A demonstration of [http://www.l2f.inesc-id.pt/projects/wfst/demo/trans-pc.smi large vocabulary translation] is also available.
The output of the WFST-based speech recognition module was translated using
a WFST-based machine translation module trained in the European Parliament domain.

[[category:Research]]
[[category:Projects]]
[[category:National Projects]]

Location

2006-07-05T15:45:07Z

Dcaseiro:

<php>
print "<script src=\"http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAiipWyrjj27U9JMBnqzAhKBRy0aIh4Aw9l5upTCWHrfmyWW9BTxTXBAH2m-5wlu8a5fw-q9XGF9QETg\" type=\"text/javascript\"></script><div id=\"map\" style=\"background: red; width: 100%; height: 500px; border-style: solid; border-width: 1px; border-color: black;\"></div><script type=\"text/javascript\">function createMarker(point,html) { var marker = new GMarker(point); GEvent.addListener(marker, \"click\", function() { marker.openInfoWindowHtml(html); }); return marker; } function load() { if (GBrowserIsCompatible()) { var map = new GMap2(document.getElementById(\"map\")); map.addControl(new GLargeMapControl()); map.addControl(new GMapTypeControl()); map.setCenter(new GLatLng(38.736436,-9.140646), 13); var ptl2f = new GLatLng(38.736436,-9.140646); var ptist = new GLatLng(38.736703,-9.139026);var mkl2f = createMarker(ptl2f,'<a href=\"http://www.l2f.inesc-id.pt/\">Spoken Language Systems Lab</a><br>Rua Alves Redol 9<br>1000-029 LISBOA<br>Portugal<br><br>Email: <a href=\"mailto:info@l2f.inesc-id.pt\">info@l2f.inesc-id.pt</a><br>Phone: +351 213 100 268<br>Fax: +351 213 145 843'); var mkist = createMarker(ptist,'<a href=\"http://www.ist.utl.pt/\">Instituto Superior Técnico</a><br>Avenida Rovisco Pais<br>1049-001 LISBOA<br>Portugal'); map.addOverlay(mkl2f); map.addOverlay(mkist); } } load();</script> ";
</php>

Speech-to-speech Translation

2006-06-29T16:31:00Z

Dcaseiro:

Speech-to-speech machine translation is one of the most strategically relevant areas for L2F.
The state of the art in speech translation is crucially dependent on the state of the art of several core technologies: speech recognition, machine translation and, to a lesser extent, text-to-speech synthesis (namely in what concerns voice morphing, in order to reproduce the source speakers’ characteristics in the target speaker’s voice). The main limitations of current machine translation systems are the lack of semantic interpretation and world knowledge as well as insufficient coverage of the large proportion of idiosyncratic linguistic phenomena in lexicon and syntax. The most promising approaches combine improved statistical methods with the improved knowledge-driven methods in a variety of clever ways.

L2F has been investing in statistically based speech-to-speech machine translation approaches based on weighted finite state transducers [Picó 2005] [Caseiro 2006], aiming at a tight integration between recognition and translation. WFSTs are especially well suited for combining different type of approaches, whether statistical or knowledge-based. The combination may be advantageous for achieving two different goals (i) include morpho-syntactic linguistic knowledge into the statistical machine translation paradigm and (ii) tackle the data sparseness problem for speech translation.

This research is carried out within the scope of a national project on “Weighted Finite State Transducers Applied to Spoken Language Processing”. Two PhD theses have recently started in this area.

File:Logo-ptfc060726.jpg

2006-06-06T07:54:56Z

Dcaseiro: Rosetta Stone

Rosetta Stone

Diamantino Caseiro

2006-04-01T15:06:17Z

Dcaseiro: /* Research Interests */

{{infobox|name=Diamantino Caseiro|username=dcaseiro|contact=dcaseiro
|phone=+351-213-100-300 ext. 2528|fax=+351-213-145-843
}}
<inesc-id what='person' id='132'></inesc-id>


== Research Interests ==

* Finite-State Methods in Speech and Language Processing
* Automatic Speech Recognition
* Machine Aided Translation
* Automatic Language Identification

== Resume ==
PDF Version [http://www.l2f.inesc-id.pt/~dcaseiro/caseiro-web.pdf]

Demos

2006-03-07T18:17:06Z

Dcaseiro:

If your computer is capable of playing audio files you can listen to some demos of our work (you can listen to [[Resources|demos of corpora collected by us]]):

{| width='95%'
! valign='top' |{{online_demo|logo=demo-ssnt-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/SSNT/SSNT-en.php SSNT]
| description=Transcription of Broadcast News
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=AmbrosioNovo1.jpg
| link=[http://www.l2f.inesc-id.pt/press/2006_01_31-Casa_do_Futuro-Telejornal.rm Ambrósio]
| description=Virtual Butler at Home of the Future
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-svit-logo.png
| link=[http://www.l2f.inesc-id.pt/~lco/svit/svitd_en.cgi SVIT]
| description=Speech synthesizer for telephone numbers
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-tumba-logo.png
| link=[[Tumba]]
| description=Adding a voice to tumba, the portuguese web search engine
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voices-logo.png
| link=[http://www.l2f.inesc-id.pt/~spaulo/voices Voices]
| description=Corpus-based speech synthesis for any voice
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-emovoice-logo.png
| link=[http://www.l2f.inesc-id.pt/~jpcabral/demo/ EmoVoice]
| description=Transform speech emotions
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voicedub-logo.png
| link=[http://www.l2f.inesc-id.pt/~cmdm/VoiceDub/ VoiceDub]
| description=Voice dub
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-ipsom-logo.png
| link=[[IPSOM#Demos|IPSOM]]
| description=Automatic alignment of spoken digital books
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-lectra-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/lectra/demo/aula1212_last.smi LECTRA]
| description=Transcription of Classroom Lectures
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-wfst-translation.png
| link=[http://www.l2f.inesc-id.pt/wiki/index.php/WFST_-_Weighted_Finite_State_Transducers_Applied_to_Spoken_Language_Processing#Demos WSFT]
| description=Speech to text translation
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|}

Demos

2006-03-07T18:15:37Z

Dcaseiro:

If your computer is capable of playing audio files you can listen to some demos of our work (you can listen to [[Resources|demos of corpora collected by us]]):

{| width='95%'
! valign='top' |{{online_demo|logo=demo-ssnt-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/SSNT/SSNT-en.php SSNT]
| description=Transcription of Broadcast News
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=AmbrosioNovo1.jpg
| link=[http://www.l2f.inesc-id.pt/press/2006_01_31-Casa_do_Futuro-Telejornal.rm Ambrósio]
| description=Virtual Butler at Home of the Future
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-svit-logo.png
| link=[http://www.l2f.inesc-id.pt/~lco/svit/svitd_en.cgi SVIT]
| description=Speech synthesizer for telephone numbers
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-tumba-logo.png
| link=[[Tumba]]
| description=Adding a voice to tumba, the portuguese web search engine
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voices-logo.png
| link=[http://www.l2f.inesc-id.pt/~spaulo/voices Voices]
| description=Corpus-based speech synthesis for any voice
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-emovoice-logo.png
| link=[http://www.l2f.inesc-id.pt/~jpcabral/demo/ EmoVoice]
| description=Transform speech emotions
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voicedub-logo.png
| link=[http://www.l2f.inesc-id.pt/~cmdm/VoiceDub/ VoiceDub]
| description=Voice dub
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-ipsom-logo.png
| link=[[IPSOM#Demos|IPSOM]]
| description=Automatic alignment of spoken digital books
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-lectra-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/lectra/demo/aula1212_last.smi LECTRA]
| description=Transcription of Classroom Lectures
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-wfst-translation.png
| link=[WFST_-_Weighted_Finite_State_Transducers_Applied_to_Spoken_Language_Processing#Demos WSFT]
| description=Speech to text translation
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|}

Demos

2006-03-07T18:12:30Z

Dcaseiro:

File:Demo-wfst-translation.png

2006-03-07T18:11:04Z

Dcaseiro:

Demos

2006-03-07T18:04:26Z

Dcaseiro:

If your computer is capable of playing audio files you can listen to some demos of our work (you can listen to [[Resources|demos of corpora collected by us]]):

{| width='95%'
! valign='top' |{{online_demo|logo=demo-ssnt-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/SSNT/SSNT-en.php SSNT]
| description=Transcription of Broadcast News
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=AmbrosioNovo1.jpg
| link=[http://www.l2f.inesc-id.pt/press/2006_01_31-Casa_do_Futuro-Telejornal.rm Ambrósio]
| description=Virtual Butler at Home of the Future
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-svit-logo.png
| link=[http://www.l2f.inesc-id.pt/~lco/svit/svitd_en.cgi SVIT]
| description=Speech synthesizer for telephone numbers
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-tumba-logo.png
| link=[[Tumba]]
| description=Adding a voice to tumba, the portuguese web search engine
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voices-logo.png
| link=[http://www.l2f.inesc-id.pt/~spaulo/voices Voices]
| description=Corpus-based speech synthesis for any voice
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-emovoice-logo.png
| link=[http://www.l2f.inesc-id.pt/~jpcabral/demo/ EmoVoice]
| description=Transform speech emotions
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voicedub-logo.png
| link=[http://www.l2f.inesc-id.pt/~cmdm/VoiceDub/ VoiceDub]
| description=Voice dub
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-ipsom-logo.png
| link=[[IPSOM#Demos|IPSOM]]
| description=Automatic alignment of spoken digital books
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-lectra-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/lectra/demo/aula1212_last.smi LECTRA]
| description=Transcription of Classroom Lectures
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-wfst-logo.png
| link=[http://www.l2f.inesc-id.pt/wiki/index.php/WFST_-_Weighted_Finite_State_Transducers_Applied_to_Spoken_Language_Processing#Demos WSFT]
| description=Speech to text translation
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|}

Demos

2006-03-07T18:02:56Z

Dcaseiro:

If your computer is capable of playing audio files you can listen to some demos of our work (you can listen to [[Resources|demos of corpora collected by us]]):

{| width='95%'
! valign='top' |{{online_demo|logo=demo-ssnt-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/SSNT/SSNT-en.php SSNT]
| description=Transcription of Broadcast News
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=AmbrosioNovo1.jpg
| link=[http://www.l2f.inesc-id.pt/press/2006_01_31-Casa_do_Futuro-Telejornal.rm Ambrósio]
| description=Virtual Butler at Home of the Future
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-svit-logo.png
| link=[http://www.l2f.inesc-id.pt/~lco/svit/svitd_en.cgi SVIT]
| description=Speech synthesizer for telephone numbers
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-tumba-logo.png
| link=[[Tumba]]
| description=Adding a voice to tumba, the portuguese web search engine
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voices-logo.png
| link=[http://www.l2f.inesc-id.pt/~spaulo/voices Voices]
| description=Corpus-based speech synthesis for any voice
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-emovoice-logo.png
| link=[http://www.l2f.inesc-id.pt/~jpcabral/demo/ EmoVoice]
| description=Transform speech emotions
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-voicedub-logo.png
| link=[http://www.l2f.inesc-id.pt/~cmdm/VoiceDub/ VoiceDub]
| description=Voice dub
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-ipsom-logo.png
| link=[[IPSOM#Demos|IPSOM]]
| description=Automatic alignment of spoken digital books
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|-
! valign='top' |{{online_demo|logo=demo-lectra-logo.png
| link=[http://www.l2f.inesc-id.pt/projects/lectra/demo/aula1212_last.smi LECTRA]
| description=Transcription of Classroom Lectures
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
! valign='top' |{{online_demo|logo=demo-wfst-logo.png
| link=[http://www.l2f.inesc-id.pt/wiki/index.php/wfst#Demos WSFT]
| description=Speech to text translation
| contact=[mailto:info@l2f.inesc-id.pt L²F]
}}
|}

WFST - Weighted Finite State Transducers Applied to Spoken Language Processing

2006-03-07T17:58:03Z

Dcaseiro: /* Demos */

== Team ==

Project Leader: [[Diamantino Caseiro]]

* [[Isabel Trancoso]]
* [[Rui Amaral]]
* [[Luís Caldas de Oliveira|Luís Oliveira]]
* [[Maria do Céu Viana|M. Céu Viana]]
* Pedro Carvalho

== Summary ==

Finite-State models (FSM) and, in particular, Weighted Finite-State transducers (WFST) have proven quite successful in many fields of written and spoken language processing. This includes in particular machine translation, large vocabulary continuous speech recognition and speech synthesis.
An interesting feature of FSMs is that they can be automatically built or "learned" from training data using corpusbased techniques. Compared to more traditional knowledgebased approaches, these techniques are attractive for their potential of much lower development costs. Another interesting property of FSMs is their feasibility for implementing or approximating knowledge-based techniques. Different knowledge sources can hence be represented via FSMs, thus allowing the integration of apriori knowledge with inductive techniques in a natural and formally elegant way. This makes the FSM framework an adequate one for language processing.

The main goal of this project is the application of this framework to speech recognition and synthesis, which will constitute the themes of the two major tasks.

The group has already acquired some experience in modeling the various components of recognition systems using WFSTs, having developed specialized algorithms for transducer composition for on-the-fly lexicon and language model integration. Preliminary experiments with the explicit integration of phonological rules have also produced encouraging results, but much remains to be done in order to be able to achieve higher recognition rates, specially in what concerns spontaneous speech, not only in terms of pronunciation modeling, but also of language modeling.

The group's experience with the application of WFSTs to different modules of text-to-speech (TTS) synthesis is much more recent. We are currently exploring the potential of WFST application, namely in terms of grapheme-to-phone conversion and variable duration segment selection in synthesis by concatenation. We plan to continue this preliminary work and extend it to other modules.

A third task will deal with more exploratory themes on which the group currently has no experience, having as a goal the integration of different knowledge sources. Potential themes are the use of transducers for topic indexation, or for integrating translation models to help speech recognition of documents which have originally been written in one language and are being dictated by a human translator in another language.

The long term goal of this project is a unified framework for spoken language processing, that would encompass not only speech recognition and synthesis but rather speech understanding and speech generation from concept, being able to include all the components of a speech-to-speech translation system. This goal is far too ambitious, but we believe that significant progress towards it can be made by learning how to represent and integrate new knowledge sources.

== Demos ==

A demonstration of [http://www.l2f.inesc-id.pt/projects/wfst/demo/trans-pc.smi speech-to-text] translation is available.
The translation module is implemented as a single WFST that is used as the language model in the speech recognizer. This architecture produces sentences in the target language directly from source language speech.

WFST - Weighted Finite State Transducers Applied to Spoken Language Processing

2006-03-07T17:55:51Z

Dcaseiro:

== Team ==

Project Leader: [[Diamantino Caseiro]]

* [[Isabel Trancoso]]
* [[Rui Amaral]]
* [[Luís Caldas de Oliveira|Luís Oliveira]]
* [[Maria do Céu Viana|M. Céu Viana]]
* Pedro Carvalho

== Summary ==

Finite-State models (FSM) and, in particular, Weighted Finite-State transducers (WFST) have proven quite successful in many fields of written and spoken language processing. This includes in particular machine translation, large vocabulary continuous speech recognition and speech synthesis.
An interesting feature of FSMs is that they can be automatically built or "learned" from training data using corpusbased techniques. Compared to more traditional knowledgebased approaches, these techniques are attractive for their potential of much lower development costs. Another interesting property of FSMs is their feasibility for implementing or approximating knowledge-based techniques. Different knowledge sources can hence be represented via FSMs, thus allowing the integration of apriori knowledge with inductive techniques in a natural and formally elegant way. This makes the FSM framework an adequate one for language processing.

The main goal of this project is the application of this framework to speech recognition and synthesis, which will constitute the themes of the two major tasks.

The group has already acquired some experience in modeling the various components of recognition systems using WFSTs, having developed specialized algorithms for transducer composition for on-the-fly lexicon and language model integration. Preliminary experiments with the explicit integration of phonological rules have also produced encouraging results, but much remains to be done in order to be able to achieve higher recognition rates, specially in what concerns spontaneous speech, not only in terms of pronunciation modeling, but also of language modeling.

The group's experience with the application of WFSTs to different modules of text-to-speech (TTS) synthesis is much more recent. We are currently exploring the potential of WFST application, namely in terms of grapheme-to-phone conversion and variable duration segment selection in synthesis by concatenation. We plan to continue this preliminary work and extend it to other modules.

A third task will deal with more exploratory themes on which the group currently has no experience, having as a goal the integration of different knowledge sources. Potential themes are the use of transducers for topic indexation, or for integrating translation models to help speech recognition of documents which have originally been written in one language and are being dictated by a human translator in another language.

The long term goal of this project is a unified framework for spoken language processing, that would encompass not only speech recognition and synthesis but rather speech understanding and speech generation from concept, being able to include all the components of a speech-to-speech translation system. This goal is far too ambitious, but we believe that significant progress towards it can be made by learning how to represent and integrate new knowledge sources.

== Demos ==

A demonstration of speech-to-text translation is available.
The translation module is implemented as a single WFST that is used as the language model in the speech recognizer. This architecture produces sentences in the target language directly from source language speech.