ELRA's SLR Validation Centre at SPEX

Validation of spoken language resources

Speech

After an open call,  the Speech Processing EXpertise centre (SPEX) in Nijmegen, the Netherlands, was selected in 1999 as ELRA's first VC for spoken language resources (SLR). SLR validation, as carried out by SPEX, concerns the quality evaluation of a database against a checklist of relevant criteria. These criteria are typically the specifications of the databases, together with some tolerance margins for deviations. General information on the topic of SLR validation can be found the article The Art of Validation in the ELRA Newsletter, Vol.5.4.

In order to avoid the duplication of efforts, ELRA uses the outcome of the SpeechDat project and its successors as the baseline for its own validation efforts. One of the objectives of the SpeechDat projects is to validate the speech databases that are collected. Specific information about the validation criteria for the various projects can be found in the Validation Standards section at ELRA's webpage (under Services around LR's).

The databases collected in varous SLR producing projects are distributed via ELRA. For each of these and later validated SLR, ELRA makes available in its catalogue the main documentation file, and the validation report as written by SPEX.

On the basis of the experiences gained in various projects, SPEX has written a validation manual for SLR. This document describes validation principles that should be taken into account by producers of new SLR, specifically those who aim to distribute SLR through ELRA.

 

In the year 2000, SPEX carried out the following tasks:

 

In the year 2001, SPEX carried out the following tasks:

  • Updating ELRA's web pages regarding SLR validation
  • Implementation for a SLR bug report procedure through ELRA's URL
  • Proposal for carrying out a quality description of selected SLR

 

In the year 2002, SPEX carried out the following tasks:

  • Monitoring the SLR bug report service
  • Implementation of bug fix actions
  • Quality description of selected SLR

 

In the year 2003, SPEX carried out the following tasks:

  • Monitoring the SLR bug report service
  • Bug fix actions; patch creation
  • Quick Quality Checks of selected SLR
  • Description of a Quick Quality Check for pronunciation lexica
  • Update of standards in SLR production and validation
  • Presentation and paper about the activities in ELRA’s VCOM at EUROSPEECH 2003 Geneva

 

In the year 2004, SPEX carried out the following tasks:

  • Monitoring the SLR bug report service
  • Bug fix actions
  • Quick Quality Checks of selected SLR
  • Description of a Quick Quality Check for speech synthesis databases
  • Update of standards in SLR validation

 

In the year 2005, SPEX carried out the following tasks:

  • Monitoring the SLR bug report service & bug fix actions
  • Quick Quality Checks of selected SLR

 

For the year 2006 and 2007, the following tasks are planned:

  • Monitoring the SLR bug report service & bug fix actions
  • Quick Quality Checks of selected SLR
  • Description of a Quick Quality Check for Multimodal Language Resources