1 / 8

Summary and Action Items from RT-03 Meeting on November 15, 2003

The RT-03F meeting on November 15, 2003, focused on the summary of discussions, action items, and upcoming tasks for RT-04. Key topics included the advancement of BN-only diarization proposals, task proposals by team members, and updates on evaluation plans and data collection. The meeting also outlined the schedule for future training and evaluation activities, including the necessity for feedback and decisions on tool frameworks, file formats, and scoring methods. Participants were encouraged to contribute suggestions by specified deadlines.

nakia
Télécharger la présentation

Summary and Action Items from RT-03 Meeting on November 15, 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RT-03F Friday Discussions:Summary, Action Items, Schedule 15 November 2003

  2. Tasks for RT-04 • SU: Like RT-03 plus four types • FWD: Like RT-03 plus three types • EWD: Like RT-03 • IPD: Like RT-03 • Diarization: BN-only, Reynolds leading team to write proposal to make it richer • SASTT: BN-only, like RT-03 • Individual sites may look at speaker characterization • 04RT: Following 03RT principles

  3. Task Actions • Update eval plan secs 3 and 4 to implement RT-04 decisions made today – NIST (by 12/15/03) • Specification of primary vs. secondary measures (including implications for 04RT) • Leave placeholder for BN diarization • BN Diarization Proposal – Reynolds, et. al. • RT-A Task Proposal – Kubala, et. al.

  4. MDE Data Prioritized Wish List for LDC(to be interleaved with STT needs, once we know costs) • New dev data annotation (by 3/04) • CTS: 3hrs of new calls consistent with STT • BN: Six shows consistent with STT 1. Old dev/eval data reannotation (by 3/04) • As/if required by minor updates to V5 spec • 6 BN shows, 72 CTS calls • Eval data annotation (by 9/04) • Selection consistent with dev data • Non-English annotation • Pilot: 10 min CTS and 10 min BN in both Arabic and Mandarin 2. Training data (by 6/04) • Up to another 60 hrs CTS (probably SWB), another 80 hrs BN (HUB4)

  5. Other Data Actions • Possible hand-mark of diarization beg/end times for BN (Sue/Doug) • BN diarization re-release of spkrsegeval ref files (NIST) • V5 small mods (LDC, all) • Suggestions to LDC from sites due 12/1/03 • Initial LDC feedback due to mactech 12/15/03 • macears call (roughly 12/16/03) • V6 due to mactech 1/31/04 • Richer edit structure (Liz/Mari) • Pilot annotation, interannotator consistency, … • BN diarization data spec (Doug, Sue, et. al.) • Anything required beyond current annotations? • RT-A data spec (Francis, et. al.) • White paper, interannotator agreement, etc.

  6. Tool Decisions & Actions • Downselect: At the Feb 2004 PI meeting, Charles will announce which of the two tool frameworks will be used in RT-04 • File formats: Francis and George will distribute short white papers by 12/1 with the pros and cons of the two approaches to data representation; sites will comment by 12/15. Charles will make a decision shortly thereafter. • Support: Both Francis and George agree to implement the changes required for the RT-04 tasks up until the downselection. Both will also fix bugs. • Scoring exclusion: Sue, Jon and Barbara will propose a better way for selecting various scoring exclusion methods (vs. “bulk” UEM approach) • AG conversion: Once Charles announces the official file format, NIST will provide and support a tool to convert AG to the official file format • Significance testing: NIST to look into it

  7. Big Picture Schedule • Workshop in October • Evaluation in September • What guidelines should NIST consider w/r/t relative timing of MDE vs STT • All training data by June • All dev data by March • Tool and format decision by February • Annotation guidelines by end of Jan

  8. Other Items • Hypothetical Spring MDE meeting • Purely a technical R&D meeting (like STT has done) • Update on new pilot initiatives? • Plan: At Lincoln in conjunction with HLT • Macears calls • As needed • First one mid-December (re V6)

More Related