RT-S: Surface rich transcription scoring, methodology, and initial results

TitleRT-S: Surface rich transcription scoring, methodology, and initial results
Publication TypeJournal Articles
Year of Publication2004
AuthorsSnover M, Schwartz R, Dorr BJ, Makhoul J
JournalProc. DARPA Rich Transcription Workshop
Date Published2004///
Abstract

In this paper we present a methodoly for the scoring ofpunctuation annotated texts, as well as a preliminary system
to perform the task. We modify SCLITE’s scoring method-
ology to support scoring of punctuation. Using this method-
ology, we show that the error rate of an initial automatic sys-
tem is comparable to annotator inconsistency. However, the
use of multiple references allows us to differentiate between
human inconsistencies and system errors.