Detecting Structural Irregularity in Electronic Dictionaries Using Language Modeling
| Title | Detecting Structural Irregularity in Electronic Dictionaries Using Language Modeling |
| Publication Type | Conference Papers |
| Year of Publication | 2011 |
| Authors | Rodrigues P, Zajic D, Bloodgood M, Ye P, Doermann D |
| Conference Name | Electronic lexicography in the 21st century: new applications for new users (eLEX2011) |
| Date Published | 2011/// |
| Abstract | Dictionaries are often developed using Extensible Markup Language (XML)-based standards. Very often, these standards allow many high-level repeating elements to represent lexical entries, and utilize descendants of these repeating elements to represent the structure within each lexical entry, in the form of an XML tree. In many cases, dictionaries are published that have errors and inconsistencies that would be too expensive to find manually. This paper discusses a method for dictionary writers to audit structural regularity across entries in a dictionary, quickly, by using statistical language modelling. The approach learns the patterns of XML nodes that could occur within an XML tree, and then calculates the probability of each XML tree in the dictionary against these patterns to look for entries that diverge from the norm. |