Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-26T08:25:31.018Z Has data issue: false hasContentIssue false
  • English
  • Français

Explorations in lexical sample and all-words lexical substitution

Published online by Cambridge University Press:  09 October 2012

RAVI SINHA  and
RADA MIHALCEA
RAVI SINHA
Affiliation:
Department of Computer Science and EngineeringUniversity of North Texas Denton, TX, USA e-mails: ravisinha@my.unt.edu, rada@cs.unt.edu
RADA MIHALCEA
Affiliation:
Department of Computer Science and EngineeringUniversity of North Texas Denton, TX, USA e-mails: ravisinha@my.unt.edu, rada@cs.unt.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we experiment with several techniques to solve the problem of lexical substitution, both in a lexical sample as well as an all-words setting, and compare the benefits of combining multiple lexical resources using both unsupervised and supervised approaches. Overall in the lexical sample setting, the results obtained through the combination of several resources exceed the current state-of-the-art when selecting the best substitute for a given target word, and place second when selecting the top ten substitutes, thus demonstrating the usefulness of the approach. Further, we put forth a novel exploration in all-words lexical substitution and set ground for further explorations of this more generalized setting.

Type
Articles
Information
Natural Language Engineering , Volume 20 , Issue 1 , January 2014 , pp. 99 - 129
Copyright
Copyright © Cambridge University Press 2012 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agirre, E., and Edmonds, P. 2006. Word Sense Disambiguation: Algorithms and Applications. New York: Springer.CrossRefGoogle Scholar
Akkaya, C., Wiebe, J., and Mihalcea, R. 2009. Subjectivity word sense disambiguation. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 190–9. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Androutsopoulos, I., and Malakasiotis, P. 2010. A survey of paraphrasing and textual entailment methods. Journal of Artificial Intelligence Research 38 (1): 135–187.CrossRefGoogle Scholar
Bangalore, S., and Rambow, O. 2000. Corpus-based lexical choice in natural language generation. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics (ACL '00), Hong Kong, pp. 464–71. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Barzilay, R., and Lee, L. 2003. Learning to paraphrase: an unsupervised approach using multiple-sequence alignment. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology – Volume 1, Edmonton, Canada. Stroudsburg, PA: Association for Computational Linguistics.Google Scholar
Beale, S., Lavoie, B., McShane, M., Nirenburg, S., and Korelsky, T. 2004. Question answering using ontological semantics. In Proceedings of the 2nd Workshop on Text Meaning and Interpretation (TextMean '04), Barcelona, Spain, pp. 41–8. Morristown, NJ, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Bergsma, S., Lin, D., and Goebel, R. 2009. Web-scale n-gram models for lexical disambiguation. In Proceedings of the International Joint Conference on Artificial Intelligence, Pasadena, CA. San Francisco, CA: Morgan Kaufmann Publishers Inc.Google Scholar
Biemann, C. 2006. Chinese whispers: an efficient graph clustering algorithm and its application to natural language processing problems. In Proceedings of the First Workshop on Graph Based Methods for Natural Language Processing (TextGraphs-1), New York, NY, USA, pp. 73–80. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Biemann, C. 2010. Co-occurrence cluster features for lexical substitutions in context. In Proceedings of the 2010 Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-5), Uppsala, Sweden, pp. 55–9. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Brants, T., and Franz, A. 2006. Web 1T 5-gram version 1. Linguistic Data Consortium, Philadelphia, PA. http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13.Google Scholar
Carpuat, M., and Wu, D. 2007. Improving statistical machine translation using word sense disambiguation. In Proceedings of the 2007 Conference on Empirical Methods in Natural Language Processing, Prague, Czech Republic, pp. 61–72. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Chan, Y. S., and Ng, H. T. 2007. Word sense disambiguation improves statistical machine translation. In 45th Annual Meeting of the Association for Computational Linguistics (ACL-07), Prague, Czech Republic, pp. 33–40. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Chang, C., and Clark, S. 2010. Practical linguistic steganography using contextual synonym substitution and vertex colour coding. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Cambridge, MA, USA, October, pp. 1194–203. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Dagan, I., Glickman, O., Gliozzo, A., Marmorshtein, E., and Strapparava, C. 2006. Direct word sense matching for lexical substitution. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, pp. 449–56. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Davidov, D., and Rappoport, A. 2009. Enhancement of lexical concepts using cross-lingual web mining. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, August, pp. 852–61. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Deléger, L., and Zweigenbaum, P. 2009. Extracting lay paraphrases of specialized expressions from monolingual comparable medical corpora. In Proceedings of the 2nd Workshop on Building and Using Comparable Corpora: From Parallel to Non-parallel Corpora, Singapore, August, pp. 2–10. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Dolan, W., Quirk, C., and Brockett, C. 2004. Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources.Proceedings of the 20th International Conference on Computational Linguistics, Geneva, Switzerland.Google Scholar
Edmonds, P. 1997. Choosing the word most typical in context using a lexical co-occurrence network. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, Madrid, Spain, pp. 507–9. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Elhadad, N., and Sutaria, K. 2007. Mining a lexicon of technical terms and lay equivalents. In Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing (BioNLP '07), Prague, Czech Republic, pp. 49–56. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Etzioni, O., Reiter, K., Soderl, S., and Sammer, M. 2007. Lexical translation with application to image search on the web. In Proceedings of the Machine Translation Summit, Copenhagen, Denmark. Geneva: International Association for Machine Translation.Google Scholar
Gabrilovich, E., and Markovitch, S. 2007. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In Proceedings of The Twentieth International Joint Conference for Artificial Intelligence, Hyderabad, Andhra Pradesh, India, pp. 1606–11. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.Google Scholar
Girju, R., Badulescu, A., and Moldovan, D. I. 2003. Learning semantic constraints for the automatic discovery of part-whole relations.Proceedings of the North American Chapter of the Association for Computational Linguistics, Edmonton, Canada. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Giuliano, C., Gliozzo, A., and Strapparava, C. 2007. Fbk-irst: lexical substitution task exploiting domain and syntagmatic coherence. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, June, pp. 145–8. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Hassan, S., Csomai, A., Banea, C., Sinha, R., and Mihalcea, R. 2007. Unt: subfinder: combining knowledge sources for automatic lexical substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, June, pp. 410–13. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Hassan, S., and Mihalcea, R. 2011. Semantic relatedness using salient semantic analysis.Proceedings of the Conference of the American Association for Artificial Intelligence, San Francisco, CA, USA. Palo Alto, CA, USA: AAAI Press.Google Scholar
Inkpen, D. 2007 (February). A statistical model for near-synonym choice. ACM Transactions on Speech and Language Processing 4: 2:1–2:17.CrossRefGoogle Scholar
Islam, A., and Inkpen, D. 2009. Semantic similarity of short texts. In Recent Advances in Natural Language Processing V, Vol. 309 of Current Issues in Linguistic Theory, pp. 227–36. Amsterdam Netherlands: John Benjamins.CrossRefGoogle Scholar
Jabbari, S., Hepple, M., and Guthrie, L. 2010. Evaluation metrics for the lexical substitution task. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, USA, June, pp. 289–92. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Kim, S., Seo, H.-C., and Rim, R.-C. 2004. Information retrieval using word senses: root sense tagging approach. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK, pp. 258–65. New York, NY, USA: ACM.Google Scholar
Krovetz, R. 1997. Homonymy and polysemy in information retrieval. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics (ACL-97), Madrid, Spain, pp. 72–9. San Francisco, CA, USA: Morgan Kaufmann Publishers.CrossRefGoogle Scholar
Kubat, M., and Matwin, S. 1997. Addressing the curse of imbalanced training sets: one-sided selection. In Proceedings of the Fourteenth International Conference on Machine Learning, Nashville, TN, USA, pp. 179–86. San Francisco, CA, USA: Morgan Kaufmann Publishers.Google Scholar
Landauer, T. K., and Dumais, S. T. 1997. Solution to Plato's problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychology Review 104 (2): 211–240. Cognitive view on LSA.CrossRefGoogle Scholar
Lin, D. 1998. An information-theoretic definition of similarity. In Proceedings of 15th International Conferences on Machine Learning, Madison, WI, USA, pp. 296–304. San Francisco, CA, USA: Morgan Kaufmann Publishers.Google Scholar
Martinez, D., Kim, S. N., and Baldwin, T. 2007. Melb-mkb: lexical substitution system based on relatives in context. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval '07), Prague, Czech Republic, pp. 237–40. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
McCarthy, D., and Navigli, R. 2007. Semeval-2007 task 10: English lexical substitution task. In Proceedings of the 4th workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, pp. 48–53. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
McCarthy, D., and Navigli, R. 2009. The English lexical substitution task. Language Resources and Evaluation 43: 139–59.CrossRefGoogle Scholar
Mihalcea, R., Corley, C., and Strapparava, C. 2006. Corpus-based and knowledge-based approaches to text semantic similarity.Proceedings of the American Association for Artificial Intelligence, Boston, MA, USA. Palo Alto, CA, USA: AAAI Press.Google Scholar
Mihalcea, R., and Edmonds, P. (eds.), 2004. Proceedings of SENSEVAL-3, Association for Computational Linguistics Workshop, Barcelona, Spain.Google Scholar
Mihalcea, R., Sinha, R., and McCarthy, D. 2010. Semeval-2010 task 2: cross-lingual lexical substitution. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden, July, pp. 9–14. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Miller, G. A. 1995. WordNet: a lexical database for English. Communications of the ACM 38: 39–41.CrossRefGoogle Scholar
Mitchell, J., and Lapata, M. 2008. Vector-based models of semantic composition.Proceedings of Association of Computational Linguistics, Columbus, OH, USA. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Monz, C. 2005. Iterative translation disambiguation for cross-language information retrieval. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, pp. 520–7. New York, NY, USA: ACM Press.CrossRefGoogle Scholar
Pradhan, S., Loper, E., Dligach, D., and Palmer, M. 2007. Semeval-2007 task-17: English lexical sample, SRL and all words. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, June, pp. 87–92. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Preiss, J., Coonce, A., and Baker, B. 2009. Hmms, grs, and n-grams as lexical substitution techniques: are they portable to other languages? In Proceedings of the Workshop on Natural Language Processing Methods and Corpora in Translation, Lexicography, and Language Learning (MCTLLL '09), Borovets, Bulgaria, pp. 21–7. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Quirk, C., Brockett, C., and Dolan, W. 2004. Monolingual machine translation for paraphrase generation.Proceedings of the 2004 Conference on Empirical Methods in Natural Language Process, Barcelona, Spain. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Shinyama, Y., Sekine, S., and Sudo, K. 2002. Automatic paraphrase acquisition from news articles.Proceedings of the Second International Conference on Human Language Technology Research, San Diego, CA, USA. San Francisco, CA, USA: Morgan Kaufmann Publishers.Google Scholar
Stokoe, C. 2005. Differentiating homonymy and polysemy in information retrieval. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT '05), Vancouver, British Columbia, Canada, pp. 403–10. Stroudsburg, PA, USA: Association for Computational Linguistics.Google Scholar
Wang, T., and Hirst, G. 2010. Near-synonym lexical choice in latent semantic space. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING '10), Beijing, China, pp. 1182–90. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Yatskar, M., Pang, B., C. Danescu-Niculescu-Mizil, and Lee, L. 2010. For the sake of simplicity: unsupervised extraction of lexical simplifications from Wikipedia. In Proceedings of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, USA, pp. 365–68. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Yu, L., Shih, H., Lai, Y., Yeh, J., and Wu, C. 2010. Discriminative training for near-synonym substitution. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING '10), Beijing, China, pp. 1254–62. Morristown, NJ, USA: Association for Computational Linguistics.Google Scholar
Yuret, D. 2007. Ku: word sense disambiguation by substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, June, pp. 207–14. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar
Zhao, S., Zhao, L., Zhang, Y., Liu, T., and Li, S. 2007. Hit: web-based scoring method for English lexical substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic, June, pp. 173–6. Stroudsburg, PA, USA: Association for Computational Linguistics.CrossRefGoogle Scholar