Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-10T21:29:46.488Z Has data issue: false hasContentIssue false
  • English
  • Français

Construction and evaluation of event graphs

Published online by Cambridge University Press:  01 May 2014

GORAN GLAVAŠ  and
JAN ŠNAJDER Open the ORCID record for JAN ŠNAJDER [Opens in a new window]
GORAN GLAVAŠ
Affiliation:
University of Zagreb, Faculty of Electrical Engineering and Computing, Text Analysis and Knowledge Engineering Lab, Unska 3, 10000 Zagreb, Croatia email: goran.glavas@fer.hr, jan.snajder@fer.hr
JAN ŠNAJDER
Affiliation:
University of Zagreb, Faculty of Electrical Engineering and Computing, Text Analysis and Knowledge Engineering Lab, Unska 3, 10000 Zagreb, Croatia email: goran.glavas@fer.hr, jan.snajder@fer.hr
Get access Rights & Permissions [Opens in a new window]

Abstract

Events play an important role in natural language processing and information retrieval due to numerous event-oriented texts and information needs. Many natural language processing and information retrieval applications could benefit from a structured event-oriented document representation. In this paper, we propose event graphs as a novel way of structuring event-based information from text. Nodes in event graphs represent the individual mentions of events, whereas edges represent the temporal and coreference relations between mentions. Contrary to previous natural language processing research, which has mainly focused on individual event extraction tasks, we describe a complete end-to-end system for event graph extraction from text. Our system is a three-stage pipeline that performs anchor extraction, argument extraction, and relation extraction (temporal relation extraction and event coreference resolution), each at a performance level comparable with the state of the art. We present EvExtra, a large newspaper corpus annotated with event mentions and event graphs, on which we train and evaluate our models. To measure the overall quality of the constructed event graphs, we propose two metrics based on the tensor product between automatically and manually constructed graphs. Finally, we evaluate the overall quality of event graphs with the proposed evaluation metrics and perform a headroom analysis of the system.

Type
Articles
Information
Natural Language Engineering , Volume 21 , Issue 4 , August 2015 , pp. 607 - 652
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

ACE. 2005. Evaluation of the Detection and Recognition of ACE: Entities, Values, Temporal Expressions, Relations, and Events. Gaithersburg, MD: NIST.Google Scholar
Agirre, E., and Soroa, A. 2009. Personalizing PageRank for word sense disambiguation. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL ’09). Athens, Greece. Stroudsburg, PA: ACL, pp. 33–41.Google Scholar
Ahn, D. 2006. The stages of event extraction. In Proceedings of COLING/ACL 2006 Workshop on Annotating and Reasoning about Time and Events, Sydney, Australia. Stroudsburg, PA: ACL, pp. 1–8.Google Scholar
Ahn, D., Schockaert, S., De Cock, M., and Kerre, E. 2006. Supporting temporal question answering: strategies for offline data collection. In Proceedings of the 5th International Workshop on Inference in Computational Semantics. Buxton, UK: ACL, pp. 127–32.Google Scholar
Allan, J. 2002. Topic Detection and Tracking: Event-Based Information Organization, vol. 12. Dordrecht, Netherlands: Kluwer.CrossRefGoogle Scholar
Allen, J. R., 1983. Maintaining knowledge about temporal intervals. Communications of the ACM 26 (11): 832–43.CrossRefGoogle Scholar
Aone, C., and Ramos-Santacruz, M. 2000. REES: a large-scale relation and event extraction system. In Proceedings of the Sixth Conference on Applied Natural Language Processing. Seattle, WA. Stroudsburg, PA: ACL, pp. 76–83.CrossRefGoogle Scholar
Bagga, A., and Baldwin, B. 1999. Cross-document event coreference: annotations, experiments, and observations. In Proceedings of the Workshop on Coreference and its Applications. Stroudsburg, Pennsylvania. Stroudsburg, PA: ACL, pp. 1–8.CrossRefGoogle Scholar
Baker, C. F., Fillmore, C. J., and Lowe, J. B., 1998. The Berkeley framenet project. In Proceedings of the 17th International Conference on Computational linguistics (COLING ’98), Montreal, Canada. Stroudsburg, PA: ACL, pp. 86–90.CrossRefGoogle Scholar
Bejan, C., and Harabagiu, S. 2008. A linguistic resource for discovering event structures and resolving event coreference. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. Paris, FranceELRA.Google Scholar
Bejan, C., and Harabagiu, S., 2010. Unsupervised event coreference resolution with rich linguistic features. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), Uppsala, Sweden. Stroudsburg, PA: ACL, pp. 1412–22.Google Scholar
Bejan, C. A., and Hathaway, C., 2007. UTD-SRL: a pipeline architecture for extracting frame semantic structures. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval 2007), Prague, Czech Republic. Stroudsburg, PA: ACL, pp. 460–63.CrossRefGoogle Scholar
Bethard, S. 2008. Finding Event, Temporal and Causal Structure in Text: A Machine Learning Approach. PhD thesis. University of Colorado at Boulder, USA.Google Scholar
Bethard, S. 2013. ClearTK-TimeML: a minimalist approach to TempEval 2013. Second Joint Conference on Lexical and Computational Semantics (*SEM), vol. 2, Atlanta, GA. Stroudsburg, PA: ACL, pp. 10–14.Google Scholar
Boguraev, B., and Ando, R. K. 2005. TimeBank-driven TimeML analysis. In Annotating, Extracting and Reasoning about Time and Events, Dagstuhl Seminar Proceedings, Dagstuhl, Germany.Google Scholar
Borgwardt, K. M. 2007. Graph Kernels. PhD thesis, Ludwig-Maximilians-Universität München, Munich, Germany.Google Scholar
Bramsen, P., Deshpande, P., Lee, Y. K., and Barzilay, R., 2006. Inducing temporal graphs. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP ’06), Sydney, Australia. Stroudsburg, PA: ACL, pp. 189–98.Google Scholar
Chambers, N., and Jurafsky, D. 2008. Unsupervised learning of narrative event chains. In Proceedings of 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT ’08), pp. 789–97.Google Scholar
Chambers, N., and Jurafsky, D., 2009. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP vol. 2, Singapore. Stroudsburg, PA: ACL, pp. 602–10.Google Scholar
Chang, C. C., and Lin, C. J. 2011. LibSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2 (27): 1–27. Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm.CrossRefGoogle Scholar
Chang, A. X., and Manning, C. D., 2012. SUTIME: A Library for Recognizing and Normalizing Time Expressions. Istanbul, Turkey: ELRA.Google Scholar
Chen, B., Su, J., Pan, S., and Tan, C. 2011. A unified event coreference resolution by integrating multiple resolvers. In Proceedings of 5th International Joint Conference on Natural Language Processing (IJCNLP 2011), Chiang Mai, Thailand. New York, NY: Curran Associates.Google Scholar
Cordella, L. P., Foggia, P., Sansone, C., and Vento, M., 2004. A subgraph isomorphism algorithm for matching large graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (10): 1367–72.CrossRefGoogle ScholarPubMed
Davidson, D., 1967. The logical form of action sentences. Essays on Actions and Events 5: 105–48.Google Scholar
De Marneffe, M. C., MacCartney, B., and Manning, C. D., 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of 5th International Conference on Language Resources and Evaluation (LREC 2006), Genoa, Italy, vol. 6. Paris, France: ELRA, pp. 449–54.Google Scholar
Derczynski, L., and Gaizauskas, R. 2013. Temporal signals help label temporal relations. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria, vol. 78. Stroudsburg, PA: ACL.Google Scholar
Fan, R. E., Chang, K., Hsieh, C. J., Wang, X. R., and Lin, C. J., 2008. LibLinear: a library for large linear classification. Journal of Machine Learning Research 9: 1871–4.Google Scholar
Fillmore, C. J., 1976. Frame semantics and the nature of language*. Annals of the New York Academy of Sciences 280 (1): 20–32.CrossRefGoogle Scholar
Finkel, J. R., Grenager, T., and Manning, C. D., 2005. Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL ’05), Ann Arbor, MI. Stroudsburg, PA: ACL, pp. 363–70.Google Scholar
Gärtner, T., Flach, P., and Wrobel, S. 2003. On graph kernels: hardness results and efficient alternatives. In Schölkopf, B. and Warmuth, M. K. (eds.), Learning Theory and Kernel Machines. Berlin, Germany: Springer-Verlag, pp. 129–43.CrossRefGoogle Scholar
Gildea, D., and Jurafsky, D., 2002. Automatic labeling of semantic roles. Computational Linguistics 28 (3): 245–88.CrossRefGoogle Scholar
Glavaš, G., and Šnajder, J. 2013. Recognizing identical events with graph kernels. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria (accepted). Stroudsburg, PA: ACL.Google Scholar
Grishman, R., and Sundheim, B., 1996. Message understanding conference-6: a brief history. In Proceedings of International Conference on Computational Linguistics (COLING 1996), Copenhagen, Denmark, vol. 96. Dresden, Germany: ICCL, pp. 466–71.CrossRefGoogle Scholar
Grover, C., Tobin, R., Alex, B., and Byrne, K., 2010. Edinburgh-LTG: TempEval-2 system description. In Proceedings of the 5th International Workshop on Semantic Evaluation (SemEval 2010), Uppsala, Sweden. Stroudsburg, PA: ACL, pp. 333–6.Google Scholar
Haghighi, A., Toutanova, K., and Manning, C. D., 2005. A joint model for semantic role labeling. In Proceedings of the Ninth Conference on Computational Natural Language Learning, Ann Arbor, MI. Stroudsburg, PA: ACL, pp. 173–6.CrossRefGoogle Scholar
Hammack, R., Imrich, W., and Klavžar, S., 2011. Handbook of Product Graphs. Discrete Mathematics and Its Applications. Boca Raton, FL: CRC Press.CrossRefGoogle Scholar
Hatzivassiloglou, V., Gravano, L., and Maganti, A. 2000. An investigation of linguistic features and clustering algorithms for topical document clustering. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM, pp. 224–31.CrossRefGoogle Scholar
Humphreys, K., Gaizauskas, R., and Azzam, S., 1997. Event coreference for information extraction. In Proceedings of a Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, Madrid, Spain. Stroudsburg, PA: ACL, pp. 75–81.Google Scholar
Humphreys, K., Gaizauskas, R., Azzam, S., Huyck, C., Mitchell, B., Cunningham, H., and Wilks, Y. 1998. University of Sheffield: description of the LaSIE-II system as used for MUC-7. In Proceedings of the Seventh Message Understanding Conferences (MUC-7), San Diego, CA. Gaithersburg, MD: NIST.Google Scholar
Jans, B., Bethard, S., Vulić, I., and Moens, M. F., 2012. Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France. Stroudsburg, PA: ACL, pp. 336–44.Google Scholar
Jun, P., and Min, Y., 2012. Improved temporal relation classication using dependency parses and selective crowdsourced annotations. In Proceedings of the International Conference on Computational Linguistics (COLING 2012), Mumbai, India, pp. 2109–24.Google Scholar
Karttunen, L., and Zaenen, A. 2005. Veridicity. Annotating, Extracting and Reasoning about Time and Events, Dagstuhl Seminar Proceedings. Dagstuhl, Germany: IBFI.Google Scholar
Kingsbury, P., and Palmer, M., 2002. From Treebank to PropBank. In Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC ’02), Las Palmas, Spain. Paris, France: ELRA, pp. 1989–93.Google Scholar
Kolomiyets, O., Bethard, S., and Moens, M., 2012. Extracting narrative timelines as temporal dependency structures. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), Jeju Island, Korea. Stroudsburg, PA: ACL, pp. 88–97.Google Scholar
Kumaran, G., and Allan, J. 2004. Text classification and named entities for new event detection. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM, pp. 297–304.CrossRefGoogle Scholar
Lavie, A., and Denkowski, M. J., 2009. The METEOR metric for automatic evaluation of machine translation. Machine Translation 23 (2–3): 105–15.CrossRefGoogle Scholar
Lee, H., Peirsman, Y., Chang, A., Chambers, N., Surdeanu, M., and Jurafsky, D., 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task (CoNLL 2011), Portland, OR. Stroudsburg, PA: ACL, pp. 28–34.Google Scholar
Lee, H., Recasens, M., Chang, A., Surdeanu, M., and Jurafsky, D., 2012. Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP 2012), Jeju Island, Korea, Stroudsburg, PA: ACL, pp. 489–500.Google Scholar
Llorens, H., Saquete, E., and Navarro, B., 2010. TIPSem (English and Spanish): evaluating CRFs and semantic roles in TempEval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden. Stroudsburg, PA: ACL, pp. 284–91.Google Scholar
Llorens, H., Saquete, E., and Navarro-Colorado, B., 2013. Applying semantic knowledge to the automatic processing of temporal expressions and events in natural language. Information Processing & Management 49 (1): 179–97.CrossRefGoogle Scholar
Makkonen, J., 2003. Investigations on event evolution in TDT. In Proceedings of the Student Research Workshop at Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (NAACL-HLT ’03), Edmonton, Canada. Stroudsburg, PA: ACL, pp. 43–48.Google Scholar
Makkonen, J., Ahonen-Myka, H., and Salmenkivi, M., 2004. Simple semantics in topic detection and tracking. Information Retrieval 7 (3): 347–68.CrossRefGoogle ScholarPubMed
McCallum, A., and Li, W., 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Sapporo, Japan, Stroudsburg, PA: ACL, pp. 188–91.CrossRefGoogle Scholar
Melli, G., Wang, Y., Liu, Y., Kashani, M. M., Shi, Z., Gu, B., Sarkar, A., and Popowich, F. 2006. Description of SQUASH, the SFU question answering summary handler for the DUC-2005 summarization task. In Proceedings of the Document Understanding Conference, New York NY. Gaithersburg, MD: NIST.Google Scholar
Menchetti, S., Costa, F., and Frasconi, P., 2005. Weighted decomposition kernels. In Proceedings of the 22nd International Conference on Machine Learning (ICML ’05), Bonn, Germany. New York, NY: ACM, pp. 585–92.CrossRefGoogle Scholar
Moreda, P., Llorens, H., Saquete, E., and Palomar, M., 2011. Combining semantic information in question answering systems. Information Processing & Management 47 (6): 870–85.CrossRefGoogle Scholar
Nadeau, D., and Sekine, S., 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes 30 (1): 3–26.CrossRefGoogle Scholar
Nallapati, R., Feng, A., Peng, F., and Allan, J. 2004. Event threading within news topics. In Proceedings of the 13th ACM International Conference on Information and Knowledge Management. New York, NY: ACM, pp. 446–53.Google Scholar
Navigli, R., and Lapata, M., 2007. Graph connectivity measures for unsupervised word sense disambiguation. In Proceedings of the 20th International Joint Conference on Artifical Intelligence, Hyderabad, India. Burlington, MA: Morgan Kaufmann, pp. 1683–8.Google Scholar
Palmer, M., Gildea, D., and Kingsbury, P., 2005. The proposition bank: an annotated corpus of semantic roles. Computational Linguistics 31 (1): 71–106.CrossRefGoogle Scholar
Polanyi, L., and Zaenen, A., 2006. Contextual valence shifters. Computing Attitude and Affect in Text: Theory and Applications 20: 1–10.Google Scholar
Pradhan, S., Ramshaw, L., Marcus, M., Palmer, M., Weischedel, R., and Xue, N., 2011. CoNLL-2011 shared task: modeling unrestricted coreference in OntoNotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task (CoNLL 2011), Portland, OR. Stroudsburg, PA: ACL, pp. 1–27.Google Scholar
Pustejovsky, J., 1991. The syntax of event structure. Cognition 41 (1): 47–81.CrossRefGoogle ScholarPubMed
Pustejovsky, J., Castano, J., Ingria, R., Sauri, R., Gaizauskas, R., Setzer, A., Katz, G., and Radev, D., 2003b. TimeML: robust specification of event and temporal expressions in text. New Directions in Question Answering 2003: 28–34.Google Scholar
Pustejovsky, J., Hanks, P., Sauri, R., See, A., Gaizauskas, R., Setzer, A., Radev, D., Sundheim, B., Day, D., and Ferro, L., 2003a. The TimeBank corpus. In Proceedings of Corpus Linguistics, vol. 2003. Lancaster, UK: UCREL, pp. 40.Google Scholar
Quine, W. 1985. Events and reification. In LePore, E. and McLaughlin, B. P. (eds.), Actions and Events: Perspectives on the Philosophy of Donald Davidson, Oxford UK: Blackwell, pp. 162–71.Google Scholar
Resnik, P., 1999. Semantic similarity in a taxonomy: an information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11: 95–130.CrossRefGoogle Scholar
Rosen, S., 1999. The syntactic representation of linguistic events. Glot International 4 (2): 3–11.Google Scholar
Salton, G., Wong, A., and Yang, C., 1975. A vector space model for automatic indexing. Communications of the ACM 18 (11): 613–20.CrossRefGoogle Scholar
Saquete, E., González, J. L. V., Martínez-Barco, P., Munoz, R., and Llorens, H., 2009. Enhancing QA systems with complex temporal question processing capabilities. Journal of Artificial Intelligence Research 35 (2): 775.CrossRefGoogle Scholar
Šarić, F., Glavaš, G., Karan, M., Šnajder, J., and Bašić, B. D. 2012. TakeLab: systems for measuring semantic text similarity. In Proceedings of the 6th International Workshop on Semantic Evaluation (SemEval 2012) in Conjunction with the First Joint Conference on Lexical and Computational Semantics (*SEM 2012), Montreal, Canada. Stroudsburg, PA: ACL.Google Scholar
Saurí, R., and Pustejovsky, J., 2012. Are you sure that this happened? Assessing the factuality degree of events in text. Computational Linguistics 38 (2): 261–99.CrossRefGoogle Scholar
Setzer, A., Gaizauskas, R., and Hepple, M. 2003. Using semantic inferences for temporal annotation comparison. In Proceedings of the Fourth International Workshop on Inference in Computational Semantics (ICOS-4). Nancy, France: INRIA, pp. 25–6.Google Scholar
Smith, C. S. 1999. Activities: states or events? Linguistics and Philosophy 22 (5): 479–508.CrossRefGoogle Scholar
Soon, W., Ng, H., and Lim, D., 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics 27 (4): 521–44.CrossRefGoogle Scholar
Sun, W., Rumshisky, A., and Uzuner, O., 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association 20 (5): 806–13.CrossRefGoogle ScholarPubMed
Surdeanu, M., Harabagiu, S., Williams, J., and Aarseth, P., 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, Sapporo, Japan, vol. 1. Stroudsburg, PA: ACL, pp. 8–15.Google Scholar
Surdeanu, M., and Turmo, J., 2005. Semantic role labeling using complete syntactic analysis. In Proceedings of the Ninth Conference on Computational Natural Language Learning, Ann Arbor, MI. Stroudsburg, PA: ACL, pp. 221–4.CrossRefGoogle Scholar
Tannier, X., and Muller, P., 2008. Evaluation metrics for automatic temporal annotation of texts. In Proceedings of the Conference on Language Resources and Evaluation, Marrakech, Morocco. Paris, France: ELRA, pp. 150–5.Google Scholar
Tenny, C., and Pustejovsky, J., 2000. A history of events in linguistic theory. Events as Grammatical Objects 32: 3–37.Google Scholar
Tong, H., Faloutsos, C., Gallagher, B., and Eliassi-Rad, T. 2007. Fast best-effort pattern matching in large attributed graphs. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: ACM, pp. 737–46.CrossRefGoogle Scholar
UzZaman, N., and Allen, J., 2010. TRIPS and TRIOS system for TempEval-2: extracting temporal information from text. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden. Stroudsburg, PA: ACL, pp. 276–83.Google Scholar
UzZaman, N., and Allen, J. F., 2011. Temporal evaluation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), Portland, OR. Stroudsburg, PA: ACL, pp. 351–6.Google Scholar
UzZaman, N., Llorens, H., Derczynski, L., Verhagen, M., Allen, J., and Pustejovsky, J. 2013. SemEval-2013 Task 1: TempEval-3: evaluating time expressions, events, and temporal relations. In Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013), in Conjunction With the Second Joint Conference on Lexical and Computational Semantcis (* SEM 2013), Association for Computational Linguistics, June, Atlanta. Stroudsburg, PA: ACL.Google Scholar
Vendler, Z., 1957. Verbs and times. The Philosophical Review 66 (2): 143–60.CrossRefGoogle Scholar
Verhagen, M. 2007. Drawing TimeML relations with TBox. In Schilder, F., Katz, G. and Pustejovsky, J. (eds.), Annotating, Extracting and Reasoning about Time and Events. Berlin, Germany: Springer-Verlag, pp. 7–28.CrossRefGoogle Scholar
Verhagen, M., Gaizauskas, R., Schilder, F., Hepple, M., Katz, G., and Pustejovsky, J., 2007. SemEval-2007 Task 15: TempEval temporal relation identification. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval 2007), Prague, Czech Republic. Stroudsburg, PA: ACL, pp. 75–80.CrossRefGoogle Scholar
Verhagen, M., Sauri, R., Caselli, T., and Pustejovsky, J., 2010. SemEval-2010 Task 13: TempEval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation (SemEval 2010), Uppsala, Sweden. Stroudsburg, PA: ACL, pp. 57–62.Google Scholar
Wei, C. P., and Chang, Y. H., 2007. Discovering event evolution patterns from document sequences. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 37 (2): 273–83.CrossRefGoogle Scholar
Wolf, F., and Gibson, E., 2005. Representing discourse coherence: a corpus-based study. Computational Linguistics 31 (2): 249–87.CrossRefGoogle Scholar
Wu, Z., and Palmer, M., 1994. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics (ACL 1994), Las Cruces, NM. Stroudsburg, PA: ACL, pp. 133–8.CrossRefGoogle Scholar
Yang, Y., Carbonell, J. G., Brown, R. D., Pierce, T., Archibald, B. T., and Liu, X., 1999. Learning approaches for detecting and tracking news events. Intelligent Systems and their Applications 14 (4): 32–43.CrossRefGoogle Scholar
Yang, C. C., Shi, X., and Wei, C. P., 2009. Discovering event evolution graphs from news corpora. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 39 (4): 850–63.CrossRefGoogle Scholar
Yangarber, R., Grishman, R., Tapanainen, P., and Huttunen, S., 2000. Automatic acquisition of domain knowledge for information extraction. In Proceedings of the 18th Conference on Computational Linguistics, Hong Kong, vol. 2. Stroudsburg, PA: ACL, pp. 940–6.CrossRefGoogle Scholar