Hostname: page-component-7dd5485656-g8tfn Total loading time: 0 Render date: 2025-10-31T11:58:08.179Z Has data issue: false hasContentIssue false
  • English
  • Français

One, no one and one hundred thousand events: Defining and processing events in an inter-disciplinary perspective*

Published online by Cambridge University Press:  25 October 2016

R. SPRUGNOLI  and
S. TONELLI
R. SPRUGNOLI
Affiliation:
Fondazione Bruno Kessler, Via Sommarive 18, Trento, Italy e-mails: sprugnoli@fbk.eu, satonelli@fbk.eu Università di Trento, Via Sommarive 9, Trento, Italy
S. TONELLI
Affiliation:
Fondazione Bruno Kessler, Via Sommarive 18, Trento, Italy e-mails: sprugnoli@fbk.eu, satonelli@fbk.eu
Get access Rights & Permissions [Opens in a new window]

Abstract

We present an overview of event definition and processing spanning 25 years of research in NLP. We first provide linguistic background to the notion of event, and then present past attempts to formalize this concept in annotation standards to foster the development of benchmarks for event extraction systems. This ranges from MUC-3 in 1991 to the Time and Space Track challenge at SemEval 2015. Besides, we shed light on other disciplines in which the notion of event plays a crucial role, with a focus on the historical domain. Our goal is to provide a comprehensive study on event definitions and investigate which potential past efforts in the NLP community may have in a different research domain. We present the results of a questionnaire, where the notion of event for historians is put in relation to the NLP perspective.

Information

Type
Articles
Information
Natural Language Engineering , Volume 23 , Issue 4 , July 2017 , pp. 485 - 506
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

*

The title is inspired by the novel ‘One, No One and One Hundred Thousand’ written by the Nobel prize winner Luigi Pirandello.

References

Aguilar, J., Beller, C., McNamee, P., Van Durme, B., Strassel, S., Song, Z., and Ellis, J., 2014. A comparison of the events and relations across ACE, ERE, TAC-KBP, and FrameNet annotation standards. In Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Baltimore, Maryland, USA: ACL, pp. 45–53.CrossRefGoogle Scholar
Allen, J. F., 1984. Towards a general theory of action and time. Artificial Intelligence 23 (2): 123–54.CrossRefGoogle Scholar
Aone, C., Halverson, L., Hampton, T., and Ramos-Santacruz, M. 1998. SRA: description of the IE2 system used for MUC-7. In Proceedings of the 7th Message Understanding Conferences (MUC-7). Fairfax, VA.Google Scholar
Appelt, D. E., Hobbs, J. R., Bear, J., Israel, D., and Tyson, M., 1993. FASTUS: a finite-state processor for information extraction from real-world text. In Proceedings of IJCAI, Chambéry, France, vol. 93, pp. 1172–1178.Google Scholar
Archer, D. 2014. Exploring verbal aggression in English historical texts using USAS. In Taavitsainen, I., Jucker, A. H. and Tuominen, J. (eds.), Diachronic Corpus Pragmatics (Pragmatics & Beyond New Series), pp. 273–302. University of Helsinki/University of Zurich: John Benjamins Publishing Company.Google Scholar
Asahara, M., Yasuda, S., Konishi, H., Imada, M., and Maekawa, K., 2013. BCCWJ-TimeBank: temporal and event information Annotation on Japanese text. In Proceedings of PACLIC 27, Taipei, Taiwan, pp. 206–214.Google Scholar
Ashish, N., Appelt, D., Freitag, D., and Zelenko, D. 2006. Papers from the AAAI workshop on event extraction and synthesis. Technical Report WS-06-07, American Association for Artificial Intelligence.Google Scholar
Bach, E. 1986. The algebra of events. Linguistics and Philosophy, 9 (1): 5–16, D. Reidel Publishing Company.Google Scholar
Baker, C. F., Fillmore, C. F., and Lowe, J. B., 1998. The Berkeley FrameNet project. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Montréal, Quebec, Canada: ACL, pp. 86–90.Google Scholar
Berr, H., 1922. L’histoire traditionnelle et la synthse historique. Revue belge de philologie et d’histoire 1 (3): 556–556.Google Scholar
Bethard, S., Derczynski, L., Savova, G., Pustejovsky, J., and Verhagen, M. 2015. SemEval-2015 Task 6: clinical TempEval. In Proceedings of SemEval 2015, Denver, Colorado, USA: ACL.Google Scholar
Bittar, A., Amsili, P., Denis, P., and Danlos, L., 2011. French TimeBank: an ISO-TimeML annotated reference corpus. In Proceedings of ACL, Portland, Oregon, USA, ACL, pp. 130–134.Google Scholar
Bjrne, J., and Salakoski, T. 2013. TEES 2.1: automated annotation scheme learning in the BioNLP 2013 Shared Task. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.Google Scholar
Bui, Q. C., Campos, D., van Mulligen, E. M., and Kors, J. A. 2013. A fast rule-based approach for biomedical event extraction. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.Google Scholar
Caselli, T., Bartalesi Lenzi, V., Sprugnoli, R., Pianta, E., and Prodanof, I. 2011. Annotating events, temporal expressions and relations in Italian: the It-TimeML experience for the Ita-TimeBank. In Proceedings of the 5th Linguistic Annotation Workshop, Portland, Oregon: ACL.Google Scholar
Caselli, T., Sprugnoli, R., Speranza, M., and Monachini, M. 2014. EVENTI EValuation of events and temporal INformation at Evalita 2014. In Proceedings of the 4th International Workshop EVALITA 2014, Pisa, Italy, Pisa University Press.Google Scholar
Chinchor, N. A. 1998. Overview of MUC-7/MET-2. In Proceedings of the 7th Message Understanding Conference (MUC-7). Fairfax, VA.Google Scholar
Cybulska, A., and Vossen, P., 2011. Historical event extraction from text. In Proceedings of the 5th ACL-HLT LaTeCH Workshop, Portland, Oregon: ACL, pp. 39–43.Google Scholar
Cybulska, A., and Vossen, P., 2014. Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. In Proceedings of LREC 2014, pp. 26–31, Reykjavik, Iceland, European Language Resources Association (ELRA), pp. 26–31.Google Scholar
Doddington, G. R., Mitchell, A., Przybocki, M. A., Ramshaw, L. A., Strassel, S., and Weischedel, R. M. 2004. The automatic content extraction (ACE) program-tasks, data, and evaluation. In Proceedings of LREC 2004, Lisbon, Portugal, European Language Resources Association (ELRA).Google Scholar
Farzindar, A., and Khreich, W., 2015. A survey of techniques for event detection in Twitter. Computational Intelligence 31.1 (2015): 132–164.Google Scholar
Febvre, L. P. V. 1953. Combats pour l’histoire. Paris: Armand Colin.Google Scholar
Fellbaum, C. 1998. WordNet. Blackwell Publishing Ltd. Cambridge: MIT Press.Google Scholar
Filatova, E., and Hovy, E. 2001. Assigning time-stamps to event-clauses. In Proceedings of the ACL-EACL 2001 Workshop for Temporal and Spatial Information Processing, Toulouse, France: ACL.Google Scholar
Forascu, C., and Tufi, D. 2012. Romanian TimeBank: an annotated parallel corpus for temporal information. In Proceedings of LREC 2012, Istanbul, Turkey. European Language Resources Association (ELRA).Google Scholar
Grishman, R. 2010. The impact of task and corpus on event extraction systems. In Proceedings of LREC 2010, Valletta, Malta, European Language Resources Association (ELRA).Google Scholar
Guldi, J., and Armitage, D., 2014. The History Manifesto. Cambridge: Cambridge University Press.Google Scholar
Hagège, C. 1996. L’homme de Paroles: Contribution Linguistique aux Sciences Humaines. Fayard, Paris.Google Scholar
Hakala, K., Van Landeghem, S., Salakoski, T., Van de Peer, Y., and Ginter, P. 2013. EVEX in ST13: application of a large-scale text mining resource to event extraction and network construction. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.Google Scholar
Hatzivassiloglou, V., and Filatova, E., 2003. Domain-independent detection, extraction, and labeling of atomic events. In Proceedings of RANLP, Borovetz, Bulgaria, pp. 145–152.Google Scholar
Hovy, E., Mitamura, T., Verdejo, F., Araki, J., and Philpot, A. 2013. Events are not simple: identity, non-identity, and quasi-identity. In Proceedings of the The 1st Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Atlanta, Georgia, USA: ACL.Google Scholar
Humphreys, K., Gaizauskas, R., Azzam, S., Huyck, C., Mitchell, B., Cunningham, H., and Wilks, Y. 1998. University of Sheffield: description of the LaSIE-II system as used for MUC-7. In Proceedings of the 7th Message Understanding Conferences (MUC-7). Fairfax, VA.Google Scholar
Ide, N., and Woolner, D. 2004. Exploiting semantic web technologies for intelligent access to historical documents. In Proceedings of LREC 2004, Reykjavik, Iceland, European Language Resources Association (ELRA).Google Scholar
Ikuta, R., Styler, W. F. IV, Hamang, M., O’Gorman, T., and Palmer, M. 2014. Challenges of adding causation to Richer Event Descriptions. In Proceedings of the The 2nd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Baltimore, Maryland, USA: ACL.Google Scholar
Im, S., You, H., Jang, H., Nam, S., and Shin, H., 2009. Ktimeml: specification of temporal and event expressions in korean text. In Proceedings of the 7th Workshop on Asian Language Resources, Suntec, Singapore: ACL, pp. 115–122.Google Scholar
Jung, H., and Stent, A., 2013. Att1: temporal annotation using big windows and rich syntactic and semantic features. In Proceedings of * SEM, Atlanta, Georgia, USA, vol. 2, pp. 20–24.Google Scholar
Kay, C., Roberts, J., Samuels, M., and Wotherspoon, I. 2009. Unlocking the OED: the story of the historical thesaurus of the OED. In Historical Thesaurus of the Oxford English Dictionary: With Additional Material from a Thesaurus of Old English. Oxford: Oxford University Press.Google Scholar
Katz, G., and Arosio, F. 2001. The annotation of temporal information in natural language sentences. In Proceedings of the ACL 2001 Workshop on Temporal and Spatial Information Processing, Toulouse, France: ACL.Google Scholar
Kim, J. D., Ohta, T., Tateisi, Y., and Tsujii, J. 2006. GENIA corpus manual. Technical Report, Citeseer.Google Scholar
Kim, J. D., Ohta, T., and Tsujii, J., 2008. Corpus annotation for mining biomedical events from literature. BMC Bioinformatics 9 (1): 10.Google Scholar
Kolomiyets, O., and Moens, M. F., 2013. KUL: a data-driven approach to temporal parsing of documents. In Proceedings of SemEval 2013, Atlanta, Georgia, USA, pp. 83–87.Google Scholar
Lee, H., Recasens, M., Chang, A., Surdeanu, M., and Jurafsky, D., 2012. Joint entity and event coreference resolution across documents. In Proceedings of EMNLP-2012, Jeju, South Korea, pp. 489–500.Google Scholar
Linguistic Data Consortium. 2005. ACE (automatic content extraction) english annotation guidelines for events, version 5.4.3 2005.07.01.Google Scholar
Llorens, H., Chambers, N., UzZaman, N., Mostafazadeh, N., Allen, J., and Pustejovsky, J. 2015. SemEval-2015 task 5: QA TEMPEVAL-Evaluating temporal information understanding with question answering. In Proceedings of SemEval 2015, Denver, Colorado: ACL.Google Scholar
Llorens, H., Saquete, E., and Navarro, B., 2010. TIPSem (English and Spanish): evaluating CRFs and semantic roles in TempEval-2. In Proceedings of SemEval-2010, Uppsala, Sweden: ACL, pp. 284–291.Google Scholar
Marrou, H.-I. 1954. De la connaissance historique. Paris: Seuil.Google Scholar
Meroño-Peñuela, A., Ashkpour, A., van Erp, M., Mandemakers, K., Breure, L., Scharnhorst, A., Schlobach, S., and van Harmelen, F. 2015. Semantic technologies for historical research: a survey. Semantic Web Journal, 6 (6): 539–64. IOS Press.Google Scholar
Metzler, D., Cai, C., and Hovy, E. 2012. Structured event retrieval over microblog archives. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, Canada: ACL.Google Scholar
Mikheev, A., Grover, C., and Moens, M. 1998. Description of the LTG system used for MUC-7. In Proceedings of 7th Message Understanding Conference (MUC-7), Fairfax, VA.Google Scholar
Minard, A. L., Speranza, M., Agirre, E., Aldabe, I., van Erp, M., Magnini, B., Rigau, G., and Urizar, R. 2015. Semeval-2015 task 4: timeline: cross-document event ordering. In Proceedings of SemEval 2015, ACL. Denver, Colorado, USA: Association for Computational Linguistics.Google Scholar
Minard, A. L., Speranza, M., Urizar, R., Altuna, B., van Erp, M., Schoen, A., and van Son, C. 2016. MEANTIME, the NewsReader multilingual event and time corpus. In Proceedings of LREC 2016, Portoro, Slovenia, European Language Resources Association (ELRA).Google Scholar
Mirza, P., and Minard, A. L. 2014. FBK-HLT-time: a complete Italian temporal processing system for EVENTI-Evalita 2014. In Proceedings of the 4th International Workshop EVALITA 2014, Pisa, Italy: Pisa University Press.Google Scholar
Mirza, P., and Tonelli, S., 2014. An analysis of causality between events and its relation to temporal information. In Proceedings of COLING 2014, Dublin, Ireland: Dublin City University and ACL, pp. 2097–2106.Google Scholar
Mitamura, T., Yamakawa, Y., Holm, S., Song, Z., Bies, A., Kulick, S., and Strassel, St., 2015. Event nugget annotation: processes and issues. In Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Denver, Colorado, USA: ACL, pp. 66–76.Google Scholar
Mott, J., Bies, A., Song, Z., and Strassel, S. 2016. Parallel Chinese-English entities, relations and events corpora. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), Portoro, Slovenia, European Language Resources Association (ELRA).Google Scholar
Nédellec, C., Bossy, R., Kim, J. D., Kim, J. J., Ohta, T., Pyysalo, S., and Zweigenbaum, P., 2013. Overview of BioNLP shared task 2013. In Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria, pp. 1–7.Google Scholar
Nieto, M. G., Saurí, R., and Bernabé Poveda, M. A., 2011. ModeS TimeBank: a modern spanish TimeBank corpus. Procesamiento del lenguaje natural 47 (2011): 259–267.Google Scholar
Palmer, M., Gildea, D., and Kingsbury, P. 2005. The proposition bank: an annotated corpus of semantic roles. Computational Linguistics, 31 (1): 71–106.Google Scholar
Petrović, S., Osborne, M., and Lavrenko, V., 2010. Streaming first story detection with application to twitter. In Proceedings of NAACL 2010, Los Angeles, California, USA, pp. 181–189.Google Scholar
Pustejovsky, J., 1991. The syntax of event structure. Cognition 41 (1–3): 47–81.Google Scholar
Pustejovsky, J. 2005. A survey of dot objects. Author’s weblog. Retrieved from URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208.7525&rep=rep1&type=pdf Google Scholar
Pustejovsky, J., Castaño, J. M., Ingria, R., Saurí, R. G., Setzer, A. and Katz, G. 2003. TimeML: robust specification of event and temporal expressions in text. In Proceedings of IWCS-5. Tilburg, The Netherlands.Google Scholar
Pustejovsky, J., Hanks, P., Saurí, R., See, A., Gaizauskas, R., Setzer, A., Radev, D., Sundheim, B., Day, D., Ferro, L., and Lazo, M., 2003. The TIMEBANK Corpus. In Proceedings of Corpus Linguistics 2003, Lancaster, UK, pp. 647–656.Google Scholar
Pustejovsky, J., Littman, J., and Saurì, R. 2007. Arguments in TimeML: events and entities. In Schilder, F., Katz, G., & Pustejovsky, J. (eds), Annotating, Extracting and Reasoning about Time and Events: International Seminar Revised Papers (Lecture Notes in Computer Science), pp. 107–126. Berlin: Springer Berlin Heidelberg.CrossRefGoogle Scholar
Rayson, P., Archer, D., Piao, S., and McEnery, A. M., 2004. The UCREL semantic analysis system. In Proceedings of the Beyond Named Entity Recognition Semantic Labelling for NLP Tasks Workshop, Lisbon, Portugal, pp. 7–12.Google Scholar
Rayson, P., Baron, A., Piao, S., and Wattam, S. 2015. Large-scale time-sensitive semantic analysis of historical corpora. In Proceedings of the 36th Meeting of ICAME, Trier, Germany.Google Scholar
Ritter, A., Mausam, E. O., and Clark, S., 2012. Open domain event extraction from Twitter. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China: ACM, pp. 1104–1112.Google Scholar
Sasse, H. J., 2002. Recent activity in the theory of aspect: accomplishments, achievements, or just non-progressive state. Linguistic Typology 6 (2): 199–271.Google Scholar
Saurı, R. 2010. Annotating temporal relations in catalan and spanish TimeML annotation guidelines. Technical Report BM 2010-04, Barcelona Media.Google Scholar
Saurí, R., and Pustejovsky, J., 2009. Factbank: a corpus annotated with event factuality. Language Resources and Evaluation 43 (3): 227–68.Google Scholar
Schilder, F., and Habel, C. 2001. From temporal expressions to temporal information: semantic tagging of news messages. In Proceedings of the ACL-EACL 2001 Workshop for Temporal and Spatial Information Processing. Toulouse, France: ACL.Google Scholar
Schilder, F., and Habel, C. 2003. Temporal information extraction for temporal question answering. In New Directions in Question Answering, AAAI, pp. 35–44.Google Scholar
SemAf/Time Working Group ISO, 2008. ISO DIS 24617-1: 2008 Language Resource Management - Semantic Annotation Framework - Part 1: Time and Events. ISO Central Secretariat, Geneva.Google Scholar
Setzer, A. 2001. Temporal Information in Newswire Articles: An Annotation Scheme and Corpus Study. PhD Thesis, University of Sheffield.Google Scholar
Simiand, F. 1960. Méthode historique et science sociale. In Annales. Histoire, Sciences Sociales 15 (1): 83–119. France: EHESS.Google Scholar
Song, Z., Bies, A., Strassel, S., Riese, T., Mott, J., Ellis, J., Wright, J., Kulick, S., Ryant, N., and Ma, X., 2015. From light to rich ERE: annotation of entities, relations, and events. In Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Denver, Colorado, USA: ACL, pp. 89–98.Google Scholar
Song, Z., Biesa, A., Strassel, S., Ellis, J., Mitamura, T., Dang, H., Yamakawa, Y., and Holm, S. 2016. Event nugget and event coreference annotation. In Proceedings of the 4th Workshop on EVENTS: Definition, Detection, Coreference, and Representation, NAACL HLT 2016.Google Scholar
Styler IV, W. F., Bethard, S., Finan, S., Palmer, M., Pradhan, S., de Groen, P.C., Erickson, B., Miller, T., Lin, C., Savova, G., and Pustejovsky, J., 2014. Temporal annotation in the clinical domain. Transactions of the Association for Computational Linguistics 2 (2014): 143–54.Google Scholar
Sun, W., Rumshisky, A., and Uzuner, O. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association. 20 (5): 806–13. Oxford University Press.Google Scholar
UzZaman, N., Llorens, H., Derczynski, L., Allen, J., Verhagen, M., and Pustejovsky, J., 2013. Semeval-2013 task 1: Tempeval-3: Evaluating time expressions, events, and temporal relations. In Proceedings of SemEval 2013, Atlanta, Georgia, USA: ACL, pp. 1–9.Google Scholar
Van Den Akker, C., Aroyo, L., Cybulska, A., Van Erp, M., Gorgels, P., Hollink, L., Jager, C., Legene, S., van der Meij, L., Oomen, J., van Ossenbruggen, J., Schreiber, G., Segers, R., Vossen, P., and Wielinga, B. 2010. Historical event-based access to museum collections. In Proceedings of EVENTS 2010, Athens, Greece: ACL.Google Scholar
van Son, C., van Erp, M., Fokkens, A., and Vossen, P., 2014. Hope and fear: interpreting perspectives by integrating sentiment and event factuality. In Proceedings of LREC 2014, Reykjavik, Iceland, European Language Resources Association (ELRA), pp. 26–31.Google Scholar
Vendler, Z. 1967. Verbs and times. In Linguistics and Philosophy, pp. 97–121. Ithaca, NY: Cornell University Press.CrossRefGoogle Scholar
Verhagen, M., Gaizauskas, R., Schilder, F., Hepple, M., Katz, G., and Pustejovsky, J., 2007. Semeval-2007 task 15: Tempeval temporal relation identification. In Proceedings of SemEval-2007, Prague, Czech Republic: ACL, pp. 75–80.Google Scholar
Verhagen, M., Sauri, R., Caselli, T., and Pustejovsky, J., 2010. Semeval-2010 task 13: Tempeval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden: ACL, pp. 57–62.Google Scholar
Vossen, P., Rigau, G., Serafini, L., Stouten, P., Irving, F., and Van Hage, W. 2014. NewsReader: recording history from daily news streams. In Proceedings of LREC 2014, Reykjavik, Iceland, European Language Resources Association (ELRA).Google Scholar
Yaghoobzadeh, J., Ghassem-Sani, G., Mirroshandel, S. A., and Eshaghzadeh, M., 2012. ISO-TimeML event extraction in Persian text. In Proceedings of COLING 2012, Mumbai, India, pp. 2931–2944.Google Scholar
Zavarella, V., and Tanev, H., 2013. FSS-TimEx for TempEval-3: extracting temporal information from text. In Proceedings of SemEval 2013, Atlanta, Georgia, USA, ACL, pp. 58–63.Google Scholar