Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-11T00:13:18.020Z Has data issue: false hasContentIssue false
  • English
  • Français

Keyword extraction: Issues and methods

Published online by Cambridge University Press:  11 November 2019

Nazanin Firoozeh Open the ORCID record for Nazanin Firoozeh [Opens in a new window] ,
Adeline Nazarenko ,
Fabrice Alizon  and
Béatrice Daille
Nazanin Firoozeh*
Affiliation:
Pixalione SAS, 75015 Paris, France Northern Paris Computer Science Laboratory (LIPN), Paris 13 University – Sorbonne Paris Cité & CNRS, 93430 Villetaneuse, France
Adeline Nazarenko
Affiliation:
Northern Paris Computer Science Laboratory (LIPN), Paris 13 University – Sorbonne Paris Cité & CNRS, 93430 Villetaneuse, France
Fabrice Alizon
Affiliation:
Pixalione SAS, 75015 Paris, France
Béatrice Daille
Affiliation:
Laboratory of Digital Sciences of Nantes (LS2N), University of Nantes, 44322 Nantes Cedex 3, France
*
*Corresponding author. Email: nazanin.firoozeh@pixalione.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Due to the considerable growth of the volume of text documents on the Internet and in digital libraries, manual analysis of these documents is no longer feasible. Having efficient approaches to keyword extraction in order to retrieve the ‘key’ elements of the studied documents is now a necessity. Keyword extraction has been an active research field for many years, covering various applications in Text Mining, Information Retrieval, and Natural Language Processing, and meeting different requirements. However, it is not a unified domain of research. In spite of the existence of many approaches in the field, there is no single approach that effectively extracts keywords from different data sources. This shows the importance of having a comprehensive review, which discusses the complexity of the task and categorizes the main approaches of the field based on the features and methods of extraction that they use. This paper presents a general introduction to the field of keyword/keyphrase extraction. Unlike the existing surveys, different aspects of the problem along with the main challenges in the field are discussed. This mainly includes the unclear definition of ‘keyness’, complexities of targeting proper features for capturing desired keyness properties and selecting efficient extraction methods, and also the evaluation issues. By classifying a broad range of state-of-the-art approaches and analysing the benefits and drawbacks of different features and methods, we provide a clearer picture of them. This review is intended to help readers find their way around all the works related to keyword extraction and guide them in choosing or designing a method that is appropriate for the application they are targeting.

Keywords

Information extractionKeyword extractionExtraction featuresExtraction methods
Type
Survey Paper
Information
Natural Language Engineering , Volume 26 , Issue 3 , May 2020 , pp. 259 - 291
Copyright
© Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abilhoa, W.D. and de Castro, L.N. (2014). A keyword extraction method from twitter messages represented as graphs. Applied Mathematics and Computation 240, 308–325.CrossRefGoogle Scholar
Ajgalík, M., Barla, M. and Bieliková, M. (2013). From ambiguous words to key-concept extraction. In 24th International Workshop on Database and Expert Systems Applications, pp. 63–67.CrossRefGoogle Scholar
Augenstein, I., Das, M., Riedel, S., Vikraman, L. and McCallum, A. (2017) SemEval 2017 Task 10: ScienceIE - Extracting keyphrases and relations from scientific publications. In Proceedings of the International Workshop on Semantic Evaluation (SemEval), Vancouver, Canada, pp. 546–555.CrossRefGoogle Scholar
Beliga, S., Meštrović, A. and Martins̆ić-Ips̆ić, S. (2015). An overview of graph-based keyword extraction methods and approaches. Journal of Information and Organizational Sciences 39, 1–20.Google Scholar
Bennani-Smires, K., Musat, C., Hossmann, A., Baeriswyl, M. and Jaggi, M. (2018). Simple unsupervised keyphrase extraction using sentence embeddings. In Proceedings of the 22nd Conference on Computational Natural Language Learning, Brussels, Belgium, pp. 221–229.CrossRefGoogle Scholar
Bharti, S.K. and Babu, K.S. (2017). Automatic keyword extraction for text summarization: A survey. In Computing Research Repository (CoRR), Volume abs/1704.03242.Google Scholar
Blei, D., Boudin, F. and Daille, B. (2013). Latent Dirichlet allocation. Journal of Machine Learning Research 3, 993–1022.Google Scholar
Boudin, F. (2013). A comparison of centrality measures for graph-based keyphrase extraction. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, Nagoya, Japan, pp. 834–838.Google Scholar
Boudin, F. and Morin, E. (2013). Keyphrase extraction for N-best reranking in multi-sentence compression. In North American Chapter of the Association for Computational Linguistics (NAACL), Atlanta, United States, pp. 298–305.Google Scholar
Bougouin, A. (2015). Automatic Domain-Specific Keyphrase Annotation. PhD thesis, Université de Nantes.Google Scholar
Bougouin, A., Boudin, F. and Daille, B. (2013). TopicRank: Graph-based topic ranking for keyphrase extraction. In Sixth International Joint Conference on Natural Language Processing (IJCNLP), Nagoya, Japan, pp. 543–551.Google Scholar
Brin, S. and Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks 30, 107–117.Google Scholar
Buckley, C. and Voorhees, E.M. (2004). Retrieval evaluation with incomplete information. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, United Kingdom, pp. 25–32.CrossRefGoogle Scholar
Budanitsky, A. and Hirst, G. (2001). Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Workshop on Wordnet and Other Lexical Resources, Second Meeting of the North American Chapter of the Association for Computational Linguistics.Google Scholar
Bulgarov, F. and Caragea, C. (2015). A comparison of supervised keyphrase extraction models. In Proceedings of the 24th International Conference on World Wide Web, Vol. 30, Florence, Italy, pp. 13–14.CrossRefGoogle Scholar
Caragea, C., Bulgarov, F., Godea, A. and Gollapalli, S. (2014). Citation-enhanced keyphrase extraction from research papers: A supervised approach. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1435–1446.CrossRefGoogle Scholar
Carretero-Campos, C., Bernaola-Galvan, P., Coronado, A. and Carpena, P. (2013). Improving statistical keyword detection in short texts: Entropic and clustering approaches. Physica A: Statistical Mechanics and its Applications, 392(6), 1481–1492.CrossRefGoogle Scholar
Cellier, P., Charnois, T., Hotho, A., Matwin, S., Moens, M. and Toussaint, Y., (eds.) (2014). Proceedings of the 1st International Workshop on Interactions between Data Mining and Natural Language Processing (DMNLP@PKDD/ECML), volume 1202 of CEUR Workshop Proceedings, Nancy, France.Google Scholar
Chen, D., Li, X., Liu, J. and Chen, X. (2009). Ranking-constrained keyword sequence extraction from web documents. In Bouguettaya, A. and Lin, X. (eds), Proceedings of the 20th Australasian Database Conference (ADC), Vol. 92. Wellington, New Zealand: ACS, pp. 161–169.Google Scholar
Chung, W., Chen, H. and Nunamaker, J.F. (2003). Business intelligence explorer: A knowledge map framework for discovering business intelligence on the Web. In Proceedings of the 36th Annual Hawaii International Conference on System Sciences.Google Scholar
Cohen, J.D. (1995). Highlights: Language- and domain-independent automatic indexing terms for abstracting. Journal of the American Society for Information Science (JASIS), 46(3), 162–174.3.0.CO;2-6>CrossRefGoogle Scholar
Danesh, S., Sumner, T. and Martin, J.H. (2015). SGRank: Combining statistical and graphical methods to improve the state of the art in unsupervised keyphrase extraction. In Palmer, M., Boleda, G. and Rosso, P. (eds), Proceedings of the 4th Joint Conference on Lexical and Computational Semantics (SEM), Denver, Colorado, USA, pp. 117–126.CrossRefGoogle Scholar
De Maio, C., Fenza, G., Loia, V. and Parente, M. (2016). Time aware knowledge extraction for microblog summarization on Twitter. Information Fusion Journal 28, 60–74.CrossRefGoogle Scholar
Frank, E., Paynter, G.W., Witten, I.H., Gutwin, C. and Nevill-Manning, C.G. (1999). Domain-specific keyphrase extraction. In Proceedings of the 6th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA: Morgan Kaufmann, pp. 668–673.Google Scholar
Grineva, M., Grinev, M. and Lizorkin, D. (2009). Extracting key terms from noisy and multi-theme documents. In Proceedings of the 18th International Conference on World Wide Web (WWW). New York, NY, USA: ACM, pp. 661–670.Google Scholar
HaCohen-Kerner, Y. (2003). Automatic extraction of keywords from abstracts. In Knowledge-Based Intelligent Information and Engineering Systems. Berlin, Heidelberg: Springer, pp. 843–849.CrossRefGoogle Scholar
Hammouda, K.M., Matute, D.N. and Kamel, M.S. (2005). CorePhrase: Keyphrase extraction for document clustering. In Perner, P. and Imiya, A. (eds), Machine Learning and Data Mining in Pattern Recognition, Springer, Berlin, Heidelberg, pp. 265–274.CrossRefGoogle Scholar
Harris, Z. (1954). Distributional structure. Word 10(23), 146–162.CrossRefGoogle Scholar
Hasan, K.S. and Ng, V. (2014). Automatic keyphrase extraction: A survey of the state of the art. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, USA: ACL, pp. 1262–1273.CrossRefGoogle Scholar
Herrera, P.J. and Pury, A.P. (2008). Statistical keyword detection in literary corpora. The European Physical Journal B 63(1), 135–146.CrossRefGoogle Scholar
Hindle, D. (1990). Noun classification from predicate-argument structures. In Proceedings of the annual meeting of the Association for Computational Linguistics (ACL), Pittsburgh, Pennsylvania, pp. 268–275.CrossRefGoogle Scholar
Huang, C., Tian, Y., Zhou, Z., Ling, C.X. and Huang, T. (2006). Keyphrase extraction using semantic networks structure analysis. In Proceedings of the 6th International Conference on Data Mining (ICDM), IEEE, pp. 275–284.CrossRefGoogle Scholar
Hulth, A. (2003). Improved automatic keyword extraction given more linguistic knowledge. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA, USA: ACL, pp. 216–223.CrossRefGoogle Scholar
Hussey, R., Williams, S. and Mitchell, R. (2012). Automatic keyphrase extraction: A comparison of methods. In Proceedings of the 4th International Conference on Information Process, and Knowledge Management (eKNOW), Valencia, Spain, pp. 18–23.Google Scholar
Jacquemin, C. and Bourigault, D. (2003). Term extraction and automatic indexing. In Mitkov, R. (ed), The Oxford Handbook of Computational Linguistics, Oxford University Press, pp. 599–615.Google Scholar
Jiang, X., Hu, Y. and Li, H. (2009). A ranking approach to keyphrase extraction. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Boston, MA, USA, pp. 756–757.CrossRefGoogle Scholar
Kaur, J. and Gupta, V. (2010). Effective approaches for extraction of keywords. International Journal of Computer Science Issues (IJCSI) 7(6), 144–148.Google Scholar
Kelleher, D. and Luz, S. (2005). Automatic hypertext keyphrase detection. In Kaelbling, L.P. and Saffiotti, A. (eds), Proceedings of the 19th International Joint Conference on Artificial intelligence (IJCAI), San Francisco, CA, USA, pp. 1608–1609.Google Scholar
Kim, S., Medelyan, O., Kan, M. and Baldwin, T. (2010). SemEval-2010 Task 5: Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden, pp. 21–26.Google Scholar
Kleinberg, J.M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632.CrossRefGoogle Scholar
Lahiri, S., Choudhury, S.R. and Caragea, C. (2014). Keyword and keyphrase extraction using centrality measures on collocation networks. In Computing Research Repository (CoRR), abs/1401.6571.Google Scholar
Litvak, M. and Last, M. (2008). Graph-based keyword extraction for single-document summarization. In Proceedings of the Workshop on Multi-source Multilingual Information Extraction and Summarization, Manchester, United Kingdom, pp. 17–24.CrossRefGoogle Scholar
Liu, Z., Huang, W., Zheng, Y. and Sun, M. (2010). Automatic keyphrase extraction via topic decomposition. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Cambridge, Massachusetts: ACL, pp. 366–376.Google Scholar
Liu, F., Pennell, D., Liu, F. and Liu, Y. (2009). Unsupervised approaches for automatic keyword extraction using meeting transcripts. In Proceedings of Human Language Technologies (NAACL). Boulder, Colorado: ACL, pp. 620–628.CrossRefGoogle Scholar
Lopez, P. and Romary, L. (2010). HUMB: Automatic key term extraction from scientific articles in GROBID. In Proceedings of the 5th International Workshop on Semantic Evaluation, pp. 248–251.Google Scholar
Lossio-Ventura, J.A., Jonquet, C., Roche, M. and Teisseire, M. (2013). Combining C-value and keyword extraction methods for biomedical terms extraction. In LBM: Languages in Biologyand Medicine, Tokyo, Japan.Google Scholar
Luhn, H.P. (1957). A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Devlopment 1(4), 309–317.CrossRefGoogle Scholar
Manning, C. and Schütze, H. (1990). Foundations of statistical Natural Language Processing. Cambridge, MA: MIT Press.Google Scholar
Martins, C.B., Pardo, T.A.S., Espina, A.P. and Rino, L.H.M. (2001). Introducão à sumarizacão automática. Technical report RT-DC 002/2001, ICMC-USP.Google Scholar
Matsuo, Y. and Ishizuka, M. (2003). Keyword extraction from a single document using word co-occurrence statistical informationl. International Journal on Artificial Intelligence Tools 13(1), 157–169.CrossRefGoogle Scholar
Matsuo, Y., Ohsawa, Y. and Ishizuka, M. (2001). Keyworld: Extracting keywords from a document as a small world. In Proceedings of the 4th International Conference on Discovery Science (DS), volume 2226 of LNCS, pp. 271–281.Google Scholar
Medelyan, O. (2009). Human-Competitive Automatic Topic Indexing. PhD thesis, The University of Waikato.Google Scholar
Medelyan, O., Frank, E. and Witten, I.H. (2009). Human-competitive tagging using automatic keyphrase extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Vol. 3, Singapore, pp. 1318–1327.CrossRefGoogle Scholar
Medelyan, O. and Witten, I.H. (2006). Thesaurus based automatic keyphrase indexing. In Proceedings of the 6th Joint Conference on Digital Libraries (JCDL), ACM, pp. 296–297.CrossRefGoogle Scholar
Mehri, A. and Darooneh, A.H. (2011). The role of entropy in word ranking. Physica A: Statistical Mechanics and its Applications, 390, 3157–3163.CrossRefGoogle Scholar
Mehri, A., Jamaati, M. and Mehri, H. (2015). Word ranking in a single document by jensen-shannon divergence. Physics Letters A 379(28), 1627–1632.CrossRefGoogle Scholar
Meng, R., Zhao, S., Han, S., He, D., Brusilovsky, P. and Chi, Y. (2017). Deep keyphrase generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, Vancouver, Canada, pp. 582–592.CrossRefGoogle Scholar
Mihalcea, R. and Tarau, P. (2004). TextRank: Bringing order into texts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Barcelona, Spain, pp. 404–411.Google Scholar
Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. In Computing Research Repository (CoRR), pp. 1–12.Google Scholar
Miller, G.A., Beckwith, R. and Ch, Fellbaum. (1990). WordNet: An on-line lexical database. International Journal of Lexicography, 3(4), 235–244.CrossRefGoogle Scholar
Mimouni, N., Nazarenko, A. and Salotti, S. (2015). Search and discovery in legal document networks. In Legal Knowledge and Information Systems (JURIX), Braga, Portugal, pp. 187–188.Google Scholar
Momtazi, S., Khudanpur, S. and Klakow, D. (2010). A comparative study of word co-occurrence for term clustering in language model-based sentence retrieval. In Proceedings of Human Language Technologies (NAACL). Los Angeles, CA, USA: ACL, pp. 325–328.Google Scholar
Mori, J., Matsuo, Y., Ishizuka, M. and Faltings, B. (2004). Keyword extraction from the web for foaf metadata. In Workshop on Friend of a Friend, Social Networking and the Semantic Web.Google Scholar
Muñoz, A. (1997). Compound key word generation from document databases using a hierarchical clustering {ART} model. Intelligent Data Analysis, 1(1–4), 25–48.CrossRefGoogle Scholar
Nadeau, D. and Sekine, S. (2007). A survey of named entity recognition and classification. Linguisticae Investigationes, 30(1), 3–26. John Benjamins.Google Scholar
Navigli, R. and Velardi, P. (2002). Semantic interpretation of terminological strings. In Proceedings of the Conference on Terminology and Knowledge Engineering (TKE), pp. 95–100.Google Scholar
Nguyen, T. and Kan, M. (2007). Keyphrase extraction in scientific publications. In Proceedings of the 10th International Conference on Asian Digital Libraries: Looking Back 10 Years and Forging New Frontiers, Hanoi, Vietnam, pp. 317–326.CrossRefGoogle Scholar
Ohsawa, Y., Benson, N.E. and Yachida, M. (1998). KeyGraph: Automatic indexing by co-occurrence graph based on building construction metaphor. In Proceedings of the Advances in Digital Libraries Conference (ADL). Washington, DC, USA: IEEE, pp. 12–18.CrossRefGoogle Scholar
Ortuño, M., Carpena, P., Bernaola-Galván, P., Muñoz, E. and Somoza, A.M. (2002). Keyword detection in natural languages and dna. EPL (Europhysics Letters) 57(5), 759–764.CrossRefGoogle Scholar
Palshikar, G.K. (2007). Keyword extraction from a single document using centrality measures. In Proceedings of the 2nd International Conference on Pattern Recognition and Machine Intelligence, Kolkata, India, pp. 503–510.CrossRefGoogle Scholar
Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann.Google Scholar
Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M.M. and Gatford, M. (1996). Okapi at TREC-3. In Overview of the Third Text REtrieval Conference (TREC-3). Gaithersburg, MD: NIST, pp. 109–126.Google Scholar
Rose, S., Engel, D., Cramer, N. and Cowley, W. (2010). Automatic keyword extraction from individual documents. In Berry, M.W. and Kogan, J. (eds) Text Mining: Applications and Theory, Chichester, UK: JohnWiley & Sons Ltd, pp. 1–20.Google Scholar
Salton, G., Yang, C.S. and Yu, C.T. (1975). A theory of term importance in automatic text analysis. Journal of the American Society for Information Science 26(1), 33–44.CrossRefGoogle Scholar
Sarkar, K., Nasipuri, M. and Ghose, S. (2010). A new approach to keyphrase extraction using neural networks. In Computing Research Repository (CoRR), abs/1004.3274.Google Scholar
SEOmoz (2012). The beginners guide to SEO. Technical report.Google Scholar
Shannon, C.E. (1948). A mathematical theory of communication. The Bell System Technical Journal 27(3), 379–423.CrossRefGoogle Scholar
Siddiqi, S. and Sharan, A. (2015). Keyword and keyphrase extraction techniques: A literature review. International Journal of Computer Applications 109(2), 18–23.CrossRefGoogle Scholar
Singhal, A., Kasturi, R., Sharma, A. and Srivastava, J. (2017). Leveraging web resources for keyword assignment to short text documents. In Computing Research Repository (CoRR).Google Scholar
Sparck Jones, K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation 28, 11–21.CrossRefGoogle Scholar
Sterckx, L., Caragea, C., Demeester, T. and Develder, C. (2016). Supervised keyphrase extraction as positive unlabeled learning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 1924–1929.CrossRefGoogle Scholar
Sterckx, L., Demeester, T., Deleu, J. and Develder, C. (2017). Creation and evaluation of large keyphrase extraction collections with multiple opinions. Language Resources and Evaluation, 52, 503–532.CrossRefGoogle Scholar
Tomokiyo, T. and Hurst, M. (2003). A language model approach to keyphrase extraction. In Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment, Sapporo, Japan, pp. 33–40.CrossRefGoogle Scholar
Tsujimura, T., Miwa, M. and Sasaki, Y. (2017). TTI-COIN at SemEval-2017 Task 10: Investigating embeddings for end-to-end relation extraction from scientific papers. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), Vancouver, Canada, pp. 985–989.CrossRefGoogle Scholar
Turney, P.D. (2000). Learning algorithms for keyphrase extraction. Information Retrieval 2(4), 303–336.CrossRefGoogle Scholar
Turney, P.D. (2003). Coherent keyphrase extraction via web mining. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI), Morgan Kaufmann, pp. 434–439.Google Scholar
Unesco. (1975). UNISIST Indexing Principle SC.75/WS/58.Google Scholar
Viera, A. and Garrett, J. (2005). Understanding interobserver agreement: The kappa statistic. Family Medicine 37(5), 360–363.Google ScholarPubMed
Voorhees, E.M. (1999). The TREC-8 question answering track report. In Proceedings of The Eighth Text REtrieval Conference, pp. 77–82.Google Scholar
Voorhees, E.M. (2001). The philosophy of information retrieval evaluation. In Evaluation of Cross-Language Information Retrieval Systems: Second Workshop of the Cross-Language Evaluation Forum, pp. 355–370.Google Scholar
Wan, X. (2007). TimedTextRank: Adding the temporal dimension to multi-document summarization. In Proceedings of the 30th Annual International Conf on Research and Development in Information Retrieval (SIGIR), ACM, Amsterdam, The Netherlands, pp. 867–868.CrossRefGoogle Scholar
Wan, X. and Xiao, J. (2008). Single document keyphrase extraction using neighborhood knowledge. In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI), Chicago, Illinois, pp. 855–860.Google Scholar
Wan, X. and Xiao, J. (2008). Collabrank: Towards a collaborative approach to single-document extraction. In Proceedings of 22nd International Conference on Computational Linguistics, pp. 969–976.CrossRefGoogle Scholar
Wan, X., Yang, J. and Xiao, J. (2007). Towards an iterative reinforcement approach for simultaneous document summarization and keyword extraction. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL). Prague, Czech Republic: ACL, pp. 552–559.Google Scholar
Wang, J., Liu, J. and Wang, C. (2007). Keyword extraction based on Pagerank summarization and keyword extraction. In Proceedings of the 11th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Minin, Nanjing, China, pp. 857–864.CrossRefGoogle Scholar
Yih, W.-t., Goodman, J. and Carvalho, V.R. (2006). Finding advertising keywords on web pages. In Proceedings of the 15th International Conference on World Wide Web (WWW), ACM, Edinburgh, Scotland, pp. 213–222.CrossRefGoogle Scholar
Zesch, T. and Gurevych, I. (2009). Approximate matching for evaluating keyphrase extraction. In Proceedings of the 7th International Conference on Recent Advances in Natural Language Processing, Borovets, Bulgaria, pp. 484–489.Google Scholar
Zhang, C., Wang, H., Liu, Y., Wu, D., Liao, Y. and Wang, B. (2008). Automatic keyword extraction from documents using conditional random fields. Computational Information Systems 4, 1169–1180.Google Scholar
Zhang, K., Xu, H., Tang, J. and Li, J. (2006). Keyword extraction using support vector machine. In Proceedings of the 7th International Conference on Advances in Web-Age Information Management (WAIM), Springer Verlag, pp. 85–96.CrossRefGoogle Scholar
Zhang, F., Huang, L. and Peng, B. (2013). WordTopic-MultiRank: A new method for automatic keyphrase extraction. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, Asian Federation of Natural Language Processing, Nagoya, Japan, pp. 10–18.Google Scholar
Zhang, Q., Wang, Y., Gong, Y. and Huang, X. (2016). Keyphrase extraction using deep recurrent neural networks on Twitter. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Austin, Texas, pp. 836–845.CrossRefGoogle Scholar