Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-27T15:16:21.940Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing text-image patent datasets with text-based metrics for engineering design applications

Published online by Cambridge University Press:  16 May 2024

Marco Consoloni ,
Vito Giordano  and
Gualtiero Fantoni
Marco Consoloni*
Affiliation:
University of Pisa, Italy Business Engineering for Data Science Lab (B4DS), Italy
Vito Giordano
Affiliation:
University of Pisa, Italy Business Engineering for Data Science Lab (B4DS), Italy
Gualtiero Fantoni
Affiliation:
University of Pisa, Italy Business Engineering for Data Science Lab (B4DS), Italy
*
marco.consoloni@phd.unipi.it

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Images provide concise representations of design artifacts and emerge as the primary mode of communication among innovators, engineers, and designers. The advanced of Artificial Intelligence tools which integrates image and textual information can significantly support the Engineering Design process. In this paper we create 5 different datasets combining both images and text of patents and we develop a set of text-based metrics to assess the quality of text for multimodal applications. Finally, we discuss the challenges arising in the development of multimodal patent datasets.

Keywords

engineering designproduct developmentnatural language processing (NLP)image datapatent analysis
Type
Artificial Intelligence and Data-Driven Design
Information
Proceedings of the Design Society , Volume 4: DESIGN 2024 , May 2024 , pp. 1969 - 1978
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2024.

References

Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., & Herrera, …, F. (2020). Explainable Artificial Intelligence (XAI): "Concepts, taxonomies, opportunities and challenges toward responsible AI". Information fusion, 58, 82-115. https://doi.org/10.1016/j.inffus.2019.12.012CrossRefGoogle Scholar
Atherton, M., Jiang, P., Harrison, D., & Malizia, A. (2018). "Design for invention: annotation of functional geometry interaction for representing novel working principles." Research in Engineering Design, 29, 245-262. https://doi.org/10.1007/s00163-017-0267-2CrossRefGoogle ScholarPubMed
Chiarello, F., Belingheri, P., & Fantoni, G. (2021). "Data science for engineering design: State of the art and future directions." Computers in Industry, 129, 103447. https://doi.org/10.1016/j.compind.2021.103447CrossRefGoogle Scholar
Giordano, V., Puccetti, G., Chiarello, F., Pavanello, T., & Fantoni, G. (2023). "Unveiling the inventive process from patents by extracting problems, solutions and advantages with natural language processing." Expert Systems with Applications, 229, 120499. https://doi.org/10.1016/j.eswa.2023.120499CrossRefGoogle Scholar
Jee, J., Park, S., & Lee, S. (2022). "Potential of patent image data as technology intelligence source." Journal of Informetrics, 16(2), 101263. https://doi.org/10.1016/j.joi.2022.101263CrossRefGoogle Scholar
Jiang, S., Luo, J., Ruiz-Pava, G., Hu, J., & Magee, C. L. (2021). "Deriving design feature vectors for patent images using convolutional neural networks." Journal of Mechanical Design, 143(6). https://doi.org/10.1115/1.4049214CrossRefGoogle Scholar
Jiang, S., Sarica, S., Song, B., Hu, J., & Luo, J. (2022). "Patent data for engineering design: A critical review and future directions." Journal of Computing and Information Science in Engineering, 22(6), 060902. https://doi.org/10.1115/1.4054802CrossRefGoogle Scholar
Kucer, M., Oyen, D., Castorena, J., & Wu, J. (2022). "DeepPatent: Large scale patent drawing recognition and retrieval." In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2309-2318).Google Scholar
Lin, W., Yu, W., & Xiao, R. (2023). "Measuring Patent Similarity Based on Text Mining and Image Recognition." Systems, 11(6), 294. https://doi.org/10.3390/systems11060294CrossRefGoogle Scholar
Pustu-Iren, K., Bruns, G., & Ewerth, R. (2021). "A multimodal approach for semantic patent image retrieval." In PatentSemTech 2021-Patent Text Mining and Semantic Technologies, July 15th 2021, online (Vol. 2909). Aachen, Germany: RWTH Aachen. https://doi.org/10.34657/6842Google Scholar
Rao, R., Rao, S., Nouri, E., Dey, D., Celikyilmaz, A., & Dolan, B. (2020). "Quality and relevance metrics for selection of multimodal pretraining data." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 956-957). https://doi.org/10.1109/CVPRW50498.2020.00486CrossRefGoogle Scholar
Sarica, S., & Luo, J. (2021). "Stopwords in technical language processing. " Plos one, 16(8), e0254937. https://doi.org/10.1371/journal.pone.0254937CrossRefGoogle ScholarPubMed
Sarica, S., Luo, J., & Wood, K. L. (2020). "TechNet: Technology semantic network based on patent data." Expert Systems with Applications, 142, 112995. https://doi.org/10.1016/j.eswa.2019.112995CrossRefGoogle Scholar
Vrochidis, S., Moumtzidou, A., & Kompatsiaris, I. (2012). "Concept-based patent image retrieval." World Patent Information, 34(4), 292-303. http://dx.doi.org/10.1016/j.wpi.2012.07.002CrossRefGoogle Scholar
Vrochidis, S., Papadopoulos, S., Moumtzidou, A., Sidiropoulos, P., Pianta, E., & Kompatsiaris, I. (2010). "Towards content-based patent image retrieval: A framework perspective." World Patent Information, 32(2), 94-106. https://doi.org/10.1016/j.wpi.2009.05.010CrossRefGoogle Scholar