Images provide concise representations of design artifacts and emerge as the primary mode of communication among innovators, engineers, and designers. The advanced of Artificial Intelligence tools which integrates image and textual information can significantly support the Engineering Design process. In this paper we create 5 different datasets combining both images and text of patents and we develop a set of text-based metrics to assess the quality of text for multimodal applications. Finally, we discuss the challenges arising in the development of multimodal patent datasets.

Patents are an invaluable source of data that can be beneficial for Engineering Design (ED). Patenting is one of the main means for disclosing the inventive process. For this reason, the description of the problem solved should also be included in any patents.

The ED literature lacks a proper definition of a problem, resulting in a fragmented scenario. Prior studies have employed Text Mining (TM) to extract problems from patents. We argue that TM can assist ED researchers in understanding how problems are articulated in text. Based on the literature, we propose two hypotheses: (1) problem-related text exhibits a negative sentiment polarity compared to other sections of patents; (2) problem-related keywords identified in the literature are predominantly used to describe problems rather than other aspects.

We analyse Japanese patents to validate our hypotheses, since they explicit Problem and Solution in the abstract. Finally, we compare our results with a set of problem-related sentences extracted from USPTO patents.

Our study reveals a higher positive sentiment in problem-related sentences compared to solution-related ones and highlights the inadequacy of using problem-related keywords alone to differentiate between the two.