Industrial evaluation of design heuristics for additive manufacturing

Abstract

Additive manufacturing (AM) has gained prominence over the last 15 years and become a viable manufacturing option. Since there is great industrial interest to implement serial production of products using AM, the education of engineers in design for additive manufacturing (DfAM) is important today. DfAM extends on design for manufacturing to provide knowledge about the new design opportunities enabled by AM. A set of design heuristics for additive manufacturing (DHAM) to assist designers with conceptual-level DfAM has previously been proposed. In this paper, these DHAM are evaluated through workshops with 3 engineering companies and 19 engineers, in which the participants re-design existing company products for AM using the DHAM as an aid, and then give feedback. The DHAM are well received by the workshop participants, and they find that the DHAM are good for teaching others about the capabilities of AM, provide a checklist of items to consider to help avoid oversights, and encourage the generation of new ideas. Criticisms include the number of examples provided and the lack of information about how to implement the ideas concretely. It is also found that the DHAM fulfil 16 of 18 criteria for early design phase DfAM methods, identified in academic literature.

1. Introduction

Additive manufacturing (AM), colloquially known as 3D printing, has gained prominence over the last 15 years and has moved out of the laboratory and into the industrial product stream, becoming a viable manufacturing option for end-use parts (Müller & Karevska 2016; Thompson et al. 2016). This advancement is significant considering the roots of AM in rapid prototyping, where the objective is to fabricate prototypes compared to the current focus on designing and producing functional parts with AM that are used directly within a product or engineering system. New knowledge in design for additive manufacturing (DfAM) is thus needed to achieve this transition successfully. Despite its viability as a new manufacturing option and growth in industry of 28% per year between 2011 and 2015 (Müller & Karevska 2016), at the time of the survey, 76% of companies have no experience with AM (Müller & Karevska 2016). Of these companies, 28% cite a lack of AM Knowledge (AMK) as their main barrier to entry (Müller & Karevska 2016). A 2016 survey found that 38% of companies worldwide wanted to implement the serial production of products using AM by 2021 (Müller & Karevska 2016), indicating that the diffusion of AMK is a pressing issue.

The diffusion of AMK into industry is a particularly difficult problem to address because of the unique capabilities afforded by AM when compared to conventional manufacturing techniques, namely shape, material, hierarchical, and functional complexity (Gibson, Rosen, & Stucker 2015). Particularly due to the shape complexity afforded by AM, many of the geometric constraints of conventional manufacturing are obsolete when designing for AM. Designers experienced with conventional manufacturing rules have difficulty breaking out of their current mind-set and require assistance to do so successfully (Seepersad 2014, Valjak et al. 2020, Prabhu et al. 2020a, Dordlofva & Törlind 2020).

Design for additive manufacturing (DfAM) education is a particularly pressing issue due to the large number of companies that would like to implement serial AM production. What makes this even more challenging is the fact that many future users are already in industry and may not have the time or capacity to participate in a lengthy training course, and they did not attend any courses in AM during their studies, as these courses have only started to emerge in the last 5 years. Additionally, novice designers, particularly those that are still in education, also need assistance to learn about the technology, but the education needs to be incorporated into the existing design and manufacturing curriculum, which can already be quite full (Borgianni et al. 2019). Subsequently, the field of DfAM has developed to address the needs of both experienced and novice designers (Thompson et al. 2016). Both industry and academia are active in this area.

As part of this movement, in 2009, the United States National Science Foundation and the United States Office of Naval Research developed a Roadmap for AM with specific goals for AM research over the subsequent 10 years. These goals included developing DfAM methods to help designers. As of 2014, a review of existing work revealed that although much progress in the areas of optimisation, computational tools, and restrictive design guidelines had been made, there was still a distinct lack of methods to assist designers in exploring the AM design space during the conceptual design phase (Bourell, Rosen, & Leu 2014). With the goal of addressing this lack of methods to assist designers during the early phases of the design process, several researchers have proposed design aids.

The authors have previously proposed a set of 25 design heuristics for additive manufacturing (DHAM) in a card-based format (Blösch-Paidosh, Ahmed-Kristensen, & Shea 2019) and with accompanying AM objects (Blösch-Paidosh & Shea 2021). The DHAM, DHAM cards, and DHAM objects have already been found to increase creativity and the inclusion of AM concepts in controlled novice user studies (Blösch-Paidosh et al. 2019; Blösch-Paidosh & Shea 2021). However, the DHAM have not been evaluated in industry-based design scenarios (i.e., in a nonfabricated design scenario) or with expert designers, and neither has open-ended feedback about the DHAM been gathered. These gaps are now addressed by conducting a series of workshops and gathering industry feedback on the DHAM, DHAM cards and DHAM objects.

It is hypothesised that similar increases in creativity and the inclusion of AM concepts will be seen in industry and workshop environments (H1). This is based on the assumption that the DHAM should perform similarly when compared to the controlled experiments. This was found by Yilmaz et al. (2013a), when comparing the use of their general design heuristics by novices and professionals. It is also hypothesised that the combination of the cards and objects will be positively received by the participants, as the combination was found to be more effective in the controlled experiments (H2; Blösch-Paidosh & Shea 2021). Finally, it is hypothesised that the industry participants will find that the DHAM fulfil all of the identified criteria for early-phase DfAM methods, except providing restrictive DfAM information (see Section 2) as they were not originally developed to provide such information (H3; Blösch-Paidosh & Shea 2017).

This paper first provides background information on relevant topics. Then, the workshop procedure and evaluation method are described in Section 3. Additionally, this section quickly reviews the DHAM cards and objects. Next, the results of the workshops are presented followed by a discussion, including a discussion of future work. Finally, the paper concludes with a summary. The contributions of this work are the evaluation of the DHAM in an industrial setting and a questionnaire to help evaluate and compare early design phase DfAM methods. By evaluating this early-phase DfAM method in an industry setting, it makes it one of the few methods that has been evaluated with both novices and experts, and in both workshops and randomised controlled studies.

2. Background

One approach to providing early-phase AM Knowledge, and the approach chosen by the authors, is through design heuristics or design principles. A heuristic is defined as ‘a context-dependent directive, based on intuition, tacit knowledge, or experiential understanding, which provides design process direction to increase the chance of reaching a satisfactory but not necessarily optimal solution (Fu, Yang, & Wood 2016)’, whereas a principle is defined as ‘a fundamental rule or law, derived inductively from extensive experience and/or empirical evidence, which provides design process guidance to increase the chance of reaching a successful solution (Fu et al. 2016)’. Design heuristics and design principles are similar to each other, but heuristics typically use less empirical evidence for derivation, are more general (i.e., higher-level), are less formal, and are usually more prescriptive as opposed to descriptive (Fu et al. 2016). Design heuristics and principles are a popular way of communicating design guidance because they are easy to use and quickly communicate the necessary information (Yilmaz & Seifert 2010; Fu et al. 2016; Roy & Warren 2019) and are prevalent across a variety of design domains (Roy & Warren 2019). One of the most well-known sets of design heuristics are the 77 design heuristics developed by Yilmaz et al. (2016). They have been validated extensively with novice and expert designers (Yilmaz, Seifert, & Gonzalez 2010; Christian et al. 2012; Daly et al. 2012; Yilmaz et al. 2013a, b; Kramer et al. 2014). Design heuristics and principles are also a popular way to communicate the design possibilities surrounding AM (Perez et al. 2015; Perez 2018; Yang, Page, & Zhao 2018; Schumacher et al. 2019; Valjak & Bojčetić 2019; Watschke et al. 2019).

Early design phase DfAM often has a strong opportunistic DfAM component (information about the unique capabilities of AM; Gibson et al. 2015) as opportunistic design support assists designers in expanding their AM design horizons, something that is key in the early design phases (Ulrich & Eppinger 2008). However, some early-phase DfAM methods also include restrictive DfAM information (information about the restrictions of the AM process; e.g., Laverne et al. 2015; Weiss, Binz, & Roth 2015; Kumke et al. 2017; Watschke et al. 2017; Perez 2018) to help users make the transition from the early design phases into the later phases of the design process (e.g., detailed design) without having to use or learn an additional method. The usefulness of these different types of design support in the early phases has not been studied in detail, and there are varying opinions on the inclusion of restrictive DfAM in early-phase DfAM methods (see Table 1). DfAM methods targeted specifically at detailed design also exist (e.g., The DfAM Worksheet; Booth et al. 2017), but are not included in this literature review as they are not focussed on a design phase under investigation in this work.

Table 1. Literature survey of important characteristics of early-phase DfAM methods

Abbreviations: AM, additive manufacturing; DfAM, design for additive manufacturing.

The evaluation of early design phase DfAM methods in an industry setting (Kumke et al. 2017; Dordlofva & Törlind 2020; Prabhu et al. 2020a) or using expert designers (Laverne et al. 2015; Lauff et al. 2019; Fillingim et al. 2020; Prabhu et al. 2020a) are few, and the descriptions of the results and feedback are usually vague and often only involve a short survey about the method following the workshop or user study. When a post-survey is used, the questions asked are also variable between studies. Sometimes they focus on helpfulness (Kumke et al. 2017), amount of information provided (Kumke et al. 2017), or ability of the participants to understand (Fillingim et al. 2020), and sometimes they ask how well a method fulfils early-phase DfAM method needs (Lauff et al. 2019). Additionally, one study has also looked at designers’ self-efficacy following the workshop (Prabhu et al. 2020a). The myriad of ways in which the methods are evaluated makes it difficult to compare the results and abilities of one method to another. A standard set of criteria should be set to ease this comparison in the future.

What exact qualities early design phase DfAM methods should embody are not clearly stated in literature. To determine what qualities are preferable, 19 academic works published between 2011 and 2019 (see Table 1) are examined for information to determine what are generally considered important characteristics in early-phase DfAM methods. Thirty-two different characteristics are found to be considered important in the literature examined; however, only 18 characteristics are consistently identified by researchers from multiple research groups. Table 1 lists these 18 characteristics and provides references to the supporting literature. Although much additional research in this area has been published since the survey was conducted in 2019 (e.g., Perez et al. 2019; Prabhu et al. 2019; Schumacher et al. 2019; Valjak & Bojčetić 2019; Watschke et al. 2019; Wiberg, Persson, & Ölvander 2019; Dordlofva & Törlind 2020; Prabhu et al. 2020b; Valjak et al. 2020), it was conciously decided not to update Table 1, because of the results’ direct connection to the development of the questionnaire used to evaluate the method (see Sections 3–3.3).

Of the 18 characteristics identified in Table 1, some have very similar wording, which may be confusing. In order to clarify the meaning of some of the similar-sounding characteristics, we include a brief description for selected characteristics:

• • Is easy to learn how to use: learning how to use the method is easy.

• • Is easy to use: once one knows how to use the method, it is easy to apply.

• • Structured in an easy to understand way: the information supplied via the method is structured in a way that is easy to understand.

• • Offers a comprehensive view of the capabilities of AM: a comprehensive view of the capabilities of AM is offered, that is, it does not focus on just one area.

• • Offers opportunistic AM information: opportunistic information about AM is supplied (i.e., information about the capabilities specific to early phase DfAM, e.g., incorporation of metamaterials).

• • Offers restrictive AM information: restrictive information about AM processes is supplied (e.g., design guideline for wall thickness or overhang angle).

3. Method

The method chosen to evaluate the DHAM in industrial design scenarios and with expert designers is a series of workshops with three engineering companies. This evaluation method and results reporting is based specifically on the work of Hamraz & Clarkson (2015), but workshops as a method for evaluation of design methods in industry are commonly reported (Laverne et al. 2015; Kumke et al. 2017; Rigger et al. 2019; Dordlofva & Törlind 2020; Prabhu et al. 2020a). This evaluation method is chosen because it focusses on evaluation of the method and not of the individual results generated by each company. It is also flexible with regard to the number of participants and the artefacts chosen.

In contrast to Hamraz & Clarkson (2015), who carried out the evaluation of an engineering design method with 1 engineering company and 10 engineers, we report the evaluation of the DHAM with 3 companies and 19 engineers in total. It is not the goal of this research to make quantitative statements about the use of DHAM in industrial situations, but rather to take more of a case-study approach (Yin 2014), which is typical for assessments of design methods in industry (Albers et al. 2019; Bertoni & Bertoni 2019; Sjögren et al. 2019; Stenholm, Catic, & Bergsjö 2019; Dordlofva & Törlind 2020). In this section, the DHAM are first briefly reviewed. Then, the workshop agenda is described. Finally, the feedback evaluation method is detailed.

3.1. Design Heuristics for Additive Manufacturing

Twenty-five DHAM were derived through the analysis and synthesis of 275 AM artefacts stemming from literature, industry sources, the popular media and hobby websites (Blösch-Paidosh & Shea 2019; Blösch-Paidosh & Shea 2021). These 25 DHAM can be broken-up into eight different categories: part consolidation, customisation, convey information, material, material distribution, embed-enclose, lightweight and reconfiguration. They are listed in Table 2.

Table 2. The 25 DHAM organised by category

Later, a set of cards was developed to communicate each heuristic (Blösch-Paidosh et al. 2019; Blösch-Paidosh & Shea 2021). The cards are multimodal and each contains seven different pieces of information. An example card with each of these pieces of information labelled is available in Figure 1. Additionally, a physical object was created to accompany each of the 25 DHAM cards (Blösch-Paidosh & Shea 2021). An example object corresponding to the card in Figure 1 can be seen in Figure 2 and a selection of the accompanying 25 DHAM objects can be seen in Figure 3. The full set of the 25 DHAM cards and objects are available to view and download at https://edac.ethz.ch/Research/Design-Heuristics-AM.html and their derivation is described in more detail in Blösch-Paidosh & Shea (2021).

Figure 1. Example of a Design Heuristic for Additive Manufacturing (DHAM) card. The front of the card is at the top and the back of the card is on the bottom.

Figure 2. Corresponding Design Heuristic for Additive Manufacturing (DHAM) object for the DHAM card shown in Figure 1.

Figure 3. A selection of the 25 Design Heuristics for Additive Manufacturing (DHAM) objects.

The DHAM cards were inspired by the design heuristics cards of Leahy et al. (2018). They differ mainly from their closest AM-specific relative, the design principles for AM cards of Perez (2018), in that they focus solely on opportunistic DfAM and that the DHAM draw from a wider derivation pool including examples from industry, academia, and the media. Additionally, the combination of the cards and objects offered by the DHAM allow the designer to not just read an instructional text and look at images, but to also interact with a physical representation of the heuristic (Blösch-Paidosh & Shea 2021).

3.2. Workshop Description

Three workshops are performed with three different industrial partners over a period of 6 months, between March and August 2019. The companies are active in the automotive, manufacturing machinery, and construction machinery industries. At each company between 5 and 8 employees with technical backgrounds take part in the workshops, for a total of 19 participants. The participating companies and employees are described in Tables 3 and 4, respectively. The self-reported DfAM experience is taken from a DfAM knowledge survey filled-out by each participant. It is based on the survey developed by Prabhu et al. (2018). Neither the workshop facilitators (researchers), the companies, nor the participants are compensated monetarily for their time or participation, and each of the workshops takes place at the participating company during normal working hours. In exchange for conducting the workshops, the workshop facilitators receive feedback on the method and the allowance to publish it as compensation for their time.

Table 3. Company participants in industry workshops

^a No Brainwriting 6-3-5 performed due to each subgroup working on different artefacts.

Table 4. Participants in the industry workshops

The artefacts used during the workshops as a design task are actual company products that are defined by each company prior to the workshop, and details about the chosen artefacts and the changes made to them are confidential and cannot be reported here, as per agreements with the participating companies. However, a short description of the artefacts worked-on by each company is available in Table 3. They were all part of Research and Development projects. The advantage of carrying out the study this way, rather than giving the same academic example to all three companies, is that the method can be tested in the context of each company, across different industry sectors, and on a product that the company and participants have direct interest and motivation to improve.

Each workshop lasts approximately 3 hours and follows the outline shown in Figure 4. The workshop begins with an introduction of the facilitator(s) and the participants to each other, followed by an overview of the agenda. Then, the workshop participants are asked to describe the current state of the product(s) or concept(s) that they would like to work on during the workshop. This is an important step, as it was found during previous failed user studies, that it is difficult to apply the DHAM without a well-defined artefact as a starting point. Then, a discussion of the current design practices at the company is led by the facilitator(s), including their general design process and how they currently incorporate DfAM into their design process. The goal of these workshops is not to gather data on DfAM or development practices in industry, but to obtain industrial feedback on the DHAM cards and objects. This discussion is included so that the facilitator knows the context of the statements made by the participants while giving feedback on the DHAM. Next, the benefits of using design heuristics (not just the DHAM) are explained to the participants along with an example DHAM card and object. The participants are also provided with some example applications of the DHAM. Following this, the participants are split into two equal subgroups and the ideation session begins, during which the participants use the DHAM as an aid to help redesign their chosen product(s) for AM. The results of the DfAM knowledge survey are not used to divide the participants into groups based on their experience during the workshop. Instead, the participants are allowed to self-divide. Group instead of individual ideation is used to keep the workshop/group brainstorming feeling. The users are split into subgroups to keep the teams small and allow each of the participants to fully participate.

Figure 4. Industry workshop agenda outline.

During the ideation session, each subgroup has access to one complete set of the DHAM cards and objects. The cards are laid out in numeric order in a 5 × 5 grid on a table with the object on top of the corresponding card. The ideation session lasts about 1 hour and is done in a group. If both subgroups are working on the same product or concept, each subgroup freely brainstorms for 45 minutes. Then, the ideas generated by one subgroup, in the form of annotated sketches, are passed to the other and each subgroup uses the other subgroup’s ideas as the basis for a Brainwriting 6-3-5 session (Rhorbach 1969), adjusted to the number of ideas and participants. This second portion of the ideation session lasts no more than 15 minutes. However, if each subgroup is working on a different product or concept, the second portion of the ideation session is eliminated and the first portion is extended to 1 hour. The companies, which do not complete a 6-3-5 session are indicated in Table 3.

There are many different creative ideation methods aside from Brainstorming including SCAMPER (Eberle 1996), the Gallery method (Vangundy 1988), Synectics (Gordon 1961), Six Thinking Hats (de Bono 1985), and Brainwriting 6-3-5 that each have their own strengths. In order to keep the focus on the DHAM, it is not desired to introduce another method, which would require lengthy explanation. Additionally, due to the limited time available for the second ideation method, some of the aforementioned methods are not suitable. This leads to the selection of Brainwriting 6-3-5, because it is simple, quickly explained, and highly flexible depending on the number of participants and ideas generated during the initial Brainstorming session.

Following the ideation session, the subgroups come back together to discuss the generated ideas as a group. Once the participants are finished discussing the ideas they generated, the participants are asked to evaluate the DHAM in various ways. In a focus group-style session (Morgan, Krueger, & King 1998), the participants are asked to give feedback on the method. They are asked about if they could see themselves using this method in conjunction with future development projects and how it would or could fit into their existing development process. Additionally, they are encouraged to give direct feedback on the DHAM cards and objects, such as, what they liked and disliked about them, what was possibly confusing and what they would change about them if given the chance. They are also encouraged to provide any open-ended feedback about the DHAM. Throughout this data collection session and the session about the company’s design process, notes on the responses are taken by the workshop facilitators. None of the companies agrees to the sessions being recorded for later analysis, so no transcript analysis is possible. Finally, each participant is asked to fill out a questionnaire in which they rate the suitability of the DHAM as an early-phase DfAM method. This questionnaire and its derivation are described in the next section.

3.3. Evaluation Method

The evaluation of the workshop data consists of two main parts: analysis of the oral feedback and analysis of the early-phase DfAM questionnaire. The analysis of the oral feedback is quite simple. Immediately following each workshop, the notes taken by the workshop facilitator(s) are transferred to a spreadsheet, where each comment, its provider, and any necessary context are recorded. Observations made by the facilitator(s) are also recorded. Then, the comments are clustered based on their content and relevance to certain topics. As more workshops are conducted, the comments are combined together in categories, and patterns in the responses are examined to draw conclusions (Yin 2014).

A questionnaire to determine if the DHAM fulfil the qualities desirable in early design phase DfAM methods is derived based on the 18 characteristics designated as important for early-phase DfAM methods based on the literature survey in the background section (see Table 1). A statement is created for each of the 18 characteristic deemed important by literature in which the DHAM are stated as fulfilling that characteristic (18 total statements). The questionnaire asks each participant to rate their agreement with the statement on a 5-point Likert scale (Strongly Disagree to Strongly Agree). Although the questionnaire is designed with the DHAM in focus, the text can be easily edited to accommodate any other early-phase DfAM method by replacing ‘DH for AM’ with the name of the chosen method and is thus suitable for the general evaluation of early-phase DfAM methods. The designed questionnaire is available in Figure 5. For analysis purposes, the Likert scale is transformed into a numeric scale (−2 to 2, with −2 = Strongly Disagree and 2 = Strongly Agree) to help visualise tendencies in the responses. A −2 to 2 conversion scale is chosen so that a neutral response by the participant (Neither Agree nor Disagree) corresponds with a neutral value (0) on the scale.

Figure 5. Early-phase design for additive manufacturing (DfAM) questionnaire based on the criteria of literature. Statements follow the same order as the characteristics listed in Table 1.

4. Results

The results of the oral feedback analysis are presented in the following tables. The analysis is split-up into multiple tables that each focus on a different aspect of the responses: the praised advantages (Table 5), suggested improvements (Table 6), and suggestions for use (Table 7). Each statement is labelled with the participant that stated it as indicated in Table 4. Some statements, however, were generally agreed on by all of the participants at a particular company. In these cases, the company is indicated instead of a specific participant.

Table 5. Praised advantages of the DHAM with supporting comments (continued on the following page)

Abbreviations: AM, additive manufacturing; DHAM, Design Heuristics for Additive Manufacturing.

Table 6. Suggested improvements to the DHAM with supporting comments (continued on the following page)

Abbreviation: DHAM, Design Heuristics for Additive Manufacturing; FDM, fused deposition modelling.

Table 7. Suggestions for use of the DHAM with supporting comments

Abbreviation: DHAM, Design Heuristics for Additive Manufacturing.

The results of the questionnaire are available in Figure 6. The participants generally agree with all but two statements: DHAM provide me with information about the restrictions of AM and DHAM are AM-process and material independent. For the former of these statements, they do give a mean negative rating, but for the latter, they are slightly positive. However, the mode responses are neither agree nor disagree. Additionally, the maximum rating for all but the aforementioned statements is strongly agree and the lowest rating for 10 of the 18 statements is neither agree nor disagree. The mode responses for 15 of the 18 statements are either agree or strongly agree. For the other three statements, it is neither agree nor disagree. None of the mode responses is negative.

Figure 6. Summary of the questionnaire results, n = 19.

5. Discussion

The discussion is organised into five sections: praised advantages, suggested improvements, suggestions for use, fulfilment of early-phase DfAM characteristics, and comparison to novice designer studies.

5.1. Praised Advantages

Overall, the reception of the DHAM by the industry participants is overwhelmingly positive. In fact, at one of the companies, one of the participants is quite sceptical of the DHAM at the beginning of the ideation session, and implies he could only apply the DHAM in a trivial manner, but by the end of the ideation session and during the discussion, he is one of the biggest supporters of the method at the workshop (A.6). The advantages of the DHAM as described by the participants are four-fold: they are good for teaching others about the capabilities of AM, they provide a checklist of things to consider to help avoid oversights, the inclusion of the DHAM objects helps to improve their understanding of the DHAM, and they encourage the generation of new ideas. The generation of new and a wider variety of ideas is something that is empirically mirrored in the previous novice user studies (Blösch-Paidosh & Shea 2021), so it is beneficial to have confirmation of this from an industrial perspective. Therefore, H1 is confirmed. A limitation on these results is that they could only be qualitatively confirmed in industry, and not also quantitatively as in previous research with novice users. Therefore, it would be beneficial in the future to confirm them in a controlled experiment using industry professionals. However, getting the support of enough industry professionals and developing a relevant and publishable design task with them is difficult.

The importance of having physical examples for each DHAM is repeatedly expounded on by the participants. Their praise of the DHAM objects falls mainly into two categories: quicker understanding of the DHAM and the ability to ‘play’ with a real example. The DHAM cards are in English, but the objects are language-less. Although all of the engineers that participate in this study speak English well, it is the native language of only one of them. This may factor into why they gravitate toward the DHAM objects: when the objects are there to supplement the information on the cards, language is less of a factor. This indicates that the DHAM may be useful in cross-cultural design situations and that significant changes to the DHAM are not needed for different cultures. Additionally, the ability to ‘play’ with an object may help the user of the DHAM understand the concept faster (Spallek & Krause 2017) and possibly help them retain the information longer (Golinkoff, Hirsh-Pasek, & Singer 2006). This is also supported by some of the statements in Table 7, in which the users indicate their ability to quickly recall the objects in the future through a small reminder, and in Table 5, in which some of the participants specifically mention liking to play with the objects. Therefore, H2 is considered confirmed.

The statements indicating that the DHAM are useful for teaching others, and implicitly oneself, about the unique capabilities of AM are direct support of the stated goal of the development of the DHAM, indicating that this goal is at least qualitatively achieved. Additionally, the participants talk about using the DHAM as a comprehensive checklist about what is possible with respect to AM, something that is mirrored in the questionnaire responses, where the lowest rating given is neither agree nor disagree. This indicates that from the industry perspective, the DHAM are providing a comprehensive view of the possibilities of AM.

5.2. Suggested Improvements

In addition to the praise, there is a variety of criticisms about the DHAM as well. In contrast to the praised abilities, most of the critical points stem from one or two comments. However, there are three criticisms that repeatedly appear. The first repeated criticism is that the real-world examples are on the back of the cards instead of the front. Although the contents of the cards, both back and front, are explained before the ideation session begins, some of the participants never turn over the cards during the session, and some do not even realise that there is a backside, despite the explanation. This may partially stem from how the cards and objects are lain out on the table during the session; because the object sits on top of the card, the participants may be discouraged from picking-up and flipping over the card. However, as the laying-out of the cards is positively commented on by the participants, it is not suggested to change this layout method. Additionally, each of these workshops is the first experience that the participants have with using the DHAM. As they use them more often, they may grow more familiar with the DHAM, which could alleviate this problem. This familiarity may also address the feeling that participant A.5 has about accidently breaking one of the objects.

Related to the concern about breaking the objects, is the concern that the participants will not be able to produce their own copy of the objects because of a lack of the necessary AM machines. As described by Blösch-Paidosh & Shea (2021), effort was taken to make as many of the objects as possible manufacturable with a dual-extruder desktop FDM printer exactly to address this accessibility concern. However, due to the nature of some objects, for example, inclusion of multiple colours, adaption was not possible for all objects. However, any of these special objects should be manufacturable through online 3D printing services. The perceived difficulty around the procurement of a set of DHAM objects may also be overcome by reducing the number of DHAM object sets needed at one company by distributing digital copies of the cards and objects to whomever would like them, but keeping only one or two of the physical object sets in a central location such as the ideation room or MakerSpace, as per the suggestion of the participants at Company B.

The second repeated criticism relates to the number and types of examples provided. Some of the participants, particularly those at Company C, would like to see more examples directly connected to specific AM-processes and materials. This desire to see more examples is a sentiment echoed by the novice designers of the previous DHAM user studies (Blösch 2020). As this feedback is coming from both novices and experts, some more examples could be added, but maximum one or two per heuristic, because if too many examples are added it becomes a different type of design support method, e.g., a design catalogue, like Weiss et al. (2015), instead of heuristics. Another option would be to include access to a database similar to that of Bin Maidin (2011) that connects each of the DHAM to the AM artefacts used to derive it, so as to give the users another way to access more examples if desired, but keep the number of immediate examples low.

Two of the companies (Companies B and C) are concerned about the lack of metal objects and metal examples, and during the ideation session, some participants sometimes seem hung-up on the material of the objects. Company B mainly manufactures products out of metal and the participants at Company C work specifically on metal products produced by the company, whereas the participants at Company A work regularly with both polymers and metal. The participants that work regularly with both materials may be better at transferring the principles indicated in the objects to different materials than those that work primarily with one material. This feeling may also be part of the reason that the statement DHAM are AM-process and material independent receives the second lowest rating. Although all of the DHAM are broadly applicable to different AM-processes and materials and include real-world examples utilising various materials and processes, the fact that all of the objects are fabricated using polymers may be biasing the responses of the participants. As one of the main goals of the DHAM is to be AM-process and material independent, this effect should be investigated in future research. This is the first indication that H3 is not fully confirmed.

The third repeated criticism is that the DHAM do not provide the user with specific next steps for implementing their AM-enabled concepts. The goal of the DHAM is to provide inspiration, not implementation instructions (Blösch-Paidosh & Shea 2019). However, if the users need assistance in taking the next steps with regard to implementing their ideas, perhaps some sort of support system for this needs to be considered. One option would be to include some suggested implementation resources such as software, complementary methods or papers as references on the back of each card. A second option would be to integrate the DHAM into a DfAM-supplemented design process framework in which methods developed by various researchers and industrialists could be recommended for use during each design phase, as an extension of the work of Pradel et al. (2018).

5.3. Suggestions for Use

Several recommendations for future use of the DHAM are also made by the participants. Generally, they like the 1-hour ideation session format and that the DHAM cards and objects are spread out on the table for them to have a good overview. Additionally, although they could see themselves using the DHAM as a sort of checklist for AM alone at their desk, they see their primary function in a group brainstorming session. They also intimate that they do not see these as useful for a blue-sky development project, as one needs a starting design or concept to which one can apply the DHAM, i.e. something to re-design. This statement is consistent with the idea that design heuristics are concept modifiers, not concept generators. This is a limitation on the usefulness of the DHAM. However, according to Jensen (1999), ‘80% of all designs are adaptive or variant, that is, a design based upon a well-known solution principle’. This is the vast majority of all design, which therefore makes them a relevant and useful design method despite the limits on their use.

Related to the types of applicable design scenarios for DHAM, Company C works on a single-part product during their workshop, but thinks that the DHAM would be more useful when developing concepts for some of their other multi-component products. In contrast to this statement, Company A, who reacts most positively to the DHAM of all of the companies involved in the workshops, also applies the DHAM to a single part product. Both of the design tasks previously employed in the controlled user studies with novice designers are multi-component products (Blösch-Paidosh et al. 2019; Blösch-Paidosh & Shea 2019; Blösch-Paidosh & Shea 2021). Conflicting evidence surrounding this statement means that no definitive recommendation can be made.

Statements from the participants also indicate that their companies could be hesitant to regularly implement the DHAM in design sessions and implement AM in general. Two suggestions are provided by the companies to address this. The first is to do some trial workshops using business unit projects and not only Research and Development projects to demonstrate how effective use of the DHAM can be in order to get buy-in from management and designers. The second recommendation, is to start attaching existing, company specific (i.e., internal) examples to each of the relevant heuristics so that users do not only have the general examples to work from, but also see how their company has applied them. Doing this could help customise the DHAM to each company that integrates it into their design process, which could increase their willingness to use it and, potentially, the effectiveness.

A final important note, is that since the conclusion of the workshops, all three companies that participated in the workshops have gone on to acquire a set of the DHAM cards and objects to use during future brainstorming sessions. Additionally, one of the companies used the objects to help instruct their employees in the capabilities of AM during a large company workshop.

Based on the feedback of the participants during the workshops, the following recommendations can be made for using the DHAM in future ideation workshops:

1. (i) Use an existing product that the participants already know as a starting point (i.e., no blue-sky projects).

2. (ii) Previous experience with the DHAM is not necessary for a successful workshop, although a basic awareness of AM is beneficial.

3. (iii) Laying the DHAM cards out in a gallery allows for a collaborative approach.

4. (iv) It is highly preferable to have a printed set of the DHAM objects available for the session.

5. (v) If the budget permits it, make sure some of the DHAM objects are printed in something other than polymers, for example, metal, to show the material independence.

6. (vi) If the company or group already has some existing examples of AM applications in their repertoire, the DHAM cards can be augmented to use these as examples.

7. (vii) The workshop does not need to be long; 1-hour of ideation time appears to be sufficient.

5.4. Fulfilment of Early-Phase DfAM Characteristics

According to the results of the questionnaire, the DHAM fulfil ¹ 12 of the 18 characteristics identified as necessary for early-phase DfAM methods, and partially fulfil ² 4 of the 18. That the vast majority of the mode responses are positive and many of the minimum response values are neutral indicates general agreement on the ability of the DHAM to fulfil the 18 characteristics of early-phase DfAM methods. As previously mentioned, it is concerning that it is not clear that the DHAM are process and material independent. However, the fact that the DHAM do not seem to fulfil the criteria of providing information about the restrictions of AM is not surprising, as this was never the intent of the DHAM (Blösch-Paidosh & Shea 2017; Blösch-Paidosh & Shea 2019). From the results of this questionnaire, it can be said that the DHAM cards and objects generally fulfil criteria for early design phase DfAM methods as defined by literature; however, H3 cannot be fully confirmed.

5.5. Comparison to Novice Designer Studies

When compared to the previous controlled user studies with novice designers, the following is found:

1. (i) The DHAM are an effective tool for both novice and experienced designers.

2. (ii) The DHAM are effective not just in fabricated design situations, but also in industry-derived design scenarios.

3. (iii) Working in teams, novice and experienced designers use the DHAM cards in a similar fashion.

4. (iv) Evidence from industry qualitatively confirms the numeric increases in creativity metrics found in controlled studies with novices.

6. Conclusion

The goal of this paper is to evaluate the DHAM card and objects (Blösch-Paidosh & Shea 2021) in an industrial setting with expert designers. A total of 3 workshops are conducted with 19 different participants at 3 companies from 3 different engineering industries. Verbal feedback on the method is collected from the participants and analysed by clustering similar responses together to determine the praised advantages, suggested improvements and suggestions for use of the DHAM. Additionally, the participants answer a questionnaire to determine how well the DHAM fulfil the characteristics desirable in early-phase DfAM methods. The verbal response to the DHAM is overwhelmingly positive, however, some suggested improvements provided by the participants should be considered in any further development. Additionally, the DHAM are considered to embody 16 of the 18 DfAM community-defined characteristics for early-phase DfAM methods. The failure to fulfil one of these criteria (providing restrictive DfAM information) is expected, but one is not (AM process and material independence). The results confirm two of the three hypotheses, and partially confirm the third.