Iterative value models generation in the engineering design process

Abstract

Value models are increasingly discussed today as a means to frontload conceptual design activities in engineering design, with the final goal of reducing cost and rework associated with sub-optimal decisions made from a system perspective. However, there is no shared agreement in the research community about what a value model exactly is, how many types of value models are there, their input–output relationships and their usage along the engineering design process timeline. Emerging from five case studies conducted in the aerospace and in the construction equipment industry, this paper describes how to tailor the development of value models in the engineering design process. The initial descriptive study findings are summarized in the form of seven lessons learned that shall be taken into account when designing value models for design decision support. From these lessons, the paper proposes a six-step framework that considers the need to update the nature and definition of value models as far as new information becomes available, moving from initial estimations based on expert judgment to detailed quantitative analysis.

1 Introduction and objectives

Solving problems – such as unexpected failures in prototype testing, manufacturability concerns or warranty issues – becomes more resource intensive and time-consuming as development projects progress and financial commitments are made. A major concern for an engineering design team is then to be able to generate knowledge about solutions as early as possible in the process. This is typically done by developing and executing ‘models’ that can inform decision makers about the behavior of a technology, product or system in the different stages of its life cycle.

In spite of the important work conducted in the domain of model-based decision support (Wierzbicki, Makowski & Wessels 2000) and model-based systems engineering (Wymore 1993), models in engineering design are still largely used to verify that a design does not fail regarding performances, rather than to learn about what to develop (Isaksson, Larsson & Rönnback 2009). Even though literature proposes several models to support design space exploration activities (e.g., Söderberg, Lindkvist & Carlson 2006; Runnemalm, Tersing & Isaksson 2009; Vallhagen et al. 2013), these are often limited to the analysis of geometrical robustness, performance-related attribute and life cycle costing.

The International Council on Systems Engineering (INCOSE) vision for 2025 highlights that this traditional performance versus cost view must evolve to support a more rapid analysis of a large number of design alternatives, with multiple variables and uncertainty (INCOSE 2014). In the same spirit, lean product development (León & Farris 2011; Siyam, Wynn & Clarkson 2015) and systems engineering literature (Solomon & Young 2007; Weiss 2013) discuss the limitations of existing early-stage models and promote an approach for design space exploration that brings together considerations related to the technical hardware, the system of services related to it and the broader business proposition of the company.

Recent literature recognizes the development of the so-called ‘value models’ as a step forward in the process of ‘frontloading’ (Thomke & Fujimoto 2000) the engineering design process with relevant knowledge for decision-making. Value models are often described as objective functions (Richardson, Penn & Collopy 2010; Collopy & Hollingsworth 2011) used to inform decision makers about the expected monetary value generated by an innovative solution concept. In spite of their appeal, the application of value models is still mainly limited to pilot studies related to satellite, rocket and aircraft design. Value models are often criticized for being of impractical use in engineering design problems. This is because the lack of confidence in the quality of the monetary functions can stimulate costly and time-consuming iterations – a phenomenon referred to as ‘tragic feedback loop’ (Lee, Binder & Paredis 2014) – in the attempt to capture any existing knowledge item, no matter how insignificant.

The main aim of this paper is to disrupt this loop by guiding design teams in tailoring the development of value models in the engineering design process. The underlying research question for the work can be then described as

1. (i) How shall value models iteratively translate customers’ desires into terms that are meaningful for engineering design decision-making?

The paper has two complementary objectives. The first objective is to present the findings of a descriptive study conducted in collaboration with Swedish manufacturing companies in the aerospace and construction sector. These findings are summarized in the form of seven lessons learned that describe how value modeling activities shall be shaped to support decision-making in early design. The second objective is to present, emerging from these lessons, a six-step framework for value model generation in engineering design. The latter considers the need to update the nature and definition of value models as far as new information becomes available to the design team, moving from initial estimations based on expert judgment to detailed quantitative analysis.

2 Decision-based engineering design

The academic discussion on value models can be traced back to the notions of decision-based engineering design (DBED) (Hazelrigg 1998) and decision-based design (Chen, Hoyle & Wassenaar 2012). Both approaches are focused on the rigorous applications of mathematical principles to improve how decision-making activities are performed in the engineering design process and to cope with the necessity of reducing time, cost and rework of realizing complex systems.

Traditionally, these systems are developed within the systems engineering paradigm through the use of a requirement-based engineering design methodology (RBED) (Pardessus 2004).

The identification and analysis of the requirements to represent the preferences of customers and stakeholders lie at the core of the RBED process. Once the engineering team agrees on, for instance, power, lifetime and/or range of a vehicle, any design configuration that is capable of meeting these requirements is considered likely to be successful (INCOSE 2014). However, as explained by Collopy & Hollingsworth (2011), the process of imposing constraints on the product design discourages the engineering team to improve a concept that already meets the requirements. For instance, two identical cars with a 400 km versus 700 km range are both acceptable and approximately equally preferable if the initial system requirement states ‘the car nominal range shall be at least 399 km’.

DBED methodologies – as opposed to RBED – have been proposed as an alternative method to create an objective decision-making process under uncertainty and risk. In DBED, multiple attributes are first drawn into a single system-level attribute of ‘value’, which is typically represented by the profit gained by a company throughout the life of a system. Decision makers shall then select the design alternative that maximizes the net present value for the company after examining their risk preferences over the singular attribute of profit. This process is advocated to offer several advantages over RBED. First, by removing requirements from the decision-making gates, it is possible to open up the design space and create an informed process where the higher the value is, the better the design. Second, it is possible to update the design as soon as the system definition matures because the team is no longer constrained by contractual requirements that, when set, cannot be changed or improved.

3 Value models in engineering design: a literature review

3.1 Value models based on monetary objective functions

Value-driven design (VDD) (Collopy & Hollingsworth 2011) is one of the most popular DBED methodologies. VDD promotes the systematic use of economic models to determine how varying design attributes affect the overall value of a system (Castagne, Curran & Collopy 2009), so to ‘compare one design to another or a design situated in one environment with a design situated in another’ (Collopy 2009, p. 2).

The VDD process is explained as a cycle. After having picked a point in the design space at which to attempt a solution, the team creates an outline of the design, which is elaborated into a detailed representation of design variables. A second description is then created in the form of a vector of attributes that mirrors the customer preferences or ‘value scale’. This vector is assessed against a monetary objective function that calculates the long-term profitability of the design. Hence, the ‘best solution’ is the one that optimizes this function for value.

Surplus value (SV) is the most common optimization function in VDD (see Curran et al. 2010; Fanthorpe et al. 2011; Cheung et al. 2012). SV is calculated as the reservation price for a system minus all the costs incurred (e.g., developmental, manufacturing, operating costs, externalities, taxes, delay costs and more). Reservation price is intended as the price paid by the customer for the product that makes the net present value of the transaction to be zero (Price et al. 2012). It represents the maximum possible price that a customer will pay before the cost of ownership and operation will result in losses. The latter accounts for all the possible revenues over the life of the product and represents the maximum possible amount of money that a customer will pay before the cost of ownership and operation will result in losses.

The long-term profitability idea can be then propagated to sub-systems and components to enable optimum solution strategies to be instantiated in an objective, repeatable and transparent manner (Collopy & Hollingsworth 2011). Once this process is completed for a single design, the team can accept the configuration as the solution or may try to produce an even better design by going around the cycle again.

3.2 Limitations of monetary objective functions for value

The main benefits of VDD lie in the ability to promote a value-based view in the engineering design decision-making process and of raising awareness about the revenue items that characterize the life of a product or service, together with other system-level phenomena that can influence long-term profitability. Yet, in spite of several instantiations and examples proposed in literature, the application of VDD in mainstream engineering design remains elusive. First, most implementations are still integrated into the existing systems engineering process, meaning that the identification of the design with the highest value among a set of concurring alternatives is still constrained by the goal of meeting the requirements threshold. Second, while it is true that monetary units are considered to be the most convenient, practical and universally understood metrics for value (Collopy 2012), it remains difficult to monetize those intangible factors that reflect the desire to obtain or retain a product or a service (Steiner & Harmon 2009; Grönroos & Voima 2013). Monceaux et al. (2014) and Siyam et al. (2015) claim that the SV function is too data intensive for the conceptual design phase. Even if full data would be available, value models could easily become incomprehensible to those stakeholders who do not possess specialist knowledge in the technical domain (Collopy 2012), hindering communication among the decision makers.

Lee et al. (2014) further notice that value functions often fail to remain consistent with the established axioms and results of decision theory. Value is often defined and calculated separately from cost and risk, and it is not often clear whose value is being captured and what the preference aggregation rationale is. Eventually, the cost of developing these models is not often considered, and doubts remain about the cost-effectiveness of applying monetary objective functions in early design.

3.3 Value models based on multi-criteria decision-making matrixes

The use of multi-criteria decision-making (MCDM) tools (see Ishizaka & Nemery 2013) is ubiquitous in product development and engineering design literature (e.g., Roozenburg & Eekels 1995, p. 332; Pahl & Beitz 1996, p. 178; Wright 1998, p. 139; Ullman 2002, p. 176; Ulrich & Eppinger 2012, p. 209) and typically precedes more monetary-based assessments. Recent years have seen the emergence of specific MCDM approaches for value modeling under the premises that, when qualitative data and assumptions prevail, a qualitative assessment of the ‘goodness’ of a design may be preferred against a numerical (and monetary-based) encoding of preferences (Soban, Price & Hollingsworth 2012). These models introduce a ‘value focus’ in concept assessment by calculating a single measure of ‘design merit’ that can be used to evaluate alternative solutions.

Quality function deployment (QFD) was identified early on as a strong value model candidate (Collopy 2009) mainly because of its transparency in mapping the relationships between engineering parameters and customer needs. The COncept Design Analysis (CODA) method (Eres et al. 2014) is designed as an extension of QFD that makes use of non-linear functions to calculate an overall ‘design merit’ score during design assessments and engineering design optimization studies. Improvements and extensions of CODA have been further proposed (Khamukhin & Eres 2015; Bertoni, Bertoni & Isaksson 2018) to leverage the use of MCDM as a model-based approach for VDD. Collopy (2009) further indicates Pugh matrixes as a tool for guiding the engineering discussion about needs and wants of customers, which is ‘a close cousin of value modeling’. An example of how a weighted-Pugh matrix can be applied to transform fundamental design objectives into the so-called ‘engineering characteristics’ in the context of VDD is proposed by Zhang et al. (2013).

The analytical hierarchy process (AHP) is a main building block of the value operations methodology (VOM) proposed by Curran et al. (2010). AHP is used in the VOM to establish expressions for operational value levers that are incorporated into a weighted value function. Even though AHP is limited by the use of simple linear weights and by the need of considering a limited number of alternatives at a time (Collopy 2009), recent applications address some of these issues and demonstrate its use for the design of unmanned air system for a defense application in the context of VDD (Papageorgiou, Eres & Scanlan 2017).

In summary, an array of methods and tools is proposed to assess the value of design concepts along the entire engineering design process, from qualitative methods to highly specific optimization functions. However, literature does not fully explain the input–output relationship between them as well as their position along the engineering design process timeline. The following sections discuss the evolution of value models as a decision support tool and propose a framework to guide the development and selection of value-based modeling support along the different moments of the engineering design process.

4 Research design

The design research methodology (DRM) proposed by Blessing & Chakrabarti (2009) was used as the main reference throughout the research. DRM consists of four stages: research clarification (RC), descriptive study I (DS-I), prescriptive study (PS) and descriptive study II (DS-II). This paper covers a review-based RC, comprehensive DS-I and PS and an initial DS-II. The research is further based on a multiple case study approach (Yin 2003). A total of five cases were selected to gather empirical data and draw cross-case conclusions.

4.1 Case study selection

Case study research (Yin 2003) was deemed suitable because the research described in this paper is largely exploratory as it focuses on the ‘why’ and ‘how’ questions related to the phenomenon of value model implementation upon which the researcher has no control over. The selection of the cases follows a logic of literal replication (Yin 2003), prioritizing cases that share similar settings and are expected to achieve similar results, so to corroborate each other. This is justified by the exploratory mode of the research, which made it not possible to determine the most appropriate theoretical base to guide project selection (which is, to follow a logic of ‘theoretical replication’). The case selection process was driven by the issues of appropriateness and adequacy (Kuzel 1992) and was guided by the three-cluster framework for a purposeful case study selection proposed by Shakir (2002) (originally: Patton 1990). This work adopts a combination strategy for case study selection, mixing significant cases and fieldwork determined cases. Only the cases that were considered critical – supporting logical generalization of the findings (Shakir 2002) – were retained. These were located using an opportunistic strategy, following leads during fieldwork in a way that likens snowballing. This strategy proved to be helpful during the iterative pilot phase of the research, especially in talking to process owners and designers when alternative value modeling approaches were suggested.

4.2 Case study description

Based on the above considerations, the authors selected five case studies (Figure 1), which differ on a range of measures, such as product type and organizational structure. Yet, all companies are active in the business-to-business sector and are familiar with systems engineering and set-based concurrent engineering. They have experience with cross-functional design teams and have grown lessons learned on the need to facilitate a participatory process in the design. At the same time, their business is facing rapid transformations, largely driven by the same macrotrends: digitalization, connectivity, electrification, artificial intelligence and resource scarcity.

Figure 1. Research approach in the selected case studies.

Case studies 1 and 2 were conducted in collaboration with a Swedish design-make supplier to major aero-engine original equipment manufacturers. The first case (Bertoni et al. 2018) dealt with the development of an intermediate compressor case, a major aero-engine component transferring the thrust from the engine to the airframe and keeping airflows separated. The objective was to analyze the potential impact on customer satisfaction of an innovative bleed air take-off solution, which off-takes compressed air from the gas turbine engine for anti-icing and/or de-icing purposes. The second case focused on the development of an aero-engine turbine rear structure, a major sub-system dedicated to transferring different loads and redirecting the aero-engine outgoing airflow (Bertoni, Amnell & Isaksson 2015). The goal was to create value models for a multitude of automatically generated variations (such as, for instance, changing the angle or the thickness of a flange), building on thermal, pressure and fluid dynamics performances obtained from computer-based simulations. Case studies 3 and 4 were conducted in collaboration with a multinational engineering manufacturer of mobile compactors for road surfaces. Case study 3 focused on the design of a major sub-system for a 9-ton double drum asphalt roller (Bertoni, Panarotto & Jonsson 2017) and specifically on the creation of value models for four sub-system concepts based on a similar product platform. Case study 4 (Panarotto et al. 2017) dealt with the development of models to inform the decision makers about the overall effect on value and costs of an innovative concept for a 1.5-ton double drum asphalt roller when considering alternative business models (from ‘one-sale’ models, to leasing, to functional results). Case study 5 (Bertoni 2019) was conducted in collaboration with a world-leading total-solution provider of construction equipment with the objective of developing a value modeling approach to capture the value- and sustainability-related consequences related to an autonomous, battery-electric load carrier for mining operations.

4.3 Sample selection

Semi-structured interviews were used as the main data gathering approach across the cases. In line with what suggested by Ritchie, Lewis & Elam (2013) for small-scale, in-depth studies, respondents were located by means of non-probability sampling. This means that interview respondents were not intended to be statistically representative, rather they were selected ‘with a purpose’. Experience with model-based support for engineering design and with design decision-making tools was considered as the main criteria for purposive selection. After initial data were analyzed, another sample was identified using snowballing techniques (Warren 2002) by asking each member of the initial set to locate other relevant individuals through his/her social network. Researchers took advantage of this opportunity and shortlisted relevant individuals, having the care to preserve diversity by including both the ‘meatiest’ respondents and the ‘peripheries’ (Miles, Huberman & Saldana 2013) in the second interview round. The benefit of such a heterogeneous sampling strategy (Ritchie et al. 2013) lies in the opportunity to uncover central themes which cut across the variety of sub-cases or people. This process continued until the researchers believed that no significantly original insights would be obtained from expanding the sample further. The final sample covers a variety of roles, from managers to computer aided design (CAD) engineers, from marketing practitioners to information technology experts.

4.4 Interview design

The semi-structured interviews were designed following the best practices from the qualitative research field (e.g., Legard, Keegan & Ward 2013). The initial exploratory and largely descriptive questions were typically followed up by more specific inquiries, with the aim of clarifying answers, requesting further examples or pursuing the implications of answers to the main question (see Warren 2002). Interviews were kicked off with ground mapping questions to ‘open up’ the subject. These featured minimal probing to allow respondents to raise issues that were most relevant for them. This content mapping activity was complemented by dimension mapping questions to focus participants more narrowly on the model- and value-related topics, stressing the impact on the applied practice on engineering design decision-making. Perspective-widening questions further encouraged respondents to look at issues from different standpoints, ensuring comprehensive coverage and stimulating further thoughts (e.g., how to visualize the results of modeling activities).

In a second stage, content mining questions were used to obtain a full description of phenomena and to understand the underpinning behavior of each respondent. This stage featured the use of explanatory probes, asking ‘why’ to reveal patterns in behaviors, uncover events and pinpoint decisions. The use of such probes was iterative and likened the laddering technique from the design thinking methodology (Lockwood 2010). Amplificatory probes were further used to obtain an in-depth understanding of the manifestation or experience of a phenomenon, while exploratory probes (i.e., sampling for ‘feelings’) were seldom used.

4.5 Demonstration and verification

The analysis stage featured different cycles of coding (Miles et al. 2013). The array of individual codes was revised as experience with coding techniques for this task grew and later arranged into patterns to uncover local factors in the study. Later in the process, interviews became more confirmatory in nature. The researchers compiled visual representations and demonstrators of the emerging modeling concepts, which were verified with company stakeholders to identify critical topics for modeling.

Multi-day physical co-creation workshops and analysis of internal company documentation were used as a triangulation method. Also, participation in regular debriefing activities with the industrial partners and other academics allowed the researchers to step back from their learning experience to develop critical thinking and improve their analytical approach. Reflective learning, i.e., the process of internal examination of an issue triggered by an experience (Boyd & Fales 1983), was further aided by participation to co-located research workshops that featured a broader set of industrial practitioners.

5 Lessons learned: characteristics of effective value models

The findings from the DS-I have been gathered in this section and presented in the form of seven lessons learned. These are described as ‘experiences’ distilled from multiple case studies, which shall be taken into account when designing value models for design decision support. In the PS stage, these lessons have contributed to the development of the proposed framework for iterative value models generation presented in Section 5.

Lessons learned 1: value models shall be designed to provide the cross-functional team with ‘boundary objects’ to facilitate the negotiation of design trade-offs.

When dealing with the development of long lead-time items, such as in the aerospace sector, solution providers must be informed as early as possible about the main value-creation features for a new product or system – often even before mature requirements are made available – so to avoid costly rework in a later stage. Value models were acknowledged early on (see Isaksson et al. 2013) as an enabling mechanism in the process of handling and dispatching information outside organizational boundaries, so to involve system integrators, suppliers and subcontractors in the concurrent development of solutions.

This study further points out the role played by value models to support the interplay between different roles and functions in the organization, including engineers, managers, technicians and sales experts when negotiating trade-offs for new solutions. Existing model-based support in the engineering domain (including computer aided engineering (CAE), knowledge based engineering and more) was found to be difficult to connect to the set of models used in the business domain. As explained by Hull, Jackson & Dick (2002), too technical models become useless if they are incomprehensible for all those stakeholders who do not possess specialist knowledge in the technical domain. A main emerging function of value models is, therefore, that of providing an understandable picture of how different disciplines (from engineering to management) contribute to the creation of value for new products.

The concept of value models as ‘boundary objects’ often emerged in the discussion. These are defined as objects that ‘sit in the middle’ among individuals and groups, eventually serving as a basis for conversation and knowledge sharing within the cross-functional design team.

The process of generating value models shall, therefore, consider the need to engage different audiences in the early-stage negotiation of the requirements, facilitating their active participation and stimulating knowledge sharing. The initial stages of the design process – likely the most ‘cross-functional’ in nature – require value models that are intuitive and seamlessly understood by all stakeholders to facilitate knowledge sharing in the design exploration activity. These shall, however, ensure a link to domain-specific models used to perform work within each different group. In a later stage of the design process, the work becomes more discipline-specific, narrowing the range of expertise needed to deliberate about a design trade-off. Cross-functional knowledge sharing at a later stage becomes less of an issue, suggesting the use of discipline-specific models as a discussion catalyst at the decision gates.

Lessons learned 2: value modes shall be developed with the objective of capturing the contextual knowledge for decision-making and the underlying rationale for value.

Process owners recognize that in the fuzzy front end, engineers are lacking tools to communicate why their work is good and to deliberate about the most value-adding design. Value models are seen as an opportunity to have ‘customers at your fingerprints’, being able to assess if a given idea, configuration or detailed design will fulfill the expected level of performance and value. The case studies highlight the opportunity of shaping the value models as hubs where argumentations related to the value of a concept can be systematically captured in a way to support the discussion on the appropriate quantification strategy.

In line with what suggested by Isaksson et al. (2013), the main function of early-stage value models shall be that of ‘communicating preferences’ rather than ‘engineering the system’. During the earliest design stages, a preference has been expressed toward applying simple, qualitative models, while in a later stage, they embed more mature knowledge and capture the rationale of the system with more granularity. The descriptive study findings in the construction sector show that several aspects related to the value-generation opportunity for the customers’ organization tend to be simplified or totally neglected when focusing on monetary functions. For instance, deterministic and probabilistic cost models were considered not to be effective enough in capturing and communicating aspects related to improved quality, lower risk for delay or higher brand acknowledgment mainly due to the high uncertainty and approximation of these models. By exercising a qualitative model, engineers can improve the way contextual design information is captured during the requirement decomposition process, growing awareness of the system-level effects triggered by a specific design variation. Practitioners further expressed a preference for following up the semi-quantitative assessment with economic analysis to enhance the decision base for gate meetings. The latter was found to be better supported if value aspects are quantified in monetary terms, meaning that opinions and intuitions must be backed up with facts and evidence-based statements. The main advantage of mixing qualitative and quantitative assessment is the possibility to conserve the link between the system representation and the design intent when requirements are decomposed and allocated to the various design teams (Monceaux et al. 2014).

Lessons learned 3: value models shall be complemented by information about their level of maturity to support the team in taking actions and making decisions.

The lack of confidence in the results of early-stage models is a common denominator across the case studies. The descriptive study findings reveal an attitude toward procrastination in decision-making (Johansson, Wall & Panarotto 2017), mainly driven by unclear expectations about how to increase the reliability of the models at hand – which is, what questions should be answered and how this information shall be gathered. Procrastination was found to be a major frustration point for several interview respondents mainly because ‘a wrong decision can be changed, but with no decision there is also no action’. In order to keep the momentum in the process, design practitioners have often discussed the opportunity of developing decision support able to suggest if a selected development direction is sufficiently agreeable or good enough.

The concept of model maturity has often emerged from the discussion, being explained as a framework where to grow knowledge about the knowledge in a way to achieve a better understanding of what early-stage uncertainties, ambiguities, and assumptions involve.

Model maturity shall assist decision makers in making more informed decisions not necessarily by highlighting that a product is better but rather by providing more knowledge about its potential imperfections. Later in the design process, knowledge maturity shall probe the extent to which a model (i.e., a value model) is valid for a certain decision. Once satisficing models are identified, engineers can move on and direct resources to the improvement of the maturity of those instances that have a greater impact on the overall results. In a later phase, closer to product release – and to important decisions with higher stakes – the engineering design team shall raise the threshold for model maturity assessment, exploiting sensitivity analysis to probe the robustness of the value models.

Lessons learned 4: the generation of value models in the engineering design process shall consider the need to provide decision makers with a ‘pool’ of representations, mixing deterministic, probabilistic and qualitative aspects.

The descriptive study findings point to the beneficial effect of navigating through a ‘pool’ of representations that mix deterministic, probabilistic and qualitative aspects. The convergence between the different models was discussed as an important aspect to mutually reinforce the results of the value analysis, coping with the lack of confidence issue raised in the lessons learned above. Even though later design stages benefit from fact-based monetary models, the descriptive study reveals a need for keeping qualitative assessment alive, both to cross-check the quantitative results and to retain the underlying rationale for the requirements.

One aspect of interest when it comes to the interpretation of the results of value models is the phenomenon of associative processing. Research in cognitive behaviors has shown that the human’s ability of processing information is increased when multiple cues are presented both across and within media/channels (Severin 1967). Presenting information across multiple channels – while having care of mitigating the issue of information overload – can ultimately lead to tasks being completed in less time and with less effort. For this reason, value models in the engineering design process shall not merely be ‘replaced’ at each step, rather it is important to ensure that a balance between qualitative and quantitative aspects is retained and communicated to the decision makers. The ability to exploit a ‘pool’ of representations is also seen as an opportunity to facilitate negotiation in the cross-functional teams, with some models being generic enough to be grasped by those stakeholders without a technical background, while others being specific enough to benchmark alternative concepts with sufficient confidence and detail.

Lessons learned 5: value models shall be designed to integrate a sustainability dimension in concept selection activities, considering it as an active driver for decision-making.

Environmental awareness and other sustainability-related trends are difficult to systematically represent in the requirement description. Yet, there is a widespread consensus among the case study companies that overlooking the role of sustainability as a value-creating factor increases business risk and may result in expensive and time-consuming re-design efforts later in the product life cycle. The answer lies in methods and tools that are able, already in a preliminary design stage, to balance sustainability requirements with economic interests, highlighting how a sustainable design choice can create value for customers and stakeholders and, hence, generate market success in the long term.

The descriptive study findings show that value models are preferred means for companies to understand how sustainability compliance shall be considered in light of the more traditional ‘goodness criteria’ for a product or system. However, while some aspects of sustainability are partially encompassed by the established drivers for design (e.g., specific fuel consumption, lifetime and weight reduction in the automotive and aerospace industry), others are less readily quantifiable (e.g., material criticality from an availability and socio-ecological sustainability perspective) and problematic to use as drivers for development.

The descriptive study shows that qualitative models are a way to overcome the difficulty of assessing and communicating sustainability to technology developers. Sustainability is rooted in a list of principles (Holmberg & Robèrt 2000) and contains a significant portion of tacit concepts. For these reasons, MCDM tools represent a good trade-off between the need of quantifying the sustainability challenge – showing numbers related to the value generated by sustainability-oriented decisions – and the need of avoiding falling into a reductionist trap, which is reducing the complex sustainability discourse to a single measurable indicator (Gasparatos, El-Haram & Horner 2008).

Lessons learned 6: value models shall be designed to support the benchmarking of alternative business strategies, e.g., one-sale models versus functional provisions.

The descriptive study shows the need to extend the system boundaries for the value modeling activity, moving from merely benchmarking different product concepts to include alternative business strategies and ecosystem configurations. The value modeling generation process needs to consider the increased complexity introduced by the notion of product service systems. A main difference in the new paradigm is the need to reinforce the capability to deal with intangible and subjective aspects of value, typical of service design when comparing solution concepts. The introduction of a servitization perspective stresses the need to highlight the provider standpoint in the value assessment exercise. The case study companies pointed to the opportunity of using value models not only to capture the added value for customers of alternative strategies for product deliveries (i.e., one-sale mode versus ownerless consumptions) but also the value generated internally for the manufacturer. This is because each business strategy is able to generate (or conversely destroy) value inside the company. For instance, a circular strategy based on the opportunity to take back a construction machine at the end of its life may negatively impact cost, while benefitting the company brand due to the increased sustainability profile. Value models shall then, across the different stages of the process, communicate both the internal and external value-generating opportunity to decision makers.

Lessons learned 7: the generation of value models in the engineering design process shall be driven by the opportunity to exploit the digital thread to populate the models in the different stages of the life cycle.

The descriptive study shows that value modeling activities are increasingly dependent on the availability of data from the different steps of the system life cycle. The digitalization trend was highlighted in the case study as a major opportunity for value-driven methodologies to capitalize upon. The opportunity to extract value-related information from both existing data sets and live data streams was frequently discussed with the interview respondents. Hence, it is important for value modeling activities to be aligned with the company’s communication framework. This means promoting an integrated view of the data across the entire life cycle of an asset, enabling the flow of information across traditionally siloed functional perspectives. The value models shall be constructed in such a way to seamlessly access, integrate, transform and analyze data from disparate systems throughout the product life cycle into actionable knowledge. They shall be designed to enable such a bottom-up extraction of field data and its conversion through appropriate data analysis techniques into functions for value to be exploited at a different level. In the road construction sector, for instance, a ‘digital thread’ shall be ensured to make possible to continuously log data from the machine in operation to collect data about usage patterns, energy consumptions, the behavior of the operator and more, so to create knowledge in the form of functions and data to populate value models.

6 A framework for the iterative definition of engineering design value models

In the PS phase, the findings from the empirical investigation brought to the definition of a framework to guide the selection/development of appropriate value models along the different stages of the engineering design process. The framework features six iterations, which have been shaped along with the knowledge value stream (KVS) and product value stream (PVS) framework (Figure 2) proposed by Kennedy, Harmon & Minnock (2008). The output of early-stage activities (KVS) is used as an input to create more detailed and robust models while moving toward the end of the PVS stage. Conversely, lessons learned from the later stage of the design process are fed back to early-stage models, refining their description and content for future projects.

Figure 2. Framework for iterative value models generation in engineering design and its link to the descriptive study results (lessons learned).

6.1 Knowledge value stream and product value stream

The empirical investigation highlighted two distinct sets of needs when approaching the value modeling activities in an early design phase. These are largely shaped by the characteristics of the KVS–PVS framework. In the process fuzzy front (KVS), value models were often discussed as a catalyst to support the systematic building of knowledge about technology over time, in a way to facilitate the set-based engineering exercise. By capturing and reusing knowledge about markets, customers, technologies, products and manufacturing capabilities – across projects and organizations – value models shall support design teams in reviewing significant aspects of value, across the whole life cycle, for a proposed technology platform. Their main function at this stage is to define the scope of the development effort and to explore the long-term consequences associated with a broad number of possible technologies. For this reason, the value models shall be generic enough to accommodate heterogeneous solution proposals and to be used/shared across people and projects. Furthermore, they shall enable the screening of candidate solution strategies within a few days or weeks mainly because preliminary system requirements shall be available shortly after these iterations.

The transition from KVS to PVS is characterized by an increased maturity in the way system concepts are described. Increased data availability makes it possible to perform more in-depth value analysis, so to select winning design options to be later followed up in the detailed design stage. The PVS is specific for each project and consists of the flow of tasks, people and equipment needed for creating, for example, drawings, bill of materials and manufacturing systems. Both product and process definitions exist at this stage, and they are refined to minimize risk, cost and any other requirements compliance. Value models shall now enable a greater depth of analysis in the given context. Practitioners highlighted how models at this stage shall be able to move over a mere cost-based view by also considering the revenue-generation opportunities for the customers and customers-of-customers (e.g., an aero-engine manufacturer and an aircraft manufacturer in case studies 1 and 2). The time frame for the usage of decision support tools is still constrained; yet, studies may now expand to several weeks. In this context, quantitative value models are suggested to support the selection of a product concept from the pot of available alternatives.

6.2 Value-creation strategy models

Value modeling activities in the KVS kick off by capturing the strategy for value creation for customers and stakeholders, which is represented by a list of value criteria with associated rank weights. A value-creation strategy (VCS) model (Isaksson et al. 2013) is used then to represent this information in terms that are meaningful for the cross-functional design teams. A feasible approach to capture such a strategy is to distill a manageable subset of linearly independent ‘value’ dimensions from the voice of the customer and need description. These can be later detailed in more specific value drivers for selected sub-systems or components. The ‘main headings’ for design evaluation proposed by Pahl & Beitz (1996, p. 179) and the hierarchical structure of needs (primary, secondary and tertiary) proposed by the voice-of-the-customer theory spotlight macro-categories from which to extract these value drivers. The feasibility–viability–desirability framework (Leavy 2010) and the triple bottom line (as used by Willard 2012) models have emerged as main support mechanisms to define the set of attributes able to capture customers’ and stakeholders’ value in its fullest. These attributes provide a common ground to elaborate on the expected capabilities of a new system and to force decision makers to reflect on value creation from a social, environmental and financial perspective. Dimensions and drivers are further rank-weighted to display which aspects of the solution are emphasized by different markets, customer types and applications. The VCS model is iterated and refined as far as new information about market conditions, competitors and expected capabilities become available. Rank-weighted dimensions and drivers represent a first output of the value modeling methodology, which is used as input in the next modeling step.

6.3 Qualitative value models

After the initial value assessment step, which is mainly aimed at communicating the opportunity for value creation, the work in the KVS moves toward measuring the ‘goodness’ of early-stage design concepts against a given baseline. A critical boundary condition for the development of modeling support at this stage is to be able to withstand situations where the information available is scarce, immature and incomplete.

Value models in the form of MCDM matrixes are proposed to support each individual in the cross-functional team in bringing along different criteria and points of view, which must be resolved through a process of mutual understanding and compromise. MCDM methods are divided into several different categories, including priority-based, out-ranking, distance-based and mixed methods. In the multiple case study, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), the weighted-Pugh matrix, the Viekriterijumsko Kompromisno Rangiranje and the ELimination and Choice Expressing REality methods have emerged as main enablers to support the translation of the VCS into a benchmarking mechanism for comparing solution directions.

These qualitative models are aimed at facilitating value negotiation during co-located focus groups in a workshop-like setting, involving participants from different organizational functions and (when possible) customers. The modeling process kicks off by requesting the workshop participants to generate a first list of design concepts, which are then compared based on multiple criteria with respect to an existing concept, called baseline. The results of the assessment along the list of criteria are then aggregated using the rank weights defined in the VCS, presented in the section above, to obtain a score (e.g., the ideality score in TOPSIS, from 0 to 1) representing the value of a design.

6.4 Semi-quantitative value models

The study reveals that value models shall evolve into more systematic representations to better capture the rationale behind the assessment and to document richer lessons learned that can be exploited in future projects. Furthermore, later in the process, the assessment task shall become a choice problem (Ishizaka & Nemery 2013), with the goal of reducing the group of options to a subset of equivalent or incomparable ‘good’ options, to be further developed in the PVS. Decision matrixes at this stage shall use target requirements to map available options against the value criteria identified in the VCS document. Increased resolution in the product description opens room for more sophisticated modeling approaches such as QFD. In QFD, systems, sub-systems or components are scored based on how much they contribute to each criterion. These qualitative scores are then mapped onto quantitative ones to determine the most value-adding solution among a set of alternatives. It is possible to further extend QFD and embed more complex relationships in the mapping to capture, for instance, the non-linearity between customer satisfaction and product requirements. The CODA (Eres et al. 2014) and the EVOKE (Bertoni et al. 2018) methods are examples of QFD extensions proposed to cope with the choice problem. When a satisfying combination of characteristics is found, the team must decide whether to invest resources in optimizing such a combination and to communicate this information back to the system integrators (i.e., the engineering characteristics become the embryo of the system requirements) or to continue working on critical areas of the system that necessitate higher value contribution.

6.5 Analogous total cost-based models

Entering in the PVS stage, life cycle cost models become appealing to raise awareness on the economic impact of alternative design concepts in the customer operational process. These models are analogous – they are based on similarity with existing solutions and platforms – and are iterated as long as the product description evolves. Operational performances (e.g., use of resources or output quality) and operational support (e.g., downtime or maintainability) are the most immediate cost items to be considered at this stage mainly because they are the ones most directly influencing customers’ purchasing behavior.

Value models in the initial stages of the PVS shall be able to run cost estimation based on an open range of possible design concepts, rather than on a predefined set of solutions. These value models shall be built on modular computational blocks, which enable design engineers to obtain relative cost comparisons between the most relevant variables for a specific context. Still, when quantitative models are approached for the first time, the design space is dominated by information volatility. For this reason, initial quantitative value models insist on a conceptual approach (Gupta 1983) that consists of a set of hypothesized relationships expressed in a qualitative framework. Main cost drivers are derived either from the literature (as happened in the road construction case studies) or from company historical data (as happened in the aerospace industry case studies).

6.6 Parametric revenue-based models

Value modeling activities later benefit from the use of net present value and SV techniques that compute revenue data together with cost data and that consider ‘ilities’, such as modifiability, changeability or scalability (McManus et al. 2007) along the entire life cycle of the system. The descriptive study further pointed to the need for applying modeling support based on actual project data. Hence, the framework prescribes at this step the usage of parametric methods (Feldman & Shtub 2006) for cost prediction. Revenue models at this stage are mainly built top-down and are based on parameters (or variables) that are derived from the CAD/CAE environment using Design of Experiment and that are modified during the project simulation process. A set of mathematical equations constitutes the core of the value model. These may be derived from reference literature, or they may be proprietary equations derived from the analysis of the data gathered throughout the product/system life cycle. A model maturity score complements the results of the value calculation to communicate the level to which the functions entering the parametric cost model may be trusted.

6.7 Detailed bottom-up value models

In the last iteration, the objective becomes that of systematically decomposing a system into its constituting parts so to eliminate or modify anything that causes unnecessary costs, without damaging essential functions. Identifying and breaking down functions allows the representation of interactions between sub-systems and components in a complex product. It also helps in cascading down value-adding functions to lower level functionalities, so to identify main areas of improvement. Analytical models are used at this stage to perform trade-space studies on alternative design configurations, so to enable optimization of the different parts of the sub-system. Methods such as finite element analysis, computational fluid dynamics or modal analysis are applied to enable the optimization of a design or of a part of it.

Models become more detailed and estimations are conducted using a bottom-up approach mainly because more reliable revenue and cost data are obtained from increasingly refined functional and analytical models. The monetary value of each component is then aggregated with those of other items to obtain a total figure of value. Sensitivity analysis is further conducted to raise the decision makers’ awareness of how the outcome of benefit–cost analysis changes with variations in inputs and assumptions.

7 Discussion and conclusions

The main purpose of the proposed framework is to guide project leaders, managers and product development specialists in introducing a value-driven approach in the design decision-making process, building a shared understanding of ‘what do we mean by value’ among the individuals working at different stages of the development process.

The notion of VDD is traditionally associated with the use of a unique optimization function to drive design decisions at a vaguely defined point in time in the engineering design process. The lessons learned collected in the paper highlight the need to overcome this static view and to consider VDD as the act of progressing from low-fidelity models to deterministic functions. This means to develop the capability of carrying on the value-related information generated during the earliest design stages to progressively build the knowledge basis upon which design decisions are made. In this view, value-oriented decision-making becomes ubiquitous in the engineering design process and does not remain confined to specific steps in the process.

When developing and/or selecting value models, decision makers shall be aware that their main function shall be that of staging discussions about the value contribution of a design, rather than to merely identify the best possible concept via optimization. Discussions trigger negotiations, forcing cross-functional team members (1) to confront each other’s perceptions on what the value of a system is, (2) to resolve conflicts where conclusions differ and (3) to progressively learn what a ‘good design’ is. Negotiation eventually mitigates the risk for rework and associated cost in the later stages of the process due to the selection of sub-optimized designs. To trigger this negotiation process, the paper reveals that value models shall encompass qualitative dimensions in early stages and that they shall move toward more quantitative assessments when information becomes available and the level of detail in the system description increases. It also shows that value modeling activities need to expand along two axes. First, they shall provide more contextual information about the underlying rationale of the function and the maturity of the information on which they are built (‘i.e., where do the results come from?’). Second, they shall suggest a course of actions and actionable measures (i.e., ‘what do we do with the results?’) so to render more value in the next iteration.

These results are considered a step forward toward a larger research effort whose purpose is to create a model-driven platform for value-based decisions in conceptual design. The purpose is to use models to capture and represent ‘value’ aspects and link these to the engineering design process. The models used in the presented case studies have been exercised in different industrial domains; still, they are comparably low fidelity and simplistic. Future research aims at applying them in more data-rich situations, integrating them with other tools to improve, for instance, the visualization of modeling results.

The proposed framework provides a blueprint for the future development of a multi-model and multi-disciplinary decision-making environment for engineering design decision-making. The purpose is to engage researchers, practitioners and managers in playing with design trade-offs, so to balance the desired properties of the conceptual solution and to understand the impact of changes and decisions over the life cycle. Value models constitute the backbone of such design decision-making support. They are used to collect and summarize the results of cost analysis produced along the engineering design process, so to enable cross-functional design teams to deliberate about value ‘conversationally’.

An interesting future research track is related to the use of data mining techniques to support decision makers in populating the value models (Isaksson et al. 2015). Nowadays, technology makes it possible to continuously log data from a system during its entire life cycle and to apply data mining algorithms to discover patterns and make predictions. A promising aspect is related to the ability to organize such patterns to reveal the structure of the decision to be made, building structures (e.g., decision trees) to populate (or complement) value models.