Validation of Adult Decision-Making Competence in Chinese college students

Abstract

The present study was conducted to validate the Chinese version of the Adult Decision-Making Competence scale. 364 college students were recruited from four universities in China. The results indicate the Chinese Adult Decision-Making Competence subscales have good internal consistency and the two-factor structure in the study of Bruine de Bruin et al. (2007) was confirmed. Gender differences were found in Resistance to Sunk Cost. Differences of Applying Decision Rules and Consistency in Risk Perception were found between participants with different education background. Overall, the Chinese Adult Decision-Making Competence scale is validated in China.

1 Introduction

People face diverse decisions every day. Effective decision-making depends on how good you are at identifying, comprehending and integrating information (Fischhoff, 2008). Early way of assessing decision making abilities concerned medical decisions, such as the MacArthur Competence Assessment Tool-Treatment (MacCAT-T; Er & Sehiralti, 2014). In behavioral decision research, decision-making abilities have been measured in various way (Dewberry, Juanchich & Narendran, 2013). The Decision-Making Competence (DMC) scale has been used to assess decision making competence of students, community residents, people on the autism spectrum and adults with ADHD. Versions of the DMC scale were categorized by age in previous studies: Adult Decision Making Competence (A-DMC; Bruine de Bruin, Parker & Fischhoff, 2007; Del Missier, Mäntylä & Bruine de Bruin, 2010, 2012; Bruine de Bruin, Del Missier & Levin, 2012; Weller, Ceschi & Randolph, 2015), Youth Decision-Making Competence (Y-DMC; Parker & Fischhoff, 2005; Parker & Weller, 2015), Preadolescent Decision Making Competence (PA-DMC; Weller, Moholy, Levin & Bossard, 2015) and Older Adult Decision Making Competence (OA-DMC; Finucane & Gullion, 2010).

In the present study, we use a Chinese version of A-DMC with six subscales, which was back translated and adapted from the version of A-DMC that Bruine de Bruin et al. (2007) and Bavolar (2013) used in their research. Six series of decision tasks were designed to represent four fundamental decision making skills. Accuracy and consistency served as evaluation criterion.

2 Adult Decision-Making Competence

Decision-making competencies include four aspects of decision-making: belief assessment, value assessment, integration and metacognition. Some previous validation studies did not confirm the categorization of A-DMC subscales according to these competencies.

2.1 Belief assessment

Belief assessment involves ability to judge events probabilities or to judge probabilities of statements being true (Parker & Fischhoff, 2005; Beyth-Marom & Fischhoff, 1983). Since probability is the mathematical expression of uncertainty, probability judgments usually reflect people’s risk perceptions and their risk behaviors (Bavolar, 2013; Parker & Fischhoff, 2005).

In previous research, two tasks were adopted to assess belief assessment. The first task, Consistency in Risk Perception (CRP), consists of 20 items. Each item asks respondents to judge probabilities of events (e.g., visit a dentist) on a numerical and linear scale (0%=No Chance, 100% =Certainty) in a shorter time period (e.g., one year) and a longer time period (e.g., five years). These pairs of events were subset/superset events (e.g., to visit a dentist to fill a cavity or to visit a dentist for any reason) or complementary events (e.g., to move or not move your permanent address). The score of CRP is the percentage of correct pairs of responses, that is, those consistent with the logical relationship.

The second task is Recognizing Social Norms (RSN). RSN task has two forms of tests, which consists of 32 items. First, respondents are asked to answer 16 yes-or-no questions about whether they “think it is sometimes OK” to engage in a series of negative behaviors (e.g., “not to tell the police when you witness a crime”). Second, respondents need to estimate how many out of 100 people their age would say it is sometimes OK to do things appeared in the first part. The proportions of “Yes” across the sample in the first part reflect the actual social norms, and answers in second part serve to compute the estimated proportions of social norms. The score of RSN is the rank-order correlations between the actual and estimated proportion.

2.2 Value assessment

Value assessment involves ability to estimate gains or losses reasonably by getting rid of irrelevant information and to make consistent choices for equivalent options (Parker & Fischhoff, 2005; Kahneman & Tversky, 1979). Sensitivity to relevant information and insensitivity to irrelevant changes refer to two psychological effects: the framing effect and the sunk costs effect. Bruine de Bruin, Parker and Fischhoff (2007) adapted two tasks to measure value assessment.

One task is Resistance to Framing (RtF). The RtF subscale consists of 28 items expressed in positive and negative frames. 14 items involved in the first part are presented as gains (positive frame), and the remaining 14 items are the same as the first part except that they are presented as losses in the second part. In both of the two parts of items (14 items in each group), half of the items (7 items) are valence framing questions, and the other half are attribute framing questions (Bavolar, 2013). The score of RtF is the mean absolute difference between options in different frames.

Another task is Resistance to Sunk Cost (RtSC). The RtSC subscale consists of 10 items, each with two options: the prior option (e.g., “continue paying at the old store you’ve paid a deposit”) and the new option (e.g., “buy from the new store for lower price and lose deposit you paid in the old store”), which is superior to the old one. Respondents are asked to mark on a 6-point scale to represent their selection preference (the prior investment=1, the new investment=6). The score of RtSC is average value of respondent’s answers across 10 items.

2.3 Integration

Integration involves the ability to combine belief and value coherently (Parker & Fischhoff, 2005; Bavolar, 2013). It is measured as the ability to apply specified decision rules correctly. The Applying Decision Rules (ADR) subscale consists of 10 items requiring respondents to choose the right product (e.g., DVD player) under specified decision rules (e.g., “the best in Picture Quality”, “no DVD players that score below ‘Medium’ on Picture Quality, no DVD players that score below ‘Medium’ on Sound Quality, and no DVD players that score ‘Very Low’ on any other feature”). Each item offers respondents 5 alternatives with 5 different attribute rating scores (picture, sound, programming, brand and price). Respondents then need to choose a single product or multiple products that meet the stated requirements. The score of ADR is the percentage of correct answers.

2.4 Metacognition

Metacognition involves the ability to know the extent of one’s decision making ability. Reasonable decision making confidence is very important to make effective decisions. If decision makers overestimating or underestimating their decision abilities, they can make wrong judgments and choices. Under/overconfidence (UOC) tasks are designed to assess decision confidence. The UOC subscale consists of 34 common-sense items about daily life. After answering “true” or “false” to 34 statements (e.g., “There is no way to improve your memory”), respondents are asked to indicate the confidence of their answers (50%= Just Guessing, 100%= Absolutely Sure). The score of UOC is computed as one minus absolute value of the difference between the mean confidence and percentage of correct choices.

So far, decision-making competence measured using versions of the DMC scale has been studied by research groups in the USA (Parker & Fischhoff, 2005; Bruine de Bruin, Parker & Fischhoff, 2007, 2012; Carnevale, Inbar & Lerner, 2011; Weller, Moholy, Levin & Bossard, 2015, Levin et al., 2015; Parker & Weller, 2015), Italy (Del Missier et al., 2010, 2012; Weller et al., 2015), Sweden (Mäntylä, Still, Gullberg & Del Missier, 2012) and Slovakia (Bavolar, 2013; Bavolar & Orosova, 2015). In China, a Chinese version of A-DMC with 26 revised items showed good reliability and validity in a male cadet sample (Feng, Peng, Zhang, Huo & Xiao, 2015). Compared to the original version of the DMC with seven subscales (Parker & Fischhoff, 2005), most of the subscales in the revised A-DMC scale with six subscales (English version, Italian version and Slovak version) revealed better internal consistence and external validity.

The purpose of this study is to validate a new Chinese version of the A-DMC. We examined reliability and validity of the Chinese version of A-DMC among college students. Because our participants had different educational backgrounds (i.e., undergraduates or junior college student who couldn’t apply for bachelor’s degree), we also examine the relation between educational background and A-DMC subscales. Academic performance is regarded as the most common index for individual abilities of college students in China. So we also explore correlations between scores of A-DMC subscales and academic performance.

3 Method

3.1 Participants

364 participants (293 females, 80.5%) were recruited from four universities in western China. The average age of the sample was 19.7 years (SD=0.91, range=18-23). Data were collected between September and November in 2016. 49.2% of participants were from public universities (50.8% of them were from private universities). 89.8% of participants were undergraduates while 10.2% of them were junior college students.

3.2 Measures

Adult Decision-Making Competence was measured using A-DMC scale with six subscales (Bruine de Bruin et al., 2007; Bavolar, 2013; Bavolar & Orosova, 2015): Resistance to Framing, Recognizing Social Norms, Under/Overconfidence, Applying Decision Rules, Consistency in Risk Perception and Resistance to Sunk Costs. The A-DMC scale is composed of ten parts: Resistance to Framing has four parts and Recognizing Social Norms has two parts. 134 items are presented in the same order as in previous studies. To ensure the accuracy and feasibility of the translated version, the A-DMC scale was translated into Chinese and translated back by different professional researchers. To take cultural background into consideration, partial revisions are made to the scale: English names are replaced by Chinese names and all prices (US dollars) in the scale are converted into Chinese currency (RMB) at roughly the current exchange rate in 2016, except in the sunk-cost scale, since the goods were commonly sold in China, in which case we used a common price.

In the present study, we use grade-point average (the GPA) of the last academic year as the measure of academic performance.

Table 1: Descriptive statistics for A-DMC subscales. For each A-DMC component, data presented in the first row is from the present study; the second row is from Feng et al. (2015), the third row is from Bavolar (2013), the fourth row is from Del Missier et al. (2012), the fifth row is from Del Missier et al. (2010), the sixth row is from Bruine de Bruin et al. (2007), and the seventh row is from Parker and Fischhoff (2005). RtF=Resistance to Framing; RSN=Recognizing Social Norms; UOC=Under/Overconficence; ADR=Applying Decision Rules; CRP=Consistency in Risk Perception; RtSC=Resistance to Sunk Costs.

Notes:

^a. In order to report higher score as better performance, RtF score equals five minus the mean absolute difference between options in different frames;^b. N/A indicate the authors did not report the value in their research

3.3 Procedure

Each volunteer participated in this study in return for a ¥15 gift. Participants were asked to complete the A-DMC scale in 45 minutes. Participants’ academic performance of the last academic year was provided by educational administrative department. 270 participants authorized the educational administrative department to provide their academic performance information.

Table 2: Correlation matrix for all components of A-DMC scale and A-DMC score. Data presented in the first row is from the present study; the second row is from Bavolar (2013), the third row is from Bruine de Bruin et al. (2007), and the fourth row is from Parker and Fischhoff (2005).

Notes:

^* p< .05

^** p< .01

^*** p<.001.

3.4 Results

Descriptive statistics of A-DMC components are presented in Table 1. The present study shows reasonable range and median close to previous studies. The mean score of all A-DMC components in Table 1 indicate participants’ performance: the higher, the better. Cronbach’s alpha of all A-DMC components in Table 1 indicate the Chinese A-DMC scale has better internal consistency in RSN and CRP. Cronbach’s alpha of UOC is above 0.7, while RtF, ADR and RtSC in the range 0.5–0.6. There are no big differences among reliability of Chinese, Slovak and English A-DMC scales.

Table 3 Loadings for the one-factor and two-factor A-DMC models. Data presented in the first row is from the present study; the second row is Bavolar (2013), the third row is from Bruine de Bruin et al. (2007), the fourth row is from Parker and Fischhoff (2005). (Parker and Fischhoff (2005) did not report the two-factor structure of A-DMC subscales.)

Pearson correlations between score of A-DMC subscales and correlation between GPA and A-DMC scores are presented in Table 2. The correlation matrix indicates strong correlations between score of RtF and RSN, RtF and ADR, RtF and CRP, RSN and ADR, ADR and CRP, ADR and RtSC. The insignificant correlations mean A-DMC subscales extract different components of decision-making competence. Correlations between GPA and A-DMC scores suggest that the academic performance cannot predict decision-making competence of college students.

EFA (using the principal factors method and oblimin rotation) results are presented in Table 3. The first column in Table 3 shows loading for the one-factor model, which explains 27.75% of the variance, which is less than 40.5% in Bavolar (2013) and 30.1% in Bruine de Bruin et al. (2007) but higher than 25.1% in Parker et al. (2005). The last two columns in Table 3 show the two-factor model with eigenvalues greater than 1, which explains 46.42% of variance, slightly higher than 46.2% in Bruine de Bruin et al. (2007) and Bavolar (2013). In the present study, the largest loading on the first factor are Apply Decision Rules, Resistance to Framing and Recognizing Social Norms. The fundamental skills that the first factor extracts are integration, value assessment and belief assessment. The first factor reflects three aspects of decision making competence. The largest loading on the second factor are for Under/Overconfidence, Consistency in Risk Perception and Resistance to Sunk Costs. The fundamental skills that the second factor extracts include metacognition, belief assessment and value assessment. The second factor reflects three aspects of decision making competence. Thus, the two-factor model covers all four constructs of decision making competence.

Table 4: Fit indices for the two-factor structure of A-DMC.

Gender, education background and A-DMC performance

The A-DMC subscales we used are the same as what Bruine de Bruin et al. (2007) and Bavolar (2007) used in their research. We used LISREL to confirm the structure of the two-factor model from the study of Bruine de Bruin et al. (2007; components of factor1: RtF, UOC, ADR and CRP; components of factor2: RSN and RtSC). Table 4 displays the goodness of fit statistics for the two-factor model. As presented in Table 4, the model structure is different from the hypothesis (χ² (7) =12.03, p=0.10), and the χ² (7) is 1.72. The goodness of fit index (GFI=0.99), comparative fit index (CFI=0.96), non-normed fit index (NNFI=0.92), root mean square residual (RMR=0.01<0.05) and the root mean square error of approximation (RMSEA=0.05<0.08) indicate a good fit of the model.

Table 5: Means (and standard deviations) of overall score of A-DMC subscales by gender.

Multivariate GLM results indicated gender difference in A-DMC scores, F(6,324)=2.67, p=0.015. Specifically, ANOVAs revealed male students got higher RtSC score than female students (Table 5). Gender differences were not found between male students and female students in RtF, RSN, UOC, ADR and CRP.

Multivariate GLM also indicate differences of education background, F(6,324)=7.25, p<0.001. Undergraduates showed better performance on ADR and CRP than junior college students (Table 6). Differences in education background were not found between undergraduates and junior college students in another four A-DMC subscales. In addition, participants from public universities showed similar A-DMC performance to those from private universities, F(6,324)=1.95, p=0.072.

Table 6: Means (and standard deviations) of overall score of A-DMC subscales by education background.

4 Discussion

The reliability and validity of the Chinese A-DMC scale have been demonstrated in the present study. Small differences were found in internal consistency compared to the Slovak, English and Italian versions of the A-DMC. Bivariate correlations indicate A-DMC subscales in Chinese reflects different aspects of the A-DMC constructs (e.g., value assessment, belief assessment, integration and metacognition). Exploratory factor analysis with the principal factors extraction method and oblimin rotation also constructed a two-factor model according to eigenvalue (>1). The two-factor model explained a close proportions of variance (45.9%) to studies of Bruine de Bruin et al. (2007) and Bavolar (2013), 46.2%. Compared to the Slovak and the English A-DMC, the Chinese A-DMC in the current study has a different structure of the two-factor model, which partly resulted from cultural differences (Bavolar, 2013). The two-factor structure of A-DMC subscales in the study of Bruine de Bruin et al. (2007) was confirmed, however.

A gender difference was found in RtSC. RtSC tasks involved ability to estimate gains or losses properly based on relevant information and to make consistent decisions when irrelevant information changed. Most female participants were majoring in liberal arts. Females may be generally less interested in solving problems using mathematical intuition. Differences of ADR and CRP were found between participants with different education background. There was a big disparity of college entrance examination performance between junior college students and undergraduates. Ability to take examinations, and more general comprehension and learning abilities could account for the differences. Bivariate correlations between participants’ GPA and A-DMC scores suggest, however, that A-DMC subscales reflect more individual cognitive abilities related to decision making than merely academic performance. Because the age range was narrow in the present study, we did not compute the correlation between age and A-DMC subscales.

There were limitations of this study. Data in the present study were collected at different times, and participants were recruited from four different universities. We collected the A-DMC data one college at a time from September to November in 2016. Data from different periods and participants sampled from different population may bring better generalizability of the results.

Age and gender are usually assumed to affect cognitive ability and decision making style. Participants in the present study were college students covering a narrow age range. The Chinese A-DMC scale should be tested in a sample of wider age range. Most of participants were from the school of Economics and Management, so the proportion of female students was high. So the Chinese A-DMCs should be tested in a sample with a more balanced gender ratio.

The A-DMC, involving a series of daily decision tasks to assess individual cognitive decision skills of Consistency in Risk Perception, Recognize Social Norms, Resistance to Sunk Costs, Resistance to Framing, Applying Decision Rules and Under/overconfidence, is not only a diagnostic tool to individual behavioral decision ability but also a predictor of human real behaviors. Existing research has demonstrated that A-DMC score had predictive effect on health-risk behavior, moral behavior and financial behavior (Bruine de Bruin et al., 2007; Parker, Bruine de Bruin & Fischhoff, 2015). Therefore, the A-DMC could be useful in a Chinese context. The A-DMC can be introduced to research on different age groups in China. Systematically learning adolescents’ decision style and consciousness (e.g., recognizing social norms and applying decision rules) in their early stages, which is important to cultivate adolescents’ independence of decision making (e.g., clearly career planning), to maintain autonomy in making decisions (e.g., freely choosing majors, universities or jobs) and to improve accurate confidence of decision making (e.g., knowing well about one’s own decision ability). For middle aged and elderly people, Resistance to Framing, Resistance to Sunk Costs and Consistency in Risk Perception performances could provide information on how to improve quality of work decision and life decision (e.g., health, finance and consumption). Inspired by the previous studies of decision making competence, the A-DMC can be applied to specific groups (Desimone, 2016; e.g., medical workers, patients and managers) in China. The A-DMC helps not only with solutions to economic decision problems but also with solutions to social problems. Although following decision rules may not bring desired results, properly using these rules will make positive results more likely (Hastie & Dawes, 2010). In general, the A-DMC is an effective tool to measure individual decision abilities (Maule & Maule, 2016; Mason, 2016), and the Chinese version of A-DMC is validated in China