Health literacy (HL) means being able to apply literacy skills to health-related materials such as prescriptions and directions for home health care( Reference Nutbeam 1 ). In recent years, the number of publications on the topic has increased, with some studies finding inverse relationships between HL and increased risk of adverse health outcomes such as hospitalization( Reference Adams, Appleton and Hill 2 ), diabetes, arthritis( Reference Adams, Appleton and Hill 2 , Reference Wolf, Gazmararian and Baker 3 ), hypertension( Reference Wolf, Gazmararian and Baker 3 ), IHD and stroke( Reference Adams, Appleton and Hill 2 ). Besides its association with health outcomes, some evidence indicates that HL is also a relevant concept to the understanding of dietary behaviours and nutrition. In a study with young adults, the low HL group had significantly lower use of food labels than the high HL group( Reference Cha, Kim and Lerner 4 ). Two recent systematic reviews summarized the evidence for nutrition literacy and food literacy, which can be seen as specific forms of HL. In both reviews, the authors concluded that the available evidence supports that literacy plays an important role in food choices and dietary intake( Reference Krause, Sommerhalder and Beer-Borst 5 , Reference Vaitkeviciute, Ball and Harris 6 ).
There is no consensus on the most appropriate method of measuring HL( Reference Taggart, Williams and Dennis 7 – Reference Altin, Finke and Kautz-Freimuth 9 ). While some instruments assess only medical words recognition( Reference Altin, Finke and Kautz-Freimuth 9 ), others have been proposed to assess related skills that make up HL (multidimensional tools) such as reading, oral communication, ability to understand terms related to health and numeracy (the ability to apply arithmetic operations and use numerical information)( Reference Taggart, Williams and Dennis 7 – Reference Lipkus and Peters 12 ). Thus, the most appropriate choice of the type of measurement instrument for HL depends on the study population and the inherent characteristics of each test. It is also important to consider the range of constructs assessed, the sensitivity of the instrument as well as how long it takes to apply the instrument( Reference Baker 10 ). Considering the above, the Newest Vital Sign (NVS)( Reference Weiss, Mays and Martz 11 ) is a multidimensional HL instrument that includes both objective (task based) and subjective (self-report based) measurements, broadly evaluates HL( Reference Weiss, Mays and Martz 11 ), is quick and easy to apply( Reference Barber, Staples and Osborne 13 ), has been validated in many languages and cultures( Reference Barber, Staples and Osborne 13 – Reference Ozdemir, Alper and Uncu 15 ), but not in Brazil, and shows good sensitivity in identifying marginal or inadequate levels of HL( Reference Baker 10 , Reference Weiss, Mays and Martz 11 , Reference Wolf, Curtis and Wilson 16 ). Moreover, it is associated with food labels usage( Reference Cha, Kim and Lerner 4 ), a known predictor of food choices and dietary intake. According to its developers, the instrument was named ‘Newest Vital Sign’ because it can be quickly applied in clinical settings to assess HL skills at the same time as the assessment of other vital signs( Reference Weiss, Mays and Martz 11 ). Briefly, the NVS tool consists of six questions that must be answered based on a nutritional label on a container of ice cream.
The main tools for measuring HL were formulated in developed countries( Reference Weiss, Mays and Martz 11 ) and more studies on the importance of HL in low- and middle-income countries are still needed( Reference Taggart, Williams and Dennis 7 , Reference Berkman, Sheridan and Donahue 8 ). In Brazil, none of the validated HL instruments is multidimensional( Reference Apolinario, Braga Rde and Magaldi 17 – Reference Apolinario, Mansur and Carthery-Goulart 19 ) and other international instruments have not been validated in the Brazilian population( Reference Paskulin, Aires and Valer 20 – Reference Carthery-Goulart, Anghinah and Areza-Fegyveres 22 ). In this country, of the three instruments found, two were validated from other languages( Reference Apolinario, Braga Rde and Magaldi 17 , Reference Rabelo, Mantovani and Aliti 18 ) and the other was originally developed in Brazilian Portuguese( Reference Apolinario, Mansur and Carthery-Goulart 19 ). In one, Apolinario et al.( Reference Apolinario, Braga Rde and Magaldi 17 ) studied a sample of 226 elderly Brazilians and validated an instrument with fifty items, which evaluate the individual’s ability to pronounce and understand common medical terms. Afterwards it was reduced to an eighteen-item test. In another study, Rabelo et al.( Reference Rabelo, Mantovani and Aliti 18 ) validated a survey to assess knowledge about heart failure in 153 patients diagnosed with this disease.
In the only tool created specifically for the Brazilian population, Apolinario et al.( Reference Apolinario, Mansur and Carthery-Goulart 19 ) proposed and validated a tool to investigate HL based on sociodemographic features, occupation, use of information and ability to read and write in 322 patients of the Hospital das Clínicas, Faculty of Medicine, University of São Paulo. Three other studies have reported the usage of different international HL assessment instruments in Brazil( Reference Paskulin, Aires and Valer 20 – Reference Carthery-Goulart, Anghinah and Areza-Fegyveres 22 ), although the validation process has not been concluded up to the present moment. In Brazil, because the population is ageing quickly and the prevalence of chronic conditions is also increasing rapidly( 23 ), it is strongly important to have a valid HL assessment tool to identify individuals with inadequate literacy levels. Identifying these individuals would favour the adoption of early action to avoid health problems in these groups or, at least, minimize their consequences.
Furthermore, considering the importance of HL as a predictor of clinical outcomes in health and the lack of a validated tool for the assessment of HL in Brazilian adults, the present study aimed at cultural adaptation to Brazilian Portuguese and validation of the NVS instrument for the general population (adult clients of community pharmacies) and also a highly educated population (public school teachers).
Method
Cross-cultural adaptation
The first step in cross-cultural adaptation of the NVS tool was to obtain authorization from the researchers who developed the instrument( Reference Weiss, Mays and Martz 11 ) and from the company that holds the copyright (Pfizer™). After obtaining authorizations, the process of cultural adaptation was based on the protocol proposed by Beaton et al.( Reference Beaton, Bombardier and Guillemin 24 ). The translations of the NVS from English to Brazilian Portuguese were carried out independently by two bilingual experts in public health, native to Brazil and literate in Portuguese. The translators were informed of the study objectives and the target population. The translated versions of the instrument were reviewed, with the translators, by a member of the research team, who resolved discrepancies and obtained the first version of the questions translated into Portuguese. Additionally, after being translated following the same process as the six questions, the nutritional information label (which is part of the tool) was adapted to the standards set out by the Brazilian legislation( Reference Brasil Ministério and Agência Nacional 25 ). This version was pre-tested in a pilot study with seven college students, chosen for convenience, to identify possible idiomatic and cultural problems. The outcome of this step brought us to version 2.
The back-translation from version 2 to English was done by an English-literate translator, who was unaware of the research objectives and did not take part in any of the previous steps. All versions, including the original, were reviewed by an expert committee consisting of six members, half of which were not part of the study group. This resulted in the final NVS-BR version, of which the semantic, idiomatic, experiential and conceptual equivalence were evaluated by the committee. Finally, the instrument was once again translated into English and submitted for approval to the original authors of the instrument, in order to identify possible changes that could compromise the new version. Only after this approval was the test validation initiated.
Test validation
The validation procedures require application of the instrument in samples with features that are similar to the target population in which it is intended to be applied( Reference Beaton, Bombardier and Guillemin 24 ), such as patients, health service users, workers and others. Studies involving HL are derived from populations with varied educational levels, such as the elderly or the general population. In these studies, there is always a positive correlation between educational level and HL( Reference Wolf, Curtis and Wilson 16 , Reference Baker, Wolf and Feinglass 26 – Reference Federman, Sano and Wolf 28 ). Therefore, beyond performing validation in a general population, it is considered important to validate the instrument in a population with high educational level since this analysis can help understanding HL in these cases.
The test validation included two different populations, both living in Londrina, a large town in southern Brazil. The first population was a convenience sample of the general population (GP), formed by adult clients in four community pharmacies. The data were collected from June to August 2015. The second, highly educated (HE) population (public school teachers) was a sub-sample of the PRO-MESTRE Study, which has been described elsewhere( Reference Camargo Koga, Nesello Melanda and dos Santos 29 ). Briefly, the PRO-MESTRE Study aimed to analyse health, lifestyle and work aspects in elementary and secondary public school teachers from the seven largest schools (i.e. those with the highest number of teachers). Between August and November 2012, a total of 330 teachers met the inclusion criteria (working in the classroom and being responsible for the taught subject) and answered questions in interviews that lasted approximately 40 min.
Both GP and HE were selected from different regions of the town. The inclusion criteria were: (i) age ≥18 years; (ii) to be able to read and speak Brazilian Portuguese; and (iii) absence of hearing or vision problems that could affect the interview process. The implementation of the NVS-BR was similar to the implementation of the original NVS, using a laminated card with the nutritional information of the ice cream label (14 point font size) printed on it. During the interview, the laminated card was given to the respondent so that he/she could answer the questions. The six instrument questions were asked orally and the responses recorded by the interviewer on a score sheet that contained the correct answers (which were not supplied to the respondent). The number of correct answers was linked directly to the level of HL. The cut-off points originally proposed, in which 0–1 correct responses suggested a high probability of limited HL, 2–3 correct responses indicated possibility of limited HL (borderline) and 4–6 correct responses indicated successful and appropriate HL( Reference Weiss, Mays and Martz 11 ), were used.
The instrument’s content and construct validity were evaluated by the expert committee, through the association between the number of correct responses on the NVS and sociodemographic variables (gender, age and education) and the teacher’s teaching area (only for HE).
Statistical analysis
The data were analysed using the statistical software package IBM SPSS Statistics version 19. The internal consistency of the tool was verified by means of Cronbach’s α coefficient, with a satisfactory result being equal to or greater than 0·70( Reference Cronbach and Meehl 30 ). The Spearman correlation test was used to verify the relationship between the score on the NVS and the age and education variables.
A cross-validation approach involving both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) was used for the factorial construct analysis of the NVS-BR instrument. In the database, individuals who answered the question NVS5 wrongly were assigned zero for question NVS6. Then, the EFA was performed using a polychoric matrix and geomin rotation. Factors with an eigenvalue greater than 1 were retained, confirmed for Very Simple Structure. The CFA was performed to confirm the number of factors on the scale. The following fit indices were examined to determine the most appropriate factor model: comparative fit index (CFI), Tucker–Lewis fit index (TLI) and the root-mean-square error of approximation (RMSEA). The average variance extracted (AVE) and composite reliability (CR) were performed to evaluate retained factors. Residual-based fit indices were measured using standardized root-mean-square residual (SRMR) and weighted root-mean-square residual (WRMR). R version 3.3.1 (package RODBC, Psych, Mvtnorm, MASS, doParallel, GPArotation and Laavan) was used to run these factor analyses.
The construct validity was tested through the analysis of the correlations of HL with education, age and gender through CFA. This included the assumption that the present cross-sectional design would not allow to infer a causal relationship between the variables.
Results
Cross-cultural adaptation
The process of cross-cultural adaptation on the NVS tool was satisfactory, requiring no changes in nutritional terms, and requiring a single change in the formation of a sentence in order to improve understanding of version 2. The committee of experts examined the original version and the translated version, during which they agreed that the translated version had good semantic, idiomatic, experiential and conceptual equivalence. There were also several studies that used the NVS, to back up their opinion( Reference Weiss, Mays and Martz 11 , Reference Wolf, Curtis and Wilson 16 , Reference Patel, Joel and Rovena 27 , Reference Kirk, Grzywacz and Arcury 31 ). All of this ensured the content validity of the instrument. The author of the original tool evaluated the back-translation of the final version and approved it. No cultural difficulty for reading the nutrition facts label on the part of respondents was noted. After all of this the final version was submitted to test validation as described (see online supplementary material, Supplemental Tables 1 and 2).
Validation
Among 247 GP who fulfilled the inclusion criteria, forty-three refused to participate and fifteen were excluded from the analyses because they did not complete the NVS test. The final population consisted of 189 GP, with a mean age of 40·2 (sd 12·5) years, 66·7 % women and 28·6 % with a high level of education. Among the sample of 330 HE who were eligible, four were impossible to get in contact with, twenty refused to participate and five were excluded from the analyses due to missing information at the NVS. The final population consisted of 301 HE, with a mean age of 42·1 (sd 10·0) years, 69·8 % women and 99·7 % with a higher education (Table 1). None of those interviewed (GP and HE) had visual or hearing impairment. In summary, gender and age distributions were similar for both populations, although frequency of age groups varied among them, and the HE participants presented higher level of education when compared with the GP (Table 1).
Table 1 Characteristics of the study participants: general population (GP) and highly educated (HE) samples of Brazilian adults, Londrina, June–November 2015 (GP); August–November 2012 (HE)
NVS-BR, Newest Vital Sign Brazilian Portuguese version.
According to the Brazilian Portuguese version of the NVS instrument (NVS-BR), 48·7 % of GP and 33·5 % of HE presented adequate health literacy. Cronbach’s α presented values of 0·75 for GP and of 0·74 for HE. For both groups, the exclusion of isolated questions in the NVS-BR resulted in values of Cronbach’s α less than or equal to that of the complete tool (Table 2). The construct validity was first verified by EFA and CFA. The initial EFA extracted a two-factor solution (eigenvalue>1), confirmed by Very Simple Structure, which explained 0·59 and 0·57 of the total variance in all items for GP and HE, respectively. The values of model fit indicated an acceptable model for GP in CFA, CFI=0·99, TLI=0·99, RMSEA=0·06, SRMR=0·06 and WRMR=0·81, and exhibited an independence between factor 1 (questions NVS1, NVS2, NVS3 and NVS4) and factor 2 (questions NVS5 and NVS6; Table 3). Similar results were found for HE: CFI=0·99, TLI=0·99, RMSEA=0·02, SRMR=0·04 and WRMR=0·51. Factor 1 showed AVE=0·62 and CR=0·86 for GP, and AVE=0·59 and CR=0·81 for HE; and factor 2 showed AVE=1·01; CR=1·00 for GP, and AVE=1·13 and CR=1·06 for HE.
Table 2 Cronbach’s α of the Newest Vital Sign Brazilian Portuguese version (NVS-BR) tool with isolated questions excluded among the general population (GP) and highly educated (HE) samples of Brazilian adults, Londrina, June–November 2015 (GP); August–November 2012 (HE)
Table 3 Description of confirmatory factor analysis for the Newest Vital Sign Brazilian Portuguese version (NVS-BR), with standardized estimates, for the general population (GP) and highly educated (HE) samples of Brazilian adults, Londrina, June–November 2015 (GP); August–November 2012 (HE)
For GP, the Spearman correlation test was used to compare the number of correct responses in the NVS tool with the level of education and age, observing ρ s=0·39 (P<0·001) and ρ s=−0·19 (P=0·009), respectively. For HE, the Spearman correlation test gave ρ s=0·07 (P=0·23) and ρ s=−0·26 (P<0·001), for education and age, respectively.
For GP, the level of education presented a statistically significant association with HL, and age showed a weak inverse correlation. For HE, the data showed a lack of correlation between educational level and number of correct responses in the NVS and a weak inverse correlation between age and the number of correct responses in the NVS. Furthermore, in this same stage of the analysis, the teacher’s teaching area, using mathematics as a reference, showed no statistical difference (P>0·05) compared with other teaching disciplines (natural sciences, humanities, languages and technical disciplines). Gender was not associated with HL in both populations.
Discussion
The results obtained in the cross-cultural adaptation stage were considered very satisfactory, confirming the objectivity and clarity of the newly translated NVS instrument. With the cross-cultural adaptation of the nutritional information of the ice cream label to the Brazilian standards and culture, possible difficulties that would arise if the original instrument was simply translated and applied, were avoided. Similar results were reported for the validation of the NVS in other countries, such as the Netherlands( Reference Fransen, Van Schaik and Twickler 14 ), Ireland( Reference Sahm, Wolf and Curtis 32 ), Canada( Reference Hudon, Fortin and Poitras 33 ), Turkey( Reference Ozdemir, Alper and Uncu 15 ), Australia( Reference Barber, Staples and Osborne 13 ), Portugal( Reference Martins and Andrade 34 ), Italy( Reference Capecchi, Guazzini and Lorini 35 ) and Japan( Reference Kogure, Sumitani and Suka 36 ), making it one of the most widespread HL assessment tools.
The cross-cultural adaptation was done according to the literature, with satisfactory validity. It is important to note that the process of cross-cultural adaptation must be carried out whenever a tool is used in a place with differences in language or culture( Reference Beaton, Bombardier and Guillemin 24 ). This can be evidenced by comparing the present study with the cross-cultural adaptation carried out in Portugal, with differences in several terms, units of measure, arrangement of the ice cream components and in the reference values for daily consumption( Reference Martins and Andrade 34 ). The NVS tool shows an important limitation in its low specificity for detecting inappropriate HL, which is inherent to the scale. Based on this, it is important to emphasize that the NVS is a good instrument for screening, but is limited in its ability to discriminate HL completely.
The psychometric properties of the NVS-BR were suitable. The internal consistency of the instrument (Cronbach’s α of 0·75 and 0·74) was satisfactory compared with other studies that used the NVS, where α ranged between 0·69 and 0·78( Reference Weiss, Mays and Martz 11 , Reference Fransen, Van Schaik and Twickler 14 , Reference Ozdemir, Alper and Uncu 15 ). Inter-rater agreement was ensured by calibration and training of interviewers. For instruments that assess knowledge, test stability is not used as a reliability measure( Reference Polit, Beck and Hungler 37 ) and, therefore, test stability was not tested in the current validation process. It is our opinion that the internal consistency of the instrument is very satisfactory since Cronbach’s α was adequately high and there was no problem in ensuring the inter-rater agreement. The content validity was verified by the expert committee, ensuring that the NVS measures what it claims to measure, i.e. HL.
The judgement of the expert committee relies both on the verification of issues identified in relation to the type of questions asked on the subject and on several studies that used the NVS as a tool for measuring HL( Reference Weiss, Mays and Martz 11 , Reference Ozdemir, Alper and Uncu 15 , Reference Wolf, Curtis and Wilson 16 ). Furthermore, the correlation between NVS and other tools for measuring HL is well established in the literature, a fact that confirms their content validity( Reference Weiss, Mays and Martz 11 , Reference Wolf, Curtis and Wilson 16 , Reference Patel, Joel and Rovena 27 , Reference Kirk, Grzywacz and Arcury 31 , Reference Dunn-Navarra, Stockwell and Meyer 38 ).
In the current study, a weak correlation was observed between lower HL and increasing age for both populations, a relationship reported in several studies using the NVS instrument( Reference Weiss, Mays and Martz 11 , Reference Barber, Staples and Osborne 13 , Reference Ozdemir, Alper and Uncu 15 ) and in others that used different instruments to measure HL( Reference Paasche-Orlow, Parker and Gazmararian 39 , Reference Baker, Williams and Parker 40 ). The correlation between appropriate HL and higher education has been demonstrated in several other studies using the NVS( Reference Barber, Staples and Osborne 13 , Reference Ozdemir, Alper and Uncu 15 , Reference Kirk, Grzywacz and Arcury 31 ), which is consistent with the present study for GP but not for HE. This lack of correlation between HL and educational level could be attributed to the homogeneous profile of the HE population regarding this characteristic.
The NVS instrument has a high sensitivity for the detection of inappropriate HL; however, it suffers from lower specificity for inappropriate HL when compared with other tools such as TOFHLA( Reference Barber, Staples and Osborne 13 ) and REALM( Reference Barber, Staples and Osborne 13 , Reference Ozdemir, Alper and Uncu 15 ). One possible explanation for this difference is the structure of the NVS tool, because four of the six items require the performance of calculations, which is directly related to numeracy ability. It can therefore be suggested that the NVS measures numeracy skills rather than HL skills. However, in the current study, there was no difference in the NVS results of mathematics teachers and teachers of other subjects. This fact does not exclude a possible overestimation of numeracy, but it does indicate that in a population who has a high school qualification, this overestimation may not be significant. Furthermore, the GP showed higher numbers of correct answers in three of four questions involving numeracy (factor 1).
Although the study of HL is receiving greater attention in recent years, it mostly focuses on patients with very specific health conditions or focuses on their own non-standard methodologies. Thus, it is not possible to properly compare the results of the present study with those found in the literature, since the differences may be explained by several different variables (age, gender distribution, presence of chronic disease, occupation and culture of the country/region, among others). In the current study, differences in HL among the GP and HE samples can result from personal experiences( Reference Nutbeam 1 ) of GP, which is apparently more important for the development of HL than education.
The results of the CFA showed acceptable models and identified two independent factors according to the relationship between components and numeracy for both GP and HE data. Criterion validity was not performed with the gold standard for measuring HL, TOFHLA, because it was not validated for Brazilian Portuguese in the survey design stage.
The instrument was used to determine the HL in convenience samples of GP and HE in Londrina, and external validity of the present findings is limited. However, this limitation is shared with other validation studies( Reference Weiss, Mays and Martz 11 , Reference Apolinario, Mansur and Carthery-Goulart 19 ); moreover, the sample size of both study populations was appropriate for the purpose of the study. It is also important to consider that, for both GP and HE participants, application of the NVS occurred in a context given for the collection of various pieces of information, rather than studies specifically designed for the present validation. Thus, we cannot guarantee that results are similar if the study occurs in more favourable and simplified conditions.
Moreover, although the comparison between these populations regarding HL was not an objective in the present study, it is remarkable that GP had a higher percentage of adequate HL than the HE population (48·7 and 33·5 %, respectively). This is possibly due to methodological differences between both sub-studies; that is, the study design and the circumstances in which data were collected, such as location and time spent in the full interview, including application of the NVS. We hypothesize that the low percentage of adequate literacy shown by the HE group is due – at least partially – to the fact that teachers were tired of answering so many questions that made up the PRO-MESTRE instrument, and consequently paid less attention to the NVS issues. In this case, it is possible that the insertion of NVS into studies with a very great number of questions and scales could impact the results by reducing the specificity in identifying individuals with inadequate or borderline HL. Nevertheless, the psychometric properties of the NVS in this group were also adequate. In addition, the factor analysis results confirmed the validity of the instrument for application in both groups. Indeed, HL is conceptually different from the formal educational level, since a highly educated population would not necessarily have to present higher HL when compared with the general population.
Lastly, the content of the NVS based on the reading, processing and understanding of a food label approximates the most frequent definition used to describe instruments specifically designed to evaluate nutrition literacy( Reference Krause, Sommerhalder and Beer-Borst 5 ). However, it is interesting that NVS is widely recognized as a tool for evaluating HL in its broader concept( Reference Baker 10 , Reference Weiss, Mays and Martz 11 ), and not specifically nutrition literacy. In fact, since the definitions of nutrition literacy are linked directly to existing definitions or concepts of HL( Reference Krause, Sommerhalder and Beer-Borst 5 ), future studies are required to examine whether there is correspondence between the results of instruments for the assessment of HL, such as NVS, and tools developed to evaluate specific forms of HL, like nutrition literacy and food literacy.
The present study provides evidence that the Brazilian Portuguese version of the NVS tool (NVS-BR) has good validity and can be used for screening inadequate HL in these populations. However, the study did not intend to exhaust the analyses of validity of the NVS for application in Brazil, since the concept of validity cannot be considered definitive and should be consolidated in future studies with different populations of Brazilian adults. Prospective cohort studies are also required to evaluate if inadequate HL could be considered a predictor of adverse health outcomes in Brazilian adults, such as obesity or CVD. Although the NVS evaluates general HL based on answers to questions about food labels, it would be of interest to nutritional epidemiology to examine the sensitivity and specificity of this tool in comparison with other tools specifically developed to access food literacy and nutritional literacy.
Acknowledgements
Financial support: This study was partially funded by the Araucaria Foundation (Funding Agency of Parana State, Brazil; grant number 168/2014). R.R. has received a scholarship from CAPES (Coordination for the Improvement of Higher Level Personnel, Ministry of Education, Brazil) as part of the funding for her Master’s dissertation. S.M.A. and A.E.M. have research productivity scholarships from CNPq (Brazilian National Research Council). Conflict of interest: The authors declare that they have no conflicts of interest. Authorship: All authors developed the idea of the manuscript. R.R. was responsible for the analysis of the data, drafted and wrote the manuscript. S.M.A., A.D.G., M.M.B. and A.E.M. contributed to the interpretation of the data and in revising the manuscript critically. All authors read and approved the final manuscript. Ethics of human subject participation: This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects/patients were approved by the Ethics Research Committee of the State University of Londrina and University of Northern Paraná. Written informed consent was obtained from all subjects/patients.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1368980017000787
