We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Volunteers are a key part of the archaeological labour force and, with the growth of digital datasets, these citizen scientists represent a vast pool of interpretive potential; yet, concerns remain about the quality and reliability of crowd-sourced data. This article evaluates the classification of prehistoric barrows on lidar images of the central Netherlands by thousands of volunteers on the Heritage Quest project. In analysing inter-user agreement and assessing results against fieldwork at 380 locations, the authors show that the probability of an accurate barrow identification is related to volunteer consensus in image classifications. Even messy data can lead to the discovery of many previously undetected prehistoric burial mounds.
Archaeologists frequently use written guidelines such as site manuals, recording forms, and digital prompts during excavations to create usable data within and across projects. Most written guidelines emphasize creating either standardized datasets or narrative summaries; however, previous research has demonstrated that the resulting datasets are often difficult to (re)use. Our study analyzed observations and interviews conducted with four archaeological excavation teams, as well as interviews with archaeological data reusers, to evaluate how archaeologists use and implement written guidelines. These excavation team and reuser experiences suggest that archaeologists need more specific best practices to create and implement written guidelines that improve the quality and usability of archaeological data. We present recommendations to improve written guidelines that focus on a project's methods, end-of-season documentation, and naming practices. We also present a Written Guidelines Checklist to help project directors improve their written guidelines before, during, and after fieldwork as part of a collaborative process. Ideally, these best practices for written guidelines will make it easier for team members and future reusers to incorporate their own and others’ archaeological data into their research.
Hidden consumption is a potential problem when consumers’ expenditure data from household surveys are used in demand analyses. A solution is to collect and use actual consumption data. This study compares demand estimation using consumption and expenditure data and evaluates meat demand in Nigeria. Data are from a nationally representative panel from Nigeria. The results show the elasticities estimated across both datasets were very similar; thus, if the only objective of data collection is to estimate elasticity using a demand system framework, collection of both types of data (consumption and expenditures) may be unnecessary. The elasticity estimates classify poultry, beef, and processed seafood as luxuries, while other meat and unprocessed seafood are classified as necessities. Own-price elasticities from both datasets indicated that poultry, beef, and processed seafood were price-elastic, and poultry was the most price-elastic.
Routine patient care data are increasingly used for biomedical research, but such “secondary use” data have known limitations, including their quality. When leveraging routine care data for observational research, developing audit protocols that can maximize informational return and minimize costs is paramount.
Methods:
For more than a decade, the Latin America and East Africa regions of the International epidemiology Databases to Evaluate AIDS (IeDEA) consortium have been auditing the observational data drawn from participating human immunodeficiency virus clinics. Since our earliest audits, where external auditors used paper forms to record audit findings from paper medical records, we have streamlined our protocols to obtain more efficient and informative audits that keep up with advancing technology while reducing travel obligations and associated costs.
Results:
We present five key lessons learned from conducting data audits of secondary-use data from resource-limited settings for more than 10 years and share eight recommendations for other consortia looking to implement data quality initiatives.
Conclusion:
After completing multiple audit cycles in both the Latin America and East Africa regions of the IeDEA consortium, we have established a rich reference for data quality in our cohorts, as well as large, audited analytical datasets that can be used to answer important clinical questions with confidence. By sharing our audit processes and how they have been adapted over time, we hope that others can develop protocols informed by our lessons learned from more than a decade of experience in these large, diverse cohorts.
Clinical trials provide the “gold standard” evidence for advancing the practice of medicine, even as they evolve to integrate real-world data sources. Modern clinical trials are increasingly incorporating real-world data sources – data not intended for research and often collected in free-living contexts. We refer to trials that incorporate real-world data sources as real-world trials. Such trials may have the potential to enhance the generalizability of findings, facilitate pragmatic study designs, and evaluate real-world effectiveness. However, key differences in the design, conduct, and implementation of real-world vs traditional trials have ramifications in data management that can threaten their desired rigor.
Methods:
Three examples of real-world trials that leverage different types of data sources – wearables, medical devices, and electronic health records are described. Key insights applicable to all three trials in their relationship to Data and Safety Monitoring Boards (DSMBs) are derived.
Results:
Insight and recommendations are given on four topic areas: A. Charge of the DSMB; B. Composition of the DSMB; C. Pre-launch Activities; and D. Post-launch Activities. We recommend stronger and additional focus on data integrity.
Conclusions:
Clinical trials can benefit from incorporating real-world data sources, potentially increasing the generalizability of findings and overall trial scale and efficiency. The data, however, present a level of informatic complexity that relies heavily on a robust data science infrastructure. The nature of monitoring the data and safety must evolve to adapt to new trial scenarios to protect the rigor of clinical trials.
Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems.
Methods:
To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality.
Results:
The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs.
Conclusion:
Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.
A stream of research on co-authorship, used as a proxy of scholars’ collaborative behavior, focuses on members of a given scientific community defined at discipline and/or national basis for which co-authorship data have to be retrieved. Recent literature pointed out that international digital libraries provide partial coverage of the entire scholar scientific production as well as under-coverage of the scholars in the community. Bias in retrieving co-authorship data of the community of interest can affect network construction and network measures in several ways, providing a partial picture of the real collaboration in writing papers among scholars. In this contribution, we collected bibliographic records of Italian academic statisticians from an online platform (IRIS) available at most universities. Even if it guarantees a high coverage rate of our population and its scientific production, it is necessary to deal with some data quality issues. Thus, a web scraping procedure based on a semi-automatic tool to retrieve publication metadata, as well as data management tools to detect duplicate records and to reconcile authors, is proposed. As a result of our procedure, it emerged that collaboration is an active and increasing practice for Italian academic statisticians with some differences according to the gender, the academic ranking, and the university location of scholars. The heuristic procedure to accomplish data quality issues in the IRIS platform can represent a working case report to adapt to other bibliographic archives with similar characteristics.
This Chapter describes principles of information management for health systems and the need to focus on key data items required to improve individual and population health. It discusses the collection and analysis of relevant, high-quality data and the importance of agreeing on health programme aims before defining the minimum data set. We review the derivation of health indicators, focusing on WHO indicators. Many indicators rely on linking data from different sources, which requires accurate personal identifiers. Data is useless unless reports based on it can be shared and understood, so data analysts should use different visualization techniques to facilitate and support user decisions such as self-service dashboards. We also review the many high quality, open source, free to use data capture, analysis and data sharing tools that can support health systems, concluding that it is rarely necessary to develop an information system from scratch. Finally, while big data analytics, artificial intelligence and machine learning capture many headlines, health system can achieve much using simple tools to capture relevant, high-quality data and turn it into actionable knowledge to support their decision makers.
The poor assessment of child malnutrition impacts both national-level trends and prioritisation of regions and vulnerable groups based on malnutrition burden. Namibia has reported a high prevalence of malnutrition among children younger than 5 years of age. The present study's aim was to identify the optimal methods for estimating child stunting and wasting prevalence in Namibia using two datasets with suspected poor data quality: Namibia Demographic and Health Surveys (NDHS) (1992–2013) and Namibia Household Income and Expenditure Survey (NHIES), 2015/16. This comparative secondary data analysis used two prevalence estimation methods: WHO flags and PROBIT. WHO flags is the standard analysis method for most national household surveys, while the PROBIT method is recommended for poor quality anthropometry. In NHIES (n 4960), the prevalence of stunting (n 4780) was 30·3 and 20·9 % for the WHO flags and PROBIT estimates, respectively, and the national wasting prevalence (n 4637) was 11·2 and 4·2 %, respectively. The trends in nutritional status from NDHS and NHIES showed improvement across WHO flags and PROBIT until 2013; however, from 2013 to 2016, PROBIT showed smaller increases in stunting and wasting prevalence (2·5 and 0·6 percentage points) than WHO flags (6·6 and 5·0 percentage points). PROBIT identified the Khoisan ethnic group and Northern geographical regions with the highest stunting and wasting prevalence, while WHO flags identified similar prevalence across most groups and regions. The present study supports the recommendation to use PROBIT when poor data quality is suspected for constructing trends, and for targeting regions and vulnerable groups.
The US government invests substantial sums to control the HIV/AIDS epidemic. To monitor progress toward epidemic control, PEPFAR, or the President’s Emergency Plan for AIDS Relief, oversees a data reporting system that includes standard indicators, reporting formats, information systems, and data warehouses. These data, reported quarterly, inform understanding of the global epidemic, resource allocation, and identification of trouble spots. PEPFAR has developed tools to assess the quality of data reported. These tools made important contributions but are limited in the methods used to identify anomalous data points. The most advanced consider univariate probability distributions, whereas correlations between indicators suggest a multivariate approach is better suited. For temporal analysis, the same tool compares values to the averages of preceding periods, though does not consider underlying trends and seasonal factors. To that end, we apply two methods to identify anomalous data points among routinely collected facility-level HIV/AIDS data. One approach is Recommender Systems, an unsupervised machine learning method that captures relationships between users and items. We apply the approach in a novel way by predicting reported values, comparing predicted to reported values, and identifying the greatest deviations. For a temporal perspective, we apply time series models that are flexible to include trend and seasonality. Results of these methods were validated against manual review (95% agreement on non-anomalies, 56% agreement on anomalies for recommender systems; 96% agreement on non-anomalies, 91% agreement on anomalies for time series). This tool will apply greater methodological sophistication to monitoring data quality in an accelerated and standardized manner.
This chapter is based on two standard reference corpora, the British National Corpus and the Corpus of Contemporary American English, as opposed to the multi-billion-word database of Google Books Ngrams, which has, despite its allure, not been used in many systematic linguistic studies so far. Focusing on indefinite article allomorphy (a vs an) as an orthographic cue to the phonological strength of ‹h›-onsets in British and American English, the size advantage of the Ngrams database expectedly plays out in larger type and token counts, more stable estimates and fewer distortions due to data sparsity. However, as metadata are extremely limited (to year and variety), a fully accountable analysis is not feasible. The case study illustrates how richly annotated corpora can shed light on potential disturbances arising from two sources: genre differences and between-author variability. A sensitivity analysis offers some degree of reassurance when extending the analysis to the Ngrams database. In this way, the authors demonstrate that the strengths and limitations of corpora and big data resources can, with due caution, be counterbalanced to answer questions of linguistic interest.
Discrete choice experiments are used to collect data that facilitates measurement and understanding of consumer preferences. A sample of 750 respondents was employed to evaluate a new method of best-worst scaling data collection. This new method decreased the number of attributes and questions while discerning preferences for a larger set of attributes through self-stated preference “filter” questions. The new best-worst method resulted in overall equivalent rates of transitivity violations and lower incidences of attribute non-attendance than standard best-worst scaling designs. The new method of best-worst scaling data collection can be successfully employed to efficiently evaluate more attributes while improving data quality.
Due to the higher costs and selection bias of directly measuring weight, the majority of body weight data are based on survey responses. However, these statements are subject to systematic biases of social desirability; therefore, it is important to evaluate the magnitude of bias through indirect indicators such as rounding of weights. Data from seven rounds of the Spanish National Health Survey from 1995 to 2017 were included in the study, with 113,284 subjects. A general rounding index of weights terminating in 0 and 5, and a partial rounding index that estimated the bias direction, were used to estimate the bias distribution in the self-reporting of body weight. All body weights were systematically rounded, although more strongly in the lower weights and even more so in the higher weights. Lower weights were rounded up, and the higher weights rounded down. Regarding gender, men had higher rounding indices than women. The subjects generally reported a weight closer to the socially desirable weight. Rounding allows estimating the historical evolution of this bias in health and nutrition surveys, having more accurate information by population segments and designing public policies against obesity aimed at the more affected social segments.
High-quality data are critical to the entire scientific enterprise, yet the complexity and effort involved in data curation are vastly under-appreciated. This is especially true for large observational, clinical studies because of the amount of multimodal data that is captured and the opportunity for addressing numerous research questions through analysis, either alone or in combination with other data sets. However, a lack of details concerning data curation methods can result in unresolved questions about the robustness of the data, its utility for addressing specific research questions or hypotheses and how to interpret the results. We aimed to develop a framework for the design, documentation and reporting of data curation methods in order to advance the scientific rigour, reproducibility and analysis of the data.
Methods:
Forty-six experts participated in a modified Delphi process to reach consensus on indicators of data curation that could be used in the design and reporting of studies.
Results:
We identified 46 indicators that are applicable to the design, training/testing, run time and post-collection phases of studies.
Conclusion:
The Data Acquisition, Quality and Curation for Observational Research Designs (DAQCORD) Guidelines are the first comprehensive set of data quality indicators for large observational studies. They were developed around the needs of neuroscience projects, but we believe they are relevant and generalisable, in whole or in part, to other fields of health research, and also to smaller observational studies and preclinical research. The DAQCORD Guidelines provide a framework for achieving high-quality data; a cornerstone of health research.
When collecting egocentric network data, visual representations of networks can function as a cognitive aid for depicting relationships, helping to maintain an overview of the relationships, and keeping the attention of the interviewees. Additionally, network maps can serve as a narration generator in qualitative and in mixed-methods studies. While varying visual instruments are used for collecting egocentric network data, little is known about differences among visual tools concerning the influence on the resulting network data, the usability for interviewees, and data validity. The article provides an overview of existing visually oriented tools that are used to collect egocentric networks and discusses their functions, advantages, and limitations. Then, we present results of an experimental study where we compare four different visual tools with regard to networks elicited, manageability, and the impact of follow-up questions. In order to assess the manageability of the four tools, we used the thinking aloud method. The results provide evidence that the decision in favor of a specific visual tool (structured vs. unstructured) can affect the size and composition of the elicited networks. Follow-up questions greatly affect the elicited networks and follow-up cues can level out differences among tools. Respondents tend to prefer the concentric circles tool, with some differences in preferences and manageability of tools between participants with low and those with high socioeconomic status. Finally, assets and drawbacks of the four instruments are discussed with regard to data quality and crucial aspects of the data collection process when using visual tools.
Audits play a critical role in maintaining the integrity of observational cohort data. While previous work has validated the audit process, sending trained auditors to sites (“travel-audits”) can be costly. We investigate the efficacy of training sites to conduct “self-audits.”
Methods:
In 2017, eight research groups in the Caribbean, Central, and South America network for HIV Epidemiology each audited a subset of their patient records randomly selected by the data coordinating center at Vanderbilt. Designated investigators at each site compared abstracted research data to the original clinical source documents and captured audit findings electronically. Additionally, two Vanderbilt investigators performed on-site travel-audits at three randomly selected sites (one adult and two pediatric) in late summer 2017.
Results:
Self- and travel-auditors, respectively, reported that 93% and 92% of 8919 data entries, captured across 28 unique clinical variables on 65 patients, were entered correctly. Across all entries, 8409 (94%) received the same assessment from self- and travel-auditors (7988 correct and 421 incorrect). Of 421 entries mutually assessed as “incorrect,” 304 (82%) were corrected by both self- and travel-auditors and 250 of these (72%) received the same corrections. Reason for changing antiretroviral therapy (ART) regimen, ART end date, viral load value, CD4%, and HIV diagnosis date had the most mismatched corrections.
Conclusions:
With similar overall error rates, findings suggest that data audits conducted by trained local investigators could provide an alternative to on-site audits by external auditors to ensure continued data quality. However, discrepancies observed between corrections illustrate challenges in determining correct values even with audits.
While many countries recognize the importance of collecting and using ART data, ART surveillance systems and other ART data collection tools, such as registries or repositories, vary according to ownership, reporting responsibility, type of data being reported, information being reported, data quality and validation activities, public reporting of success rates, reporting requirements, and data protection around the world. However, there is a need for ART surveillance systems to be simple, flexible, acceptable, representative, timely, and stable as well as to have high data quality, sensitivity, and positive predictive values. This chapter will explore each of these areas as they relate to ART surveillance.
Since the introduction of laser-assisted atom probe, analysis of nonconductive materials by atom probe tomography (APT) has become more routine. To obtain high-quality data, a number of acquisition variables needs to be optimized for the material of interest, and for the specific question being addressed. Here, the rutile (TiO2) reference material ‘Windmill Hill Quartzite,’ used for secondary ion mass spectrometry U–Pb dating and laser-ablation inductively coupled plasma mass spectrometry, was analyzed by laser-assisted APT to constrain optimal running conditions. Changes in acquisition parameters such as laser energy and detection rate are evaluated in terms of their effect on background noise, ionization state, hit-multiplicity, and thermal tails. Higher laser energy results in the formation of more complex molecular ions and affects the ionization charge state. At lower energies, background noise and hit-multiplicity increase, but thermal tails shorten. There are also correlations between the acquisition voltage and several of these metrics, which remain to be fully understood. The results observed when varying the acquisition parameters will be discussed in detail in the context of utilizing APT analysis of rutile within geology.
The aim of this study was to identify guidelines and assessment tools used by health technology agencies for quality assurance of registries and investigate the current use of registry data by HTA organizations worldwide.
Methods:
As part of a European Network for Health Technology Assessment Joint Action work package, we undertook a literature search and sent a questionnaire to all partner organizations on the work package and all organizations listed in the International Society for Pharmaco-economics and Outcomes Research directory.
Results:
We identified thirteen relevant documents relating to quality assurance of registries. We received fifty-five responses from organizations representing twenty-one different countries, a response rate of 40.5 percent (43/110). Many agencies, particularly in Europe, are already drawing on a range of registries to provide data for their HTA. Less than half, however, use criteria or standards to assess the quality of registry data. Nearly all criteria or standards in use have been internally defined by organizations rather than referring to those produced by an external body. A comparison of internal and external standards identified consistency in several quality dimensions, which can be used as a starting point for the development of a standardized tool.
Conclusion:
The use of registry data is more prevalent than expected, strengthening the need for a standardized registry quality assessment tool. A user-friendly tool developed in conjunction with stakeholders will support the consistent application of approved quality standards, and reassure critics who have traditionally considered registry data to be unreliable.
Within the NHS health check (NHSHC) programme, there is evidence of marked inconsistencies and challenges in practice-level self-reporting of uptake. Consequently, we explored the perceptions of those involved in commissioning of NHSHC to better understand the implications for local and national monitoring and evaluation of programme uptake. Semi-structured, one-to-one, telephone interviews (n=15) were conducted with NHSHC commissioners and leads, and were analysed using inductive thematic analysis. NHSHC data were often collected from practices using online extraction systems but many still relied on self-reported data. Performance targets and indicators used to monitor and feedback to general practices varied between localities. Participants reported a number of issues when collecting and reporting data for NHSHC, namely because of opportunistic checks. Owing to the perceived inaccuracies in reporting, there was concern about the credibility and relevance of national uptake figures. The general practice extraction service will be important to fully understand uptake of NHSHC.