This study uses KL2 scholars’ publications to evaluate the types of research the KL2 program supports and to assess the initial productivity and impact of its scholars.

We illustrate the feasibility of 3 different approaches to bibliometrics, one viable method for determining the types of research a program or hub supports, and demonstrate how these data can be further combined with internal data records.

Gender differences were observed in the types of research scholars undertake. Overall KL2 scholars are performing well, with their publications being cited more than the norm for National Institutes of Health publications. Favorable results were also observed in scholars’ continued engagement in research.

This study illustrates that linking bibliometric data and data categorizing publications along the translational spectrum with a Clinical and Translational Science Award hub’s internal data records is feasible and offers a number of innovative possibilities for the evaluation of a Clinical and Translational Science Award hub’s programs and investigators.

High-quality data are critical to the entire scientific enterprise, yet the complexity and effort involved in data curation are vastly under-appreciated. This is especially true for large observational, clinical studies because of the amount of multimodal data that is captured and the opportunity for addressing numerous research questions through analysis, either alone or in combination with other data sets. However, a lack of details concerning data curation methods can result in unresolved questions about the robustness of the data, its utility for addressing specific research questions or hypotheses and how to interpret the results. We aimed to develop a framework for the design, documentation and reporting of data curation methods in order to advance the scientific rigour, reproducibility and analysis of the data.

Forty-six experts participated in a modified Delphi process to reach consensus on indicators of data curation that could be used in the design and reporting of studies.

We identified 46 indicators that are applicable to the design, training/testing, run time and post-collection phases of studies.

The Data Acquisition, Quality and Curation for Observational Research Designs (DAQCORD) Guidelines are the first comprehensive set of data quality indicators for large observational studies. They were developed around the needs of neuroscience projects, but we believe they are relevant and generalisable, in whole or in part, to other fields of health research, and also to smaller observational studies and preclinical research. The DAQCORD Guidelines provide a framework for achieving high-quality data; a cornerstone of health research.

The purpose of this research was to understand the preferences of patients receiving integrative medicine services for return of aggregate study results.

A brief online survey (survey 1) was sent to 341 cancer patients receiving integrative medicine interventions; subsequently, a minimally revised survey (survey 2) was sent to 812 individuals with various medical conditions who had been either research participants in integrative medicine studies (n = 446) or patients (n = 346) of mind–body medicine.

Feedback to a model plain language summary was elicited from survey 1 and survey 2 respondents. Seventy-seven survey recipients (23%) responded to survey 1, and 134 survey recipients (17%) responded to survey 2. The majority of respondents to the surveys were female and 51–70 years of age. Ninety percent of responders to survey 1 and 89% of responders to survey 2 indicated that researchers should share overall results of a study with participants. In terms of the means of result distribution, 37%–47% preferred email, while 22%–27% indicated that, as long as the results are shared, it did not matter how this occurred. Of 38 survey 1 respondents who had previously participated in a clinical trial, 37% had received the results of their study. In survey 2, 63 individuals indicated that they previously participated in clinical trials, but only 16% recalled receiving results.

These results confirm that the majority (89%–90%) of integrative medicine patients are interested in receiving the results of clinical trials. The majority (82%–94%) of respondents felt the model plain language summary of results provided was helpful.

The final rule for the protection of human subjects requires that informed consent be “in language understandable to the subject” and mandates that “the informed consent must be organized in such a way that facilitates comprehension.” This study assessed the readability of Institutional Review Board-approved informed consent forms at our institution, implemented an intervention to improve the readability of consent forms, and measured the first year impact of the intervention.

Readability assessment was conducted on a sample of 217 Institutional Review Board-approved informed consents from 2013 to 2015. A plain language informed consent template was developed and implemented and readability was assessed again after 1 year.

The mean readability of the baseline sample was 10th grade. The mean readability of the post-intervention sample (n=82) was seventh grade.

Providing investigators with a plain language informed consent template and training can promote improved readability of informed consents for research.

It is not clear how to effectively recruit healthy research volunteers.

We developed an electronic health record (EHR)-based algorithm to identify healthy subjects, who were randomly assigned to receive an invitation to join a research registry via the EHR’s patient portal, letters, or phone calls. A follow-up survey assessed contact preferences.

The EHR algorithm accurately identified 858 healthy subjects. Recruitment rates were low, but occurred more quickly via the EHR patient portal than letters or phone calls (2.7 vs. 19.3 or 10.4 d). Effort and costs per enrolled subject were lower for the EHR patient portal (3.0 vs. 17.3 or 13.6 h, $113 vs. $559 or $435). Most healthy subjects indicated a preference for contact via electronic methods.

Healthy subjects can be accurately identified from EHR data, and it is faster and more cost-effective to recruit healthy research volunteers using an EHR patient portal.

This study employed the Delphi method, an exploratory method used for group consensus building, to determine the benefits and challenges associated with community engagement in patient-centered outcomes research.

A series of email surveys were sent to the Patient-Centered Outcomes Research Institute (PCORI)-funded researchers (n = 103) in New England. Consensus was achieved through gathering themes and engaging participants in ranking their level of agreement over three rounds. In round one, participant responses were coded thematically and then tallied. In round two participants were asked to state their level of agreement with each of the themes using a Likert scale. Finally, in round three, the group was asked to rank the round two themes based on potential impact.

Results suggested the greatest benefit of community engagement is that it brings multiple perspectives to the table, with 92% ranking it as the first or second most important contribution. Time was ranked as the most significant barrier to engaging community. Strategies to overcome barriers to community engagement include engaging key stakeholders early in the research, being kind and respectful and spending time with stakeholders. The most significant finding was that no researchers reported having specific measures to evaluate community engagement.

Community engagement can enhance both research relevance and methodology when researchers are engaged in meaningful collaborations. Advancing the science of community engagement will require the development of evaluation metrics to examine the multiple domains of partnership.

The application of digital monitoring biomarkers in health, wellness and disease management is reviewed. Harnessing the near limitless capacity of these approaches in the managed healthcare continuum will benefit from a systems-based architecture which presents data quality, quantity, and ease of capture within a decision-making dashboard.

A framework was developed which stratifies key components and advances the concept of contextualized biomarkers. The framework codifies how direct, indirect, composite, and contextualized composite data can drive innovation for the application of digital biomarkers in healthcare.

The de novo framework implies consideration of physiological, behavioral, and environmental factors in the context of biomarker capture and analysis. Application in disease and wellness is highlighted, and incorporation in clinical feedback loops and closed-loop systems is illustrated.

The study of contextualized biomarkers has the potential to offer rich and insightful data for clinical decision making. Moreover, advancement of the field will benefit from innovation at the intersection of medicine, engineering, and science. Technological developments in this dynamic field will thus fuel its logical evolution guided by inputs from patients, physicians, healthcare providers, end-payors, actuarists, medical device manufacturers, and drug companies.

To identify potential participants for clinical trials, electronic health records (EHRs) are searched at potential sites. As an alternative, we investigated using medical devices used for real-time diagnostic decisions for trial enrollment.

To project cohorts for a trial in acute coronary syndromes (ACS), we used electrocardiograph-based algorithms that identify ACS or ST elevation myocardial infarction (STEMI) that prompt clinicians to offer patients trial enrollment. We searched six hospitals’ electrocardiograph systems for electrocardiograms (ECGs) meeting the planned trial’s enrollment criterion: ECGs with STEMI or > 75% probability of ACS by the acute cardiac ischemia time-insensitive predictive instrument (ACI-TIPI). We revised the ACI-TIPI regression to require only data directly from the electrocardiograph, the e-ACI-TIPI using the same data used for the original ACI-TIPI (development set n = 3,453; test set n = 2,315). We also tested both on data from emergency department electrocardiographs from across the US (n = 8,556). We then used ACI-TIPI and e-ACI-TIPI to identify potential cohorts for the ACS trial and compared performance to cohorts from EHR data at the hospitals.

Receiver-operating characteristic (ROC) curve areas on the test set were excellent, 0.89 for ACI-TIPI and 0.84 for the e-ACI-TIPI, as was calibration. On the national electrocardiographic database, ROC areas were 0.78 and 0.69, respectively, and with very good calibration. When tested for detection of patients with > 75% ACS probability, both electrocardiograph-based methods identified eligible patients well, and better than did EHRs.

Using data from medical devices such as electrocardiographs may provide accurate projections of available cohorts for clinical trials.

Computational analysis of genome or exome sequences may improve inherited disease diagnosis, but is costly and time-consuming.

We describe the use of iobio, a web-based tool suite for intuitive, real-time genome diagnostic analyses.

We used iobio to identify the disease-causing variant in a patient with early infantile epileptic encephalopathy with prior nondiagnostic genetic testing.

Iobio tools can be used by clinicians to rapidly identify disease-causing variants from genomic patient sequencing data.