To the Authors:
Many thanks to van der Hoorn and colleagues for their interest in our article and the interesting discussion points. Their comments are very important and deserve to be answered.
We completely agree that reprocessing of existing search histories is easier and time saving if they are available as one-line strings. However, there are some perils one should be aware of. First, in PubMed results are often not the same for long one-line entries and identical queries which were processed in several steps. This may in part be dependent on connection issues and network performance, but in our testing, when using the same connection several times, we still got differing results with some excessively long and complicated queries.
Algorithmically one-line strings seem to be fully equivalent, but practically and technically speaking one sometimes does get different results, depending on query length and structure. Second, quality control and strategy refinement and optimization are greatly supported by a process of stepwise modification and testing.
Additionally, if one uses preserved search strings for automatic re-processing and updating of searches it would be advisable to always check for changes in Medical
Subject Headings (MeSH) definitions, new or deleted terms or changes in indexing. One-line strings do have the advantage of being re-processable at a mouse click, but one will not be able to identify the aforementioned changes. If one is aware of the danger and if the search results are noncritical, like in a search performed for personal purposes of staying up-to-date on a topic, a one-line string can be a perfect approach. One of uses this quite extensively for the exact purpose of staying personally informed.
For the purpose of health technology assessment (HTA) we would not really recommended to exactly reprocess existing one-line strings, at least not without performing checks for any of the mentioned changes.
Your second point of discussion is the reporting of resulting hits in every query row. We do not agree with the statement that these are irrelevant numbers. On one hand there is the formal demand of maximum transparency in HTA which asks for these numbers. On the other hand it is very helpful when working on a search string to enhance sensitivity and precision of the strategy.
Sensitivity or precision-relevant changes at one step, that is, within one row, lead to immediate changes in the number of hits for this row. Thus the identification of those parts of the strategy which have the most influence on the effectiveness and precision of the whole strategy become immediately visible. Search terms that lead to imprecise or ambiguous results tend to increase the number of hits and thus can become suspect more easily.
Furthermore, in published search histories quoting of hits for each query line is very important and helpful. Errors in search strategies or incompleteness in reporting can be uncovered easier. Thus, this seemingly awkward way of reporting does lead to more efficiency when critically judging or reusing existing results and thus also helps with sharing workload.
If hit nominations are restricted to topical search components, as suggested by you, these advantages are lacking.
Apart from the above considerations, we do not suggest making the reporting of field tags for every search term compulsory. Indicating only those entries that differ from the default (in PubMed “All fields” is default) is probably self-explanatory and transparent for all readers. Nominating default field tags, in our opinion, may lead to confusion for some readers. We agree, however, that it would provide some consistency and the provision of default field tags might be quite ok if you want to use them.
Regarding the indication of the number of results for each aspect one could get that quite easily if all search terms were entered line by line and then every search component were finished by one line combining the search queries by Booleans “OR.” Thus, one result would be presented for each topical search component or aspect.
Applying this method one could construct the complete search strategy by combining the results of the topical search components by Boolean “AND.” Related to your example this would translate as: result of search component “postpartum bleeding” AND result of search component “surgery” AND result of search component “complications.”
Your third point on color coding: That seems to us in fact a very good idea for additional differentiation of search components and we welcome your suggestion.
Table 1 in our article was presented for illustrational purposes only. This search strategy—generated by PubMed Search Builder—is not a systematic search on the topic. Comprehensive information retrieval processes for HTAs exceed the limits of on-paper tabular presentations. To make amends for this, we added complete documentation of a “real life” example by means of supplemental material on the journal Web page.
We are looking forward to further discussions on enhancement of reporting and presenting comprehensive search strategies!
CONFLICT OF INTEREST
The authors do not report any potential conflicts of interest.
