The following journals are being searched prospectively by hand:
Advances in Contraception:
The Journal of Family Planning and Reproductive Health Care
[formerly Journal of Family Planning Doctors and British Journal of Family Planning]
Contraception
European Journal of Contraception and Reproductive Health Care
Family Planning Perspectives
Studies in Family Planning

Population Reports: Series A Oral Contraceptives
Series B Intrauterine Devices
Series C Female Sterilization
Series D Male Sterilization
Series F Pregnancy Termination
Series G Prostaglandins
Series H Barrier Methods
Series I Periodic Abstinence
Series J Family Planning Programs
Series K Injectables and Implants
Series M Special Topic Monographs

Cochrane Hand Search Manual
Part I: Overview of Searching Activities

• 1. Introduction
• 2. International Register of RCTs of Health Care
  • 2.1 The need for a register of randomized clinical trials (RCTs)
  • 2.2 A "virtual" register of randomized trials within MEDLINE
  • 2.3 Eligibility criteria
  • 2.4 Classification of eligible trials
  • 2.5 Searching for RCTs and CCTs
  • 2.6 Retrospective Re-tagging of Existing Records in MEDLINE
  • 2.7 Bibliographic Records for Identified RCTs not in MEDLINE
  • 2.8 Creating Specialist Registers
• Figure 1
• Figure 2
• Figure 3
• 3. Procedures for Journal Searching
  • 3.1 Checklist of steps involved in searching individual journals
  • 3.2 Preliminary activities
  • 3.3 Search the journal(s)
  • 3.4 Forward citations to the New England Cochrane Center at Providence
• 4. Quality Control Strategies
  • 4.1 Reasons for quality control
  • 4.2 Unnecessary duplication of effort
  • 4.3 Incomplete searches - unavailable journal materials
  • 4.4 Incomplete searches - lost photocopies
  • 4.5 Incomplete searches - missed reports
  • 4.6 Classification errors
  • 4.7 Typographical errors
• References
• Appendix 1
• Criteria for registering studies
  • Overarching principle:
  • Eligibility Criteria
• Appendix 2: Journal Hand Search Registration Form

1. Introduction

The aims of the Cochrane Collaboration are "preparing, maintaining and disseminating systematic reviews of the effects of health care". Before one can prepare a systematic review, one has to find the relevant research reports; in the case of the Cochrane Collaboration, this usually means reports of randomized clinical trials [1]. This can be surprisingly difficult [2,3]. Cochrane reviewers rely on several means of searching for relevant reports, including both electronic and manual methods. For complete identification of published reports, there appears to be no alternative to a page-by-page search of the literature. This Manual is designed to aid this type of manual searching.

The Manual chapters are written primarily with members of Cochrane Centres, Collaborative Review Groups, Fields and Networks in mind, but we hope that the information presented will be helpful to all involved with searching for trials. Chapter 2 describes the development of the International Register of RCTs of Health Care, and how Cochrane searching activities as a whole will be coordinated to achieve its completion. Chapter 3 presents a step-by-step checklist of searching activities, while Chapter 4 addresses concerns regarding quality control. Part II of this Manual contains an ever-growing set of examples, step-by-step instructions, and model documents prepared for and used by selected Cochrane searching teams who are carrying out the tasks discussed here. Part III contains a complete training course in hand searching that can be customized to suit the particular needs of any Cochrane searching group.

Note:Parts II and III of the Hand Search Manual are not available in electronic form. Contact the New England Cochrane Center at Providence for a complete printed copy of Part II and Part III of the Hand Search Manual.

Hundreds of journals have been or are being hand searched by members of the Cochrane Collaboration, some journals by as few as one and others by as many as 32 individual searchers. As of this writing, an estimated 1,000 searchers are contributing to this effort. As more trials are identified, it will be possible to prepare, maintain and disseminate increasing numbers of systematic reviews.

 2. International Register of RCTs of Health Care

 2.1 The need for a register of randomized clinical trials (RCTs)

A necessary starting point for performing systematic reviews of RCTs is identification of the maximum possible number of potentially relevant trials. Data collection for a systematic review involves setting eligibility criteria for all RCTs to be included in the review, identification of the relevant RCTs, and extraction of relevant data from the individual trials. Complete identification of RCTs for a systematic review, even when fully published reports are available, continues to be remarkably difficult [2,3]. In addition, the limitations of existing systems for electronically and manually searching bibliographic databases make the process time consuming and inefficient [4,5]. Therefore, it is essential to develop a register or registers of all existing RCTs to reduce the searching burden on individual reviewers, to help ensure that RCTs are identified as completely as possible, and to facilitate the production of increasing numbers of systematic reviews.

 2.2 A "virtual" register of randomized trials within MEDLINE

The Cochrane Collaboration is working with the National Library of Medicine (NLM) to create an International Register of RCTs of Health Care, which will be mounted and maintained as a "virtual" register within NLM's MEDLARS system of bibliographic databases. The project is being coordinated by the New England Cochrane Center at Providence.

Cochrane collaborators are systematically searching the medical literature to identify trials that meet the Cochrane inclusion criteria for the International Register (see Appendix 1). Trial reports will be sent to staff at the New England Cochrane Center at Providence, who will in turn communicate this information to the NLM, where citation records for identified trials will be assigned specified Medical Subject Headings (MeSH) tags within MEDLINE. Those interested in identifying trials for a systematic review will be able to search MEDLINE for citations with these MeSH tags and have some confidence that the search will identify all of the relevant trials. A diagram of the register-building process, as it is currently envisioned, is provided in Figure 1. Activities shown on the left side of the figure are currently in progress; those shown on the right are proposed for the future.

 Figure 1

 2.3 Eligibility criteria

The eligibility criteria established by the Cochrane Collaboration for reports to be included in the International Register of RCTs of Health Care are provided in Appendix 1. To ensure that the greatest proportion of published randomized trials will be identified, the agreed eligibility criteria are designed to be sensitive rather than specific. That is, all articles that definitely or possibly describe trials that prospectively assigned participants to one of two (or more) alternative forms of health care using either random allocation or some quasi-random method of allocation (such as alternation or medical record number) are eligible for inclusion.

The term "trial" is used in its broadest sense: it includes any prospective study that compared two or more interventions concurrently and was performed in humans. Even when a study was performed in healthy people and/or may not concern health care directly, it should be included in the International Register because it may contain information that is relevant to the evaluation of a health care intervention.

 2.4 Classification of eligible trials

Trials eligible for inclusion in the International Register of RCTs of Health Care are classified according to the reader's degree of certainty that random allocation was used to form the comparison groups in the trial. If the author(s) state explicitly (usually by some variant of the term "random" to describe the allocation procedure used) that the groups compared in the trial were established by random allocation, then the trial is classified as an "RCT" (randomized controlled trial). If an eligible trial has not been explicitly described by the author as randomized, then there is less certainty that it is, in fact, an RCT. This uncertainty is reflected in a different classification: "CCT" (controlled clinical trial). The classification "CCT" is also applied to quasi-randomized studies where the method of allocation is known but is not considered strictly random. The classification is based solely on what the author has written, not on the reader's interpretation.

The Cochrane classification terms "RCT" and "CCT" are intended to conform to the NLM's definitions of the publication type MeSH terms RANDOMIZED-CONTROLLED-TRIAL and CONTROLLED-CLINICAL-TRIAL (see Figures 2 and 3). The use of two classification terms balances the NLM's need for precision in the application of index terminology with the Collaboration's need for comprehensiveness in the selection of reports to be included in the International Register.

Since the classification terms are based on the author's description of the method of allocation used, they are not meant to reflect the actual quality of the allocation procedure. A trial report which states that the study participants were randomized is classified as "RCT", despite the fact that quasi-random methods may actually have been used. Conversely, although double-blind trials are nearly always randomized, many trial reports fail to mention this explicitly and such reports are classified as "CCT". It is the responsibility of those performing systematic reviews to decide which trials to include and exclude in any given review.

 Figure 2

National Library of Medicine Definition
PT RANDOMIZED-CONTROLLED-TRIAL

 Figure 3

National Library of Medicine Definition
PT CONTROLLED-CLINICAL-TRIAL

 2.5 Searching for RCTs and CCTs

Prospective identification and tagging of RCTs and CCTs will continue to be part of regular MEDLINE indexing operations. RCTs and CCTs will also be identified prospectively by Cochrane collaborators and journal editors who have agreed to search individual journals prospectively.

Both electronic (of MEDLINE and other bibliographic databases) and manual searching of individual journals and other literature, is being done by the Collaboration with the assistance of volunteers, editors and publishers of biomedical journals around the world. Each of the Collaborative Review Groups, Fields and Networks are responsible for searching the specialist literature in their particular area of interest, while the Cochrane Centres are responsible for searching the general medical literature published in their country or region. The overall coordination of all Cochrane search efforts is being handled by the New England Cochrane Center at Providence.

During 1994, Carol Lefebvre and staff of the UK Cochrane Center electronically searched MEDLINE's publication years 1985 through 1993, scanned the search output, and identified 20,000 "new" RCTs for the Register. Concurrently, Cochrane Collaborators were engaged in hand searching: by January, 1995, the search of nearly 500 journals was either underway or completed.

 2.6 Retrospective Re-tagging of Existing Records in MEDLINE

Cochrane hand searching groups will prepare and document electronic files of all citations to identified RCTs and CCTs, and mail them on floppy disks to the New England Cochrane Center at Providence (see Chapter 3). The New England Cochrane Center at Providence will merge the file with others received and prepare a comprehensive electronic re-tagging request file for the NLM. As part of its annual MEDLINE maintenance procedures, the NLM will tag all citations in the MEDLINE database (1966 forward) with the publication types "RANDOMIZED-CONTROLLED-TRIAL" and "CONTROLLED-CLINICAL-TRIAL" as appropriate.

Individual Collaborative Review Groups, Fields and Networks can search MEDLINE using subject matter terms plus the two publication type terms, "RANDOMIZED-CONTROLLED-TRIAL" and "CONTROLLED-CLINICAL-TRIAL", to build their own specialized registers.

 2.7 Bibliographic Records for Identified RCTs not in MEDLINE

We plan that Cochrane-identified RCTs and CCTs not in MEDLINE will be added manually by the NLM, either to MEDLINE or to an ancillary database. The development of an ancillary database to MEDLINE to hold newly created bibliographic records for reports of RCTs and CCTs not currently in MEDLINE is in the planning stage. Ideally, this ancillary database will be able to adapt or make use of information already present in other bibliographic databases, such as PsychLit, whenever possible or permissible by copyright.

 2.8 Creating Specialist Registers

The intent of building a "virtual register" of published reports of RCTs and CCTs in all subject areas within MEDLINE is to enable the Collaborative Review Groups, Networks and Fields to create specialist registers of trials in their interest areas in as simple and efficient a manner as possible. This is, however, a long term goal that will take a number of years to attain. In the meantime, all searching activities throughout the Collaboration must be seen as serving a dual purpose: to contribute to the comprehensive International Register and to contribute to one or more individual specialist registers. This is why it is essential that all journals and other publications be searched comprehensively for all eligible reports, and not merely for reports relevant to a particular topic or specialist register.

3. Procedures for Journal Searching

 3.1 Checklist of steps involved in searching individual journals

1. Identify a search coordinator.

2. Determine which journal(s) to search.

3. Register the intent to search a journal with the New England Cochrane Center at Providence, using the Journal Hand Search Registration Form (Appendix 2).

4.Train and test searchers, using training materials custom developed by your searching group or adapted from materials given in Parts II and III of this Manual.

5. Search journal(s):

a. Examine each issue, page by page, to identify ALL reports of RCTs and CCTs, whether they appear in articles, abstracts, news columns, editorials, letters or any other text, and regardless of the relevance of the subject matter to the searcher.

b. Photocopy the first page of each report identified, as well as any subsequent pages requested by a group's search coordinator.

c. Annotate each photocopy, on either the front or back, with complete citation information, wherever necessary.

d. Classify each report as RCT or CCT (or other as relevant to the needs of the search group) and annotate each photocopy with the classification codes. Highlighting (e.g., with a yellow marker) or underlining the key terms which justify the classification may be required by a group's search coordinator.

6. Whenever such citation records exist, download the citation records corresponding with the search results directly from MEDLINE.

7. Transfer (or enter, when there is none to transfer) citations into Pro-Cite and add classification codes to any specified, unused field.

8. Forward citations on 3 1/2" IBM-compatible floppy disk to the New England Cochrane Center at Providence. Include annotated photocopies of any reports without a corresponding MEDLINE citation record.

Each of these steps is described in more detail in the following sections.

 3.2 Preliminary activities

Each of the Cochrane Centres has the responsibility of searching the general medical literature of their country or region, and most have now designated one or more staff members to coordinate this effort. The CRGs, Fields and Networks are responsible for establishing the mechanisms for searching the specialist literature in their areas of interest, and should coordinate these activities.

It is the responsibility of the editorial team of any Collaborative Review Group (CRG) to coordinate searching activities of specialist health care journals and to ensure that all RCTs and CCTs within these journals are identified as completely as possible and registered for their own benefit and that of others in the Collaboration. This essential task can and should be pursued concurrently with efforts to establish the CRG.

Designating a search coordinator for the group should be among the first actions taken by those planning to develop a specialized register. Coordination of searching activities is essential to avoid unnecessary duplication of effort, to maximize the completeness with which trials are identified, and to minimize the resources used to do this. The overall coordination of searching activities is the responsibility of the New England Cochrane Center at Providence. The New England Cochrane Center at Providence is able to provide guidance for organizing the work of assembling a register based on the experience of other groups, and from the results of methodological research relating to the identification and registration of trials. The aim of the New England Cochrane Center at Providence is not to assume responsibility for searching activities, but to assist those with interest in specific subject areas to do this work as effectively and efficiently as possible.

A growing number of journal editors have agreed to search their own journals, in cooperation with the Collaboration. In addition, many individuals have volunteered to search journals, independent of any Cochrane group, usually because Review Groups or Fields have not yet been established in their areas of interest. Most often, these searches are coordinated by a Cochrane Center, but some are coordinated by Review Groups, Networks or Fields. While the actual methods of search coordination will vary according to the resources and preferences of individual search teams, every individual who hand searches a journal on behalf of the Cochrane Collaboration should rely on and receive training, guidance and support from a search coordinator.

Where individual members of searching teams are working in close proximity to one another, communication is less of a problem. However, the membership of many groups span the continents. Preparing and sending a newsletter, whether by electronic or regular mail, can be an excellent way to keep members involved, encouraged and informed as progress is made. The New England Cochrane Center at Providence would appreciate receiving copies of newsletters or other informational mailings sent out by your group.

3.2.2 Determine which journal(s) to search

Another essential preliminary activity that can and should be pursued concurrently with efforts to establish the CRG, Field or Network is the preparation of a listing and prioritization of all of the journals the group hopes to search. Directories of journals exist both as electronic databases and in book form, from which subject listings of health care journals can be derived.

A Master List of Journals Being Searched is maintained by the New England Cochrane Center at Providence as part of an overall progress tracking database of activities towards building the International Register of RCTs of Health Care. The Master List is updated regularly as new journal searches are registered. Those planning or coordinating journal searches for Centres, CRGs, Fields or Networks, especially in interest areas likely to overlap with others, should consult the Master List to avoid duplication of effort.

Copies of the most recent Master List may be downloaded from this site or explored on-line.

3.2.3 Register the journal search

Once a decision has been made to search a particular journal, whether for all years from 1948 (or first issue) to the present, from this time forward only, or for some other specified interval of years, a Journal Hand Search Registration Form (see Appendix 2) must be forwarded to the New England Cochrane Center at Providence. This form, which is also available as an Appendix to Section V of the Cochrane Collaboration Handbook, should be completed either by the person agreeing to search the journal or the person responsible for coordinating hand searching activities for that journal.

3.2.4 Train and test searchers

Whether or not searchers have prior experience with identifying published reports of RCTs, all who plan to hand search literature on behalf of the Cochrane Collaboration should complete a training course tailored to the requirements and needs of the Cochrane Collaboration. The purpose of training is to ensure that all searchers apply standard eligibility criteria (discussed in Chapter 2) and classify reports in the same manner.

The Training Notes for Hand Searchers (Part III of this Manual) is a training course designed for students, volunteers, and others previously unfamiliar with randomized trials and literature searching. Alexia Antczak-Bouckoms of the Cochrane Oral Health Group has adapted this course to produce a hand search manual appropriate for training collaborators familiar with randomized trials (see Chapter 3 of Part II). Either of these courses can be further modified to suit the requirements of a particular searching group.

It is strongly recommended that all searchers be required to complete an appropriate training program and complete a self-test, prior to embarking on their task. To prepare for the self-test, search coordinators should develop a "gold standard" search of one or several publication years of a typical journal relevant to their subject area. They will then be able to compare the results of each searcher's search with the "gold standard".

 3.3 Search the journal(s)

3.3.1 Scope of the search

Full-text searching of journals and other literature involves the page-by-page examination of the entire text (i.e., articles, abstracts, letters, editorials, news items, etc.) of the publication. All RCTs and CCTs contained in the publication are to be identified, classified and prepared for forwarding to the New England Cochrane Center at Providence, irrespective of the relevance of the subject matter to the individual or group performing the search.

Groups performing journal searches are free to develop the administrative structure and operating procedures most comfortable and convenient for them. Some examples of search procedures used by individual Cochrane groups are provided in Part II.

3.3.2 Make and annotate photocopies of eligible reports

When a CRG, Field or Network has sufficient resources to do so, it can be helpful to reviewers if photocopies of entire articles are made as the articles are identified as RCTs or CCTs, and several groups have elected to do this. At a minimum, each searcher should photocopy the first page of each eligible report identified, annotating each photocopy, on either the front or back, with the full reference and the report classification (i.e., RCT or CCT). Some groups have chosen to widen the scope their search to include review articles, meta-analyses or other sources of references to RCTs and CCTs, and have developed additional classification codes to accommodate these reports. Only the RCTs and CCTs are to be sent to the New England Cochrane Center at Providence for inclusion in the International Register.

It will be helpful to those who prepare the search results for transfer to the New England Cochrane Center at Providence if searchers underline or highlight (e.g., with a permanent, yellow marking pen) the text that lead him or her to classify the report as an RCT or CCT. If this is done, then searchers should also photocopy additional relevant pages when the key text occurs after the first page.

Once the page-by-page search of some number of journal issues has been accomplished, the set of photocopies resulting from the search must be checked for completeness and for proper annotation and classification. Some methods for doing this are discussed in Chapter 4. After it has been verified that the search results are complete and accurate, they must be prepared for forwarding to the New England Cochrane Center at Providence for inclusion in the International Register, as well as for inclusion in one or more specialist register, when appropriate.

 3.4 Forward citations to the New England Cochrane Center at Providence

The appropriate procedure for forwarding reports of RCTs and CCTs (see Chapter 2 for definitions), to the New England Cochrane Center at Providence depends upon whether a bibliographic record currently exists in MEDLINE for each of the identified reports. If bibliographic citations to reports of trials that have been identified through hand searching are present in MEDLINE, the citations must be downloaded directly from MEDLINE in the manner described in Section 3.4.1. Information on accessing MEDLINE and on procedures for using the downloading software can be obtained from your local medical librarian.

If a journal is not currently indexed in MEDLINE (or was not at the time a particular issue was published), there will be no MEDLINE records to update for reports of RCTs and CCTs found in that journal. In addition, some reports that were published in MEDLINE-indexed journals do not have corresponding MEDLINE records. There are several reasons why this can occur. It may be that the report in question was published during a year (e.g., prior to 1966) when the journal was not indexed. It may be that the report is of a type not generally indexed separately by NLM (e.g., abstracts from conference proceedings). Very rarely, a report or issue may have been omitted by mistake. Regardless of the reason, whenever there is no MEDLINE citation record for an identified RCT or CCT report, a bibliographic record for the report will have to be created and indexed manually by NLM staff for future inclusion in a proposed ancillary database. These citations must be documented in the manner described in section 3.4.3 and sent to the New England Cochrane Center at Providence.

3.4.1 Download MEDLINE citations

The entire MEDLINE record (all fields) for each report is required for processing by the New England Cochrane Center at Providence. The specific contents of the entire MEDLINE record will vary according to the method used to access MEDLINE. Please be aware that some versions of search/download software used to access MEDLINE will automatically default to an incomplete record unless explicitly instructed otherwise.

Most versions of search/download software used to access MEDLINE offer a choice of two or more record formats when downloading. Choose the "MEDLARS" (sometimes called "Reprint") option, which is the format designed for transfer of the records into Pro-Cite or other bibliographic database software. Each line of the downloaded records will then be preceded by a two-letter field label, needed to facilitate transfer of the record into Pro-Cite.

The following MEDLINE record fields are essential:

• Unique Identifier This is the eight-digit number which uniquely identifies the record to the MEDLINE database management software. Depending upon the search/download software and MEDLINE access method used, this could also be called the "Accession Number" (AN) or the "Call Number". No re-tagging request can be sent to NLM without this number.
• Publication Year This may be captured as a separate field or in combination with the journal name and citation information in a field called "Source".
• Journal Name (at the time the report was published)
• Citation Information This should contain the volume number, issue number, and both initial and concluding page numbers.
• Name(s) of Author(s) Some RCT reports are indexed as "anonymous".
• Title of Article Some letters and editorials do not have titles.
• Abstract The text of the abstract is often truncated to 250 words.
• MeSH Index Terms These will allow the records to be copied into searchable Pro-Cite database(s) for interim use by Collaborators and provide data for research in coding and classification.
• Publication Types Records cannot be processed for re-tagging requests without information about the PT tags that have already been applied. Some MEDLINE formats store this information in a field called "Notes".

The downloaded citation records may be grouped into files in whatever manner is convenient. Records need not be separated according to journal, publication year, or type of report, unless such an arrangement would be useful to the search group.

3.4.2 Prepare Pro-Cite database(s)

There are a number of different software applications available for managing bibliographic references, and each has different strengths and weaknesses. Pro-Cite provides the greatest flexibility for importing references from any tagged external source and consequently the Cochrane Collaboration has decided to standardize on Pro-Cite for managing the references being incorporated into the International Register of Published RCTs of Health Care. Therefore, the New England Cochrane Center at Providence requests that, if at all possible, those working with the Collaboration use this software to transfer data to the New England Cochrane Center at Providence for inclusion in the Register. Groups choosing to use other software should format their transfer data so that it may be easily exported into Pro-Cite.

Transfer of citations downloaded from MEDLINE in MEDLARS format into a Pro-Cite database is easily accomplished using the "Biblio-Links" software designed for this purpose. The appropriate linking application, "SPLINK" for use with SilverPlatter or "NLMLINK" for use with CD-PLUS Ovid or Grateful Med, is available for purchase along with Pro-Cite. Detailed instructions for using NLMLINK are provided in Part II. The steps needed to use SPLINK are virtually identical.

Once the citations have been transferred into one or more Pro-Cite databases, the classification code (e.g., RCT or CCT) may be entered into any empty field, by editing the individual records. At the New England Cochrane Center at Providence, we use the "Loc" field for this purpose, but any empty field will do. Some searching groups have chosen to design their own Pro-Cite formats (called "Workforms") to accommodate their own classification systems. Instructions for creating and using the customized Pro-Cite Workform used by the Cochrane Field Coordination in Emergency Medicine and Intensive/Critical Care Medicine are included in Part II. The Pro-Cite database(s), once the classification codes have been entered, can form the basis for your group's specialized register.

We recognize that some searching groups have already begun to construct their specialized registers using application software other than Pro-Cite, and that conversion to Pro-Cite at this time would present a hardship. When this is the case, the citations downloaded from MEDLINE may be forwarded to the New England Cochrane Center at Providence in ASCII files (in MEDLARS format so that we may transfer the citations into Pro-Cite for you). If your group requires this accommodation, please notify the New England Cochrane Center at Providence in advance, so that the additional effort can be scheduled. In addition, we request that the RCTs and CCTs be forwarded to us in separate files, and that a hard copy bibliography indicating the report classifications be included with the transfer package.

Regardless of the format used, the citations must be forwarded to the New England Cochrane Center at Providence on 3 1/2", DOS-formatted floppy disks. At present, the New England Cochrane Center at Providence lacks the facilities to process citations sent in any other format (e.g., MacIntosh). Each report must be classified as either an RCT or a CCT in an accompanying hard copy bibliography (if the data files are in ASCII format) or in any unused, specified data field of the Pro-Cite database.

3.4.3 Document reports not in MEDLINE

As stated previously, whenever existing MEDLINE citation records for identified RCT or CCT reports are unavailable, bibliographic records for each report must be created and indexed manually by NLM staff for future inclusion in a proposed ancillary database. To accommodate their needs and to make interim usage possible within the Collaboration, each report must be documented in the following manner:

• Photocopy the title and/or Table of Contents page, showing the authors, journal title and volume and issue numbers, for the journal issue containing the report.
• Photocopy the title page of the report itself, annotated to show the complete bibliographic citation (authors, title, journal name, volume, pages, and year) and the classification of the report as either an RCT or a CCT.
• Provide the bibliographic reference for each report in a WordPerfect or other word processing software document. Label the disk as to software used and file contents.

The annotated photocopies, along with the bibliography on 3 1/2" DOS-formatted floppy disk, should be forwarded to the New England Cochrane Center at Providence. In addition to the items listed above, electronic copies of citations downloaded from other bibliographic databases (e.g., PsychLit) should also be included, whenever available, in the transfer package sent to the New England Cochrane Center at Providence. These may be submitted as either a set of Pro-Cite files, an ASCII-formatted file or a word processing software document (e.g., WordPerfect) in addition to or instead of the file of bibliographic information described above.

3.4.4 Forward the citation package to the New England Cochrane Center at Providence

At present, the preferred method for transfer of citations to the Baltimore Cochrane Center is by mail or courier service, on 3 1/2" DOS-formatted floppy disk(s). Alternative arrangements can be made (e.g., transfer via e-mail of very small files) by contacting, in advance, a member of the New England Cochrane Center at Providence staff. Accommodation of other forms of electronic transfer (e.g., by ftp) is planned for the future.

New England Cochrane Center at Providence staff include:

Kay Dickersin, PhD
Co-Director
Tel: 1 401-863-9950
Fax: 1 401-863-9944

Jeanette Downing-Park
Coordinator
Tel: 1 401-863-9950
Fax: 1 401-863-9944

Eric Manheimer
Methodologist
Tel: 1 401-863-9950
Fax: 1 401-863-9944

e-mail: cochrane@brown.edu

Citation packages should be sent to our new address:

New England Cochrane Center at Providence
Department of Community Health
Brown University
Box G-S2
Providence, RI 02912

tel: +1 401-863-9950
fax: +1 401-863-9944
e-mail: cochrane@brown.edu
Web: https://www.cochrane.org/neccp.htm

4. Quality Control Strategies

4.1 Reasons for quality control

At first, it may appear that hand searching a journal is more tedious than difficult. However, this effort is as subject to human error as any other, perhaps even more so because of the nature of the task. In our experience, the progress lost to mislaid or illegible pieces of paper, typographical errors, missed or missing citations, or simple failure to follow directions, can be considerable. This chapter lists some of the problems that have been encountered by Cochrane searching groups in the past, and gives some suggestions of ways to minimize or avoid them.

 4.2 Unnecessary duplication of effort

There are several ways that hand searchers can needlessly duplicate another's effort. Fortunately, these situations can be easily avoided.

If a searcher or search coordinator does not consult the Master List of Journals Being Searched before beginning a search, then effort can be wasted on searching a journal that has already been searched by someone else. If a searcher or search coordinator does not submit a Journal Hand Search Registration Form (see Appendix 2) for each journal they plan to search, then the Master List will be incomplete, and someone else could waste effort searching a journal already searched by you. The remedy is simple: always consult the Master List and submit a Registration Form before beginning a journal search.

Searchers should not conduct an incomplete search, e.g., selecting only full length "original" articles or reports relevant to a single topic area). When this has been done, it has been necessary for someone else to repeat the entire search of the journal. Examine every page of journals searched and retrieve every eligible report.

Search coordinators may assign several searchers to a single journal, but it may be difficult to keep track of who is searching which publication years. Duplicate search assignments can be made unintentionally. Keep a record of search assignments in a spreadsheet to simplify the necessary record-keeping. Some examples are provided in Part II.

 4.3 Incomplete searches - unavailable journal materials

You may find that a journal issue that you have agreed to search is unavailable in your local library. Sometimes journal volumes can be bound with sections removed, pages missing, and even whole issues omitted. When this occurs, a note should be made in a spreadsheet or on a form designed for this purpose. Sometimes the missing materials can be obtained through an interlibrary loan or copying service. Please report all unavoidably incomplete searches to the New England Cochrane Center at Providence.

 4.4 Incomplete searches - lost photocopies

Searching activities can generate quite a lot of paper, and pieces of paper can get lost or mislaid. We recommend that searchers photocopy the Table of Contents of each issue searched and "check off" each article examined while searching. The marked Tables of Contents help guard against lost photocopies and can also serve as evidence that an issue was searched if no eligible reports were found.

4.5 Incomplete searches - missed reports

Once a journal issue has been hand searched, we hope that it will never need to be searched again. If an eligible report is missed by a hand search, it may be lost forever. Therefore, regardless of whether searchers are experienced collaborators, journal editors, students or volunteers, each searcher should be required to undergo a training exercise. Establishing a "gold standard" search of at least several issues of a journal relevant to the search team should be one of the first responsibilities of each search coordinator. Every searcher should conduct a test search to be compared against the group's "gold standard" search. The results of the comparison should be discussed with the searcher to resolve any misunderstandings before he or she begins an actual journal search.

As several groups have discovered to their disappointment, sometimes initial searcher training is not enough to ensure that all eligible reports are being retrieved. We recommend that search coordinators give a certain proportion of duplicate search assignments for ongoing comparison between searchers. Twenty percent duplication is reasonable, unless a problem is uncovered. This can be done by giving search assignments of five-year intervals that overlap between searchers by one year, or by selecting a 20% sample of completed issues for duplicate searching by another searcher. Whatever method is chosen, this should be done in a timely manner and on a continuing basis, so that feedback (both positive and negative) on the result of each comparison can be given to the searchers and misunderstandings can be resolved promptly.

 4.6 Classification errors

Even when all eligible reports have been identified and retrieved by a searcher, errors can be made in classifying them as either "RCT" or "CCT". These errors can cause serious difficulties later on when the citations are forwarded to the New England Cochrane Center at Providence for inclusion in the International Register. Search coordinators should check the photocopies to ensure that each has been annotated with the correct classification and complete reference information, before preparing the citations for transfer to NECC at Providence.

 4.7 Typographical errors

Materials needed to document search results and permit their inclusion in the International Register were discussed in Chapter 3.6. Citations to reports of RCTs and CCTs that do not already have MEDLINE records must be entered manually into a Pro-Cite database or electronic word processing file. Please proofread all bibliographic lists of search results for typographical errors before sending them to NECC at Providence.

 References

(1)Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane's agenda. BMJ 1992;305:7868.

(2)Dickersin K, Scherer R, Lefebvre C. Identification of relevant studies for systematic reviews. BMJ 1994;309:1286-91.

(3)Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med 1994;24:741-8.

(4)Schuyler PL, Dickersin K, Scherer R, Wright ND. Identification of randomized clinical trials using MEDLINE. (unpublished material).

(5)Lefebvre C, Kelleher TA, Tihanov G. Identification of randomized controlled trials using MEDLINE: the situation in 1993. Presented at "An Evidence-Based Health Care System: The Case for Clinical Trial Registries". National Institutes of Health, Office of Medical Applications of Research, Bethesda, MD, Dec. 6, 1993.

 Appendix 1

Criteria for registering studies with the Cochrane Collaboration's International Register of Published RCTs of Health Care

Overarching principle:

The highest possible proportion of all reports of RCTs of health care should be included in the International Register. Thus, those searching the literature to identify trials should give reports the benefit of any doubts. Publications which simply mention the possibility of undertaking an RCT should not be included, however.

Reviewers will decide whether to include a particular report in a review. The aim of the Register is to provide reviewers with all possible trials they may wish to include in a review, not to decide whether a report is worthy or relevant for inclusion.

Eligibility Criteria:

Relevant reports are reports published in any year, of studies comparing at least two forms of health care (medical treatment, medical education, diagnostic tests or techniques, a preventive intervention, etc.) where the study is on either living humans or parts of their body or human parts that will be replaced in living humans (e.g., donor kidneys). Studies on cadavers, extracted teeth, cell lines, etc. are not relevant.

Studies are eligible for inclusion in the Register if allocation to the intervention was random or intended-to-be-random (e.g., alternation), or if a concurrent control group was used in the trial and it is possible that a random or intended-to-be-random method was used to allocate participants to the study groups. Judgements as to the quality of the methods used or whether the authors actually did what they claimed should not be used to decide eligibility for inclusion in the Register.

A trial should thus be included in the Register if, on the basis of the best available information, it is judged that:

• the individuals (or other units) followed in the trial were definitely or possibly assigned prospectively to one of two (or more) alternative forms of health care using random allocation or some quasi-random method of allocation (such as alternation, date of birth, or case record number).

In addition:

• If one or more outcomes were assessed using "double blinding" or "double masking" such that neither the participant/patient nor the assessor was aware of the intervention received, but randomization is not mentioned explicitly in the text, a trial should be included.
• Crossover trials, in which patients have been assigned to the first intervention using random or quasi-random allocation, should be included.
• Reports of trials dealing only with animals should not be included.
• Units of randomization may be individuals, groups (such as communities or hospitals), organs (such as eyes) or other parts of the body (such as teeth).
• A report of a randomized trial should be included even when no results are presented or when results are limited to the analyses of baseline variables.
• Articles describing an intended or ongoing trial or commenting on a trial (such as in an editorial)should be brought to the attention of the New England Cochrane Center at Providence, but are not eligible for inclusion in the International Register of Published RCTs.

 Appendix 2: