Tool for international collaboration in guideline development

1. Introduction

In most Western countries, guidelines are developed within structured programmes. Many guidelines address the same topics and follow similar procedures. The literature review is a standard part of this procedure. If they all use the same review questions, guideline developers will be doing double work, because the ‘body of evidence’ is globally shared. International collaboration can improve the efficiency of the guideline development process. This is especially true with regard to the literature review and monitoring the literature to ensure it is kept updated. For this purpose, agreements on literature research methods are needed between countries, including the research strategy, selection, assessment and summary of the literature.

However, there are very few instances of international collaboration, and the few collaborations there have been were not entirely positive. When collaborating, differences in health care systems need to be taken into account along with the different contexts in which guidelines are developed and applied and the interests of the parties involved. The European CoCanCPG project has established good collaboration between the Scottish Intercollegiate Guidelines Network (SIGN), the Comprehensive Cancer Centre of the Netherlands (IKNL) and the Belgian Federal Health Care Knowledge Centre (KCE). These three parties have formed a consortium in which they jointly develop and revise guidelines (particularly oncology guidelines). The experiences from this collaboration form the basis for the development of this tool on international collaboration.

2. Objective of this tool

This tool formulates the conditions for successful international collaboration and the steps that guideline developers need to follow when collaborating at an international level. The tool can also assist in collaboration at a national level.

3. Step-by-step plan for international collaboration

Step 1: Get to know each other, create a foundation of trust and set goals for collaboration

Good collaboration between organisations begins by getting to know one another. People from the different organisations often know each other from attending international conferences or symposia together. It is important that these people work at similar levels within their organisations and that the organisations’ leaders and CEOs support the collaboration. It is strongly recommended to exchange relevant policy documents and annual reports during mutual working visits where the organisations’ leaders are present or on days when their staff work together. During these visits, it is important to discuss the common goal of the collaboration.

Step 2: Analyse the work methods, compare the delivered products and discuss possible differences

A detailed analysis of the implementation of all steps is needed to determine for which components cooperation will and will not be possible. Manuals or procedures that describe the guideline development process may provide useful information during this step. There may be differences in how literature research is conducted and it is important to determine whether these differences constitute a barrier to collaboration. In any case, it should be clear how to deal with existing differences. To get an impression of the work methods, the end products (i.e. the guidelines) can be compared in terms of quality and design. The AGREE instrument may be a useful tool for this comparison. Organisations may be able to complement each other in areas where their scores vary. The design relates to the composition and structure of the guideline and summary method. Many organisations have made their own decisions with regard to these issues and rarely like to deviate from these. It must be clear which work methods the parties deem appropriate for the collaboration, which will depend partly on the available budgets. In any case, all parties need to endorse the principles of evidence-based guideline development. The final result will be that the different parties will gain a clear overview of the following essential elements of the guideline development process:

  1. Drafting the review questions
    • Is this a standard component?
    • Will content experts be involved?
    • How detailed do the questions need to be (PICO)? Will these questions guide the literature research?
  2. Literature research
    • Who performs the research? Are these external parties or internal people? Have these people received training?
    • Which databases will be used and with which search criteria/filters?
    • Will guideline databases also be searched? What role will existing guidelines on the topic in question play?
    • To what extent will systematic reviews be used (e.g. clinical evidence, Cochrane reviews, York database, Prospero database)?
    • Which selection criteria will be applied?
  3. Selection and analysis of literature
    • Who will select and analyse the literature? Will clinical experts be involved in this step? Will they be able to supply their own literature?
    • What role will unpublished literature play (e.g. abstracts of conferences)?
    • In how much detail will the process be documented? Will it be reproducible?
  4. Summary of the literature
    • How will the literature be summarised? Will evidence tables be used and how will they be laid out? Will the tables be created in English?
    • Will an evidence grading system be used?
    • Can the evidence report (literature description) be accessed by and shared with the public?

Step 3: Determine which components the development groups wish to collaborate on and come to a documented agreement, including agreements on possible differences

On the basis of the work method analysis, the parties will decide which parts they want to share. This will involve making choices about the following: review questions, inclusion and exclusion criteria, agreements about whether or not to adapt guidelines, details about the search strategy, the research design, databases, the time period, assigning evidence levels, striving for the same format, preparing evidence reports and deciding whether these should be from the perspective of the intervention or the patient population, the cost and publications.

This step will usually involve sharing literature. If there are no specific national differences between how the parties organise or fund health care, international recommendations can also be drawn up. Figure 1 can be used as a guide for deciding what can be established internationally and what is best determined at a national level.

Figure 1: What can be done internationally and what is best done nationally?

It is advisable to conduct a pilot study: for example, parties could jointly elaborate on the same review question as a way of building trust in each other and each other's products. The feasibility and workload should also be looked into at this stage. Often, a first-time collaboration will require extra time to be invested and therefore the schedule should be somewhat flexible. The process facilitator should discuss these and other steps with the rest of the development group.

The objective of the collaboration should also be documented in writing. This step may be appealing not just for the additional time and cost savings, but also because it can improve the guideline’s quality. Preferably, the collaboration is defined in an agreement that is signed by the organisations’ leaders. For background information about collaborations see Tool for organisation and cooperation in multidisciplinary guideline development.

Step 4: Draw up an plan of approach based on the agreement, use a uniform format and establish communication channels

Formats need to be prepared for the components on which the development groups are collaborating, for example, formats for searching and summarising literature and assessing the quality of the evidence. The CoCanCPG website provides a number of examples of their European network. The development groups can also use an international system for grading the quality of the evidence and the strength of the recommendations: Grades of Recommendations, Assessment, Development and Evaluation (GRADE). In addition, the G-I-N Evidence Tables Working Group has developed international formats for evidence tables for diagnostic and therapeutic interventions. Formats for prognostic research and economic evaluations are currently being developed. All formats can be found on the website.

The parties will make concrete agreements about who delivers which products and establish channels of communication regarding the work products (e.g. via email or a joint website). If there are language differences, all parties will need to commit to one language, usually English. Teleconferences or video conferences will be scheduled regularly, preferably at least once a month, to review progress and discuss any bottlenecks. A staff exchange can be considered to intensify the collaboration. Finally, the development group will make agreements on publications, attribution and copyright.

Step 5: Make agreements in this plan of approach on how the evaluation of each other's products and insight differences will be handled

Sharing each other’s products implies that there are agreements about the content and work method. This can only be achieved if the development groups have made earlier drafts available to each other for commentary and regularly discussed them. They need clear agreements about, for example, a response time period. The commentary should always be constructive and helpful; otherwise, mutual trust can be damaged. It is important that no unwanted levels of hierarchy are created which may cause one of the parties to feel superior, unless this has been explicitly agreed upon by the other party. A problem solving strategy should be available for situations where there are continuing differences of opinion (e.g. involving an independent ‘third party’). Differences in opinion are expected and should not pose major problems, provided none of the parties lose face.

Step 6: Implement the plan of approach and make arrangements about the products’ revision procedure

Once the products are completed, a procedure for maintaining them needs to be established. This will often be a strategy for keeping the literature up-to-date which has been agreed upon by the development groups. To distribute the workload, the parties will divide the topics based partly on expertise or interest. If there are new developments, a procedure needs to be established on how they should inform each other. This procedure will include the format and the information deadline. ‘Living guidelines’ can be used to provide a continuous ‘alert service’, by incorporating new relevant developments as soon as possible by adding to or revising the guideline text and annual plans.

It is also desirable to have an overall evaluation of the collaboration that includes agreements for future joint project.

4. Literature

Ollenschläger G, Marshall C, Qureshi S, Rosenbrand K, Burgers J, Mäkelä M, Slutsky J; Board of Trustees 2002, Guidelines International Network (G-I-N). Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004 Dec;13(6):455-60.

Mlika-Cabanne N, Harbour R, de Beer H, Laurence M, Cook R, Twaddle S; on behalf of the Guidelines International Network (G-I-N) Working Group on Evidence Tables. Sharing hard labour: developing a standard template for data summaries in guideline development. Qual Saf Health Care. 2011 Feb;20(2):141-145.

Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knotterus A. GRADE guidelines: A new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2010 Dec 23. [Epub ahead of print]

Brouwers M, Kho ME, Browman GP, Cluzeau F, Feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010;182(18):E839-42.

5. Websites




European Society of Cardiology: