Tool for project management in guideline development

1. Introduction

The method of project management used in guideline development strongly influences the course of the process itself and the quality of the final result. In addition, the focus on the essence of a guideline and the desired final result strongly influence the project management structure used during guideline development. Information and tips about efficient organisation, coordination and monitoring of the guideline process can help to complete the development or revision within the scheduled time period and with the available budget while, of course, maintaining quality. This relates to project management (time, money, quality, information and organisation) that is specifically focused on guideline development and implementation. Cooperative and group dynamic aspects are discussed in the Tool for organisation and cooperation in multidisciplinary guideline development.

A distinction is often made between monodisciplinary and multidisciplinary guidelines. The current trend is that all guidelines have a more or less multidisciplinary character. In addition, nowadays smaller guidelines are ‘tied together’ with larger multidisciplinary guidelines. The term ‘network guideline’ is used for this concept. The word ‘guideline’ refers to a nationwide, evidence-based guideline with a multidisciplinary character in which the complexity increases with the number of stakeholders and interests involved with the guideline (see also Code for the prevention of improper influence due to conflicts of interest).

An important aspect of coordination, organisation and implementation is the various parties’ support of the decisions made during the preparation of the guideline and the development of its contents. Commitment from all the relevant parties is crucial to gain optimum effort from development group members, clear communication to stakeholders and acceptance of the recommendations.

This tool connects the different development phases of the Guide to Guidelines. These are summarised in the figure below:

Timeline for guideline development

When beginning guideline development, it is recommended that this tool be studied as a whole, so that during each phase timely attention can be paid to subsequent phases. In addition, the tool provides information about possible opportunities for revising (tips and tricks) and budgeting (time and money) the guideline development process.

2. Objective of this tool

The objective of this tool is to provide attention points, tips, instruments and helpful tools for efficient project management of the development or revision of guidelines and/or products derived from them. This tool focuses on four roles: the chair, the project leader, the initiating party (usually the service provider) and the stakeholders.

3. Preparation stage

3.1. Guide to Guidelines

The Guide to Guidelines states that the following aspects are addressed during the preparation phase:

  • The guideline’s topic, objective and target group are determined.
  • The initiator of the guideline development process identifies the primary professional organisations and patient/client organisations involved.
  • The primary professional organisations select the chair and are responsible for his or her performance.
  • The primary professional organisations put together the development group.
  • Content experts and methodological experts are involved in all phases of guideline development.
  • Representatives of health care users are involved in developing and maintaining the guideline.
  • The impact of conflicts of interest is minimised as much as possible.

3.2. Implications for project management

The preparation phase is crucial for the design of the project management. This is the phase in which the elements that will determine the future course of guideline development should become clear. To implement the abovementioned points into the preparation phase, it is good to make a distinction of two steps:

3.2.1. Step 1: From idea to proposal

At the moment that an idea for guideline development is first proposed, the feasibility of that idea should be tested by the initiator(s). In the implementation and use of the guidelines there are always several involved parties who must see the guideline as useful and necessary. The feasibility has to do with questions such as:

  • How can the guideline’s topic, objective and target group be established?
  • Is development of a guideline feasible for this topic?
  • Does the guideline focus on the complete care process or is it focused on the most important clinical care gaps in practice (including substantiation for choices)? In the Guide to Guidelines, the analysis of clinical care gaps is part of the development phase. If the budget and time allow, development groups may consider performing the analysis of clinical care gaps during the preparation phase since the results may have a significant impact on the subsequent stages. The Tool for the analysis of clinical care gaps contains more information about this.
  • What result should be delivered?
  • Is there support for the topic?
  • Where is the necessary substantive, methodological and process-related expertise coming from? Who are the leading experts on this topic (verify this with the proposed chair, literature, client organisations and professional organisations)?
  • How can adequate client representation be arranged? The patient participation blueprint gives more information about this. Information about client organisations can also be obtained through NPCF.
  • Which obstaclesto the application of the guideline can be foreseen so preventive measure can be taken?
  • How many resources are required for each component (divided between human resources, time, material and money) so that the commissioner can make choices if the budget seems inadequate? From where can these resources be acquired and how can the development group monitor the allocation of these resources?

3.2.2. Step 2: From proposal to application

If the relevant parties agree that a guideline should be developed or revised for the specified topic, the development group should write a plan of approach detailing how they intend to carry out the project. Developing a concrete plan is largely an explication of the questions defined in Step 1. In addition, the following questions arise:

  • How does the general playing field look when it comes to the relevant information for this guideline regarding, for example, scientific research, domestic and international guidelines, technical expertise and the client’s perspective?
  • Who are the primary parties (stakeholders) in this guideline? Who is the guideline owner and thus its driver? Who will submit the application and take on the project leadership?
  • How will the project’s tasks, roles and responsibilities be divided and what information is important to which parties (‘communication plan’)? The development group can use the RASCI method described in the Tool for Organisation and Collaboration in multidisciplinary guideline development. Roles other than those contained in the tool may also need to be addressed.
  • Is there a suitable chair who the other primary participants find substantial and process-oriented enough to accept?
  • The chairmanship requires extensive skills and a clear commitment from the start of the guideline development process. For example, these skills may be objectively tested by putting various suitable candidates for the role of chair through an application process. In practice, the chair is often chosen based on the recommendation of the most important scientific association related to the topic in question. For the profile of the chair see the Tool for Organisation and Collaboration in multidisciplinary guideline development.
  • How can the other stakeholders be properly represented? Should they take part in the development group or is their expertise better used in other ways (for example, in the implementation group or as part of a feedback group)?
  • Who is the project leader? Who is providing methodological and process-related support? How are the tasks divided in each component of the project plan and who is responsible for the project’s implementation? Who has what mandate? How does the external communication function?
  • How will a balance be found between quality and completeness on the one hand and time and budget on the other? What is the timeframe and budget? How will the project leader adhere to the planned timeframe and budget? Are there margins to work within and what can the leader do if a budget deficit arises (for example, are there options for dropping or accelerating parts of the plan)? And if changes are needed, what are the consequences of these on the guideline development process and who is authorised to decide whether they can or will be made? Who will check whether the project remains within the prescribed time and budget and how often will this be done? If there is external financing, other requirements/conditions may apply.

3.3. Tips and tricks

  • The chair plays an essential role in controlling the outcome. The chair is chosen by the service provider in consultation with the primary target groups. Together, the chair and project leader form the core of a successful project. The chair must conform to the established project plan, including the timeline and financial pre-conditions. The chair must have strong process-oriented skills and the experts in the development group must respect him or her. The chair must be aware of possible limitations of support and see it as his or her task to effectively present the client’s perspective. It is recommended that the project activities be derived from the desired end product by regularly stating the results and emphasising how the client and other stakeholders will profit from the development and implementation of the guideline. The chair also has the task using this perspective in development group discussions. If after time the chair does not appear to be effective, resulting in too much delay, an additional evaluation with the chair may be necessary. If the chair ultimately proves unable to lead, it will be necessary to replace the chair in consultation with the primary stakeholders and the commissioner. The project leader will play a signalling role in this process.
  • The development, revision and/or implementation of a guideline, especially for a large multidisciplinary topic, sometimes requires developing a comprehensive project plan with many parties. To make a project manageable, continuous control of the result is very important. It should be kept in mind what the guideline must provide for the professional, with different results for different professionals. The essential stakeholders should be closely involved with the development group throughout the project. It is often an option to inform less essential stakeholders in other ways and to involve them, for example, as a co-reader, in the commentary round, in a feedback group or in an implementation group (see also RASCI).
  • In the preparation phase, the development group must anticipate two important reasons for extending the project: the availability of content experts and the commentary and authorisation phase. For a manageable commentary and authorisation phase, it is recommended to first agree with participating organisations on how, with whom and in what period the commentary and authorisation phase will take place. To anticipate the changing availability of experts, the development group can make agreements with the experts and their stakeholders about what they expect from each other. By doing this, the experts will shoulder some of the responsibility for the timely completion of the guideline. Drafting a contract can be helpful in this process. The project leader is primarily responsible for time management and therefore should communicate with authorised organisations when deadlines are not met. At that time, new agreements must be made. If authorised organisations fail to keep their agreements, the project leader may have to discuss options with the relevant development group members and the chair. One option is to change this organisation to a participating rather than authorising party for this guideline. It also helps to jointly agree from the beginning on who is responsible for authorising the content and who for approving it (but not necessarily providing authorisation).
  • Communication is important but not always an obvious part of the preparation and further steps of the guideline development. It is recommended that a fixed contact point be appointed within the participating association whose task would be assisting with matters to do with the guidelines, for both the development group and the commentary round. In a small association, this can be a designated board member, but a larger association may propose a different approach (for example, a separate committee). The project plan can describe what information and through which channels they can be communicated to each target group. It can also establish the frequency and organisation of communication.
  • In the development of a guideline, conflicts of interest cannot be prevented. However, there are rules that can limit the influence of interests and make them transparent (see Code for the prevention of improper influence due to conflicts of interest). The participation and commitment of a guideline’s development group members can be rewarded in different ways. For example, professional associations can award accreditation points for working on a guideline and the budget may take into account attendance fees and, possibly, travel expenses for meetings or a fixed fee for work delivered.
  • The development group closes every phase with an evaluation. At least the chair and the project leader should take part in this evaluation, but it is recommended to also involve or inform the development group members and the responsible administrator. Use the outcomes to tweak the project when necessary and possible.

3.4. Budgeting and creating a timetable

The preparation phase is essential to the progress of the project and the trick is to find a balance between quality and completeness on one hand and time and budget on the other. Translating the preparation phase into time and money can greatly increase the controllability of the project. This is often called a ‘preliminary study’ or a ‘feasibility study’. During guideline development, adjustments to the budget and timetable are required if the development group makes substantially different choices than expected: for example, it can add an extra review question or decide that the literature review should be performed differently. When the margins are not taken into account, the development group and commissioner can make changes in joint consultation. Opportunities to accelerate or to make the budget more manageable can lie, for example, in a more stringent selection of the literature, partial electronic/telephone conferencing and the organisation of ‘pressure cooker sessions’ with an extended meeting time.

This process will take roughly two to six months or 150 to 300 hours for the organising/initiating party to complete. This depends on whether the analysis  of clinical care gaps is included, whether client focus groups are organised, and whether there is a general literature review and risk analysis that focuses on important international guidelines and systematic reviews. It is difficult to determine how to process this investment in the budget. Grantors generally reimburse projects from the actual start of the process. This means that the investment must be budgeted under acquisition or a staff task, or the fee for carrying out the project should include the hours invested in the preparation. Given the importance of the preliminary process and the required pre-investment, it is good to consider the various options and their consequences. Who determines the preliminary process? Who pays for the preliminary process? Are the risks of a preliminary process that is not well conducted sufficiently recognised? And who shoulders the financial risks? With good preparation, the risks to both the party conducting the work and the commissioning party are certainly reduced.

4. Development stage

4.1. Guide to Guidelines

The Guide to Guidelines states that the following aspects are addressed during the development phase:

  • An analysis of clinical care gaps is performed at the start of the guideline development process.
  • Specific review questions are prepared on the basis of the analysis of clinical care gaps.
  • The literature is systematically summarised and transparently presented by a methodologist and a content expert.
  • The search method, review and literature summary are transparently presented in the guideline.
  • The guideline text has standard parts.
  • The guideline pays attention to knowledge gaps.
  • The guideline development supplies products tailored to the needs of both the relevant health care professionals and health care users.

4.2. Implications for project management

In the development phase, all the elements necessary for the development of a good guideline process are known except for the exact focus of the topic. Therefore, this is the most important step at the start of the development phase. The development group determines exactly where the focus should lie. This can involve making use of development group meetings. Focus groups, especially from the client perspective, are another appropriate method. When it comes to prioritisation, the simplest method is to have each development group member assign a score to a clinical care gap or review question. The development group can then deal with the clinical care gaps or review questions that have the highest score. It is possible to deliver a consensus statement for the review questions that were not prioritised. It is important to realise that some clinical care gaps or review questions are more suitable than others for addressing a guideline, as described in the Tool for the analysis of clinical care gaps. This has to do with whether there is a recognised profit in implementing the recommendation as well as how much knowledge is available from research and practice, for example, instances where there are many unwanted variances in practice between professionals’ procedures.

If the analysis of clinical care gaps for end users (professionals and clients) takes place during the development phase, the project plan should take into account that this will lead to adjustments in the content and the final product. This is only possible if the project leader has also arranged the other elements in a result-oriented and flexible manner. The inclusion of more review questions means, for example, that there may be a different distribution of work activities or a stricter selection of literature.

Furthermore, the development phase is especially an interaction between the project leader, the chair, development group members and methodologists who are all focussed on steering toward the final result. Based on this end result, they will determine how the various parts are best accomplished:

Activity Objective
Planning Developed and established in the preparation phase.
Literature search Identify existing guidelines and systematic previews
Meeting 1
in person
Establish the purpose, scope, timeframe and substantive focus. Discuss the tasks and responsibilities of all the parties. Discuss possible interests from the stakeholders and conflicts of interest from development group members; determine what additional exploratory literature review should take place.
Analysis of clinical care gaps Establish the most important clinical care gaps for the end users (professionals and clients) with a special focus on how the recommendations should be formatted to ensure the relevant guideline is properly applied in practice
Meeting 2
If necessary, by telephone, Skype, video conference
Establish the substantive focus and review questions. Determine which parts may be consensus based or can be developed during the editing stage.
Meeting 3
in person
Establish an initial working version of the guideline with the review questions, related action points and initial proposals for portions of the text. If necessary, discuss how to distribute the writing work.
Literature search Determine requirements for the literature research. Carry out in-depth literature research and identify literature based on the review questions. Make a clear selection.
Writing Write the first drafts and evidence overviews together with the chair and content experts (where necessary).
Meeting 4
in person
Discuss the drafts, evidence profiles and the classification of the studies in evidential value/GRADE; formulate the remaining considerations and recommendations
Writing Edit the guideline
Review the implementability Look specifically at issues such as definitiveness, clarity, text quality, consistency and the chance of successful implementation of the guideline, for example, with a trial implementation. Develop any derived products and link them to related parts in other guidelines.
Meeting 5
If necessary, by telephone, Skype, video conference
See what implications the review of implementability has raised; address deficiencies
Consultation round See Completion phase

Methods such as video conferences, retreats, pressure cookers or digital platforms can be useful in many cases.

National and international agreements have been made for systematic summaries and transparent representations of the literature. The Evidence Tables Working Group is working on this with the Guidelines International Network (GIN). In addition, the GRADE Working Group has been working on harmonising the assessment and weighing of scientific evidence. Examples also exist of the form in which guideline texts have been presented. However, there are differences in the format; it is wise to take time to reach an agreement on this at an early stage.

During the development of the guideline, you will regularly come across questions that were not initially identified as review questions. Also during the guideline development process, new obstacles to implementing the guidelines may emerge. The risk is that the guideline development group will want to immediately answer or resolve these, but this is unlikely to succeed unless the process is restarted for certain questions. The development group cannot actually address all the questions and obstacles that they identify during the development process. It is important to name these points in the guideline.

The guideline text is the primary product to be delivered. Various stakeholders may, however, need other derivative products and clients may also set certain requirements. From the point of view of a good implementation in practice, it may also be necessary to develop different products for different end users. For a controllable project management, this means that the final products must be considered during the preparation phase, or that the final products must be budgeted and implemented as parallel components. For suggestions about this, please see the Tool for implementation of guidelines.

4.3. Tips and tricks

Explicit and implicit interests can play a large role in the guideline development process. The individual interests of development group members can be made explicit in a declaration of interests. In addition, it is good to know more about the backgrounds and agendas the individual development group members or their stakeholder may have. The Tool for Organisation and Cooperation in Multidisciplinary Guideline Development delves further into this topic.

4.4. Budgeting and creating a timetable

The development phase of a guideline usually takes one to two years, depending on the number of review questions, the chosen approach for the literature research and the different aspects that have to be considered, such as knowledge gaps, diversity, client version, trial implementation, budget impact analysis and so forth. This greatly determines the budget. Issues such as whether or not to pay attendance fees, the required amount of process and methodological support and so forth also determine the size of the budget.

5. Completion stage

5.1. Guide to Guidelines

The Guide to Guidelines states that the following aspects are addressed during the completion phase:

  • Before the guideline is published, a commentary round takes place in which experts and future users of the guideline are consulted
  • The guideline is authorised by all the primary professional groups at least
  • A procedure for revising the guideline is specified
  • The primary relevant organisations inventory the implementation possibilities for all phases of the guideline development process and actively work to introduce the guideline in practice.

5.2. Implications for project management

A good commentary round serves several purposes. Obviously the primary goal is obtaining well-reasoned, substantive commentary on the guideline. A good and broadly inclusive commentary round also increases support for implementation and provides tips for better application in practice. It also contributes to a smooth authorisation process by helping the development group anticipate barriers to implementation. Also the professional association can trust that the stakeholders recognise the guideline and support its content. Guideline commentary may be of a substantive, organisational, financial and/or legal nature. In practice this can be tricky because the guideline itself only discusses the content, which does not alter the fact that commentary of an organisational, financial and/or legal nature is also welcome. It is therefore also important to give other meaningful comments a place in the guideline or to provide the most important stakeholders with information about how to use the guideline in practice and how it can best be optimised. More information can be found in the Tool for implementation of guidelines. A trial implementation may also be part of this.

Once commentary on the guideline has been obtained, it should be transparently summarised and presented to the development group. The development group determines what adjustments are needed to the final version of the guideline. This can possibly be done on the basis of a proposal by the project leader and the chair and any subgroup chair. The guideline or attachment can show the most important comments, who provided these comments and what changes the development group made based on them.

In cases where the commentary leads to a thorough reconsideration of the guideline’s content, the chair, in consultation with the project leader and the most important stakeholders, must determine whether the guideline can be completed and in what form. It is most transparent when the people who submitted commentary can see what was done with their comments and why.

Many associations only have meetings in which they discuss and approve guidelines a few times a year. Political considerations or conflicts of interest can play a role, leading to a situation where an association needs more time to accept a guideline. The project leader should make clear agreements with the participating organisations about the authorisation arrangements. By properly communicating the progress of the project, conducting a transparent and broad commentary round and making clear agreements about who in the association will perform the authorisation and when, the decisiveness increases. In addition, a participating association can be advised to delegate the authorisation to a guidelines committee or a separate authorisation committee. In the event that an association refuses to authorise a guideline, they should first try to involve a development group member delegated by the relevant association. If this is also unsuccessful, an independent mediator can be called upon.

Most guidelines indicate that a review is needed within five years. To date, however, there are few professional associations that pursue an active policy or have a set method for updating and revising guidelines or parts thereof. On behalf of the Dutch Council for Quality of Healthcare (Regieraad Kwaliteit van Zorg), an inventory is being made of which methods have been applied at home and abroad and what the pros and cons are. Guideline owners must have a policy for updating the guideline and who is responsible for it.

5.3. Tips and tricks

  • It causes delays when basic matters come up during the commentary phase. For that reason, it is in the interest of the project management process to make development group members aware that they should consider compromising until their stakeholders do, and that they should discuss issues with them. Second, they should be open to the possibility of disagreement within the organisation. Finally, they should consider whether one of their own monodisciplinary guidelines is in conflict with the guideline on which they are working.
  • The part about the revision of the guideline is often seen as a necessary but limited part of the guideline. Yet it is good for the development group to indicate how they think a periodic review can take place in a pragmatic manner. For example, this can be a simplified procedure with a panel of experts. The development group can also get a mandate to carry out maintenance on the guideline, which would then be approved through a formal consultation round. The construction of a modular guideline database could be an important infrastructural impulse. This process is discussed in the Tool for electronic disclosure of guidelines.

5.4. Budgeting and creating a timetable

Control over the timetable is difficult in this phase of the process. Therefore it is advisable to have made agreements in the initial phase of the project with the primary professional associations involved. For example, a three-month target period could be observed.