Tool for cost-effectiveness in guidelines

1. Introduction

In the nineties, it became clear that to ensure health care access for everyone, choices also needed to be based on cost-effectiveness. ‘Dunning’s Funnel’ showed that not everything that is possible is actually necessary. The Royal Dutch Medical Association (Koninklijke Nederlandsche Maatschappij tot bevordering der Geneeskunst, KNMG) took the lead in the ‘Appropriate medical care’ project, which analysed whether guidelines are an adequate tool for evaluating cost-effectiveness. The project concluded that specifying the norms and values held by care professionals as well as patients and consumers is extremely important when justifying choices.

Since then, the methodology of evidence-based guideline development has taken a great leap forward and there is a growing consensus of the need to consider cost-effectiveness in the proposed interventions. The aim of this is to ensure that guidelines contribute to keeping high quality of care affordable in the Netherlands. In addition, it is important to have insight into the feasibility and financial implications of the recommendations provided in the guidelines. The Dutch Council for Quality of Healthcare (Regieraad Kwaliteit van Zorg) states that cost-effectiveness needs to be made a standard component in guideline development. Insight into the financial implications of recommendations can help people make socially responsible choices about health care resource allocation.

2. Definitions and basic principles

The basic principle is that cost-effectiveness should always be considered when preparing recommendations in guidelines. Its relevance may differ depending on a number of factors, which are listed in this tool. Interventions will first be evaluated in terms of cost-effectiveness and safety. In case of effective and safe interventions, cost-effectiveness is a relevant criterion.

Cost-effectiveness is the relationship between the resources employed and the results achieved with those resources. This relates to all aspects of health care, both diagnostics and therapy, but also prevention, screening and aftercare.

A cost-effectiveness analysis (CEA) can provide information about the cost-effectiveness of an intervention. A full CEA involves a comparison between two or more alternatives and considers both the costs and effects (e.g. quality of life or labour participation1). The cost-effectiveness of an intervention is always relative in comparison to another possible alternative. The results of a CEA show the extra investments needed for the new intervention to accomplish a specific result (e.g. an extra year of life).

Budget impact analyses are also discussed in the context of this tool. A budget impact analysis (BIA) describes the financial implications and changes in budgets and cash flows following a health care change that is expected to result from implementing the guideline.

1 A full CEA can be performed based on the International Classification of Functioning, Disability and Health (ICF) [Dutch translation, 2002], which describes human functioning from the perspectives of (1) the human organism, (2) human activities and (3) people as social participants. The first perspective is elaborated in two separate classifications: the classification of the organism’s functions and the classification of its anatomical features. The second and third perspectives are elaborated in the classification of activities and participation.

3. Objective of this tool

This tool shows how cost-effectiveness and/or budget impact can be involved in drafting a guideline and deciding which choices to make. It also helps in evaluating whether additional analyses are needed and what type of expertise this will require.

This relates to the following:

  • When to incorporate cost-effectiveness or budget impact?
  • How? Focal points for each phase in the guideline development process
  • The use of internal or external expertise.

For detailed information about how a CEA or BIA should be performed, please refer to the Dutch ‘Manual for Cost Research’ (Handleiding voor kostenonderzoek) and economic evaluation handbooks.

4. When should cost-effectiveness or budget impact be incorporated?

In all guidelines? In principle, cost-effectiveness should be considered for all guidelines, both new and revised. The degree to which this is done may vary per guideline. The Guide to Guidelines states that for topics with major organisational and financial consequences, the literature on cost-effectiveness should be consulted and, if necessary, an economic evaluation should be performed.

In all recommendations? At the start of the guideline process, it is important that the development group, in consultation with a Health Technology Assessment (HTA) expert, determines the topics and questions for which cost-effectiveness and/or budget impact play an important role.

CEA and/or BIA? In general, a CEA is considered important for new interventions that are more expensive and more effective than the current health care; a BIA may be important in recommendations that lead to changes in health care, especially if a recommendation concerns a large number of patients and a cost increase (or reduction) is expected, even if the change in costs per patient is limited. If a task reallocation is proposed or a change in health care occurs that is funded differently, a BIA may also be considered important and the different budgets should be specifically considered. In many cases, both a CEA and a BIA will be useful.

5. How? Attention points for each phase of guideline development

5.1. Analysis of clinical care gaps

An analysis of clinical care gaps is performed at the start of the guideline development process. When surveying organisations and associations to identify clinical care gaps, it is also important to ask about issues related to cost-effectiveness, for example, with regard to efficiency and the deployment of personnel and resources.
The following aspects are important when identifying clinical care gaps in the
field of cost-effectiveness:

  • Have there been any new developments, such as new treatments or diagnostic interventions since the previous guidelines?
  • Are there any treatments or diagnostic interventions that were included in the previous guidelines and need to be revised for cost-effectiveness?
  • Have there been discussions about task reallocation, for example, between providers or between primary and secondary care? Collaboration between primary and secondary care?
  • Are there policy-related questions about the use of expensive drugs or tools?
  • Are there any potential problems related to the staff capacity or resources?

Later in the programme, an additional analysis can be performed with financial stakeholders such as insurers.

5.2. Drafting the review questions

The guideline development group drafts the review questions based on the Population, Intervention, Comparison, and Outcome (PICO) model and describes the outcome measures in terms of cost-effectiveness. The following considerations should be taken into account:

The outcomes of a CEA are highly dependent on the study’s perspective. The pharmacoeconomic guidelines state that a societal perspective should be applied. When developing guidelines, it may also be important to present a different perspective, for example, that of the health insurer or the institution. This is particularly important when the results of the different perspectives are expected to differ significantly.

The results of an economic evaluation are preferably expressed in euros or quality-adjusted life years (QALY). Though no clear threshold value has been set in the Netherlands, in 2006 the Council for Public Health and Health Care (Raad voor de Volksgezondheid en Zorg, RVZ) proposed a threshold value that varies with the severity of the condition and may be as much as €80,000 per QALY. However, the applicability of this threshold value is under discussion.

When comparing interventions for a syndrome, it is also possible to apply disease-specific outcome measures (e.g. the number of diabetes patients with HBA1c below a certain target value). In connection with the review of clinical effectiveness, the effects can be expressed in the number needed to treat (NNT), which results in the additional cost per successfully treated patient.

5.3. Searching for literature

Ideally, an information specialist searches for literature about cost-effectiveness while also searching for clinical evidence either independently or together with the HTA expert. Some of the search terms will be identical, such as patient population, intervention and, possibly, comparison of alternatives. The HTA expert will discuss the search terms, search filters and database use with the information specialist.

If new analyses (CEA and/or BIA) are expected to be needed for the new guideline, it is advisable to simultaneously look for existing analytical models.

Since a BIA relates to the financial implications of a recommendation in a specific context, it is unlikely that BIAs relevant to the Dutch situation will be available in the literature. Therefore, additional analyses will generally be needed (see paragraph 5.5).

5.4. Judging the information

In line with the Evidence-Based Medicine methodology, the literature found by the development group on cost-effectiveness is analysed for validity, relevance and applicability. Available checklists can be used for analysing the validity of literature. The assessments look at aspects such as the comparison of alternatives and the analysis period (time horizon). In order to properly assess the relevance of the research choices made, it is important to have expertise in the field of cost-effectiveness research as well as clinical expertise. An important aspect in interpreting data on cost-effectiveness is uncertainty about the results. Well-performed analyses can identify any uncertainties regarding cost-effectiveness.

When it comes to testing the applicability of the results of cost-effectiveness research, special attention needs to be paid to how applicable the results of other research are to the current situation (transferability). Since the organisation of health care and medical costs can vary per country, Dutch information is preferable. At times, the development group will need to choose between using outdated Dutch data or more recent international data. This decision needs to be made together with the HTA expert to ensure it is carefully chosen and the reasons substantiated.

5.5. Additional analyses, if needed

If there is insufficient information from which to draw a conclusion, then the development group will need to decide whether to collect additional Dutch data and, subsequently, whether a CEA and/or BIA will need to be performed. An HTA expert will be needed to specify whether existing models can be used and to estimate the time and budget requirements. If the development group decides against performing an analysis, they must provide clear motivation for this decision in the guideline and may recommend that others carry out the necessary research. If the decision to perform additional analyses is made, it is advisable to specify the chosen perspective for these analyses (e.g., social, institutional, professional or patient).

Examples of considerations when deciding whether to perform additional analyses:

  • What are the expected costs for the analysis and what consequences does this have for the guideline planning?
  • How many patients does it involve and what is the disease burden?
  • How much uncertainty is there currently about the cost-effectiveness?
  • Will sufficient valid information be available in the short term?
  • How likely is it that this information will reduce the uncertainty?
  • How quickly will the changes occur and how long will the cost-effectiveness considerations remain valid (e.g. rapidly falling price of medicine, new intervention anticipated within one year, a different reimbursement system)?
  • Can an estimate be made of the differences in cost-effectiveness between the different policy options?

5.6. Formulating conclusions and recommendations

Based on the available literature, the development group will formulate one or more conclusions, which they will grade according to the amount of evidence for this classification. Based on the number of investigations and the amount of evidence, they will assign a level of evidential value to the conclusion. In a similar manner, the evidence regarding the care’s cost-effectiveness can be graded. However, there are still no generally valid diagrams available.

When formulating their recommendations, the development group will focus primarily on the effectiveness and safety of interventions and on providing evidence to substantiate this cost-effectiveness. If there is insufficient evidence for the cost-effectiveness, this will be explicitly mentioned. The Grades of Recommendations, Assessment, Development and Evaluation (GRADE) system proposes that the gains in health benefits (effects) are determined first. Next, the gain in health benefits must be weighed against incremental costs (resource use) in order to formulate recommendations.

The development group members and the HTA expert, preferably together, will interpret the cost-effectiveness and results of additional analyses. The HTA expert will provide the development group with sufficient information (including information about the uncertainty of the results) on which to base their assessment. In principle, the development group will choose the alternative with the most favourable cost-efficiency ratio.

It is not clear precisely how budget impact and capacity costs should be weighed. A large budget impact is not necessarily undesirable if major health benefits can be achieved. Insight into the anticipated overall impact, total costs and capacity costs may be important for proper planning at the national level. New, sustainable equipment (e.g. PET scan) will not immediately be available at all institutions, making it necessary to consider the need for centralisation. Above all, the role of economic evaluations will also be to identify bottlenecks in our knowledge about the cost-effectiveness of the treatment and diagnostic interventions discussed in the guideline.

5.7. Agreements regarding guideline updates

Detailing the cost-effectiveness of a guideline can be very labour intensive and therefore costly. It is preferable to archive the information collected and any models used in such a way that new calculations, or significant changes in practice or in a new guideline update are available to the development group. In principle, the development group should involve the HTA expert when updating the guidelines.

6. Expertise in the field of cost-effectiveness

Judging research on the cost-effectiveness of interventions and the possible implementation of additional analyses requires specific expertise. The starting point is the development group appointing someone with expertise to act as advisor. This expert will be responsible for the following parts of the process: formulating clinical care gaps and review questions, searching and reviewing the literature and/or performing a CEA and/or BIA. Depending on the available expertise in a development group, members of that group or guideline advisors can perform certain parts of the cost-effectiveness research. An HTA expert can be involved to varying degrees.

It is strongly recommended to involve the HTA expert from the start of the guideline development process to prevent him or her from having to start with a poorly formulated review question.

The development group will come to an agreement about the HTA expert’s role and the exact division of tasks in cost-effectiveness research, the presence of the HTA expert at development group meetings and the management and ownership rights of any developed models and their use in updating the guidelines. Large guideline development groups may consider forming a subgroup to develop proposals on the theme of cost-effectiveness.

When preparing the budget, the need to hire HTA expertise must be taken into account. The costs largely depend on the size of the analysis. The costs for an economic analysis, including literature study and interpretation of the data, are estimated at €20,000, and may increase to €60,000 if a CEA and/or BIA is performed.

6.1. Competence profile of an HTA expert

The HTA experts working in the Netherlands (about 50) have very diverse backgrounds (economists, health scientists, epidemiologists, clinicians, etc.). Publications in national and international journals have shown that it is desirable for the person concerned to have experience in searching for and evaluating evidence of cost-effectiveness and performing cost analyses and budget impact analyses. Preferably, the HTA expert will also have knowledge of and experience in the topic concerned and in developing guidelines. If the HTA expert has not been involved in developing guidelines before, it is recommended that the expert follow a course on evidence-based guideline development.

6.2. NVTAG

Many of the HTA experts are registered with The Dutch Society for Technology Assessment in Health Care (De Nederlandse Vereniging voor Technology Assessment in de Gezondheidszorg, NVTAG) ( This professional association can help select people with the required expertise.

7. Literature

  • Adviescommissie Richtlijnen – Raad Kwaliteit OMS. Rapport Medisch Specialistische Richtlijnen 2.0. Utrecht: Orde van Medisch Specialisten, 2011.
  • Casparie AF, Hout BA van, Simoons ML. Richtlijnen en kosten. Ned Tijdschr Geneeskd. 1998;142:2075-7.
  • Commissie Keuzen in de zorg. Rapport Kiezen en delen, advies in hoofdzaken, Zoetermeer: Ministerie van VWS, 1991.
  • College voor Zorgverzekeringen (CVZ). Handleiding voor kostenonderzoek, 2010.
  • College voor Zorgverzekeringen (CVZ). Richtlijnen voor farmaco-economisch onderzoek, 2006.
  • De Neeling JND, namens de projectgroep Passende Medische Zorg. Eindrapport ‘Passende medische zorg. Over keuzen en richtlijnen’ Utrecht: KNMG, 2000
  • Drummond ME, Sculpher MJ, Torrance GW, O'Brien BJ. Methods for the Economic Evaluation of Health Care Programmes. Oxford: Oxford University Press, 2005.
  • Essers BA, Seferina SC, Tjan-Heijnen VC, Severens JL, Novák A, Pompen M, et al. Transferability of model-based economic evaluations: the case of trastuzumab for the adjuvant treatment of HER2-positive early breast cancer in the Netherlands. Value Health 2010;13(4):375-80.
  • Nederlandse vertaling van de WHO-publicatie: International Classification of Functioning, Disability and Health: ICF, Geneva 2001, Houten: BSL, 2002. Voor een compilatie zie
  • Giepmans L, Moret‐Hartman M, Burgers J., et al. Inventarisatie kosten corpus richtlijnen. Regieraad Kwaliteit van Zorg, 2010.
  • GRADE working group. Grading quality of evidence and strength of recommendations. BMJ 2004.
  • Regieraad Kwaliteit van zorg. Gaan richtlijnen en doelmatigheid samen? Welke kosten(berekeningen) horen thuis in een richtlijn. Regieraad Kwaliteit van zorg, 2010.
  • Raad voor Volksgezondheid & Zorg (RVZ). Zinnige en duurzame zorg, 2006
  • Rutten-van Mölken M, Busschbach J, Rutten F. Van kosten tot effecten: een handleiding voor economische evaluatiestudies in de gezondheidszorg (tweede, herziene druk). Elsevier Gezondheidszorg, 2010.
  • Welte R, Feenstra T, Jager H, Leidl R. A decision chart for assessing and improving the transferability of economic evaluation results between countries. Pharmacoeconomics 2004;22: 857–76.
  • Werkgroep Richtlijn voor Richtlijnen. Richtlijn voor Richtlijnen. Den Haag: Regieraad Kwaliteit van Zorg, 2012.