Health Policy Monitor
Skip Navigation

Evaluation of health prevention projects - Estonia

Country: 
Estland
Partner Institute: 
PRAXIS Center for Policy Studies, Tallinn
Survey no: 
(10)2007
Author(s): 
Agris Koppel, Ain Aaviksoo
Health Policy Issues: 
Prävention
Current Process Stages
Idee Pilotprojekt Strategiepapier Gesetzgebung Umsetzung Evaluation Veränderung/Richtungswechsel
Implemented in this survey? nein nein nein nein nein ja nein

Abstract

In 2006 the Estonian Health insurance fund (EHIF) started to evaluate performance and management of disease prevention projects. The evaluation is performed according to the EHIF aim of financing evidence based and cost-efficient preventive services and to ensure the sustainability of service provider organisations. In 2006 there were three projects evaluated and suggestions made for changing the current project objectives, activities and management processes upon the evaluation results.

Purpose of health policy or idea

EHIF has the legal responsibility for financing cost-efficient health services. According to the Health Insurance Law and the Estonian Health Insurance Fund Law the EHIF can adopt control mechanisms to monitor and audit all aspects of health services provision by providers having a service financing contract with EHIF. EHIF has started to finance disease prevention projects in a reasonable volume in 2001. In 2006 a first external audit process of 3 projects was ordered, which started in 2002.

The main purpose of the external evaluation is to monitor how the health provider has fullfilled the contract objectives, but  an assessment of performance indicators and organisational management issues is also conducted. The aim is to ensure the project organisations sustainability and achieve the national disease prevention targets.

Main points

Main objectives

The objective of EHIF is to encourage, through various quality developing activities, providers of health services to constantly improve the quality of their activity. Three long-term prevention projects were assessed in order to improve the quality of
prevention activities: the projects for the early detection of breast cancer, prevention of
heart diseases and early detection of osteoporosis.

Type of incentives

There are no direct financial incentives in conducting the evaluation of prevention projects, but internal and external evaluation can give inputs for improvement of organisational capacity (including cost-efficient resource management). The most important non-financial improvement could be seen from improving the health of the population.

Groups affected

Disease prevention services providers, Patients, Estonian health insurance fund

 Suchhilfe

Characteristics of this policy

Innovationsgrad traditionell recht traditionell innovativ
Kontroversität unumstritten kaum umstritten kontrovers
Strukturelle Wirkung marginal recht fundamental fundamental
Medienpräsenz sehr gering neutral sehr hoch
Übertragbarkeit sehr systemabhängig systemneutral systemneutral

The assessment of preventive services is highly suggested by many health economists and public health experts and the current process impemented on consensual basis by EHIF is rather traditional. But it is remarkable that at the current situation the external project  evaluation is an important amendment to the internal evaluations done by the project organisations. Also, the structural and systemic impact is rather fundamental, because the external evaluation of disease prevention projects has a positive impact on the development of contractual relationships between EHIF and providers, and furthermore gives a set of tools to the provider organisations to develop their organisation and activities.

Political and economic background

Financing disease prevention activities is the legal responsibility of EHIF since 2002. The share of the disease prevention budget out of the total health services budget is approximately 1%. The EHIF anually evaluates the status of populations health together with Ministry of Social Affairs and decisions about projects and their volume of services and financing are made in accordance with all other public health activities in Estonia.

Change based on an overall national health policy statement

The service implementation is in accordance with the general strategic goal of the Ministry of Social Affairs ? ?better health care service accessibility and quality?.

Purpose and process analysis

Current Process Stages

Idee Pilotprojekt Strategiepapier Gesetzgebung Umsetzung Evaluation Veränderung/Richtungswechsel
Implemented in this survey? nein nein nein nein nein ja nein

Origins of health policy idea

EHIF is financing disease prevention activities according to the Public Health Law, which came into force in 1995. Estonia applied for an international loan for health care in 1994 and the World Bank experts suggested that there is a need for a public health fund for financing public health activities in Estonia. The fund was implemented according to the contract agreement between Estonia and International Bank for Reconstruction and Development. Therefore, since 1995 public health services have been financed with a volume of up to 1% of the health insurance budget.

Until 2002 the choice of health promotion and prevention projects was the task of the bodies under the responsibility of MoSA. In 2002 the Commission of Health Promotion in EHIF has taken up its work and is responsible for deciding anually which projects are financed. The choice of health promotion projects is based on open competition, but strategically important disease prevention projects are financed on a long-term basis in a 5 year cycle. Beside the disease prevention projects there are also long-term disease prevention programs (eg. HIV/AIDS strategy or prevention of cardiovascular diseases etc.) financed by MoSA out of the state budget. Some of the programs have been taken over by MoSA after their successful implementation under EHIF finaincing (eg. the cancer prevention strategy).

As EHIF is legally responsible for using health inusrance resources in an efficient way, during the last decade many short-term but continous evaluation processes have been implemented, which have created the basis for development and improvement of health promotion and prevention service providers organisations. Also, the well-functioning network of local service providers together with county council representatives have been developed during last decade.

Initiators of idea/main actors

  • Regierung
  • Leistungserbringer
  • Kostenträger
  • Internationale Organisationen

Stakeholder positions

In the last 15 years, public health services and organisation in Estonia has developed in a remarkable way and all counterparts have contributed to these processes according to their ability and level of authority. Even though the specific strategic purposes are not always based on consensuality, the overall objectives and the willingness to invest into the improvement of public health have been directing the changes in the field.

Public health financing was made priority by international experts in Estonia and local institutions and individuals have been putting much effort into the development of the vertical and horizontal network of health promotion and prevention activities. Bottom-up and top-down initiatives together have created a good atmosphere for collaboration and integration.

Also, the fair of service providers about supervision, monitoring and negotiations about health services provision and financing have been quite well accepted as there benefits in organisational development and sustainability are seen as well. Therefore, both internal and external evaluations have become a stable part of financing and providing disease prevention and health promotion services.

Actors and positions

Description of actors and their positions
Regierung
Ministry of Social Affairssehr unterstützendunterstützend stark dagegen
Local municipalitiessehr unterstützendunterstützend stark dagegen
Leistungserbringer
Service providerssehr unterstützendunterstützend stark dagegen
Kostenträger
Estonian Health Insurance Fundsehr unterstützendsehr unterstützend stark dagegen
Internationale Organisationen
World Health Organisationsehr unterstützendsehr unterstützend stark dagegen
World Banksehr unterstützendsehr unterstützend stark dagegen

Influences in policy making and legislation

Health promotion and prevention services are provided in accordance to the Public Health Law which came into force already in 1995. In 2002 EHIF has taken responsibility for financing disease prevention services and health promotion projects thorugh a health insurance fund according to the new Health Insurance Law. The implementing and financing of disease prevention and health promotion programs by the MoSa is stated by parliament or government decisions.

Legislative outcome

n/a

Actors and influence

Description of actors and their influence

Regierung
Ministry of Social Affairssehr großgroß kein
Local municipalitiessehr großgroß kein
Leistungserbringer
Service providerssehr großgroß kein
Kostenträger
Estonian Health Insurance Fundsehr großsehr groß kein
Internationale Organisationen
World Health Organisationsehr großgroß kein
World Banksehr großneutral kein
World BankWorld Health OrganisationEstonian Health Insurance FundMinistry of Social Affairs, Local municipalities, Service providers

Positions and Influences at a glance

Graphical actors vs. influence map representing the above actors vs. influences table.

Adoption and implementation

EHIF has the following criteria for assessing a health promotion or prevention project's eligibility for financing out of the health insurance budget:

  • increase the public's or target group's awareness
  • monitor the risk factors
  • ensure the early detection of diseases
  • promote healthy behaviour.

Also, the projects cannot duplicate other projects or programmes implemented by other organisations. Obviously the projects or programmes have to be well targeted and evidence-based. Performance indicators should be applied to clearly present the purpose-outcome linkage in order to ensure cost-efficient resource utilisation in solving health problems through the health insurance budget.

The external evaluation of project impact and the performance and organisation audit are based on the objectives agreed between EHIF and project organisation before start of the project. As EHIF also covers the costs for management beside health services financing, the comprehensive evaluation covers the next important aspects of the project:

  • assessment of the level of performance in accordance to the project objectives and outcome indicators
  • evaluation of project outcomes to the control group outcomes
  • analysis of participants' satisfaction with services
  • conducting cost-efficiency analysis
  • auditing the organisational structure and processes according to the project needs and volume of services
  • assessment of project sustainability.

 

Monitoring and evaluation

 In 2006 three projects were evaluated:

  • early detection of breast cancer
  • prevention of heart diseases
  • early detection of osteoporosis.

The evaluation was carried out by two different institutions using different approaches, but the main results on the projects' activities and performance over the last 4 years were quite similar. The methodolody of evaluations included a detailed analysis of existing project documentation and reporting systems, analysing the data collection for performance assessment, but also interviews held with key persons from project organisations and EHIF.

 

Review mechanisms

Halbzeitevaluation

Dimensions of evaluation

Struktur, Prozess, Ergebnis

Results of evaluation

 The main results of evaluation of all three projects could be described as follows:

  • projects have been carried out in accordance to the criteria agreed on in the contract between EHIF and project organisation
  • organisations responsible for project management are built up in different structures. While one is a well-organised formal institution with clear processes, another is based more on a network of specialists lead by the specialists' association
  • there has to be done much more work on specifying the target group and searching for more efficient methods to increase the participation rate
  • collecting project relevant information is not always well organised as project leaders do not have full understanding of the need for mid-term performance measurement
  • comprehensive cost-efficiency studies cannot be performed at present because of small target groups volume and lack of detailed cost and outcomes data
  • much more effort has to be put into the training of project staff and organisational development
  • public visibility is not well ensured as participant satisfaction is not measured regularly (at least in the beginning of the project)

Expected outcome

The first external evaluation process involved three specific projects in the field of disease prevention. The results show that there are many questions which could be solved during the next phases of project planning. According to expert opinion the main effort should be channeled into creating a common understanding and awareness about the importance of performance monitoring as it seems to be the weakest part of project implementation processes. Without performance data there is no chance to conduct any cost-efficiency analysis even after a project period of ten years. This means that EHIF cannot ensure that the objective of financing only projects which contribute the most to improving public health indicators can be achieved.

Certainly comprehensive evaluation of disease prevention projects and programmes has to be a continous process and a natural part of project implementation. In connection with the internal assessment the external audit can ensure the development and sustainability of project organisations, which in countries like Estonia tend to be struggling mainly with basic services and the surviving of the organisations.

Impact of this policy

Qualität kaum Einfluss relativ starker Einfluss starker Einfluss
Gerechtigkeit System weniger gerecht System gerechter System gerechter
Kosteneffizienz sehr gering high sehr hoch

The external project evaluation is perceived as very useful by all actors, as the assessment results are creating a basis for improving both the formal processes of provision of preventive services and also ensure further quality management of these services. Regarding the evaluation of the osteoporosis project it was suggested to find out if broadening the target group would improve the health of more people, as it means that the system could become more equitable. As it was seen in all projects, improving management processes could also lead to better resource management and therefore ensure higher cost-efficiency, eg. if while using the same resources the involvement rate of target individuals is increased.

References

Sources of Information

Author/s and/or contributors to this survey

Agris Koppel, Ain Aaviksoo

Empfohlene Zitierweise für diesen Online-Artikel:

Agris Koppel, Ain Aaviksoo. "Evaluation of health prevention projects - Estonia". Health Policy Monitor, October 2007. Available at http://www.hpm.org/survey/ee/a10/1