Guidelines for the use of Performance Indicators A part of rinderpest surveillance for the Global Rinderpest Eradication Programme (GREP)

Table of Contents

1. Introduction

2. Disease surveillance in eradication programmes

3. Real-time monitoring of sensitivity, specificity and quality of rinderpest surveillance

4. Performance indicators for rinderpest in practice

5. How countries in GREP can use performance indicators to support their application to OIE for recognition of freedom from rinderpest

6. Conclusion

7. Further reading

Appendices

Appendix 1. Examples of Performance indicators for rinderpest and their application

Appendix II. Examples of Diagnostic Indicators for assessing performance of a surveillance system

Appendix III. Examples of performance indicators

List of figures

Figure 1. Application of disease surveillance

Figure 2. Components of a rinderpest surveillance system

Figure 3. Reporting and feedback in an animal disease surveillance system

Figure 4. Flowchart to describe response to a stomatitis enteritis clinical report

Figure 5. Stomatitis-enteritis syndrome

Figure 6. The OIE pathway and the relevance of performance indicators 

Acknowledgements

This document was compiled and edited by Bernadette Abela.

Documentation and contributions were provided by Jeffrey C. Mariner, RDP Livestock Services B.V, Gijs van ’t Klooster, RDP Livestock Services B.V., Martyn H. Jeggo, Joint FAO/IAEA Division, Peter Roeder, FAO/EMPRES and Roland Geiger, Joint FAO/IAEA Division.

Current mailing addresses

Jeffrey C. Mariner, Veterinary Services, Animal and Plant Health Inspection Service, USDA, 555 S. Howes, Fort Collins, CO 80521, USA. E-mail:mailto:jeffrey.c.mariner@usda.gov

Gijs van ’t Klooster, Technical Advisor to PARC Ethiopia, RDP Services B.V. P.O. Box 5507, Addis Ababa, Ethiopia. E-mail: gijsvan.tklooster@telecomnet.et

Martyn H. Jeggo, Joint FAO/IAEA Division, P.O. Box 100, Vienna, Austria E-mail: m.h.jeggo@iaea.org

Peter Roeder, Animal Production and Health Service, FAO, Viale delle Terme di Caracalla 00100 Rome, Italy E-mail: peter.roeder@fao.org

Roland Geiger, PARC, P.O. Box 30786, Nairobi, Kenya.E-mail: rg.parc@africaonline.co.ke

List of Abbreviations

1. Introduction

This publication will provide practical guidelines for Chief Veterinary Officers (Directors of Veterinary Services etc.), rinderpest control co-ordinators, sero-surveillance co-ordinators and laboratory directors at national level on the use of performance indicators designed for rinderpest surveillance, as a part of the Global Rinderpest Eradication Programme (GREP).

It aims to guide the readers through a comprehensive indicator system capable of assisting routine monitoring of a national surveillance programme and identifying any deficiencies the programme may possess to prompt appropriate action.

1.1 Performance indicators

Performance indicators provide assurance that a surveillance system, consisting of both active and passive surveillance (refer to section 2), would be able to detect disease or virus if these were present in a population or country.

Moreover, performance indicators are specifically designed key measures of quality, sensitivity and quantity of a surveillance system, which evaluate whether achievements of a national disease surveillance programme are on target. They comprise time-delimited, denominator-based statistics.

1.2 Rinderpest specific performance indicators

Rinderpest specific performance indicators are management tools for countries to objectively measure and assess their ability to detect rinderpest disease or virus. Furthermore, performance indicators would assist GREP countries in demonstrating the presence of statistically valid, passive and active rinderpest disease surveillance capable of rapidly recognising a disease outbreak.

Indicators are system dependent, therefore, in order to formulate performance indicators for a rinderpest surveillance system, the objectives and methodologies of the surveillance system must be clearly defined.

All rinderpest surveillance activities, with the exception of sero-surveillance and possibly passive disease reporting, should be targeted to detect and diagnose field outbreaks of stomatitis-enteritis rather than rinderpest. It can be assumed that all countries experience outbreaks of stomatitis-enteritis and therefore should be finding and investigating these outbreaks, if surveillance programmes are sensitive (refer to section 3.1).

2. Disease surveillance in eradication programmes

Disease surveillance is the continuous investigation of a given population to detect occurrence of disease, which would prompt required action. It involves continuous systematic collection, analysis and interpretation of animal health data which is essential to planning, implementation and evaluation of animal health systems, closely integrated to their timely dissemination.

Figure 1. Application of disease surveillance

The application of surveillance data is essential for prevention and control. A surveillance system requires a functional capacity for data collection, analysis and dissemination.

2.1 Major components for rinderpest surveillance system

The major components for rinderpest surveillance would include the following: passive disease surveillance, active disease reporting, investigation and laboratory diagnosis, active disease search, sero-surveillance and wildlife surveillance.

Figure 2. Components of a rinderpest surveillance system

2.1.1 Passive disease surveillance

Passive disease surveillance functions within a routine national disease reporting system. It is a system, which collates monthly reports on significant disease occurrence observed mainly by veterinarians during fieldwork and through communication with livestock owners. Laboratory data generated as a result of routine activities may supplement the reports. Data collected at abattoirs through meat inspection also belong to the passive surveillance system.

Passive disease reporting systems should cover all OIE List A and List B diseases as well as any diseases of local significance.

2.1.2 Active surveillance

Active disease surveillance is intended to determine the occurrence of specific clinical syndromes.

2.1.2.1 Active disease reporting

There are different groups that contribute to a well functioning reporting system including livestock owners, veterinary staff and animal health managers.

Figure 3. Reporting and feedback in an animal disease surveillance system

Livestock owners need to be sensitised to report any cases of stomatitis-enteritis. In order to sustain data flow, feedback of results of investigation and testing of reported cases should be supplied back to livestock owners.

Veterinary staff need to be aware of the stomatitis-enteritis case definition, the importance of recording, investigation and sampling.

For managerial and decision making purposes, stomatitis-enteritis occurrence should provide an alert system and allow for prompt response and action.

2.1.2.2 Active disease investigation and laboratory diagnosis

In the framework of a rinderpest eradication programme, sampling of all stomatitis-enteritis cases should be emphasised. Additionally, stomatitis-enteritis cases can only be confirmed and ultimately diagnosed through laboratory diagnosis.

To secure final diagnosis through laboratory testing, staff should be trained to take, handle and dispatch samples appropriately. Reference laboratories play a role in assisting with final diagnosis of new rinderpest outbreaks and in periodic monitoring.

2.1.2.3 Active disease search

In the context of a rinderpest surveillance programme, active surveillance would encompass a search for a syndrome associated with stomatitis and enteritis. Such a surveillance system initiates and maintains field level surveillance with the sole purpose of detecting this clinical syndrome in animal populations. Consequently, active disease surveillance is not a search for rinderpest disease; it is a search for clinical syndromes associated with stomatitis and enteritis.

The definitive diagnosis relies on laboratory investigation (Refer to Figure 4).

An event requires reviewing if it has not been definitively diagnosed as either being rinderpest or a confirmed differential disease. A well functioning surveillance system should only have stomatitis-enteritis events in review classification as a transient category.

Stomatitis-enteritis events that are not concluded would indicate a weak point in a surveillance system.

Figure 4. Flowchart to describe response to a stomatitis enteritis clinical report

Stomatitis-enteritis syndrome has been defined as including ocular and nasal discharge with any two of the following signs: fever, oral erosions or lesions, excessive salivation, corneal opacity, diarrhoea and death (Refer to Figure 5). Although corneal opacity is not a usual rinderpest sign, it is included in the stomatitis-enteritis complex since this sign has been known to complicate diagnosis. These criteria should be applied at population level rather than at the level of individual animals.

If active disease surveillance is functioning correctly, outbreaks of diseases such as bovine viral diarrhoea (BVD), infectious bovine rhino-tracheitis (IBR) and malignant catarrhal fever (MCF) should be detected, sampled, diagnosed on the basis of confirmatory laboratory results, and reported.

2.1.3 Sero-surveillance

Rinderpest sero-surveillance is a means of illustrating the presence of virus in a population by detecting specific antibody to rinderpest virus in serum, however serological tests do not distinguish between serological response to infection with wild virus or inoculation with vaccine virus. This therefore limits the specificity of serological testing (refer to section 3.2).

However, in populations, which are not vaccinated sero-surveillance, may confirm the absence of rinderpest virus.

2.1.3.1 Wildlife surveillance

Surveillance of wildlife is only applicable to those countries with a significant wildlife population capable of contributing to the maintenance of rinderpest virus. Wildlife would then represent a non-vaccinated sentinel population.

Figure 5. Stomatitis-enteritis syndrome Ocular & nasal discharge Diarrhoea Fever Salivation Oral erosions Death

3. Real-time monitoring of sensitivity, specificity and quality of rinderpest surveillance

A surveillance system provides a method of detection, investigation and confirmation of a chain of events that should take place in a timely manner. Subsequently, such a system should facilitate a rapid response. The purpose of performance indicators is to measure the sensitivity, specificity and timeliness of a national surveillance system.

3.1. Sensitive

A sensitive surveillance system must be able to detect a high percentage of field events that are clinically or epidemiologically compatible with rinderpest.

3.2. Specific

A specific surveillance system must be able to provide a definitive diagnosis for a high percentage of rinderpest-like field events after investigation.

3.3. Timely

The system must be able to detect, diagnose and report results within a time frame that permits prompt field response.

3.4 Zero reporting

Apparent lack of a disease, in this case rinderpest, could be either due to the absence of rinderpest disease or ineffective disease surveillance. Zero reporting distinguishes the two categories by ensuring an active reporting system within which negative reports or reports of the absence of rinderpest compatible outbreaks are documented.

A zero report would imply that a search was conducted but no evidence of rinderpest was found.

3.5 Rapid response

One of the main objectives of surveillance is to provide sufficient information to decision-makers to decide on appropriate interventions and facilitate rapid response. Rapid response is dependant on a sensitive, specific and timely system.

In the case of a rinderpest emergency, the expected response should be documented in a rinderpest contingency plan where action is considered and detailed in advance and resource requirements are defined and readily available for efficient use.

4. Performance indicators for rinderpest in practice

Suggestions for performance indicators are provided below (for more detail refer to Appendix I). Since performance indicators are system dependent these should be adapted and designed to fit the objectives and methodologies of the surveillance system for rinderpest in each country.

Performance indicators should provide tools to assess the sensitivity, specificity and timeliness of the major components of rinderpest disease surveillance. The examples of performance indicators listed below make use of the number of administrative districts, population of susceptible species and numbers of identified population strata for the proportion calculation. The calculations are also assessed subject to time. Two worked examples can be found in Appendix III.

4.1 Prerequisite criteria

To achieve performance at different levels of the chain of events of surveillance - at sample collection, testing and reporting, and to detect why efficacy and efficiency of a system may be lacking, it is necessary to identify upon which criteria, performance is dependant. In identifying the underlying criteria it is then possible to diagnose why performance may be below expectation to subsequently allow for remedial action.

Examples of diagnostic indicators that assist in identifying shortcomings in performance, based upon the pre-requisite presence of criteria, are provided in detail in Appendix II.

Furthermore, tangible inputs e.g. transportation, laboratory equipment, or less tangible inputs e.g. training, are required for a surveillance system to function. These may be summarised in checklists that are critical in the design and evaluation of performance and diagnostic indicators.

4.2 Suggestions for performance indicators

Below are eight suggested performance indicators for the evaluation of rinderpest surveillance.

Timeliness, manpower, data quality, effective data analysis and their utilisation are common requirements for the implementation of all the performance indicators suggested below. Emphasised with each proposed performance indicator are some additional points that are necessary for performance to be acquired.

The examples of PI 3-7 require accurate estimates of population size in order to draw any reasonable conclusions.

PI 1 - Performance indicator for general disease surveillance

Number of districts forwarding general disease reporting forms within 30 days at least 10 months of the year per total number of districts.

This indicator relies heavily on

• the passive surveillance structure, trained staff and materials

• the capability of districts completing a disease report

• the postal system or other method of transmission of reports

• motivation of staff

PI 2 - Performance indicator for active disease search

Number of districts surveyed using active disease search techniques (participatory, questionnaire-based and clinical) with results reported within 90 days per total number of districts.

In addition to the basic common requirements mentioned above, active disease search requires the following conditions

• availability of specialised resources and a recording system

• strength in the activity chain from data collection to analysis and reporting

PI 3 - Performance indicator for active disease reporting

Number of reports of stomatitis-enteritis outbreaks received, recorded and forwarded within 30 days per 100000 heads of susceptible species.

This indicator measures the reporting of outbreaks involving one or more animals, and requires the following

• recorded, noted or at least communicated reports

• the implementation of the district report recording system

PI 4 - Performance indicator for stomatitis-enteritis outbreak investigation and sample submission to laboratories

Number of reports of stomatitis-enteritis outbreaks investigated and appropriately sampled by veterinary professionals trained in rinderpest surveillance within 7 days of report per 100000 heads of susceptible species.

The evaluation of stomatitis-enteritis outbreak investigation, involving one or more animals, assuming that all animals involved in an outbreak are sampled, is dependent on

• timeliness of the investigation especially since clinical signs of individual cases normally have a duration of up to 7 days.

• impartial surveillance throughout the country

• formally documented cases

• availability of equipment and trained staff

• Communications reaching livestock owners for a complete cycle of back and forward tracing and link to veterinary services for reporting

• the timeliness of sample collection in relation to epidemic curve and submission to laboratories

PI 5 - Performance indicator for preliminary rinderpest diagnostic testing

Number of cases examined by rinderpest antigen, serological, immuno-histopathological and/or RNA detection techniques with preliminary results reported within 3 days of receipt of samples per 100000 heads of susceptible species.

The performance of rinderpest diagnostic testing is based upon

• availability and implementation of techniques

• samples reaching the laboratory

• condition of the sample on arrival

• suitability of the samples submitted

• laboratory performing the tests

PI 6 - Performance indicator for stomatitis-enteritis case definitive diagnosis

Number of stomatitis-enteritis cases diagnosed definitively by laboratory methods at national and/or reference laboratories within 60 days of receipt of samples per 100000 heads of susceptible species (e.g. RP, BVD, MCF, ECF, FMD etc.).

Definitive diagnostic performance is dependent on

• capability of establishing differential diagnostic tests

• number of samples arriving and the time required to reach diagnosis

• the use of international reference laboratories

PI 7 - Performance indicator for sero-surveillance

Number of serum samples collected and tested with results reported within 120 days of collection per total number of populations identified in the country.

The outcome of sero-surveillance relies on

• quantity and quality of samples collected and submitted

• timeliness of sample collection, submission, testing and reporting

• availability of reagents

• conformation to the random sampling plan and reliability of estimates

Annual sample sizes should be sufficient to provide 95% probability of detecting evidence of rinderpest if present at a prevalence of 1% of herds (or other sampling units), and 5% within herds (or other sampling units).

This can typically be achieved by random sampling of 300 herds in a population, where 15 to 20 animals per herd are tested.

Procedures for sampling should be in accordance with the Guide to Epidemiological Surveillance for Rinderpest published by OIE, 1989 and updated in 1998 or any other procedure that would achieve the same probability of detection.

PI 8 - Performance indicator for wildlife surveillance (special indicator)

Number of serum samples collected and tested with results reported within 90 days of collection per thousand heads of susceptible species.

This performance indicator is only applicable in countries where wildlife surveillance is appropriate and would be dependent on

• available resources: trained staff, funding

• samples collected, tested and with definitive diagnosis

4.2 Setting the targets in individual countries

Every country will need to establish expected targets within realistically defined time frames of one to two years, for the performance of their rinderpest surveillance programme.

Actual performance results may then be compared directly to these estimated targets (Refer to Appendix 1).

This, however, assumes that countries are in possession of data required to draw a valid conclusion on each performance indicator. If data is not available, countries would have to carry out background research or extrapolate from information of other countries with similar conditions to establish realistic estimates before they can implement a system of performance indicators. Basic information regarding for example the estimated occurrences of rinderpest-like diseases such as BVD, MCF, FMD or ECF would be needed.

If targets are established but are not being achieved within a surveillance programme, specific remedial action should be triggered off to strengthen the identified weak link. When integrated in a surveillance system, performance indicators should be used to direct a system rather than generate additional, frustrating workload for veterinary services.

5. How countries in GREP can use performance indicators to support their application to OIE for recognition of freedom from rinderpest

Countries progressing along the OIE pathway from ‘provisional freedom from rinderpest’ through ‘freedom from rinderpest disease’ and finally to ‘freedom from rinderpest infection’ shall be required to demonstrate the presence of a well functioning veterinary service and provide details of their disease surveillance and reporting system. These are detailed in ‘Recommended Standards For Epidemiological Surveillance Systems For Rinderpest’ which was adopted by the International Committee of the OIE in May 1998 (Refer to Figure 5).

Countries may use the concept of performance indicators to provide evidence and documentation to the OIE for official international recognition of freedom-from-rinderpest in order to benefit from increased access to international export markets for livestock and livestock products.

Figure 6. The OIE pathway and the relevance of performance indicators

6. Conclusion

The concept of performance indicators should provide countries with a management tool to assist in operating their disease surveillance systems efficiently and effectively but the implementation of performance indicators requires a functional capacity for data collection, analysis and dissemination.

The use of performance indicators should assist countries in verifying freedom from rinderpest in order to proceed efficiently along the OIE Pathway. Thereby, countries will be able to take advantage of potentially lucrative trade opportunities by credibly demonstrating the capability to declare freedom from rinderpest disease and/or infection to the OIE and GREP.

7. Further reading

OIE. 1999. International Animal Health Code. Eight edition.

International Committee of the OIE (1998). Recommended standards for epidemiological surveillance systems for rinderpest. Also available at http://www.oie.int/norms/a_rinder.htm

USDA, Centers for Disease Control and Prevention. 1997. Manual for the surveillance of vaccine preventable diseases. Centers for Disease Control and Prevention: Atlanta, USA. Also available at http://www.cdc.gov/nip/publications/manual.htm

USDA, Centers for Disease Control and Prevention. 1997. Case Definitions for Infectious Conditions under Public Health Surveillance. Centers for Disease Control and Prevention: Atlanta, USA. Also available at http://www.cdc.gov/epo/dphsi/casedef/cover97.htm#top

WHO. 1997. Field guide for supplementary activities aimed at achieving polio eradication, Global Programme for Vaccines and Immunisation, Expanded Programme on Immunisation, World Health Organisation, Geneva.

FAO/IAEA. 1994. Recommended procedures for disease and serological surveillance as part of the Global Rinderpest Eradication Programme (GREP). IAEA-TECDOC-747.

OIE, International Committee of the OIE. 1989. Recommended standards for epidemiological surveillance systems for rinderpest.

FAO. 1999. Manual on preparation of national animal disease emergency preparedness plans.

FAO. 1999. Manual for preparation of rinderpest contingency plans.

James , A. D. (1999). Guide to epidemiological surveillance for rinderpest. Rev. sci. tech. Off. int. Epiz. , 17 (3),

The following references may be of interest to epidemiologists and laboratory diagnosticians interested in differential diagnosis of stomatitis-enteritis:

Rinderpest

Anderson, J., Barrett, T. and Scott, G. R. 1996. Manual on the diagnosis of rinderpest, FAO, Rome.

Bovine viral diarrhoea and mucosal disease

Laamanen, U. I., Neuvonen, E. P., Yliviuhkola, E. M. and Veijalainen, P. M. L. 1997. Comparison of RT-PCR assay and virus isolation in cell cultures for the detection of bovine viral diarrhoea virus (BVDV) in field samples, Research in Veterinary Science 63: 199-203.

Wilhelmsen, C. L., Bolin, S. R., Ridpath, J. F., Cheville, N. F. and Kluge, J. P. 1990. Experimental primary postnatal bovine diarrhoea viral infections in six-month-old calves, Vet. Pathol 27: 235-243.

Brownlie, J. 1985. Clinical aspects of the bovine virus diarrhoea/mucosal disease complex in cattle, In Practice, 7: 195-202.

Malignant catarrhal fever

O’Toole, D., Li, H., Miller, D., Williams, W. R. and Crawford T. B. 1997. Chronic and recovered cases of sheep-associated malignant catarrhal fever in cattle, Vet Record 140: 519-524.

Hong, L. I., Shen, D. T., O’Toole D., Knowles, D. P., Gorham, J. R. and Crawford, T. B. (1995). Investigation of sheep-associated malignant catarrhal fever virus infection in ruminants by PCR and competitive inhibition enzyme-linked immunosorbent assay, Journal of Clinical Microbiology 33: 2048-2053.

Hong, L. I., Shen, D. T., Knowles, D. P., Gorham, J. R. and Crawford, T. B. (1994). Competitive inhibition enzyme-linked immunosorbent assay for antibody in sheep and other ruminants to a conserved epitope of malignant catarrhal fever virus, Journal of Clinical Microbiology 32: 1674-1679.

Baxter, S. I. F., Pow, I., Bridgen, A., and Reid, H. W. (1993). PCR detection of the sheep-associated agent of malignant catarrhal fever, Arch. Virol. 132: 145-159.

Appendix 1. Examples of Performance indicators for rinderpest and their application

*Refer to OIE (1989, 1998). Recommended standards for epidemiological surveillance systems for rinderpest.

and

FAO/IAEA (1994). Recommended procedures for disease and serological surveillance as part of the Global Rinderpest Eradication Programme (GREP). IAEA-TECDOC-747.

---

Appendix II. Examples of Diagnostic Indicators for assessing performance of a surveillance system

Appendix III. Examples of performance indicators

Example 1: PI 1: Performance indicator for general disease surveillance

Country X has a total of 50 districts. Four hundred and twenty reports were filed in a year for the whole country. Fifteen districts forwarded general disease reports for at least 10 months of the year but only 12 of them reported within 30 days from the end of the month being reported, while 35 districts forwarded less than 10 reports

PI 1: Number of districts forwarding general disease reporting formats within 30 days at least 10 months of the year per total number of districts.
Calculation PI 1	= 12 / 50 x 100% = 24% (target 80%)

Diagnosing why performance is low

Possible interpretation:

The low performance may be attributable to the lack of veterinary infrastructure to perform daily fieldwork, which is made more severe, by the lack of motivated staff. Additionally, the diagnostic indicators suggest the availability of trained staff and standard forms but a possible lack of resources to transfer the report from district to national level in a timely manner.

Remedial action:

Invest in veterinary infrastructure. Facilitate and provide means of form submission from districts to national offices. Enhance motivation of staff by supply of feedback.

Example 2: PI 5: Performance indicator for preliminary rinderpest diagnostic testing

Country X has 2 stomatitis-enteritis cases diagnosed definitively by the laboratory within 3 days of receipt of samples. Their total unstratified cattle population is 3,000,000. Both AGID and immunocapture ELISA are carried out in the laboratory.

Number of cases examined by rinderpest antigen, serological, immuno-histopathological and/or RNA detection techniques with preliminary results reported within 3 days of receipt of samples per 100000 heads of susceptible species.
Calculation PI 5	= 2 / 3000000 x 100000 = 0.0067	(target would be the estimated number of stomatitis-enteritis cases to be expected in the country population per 100000 heads of susceptible species)

Diagnosing why performance is low

Interpretation

Specialised and formalised system for test and laboratory data management is insufficient although laboratory know-how and tests are available.

Remedial action

Introduce systematic recording methods - manual or computerised, for storage of data and follow up.