1.1 Background and Rationale

In the context of the world-wide push to achieve HIV epidemic control, quality health information is essential to benchmark and monitor progress to achieving 95-95-95 care cascade targets. Evidence from research and practices suggests that electronic patient tracking systems (EPTS) improve the quality and use of clinical data. To this end, Mozambique has rapidly scaled-up the use of EPTS in public health clinics, with funding from the US government President’s Plan for Emergency AIDS Relief (PEPFAR). Nonetheless, the Mozambique Ministry of Health (MoH) relies on paper-based reporting systems at the health facility level, which the district and province statistical centers aggregate into an electronic DHIS2-based national health information system (SIS-MA). MoH uses this information to monitor HIV epidemic trends and allocate resources. Given the reliance on both electronic and paper-based reporting systems at the site level, concordance between the two datasets takes on high importance to ensure accurate, timely, complete, and reliable data for decision-making.

Since 2015, PEFPAR clinical/implementing partners (IPs) have conducted EPTS data quality assessments (DQA) to evaluate and improve concordance between patient health information in EPTS and paper files. In total, PEPFAR IPs offer technical assistance to 470 health facilities, which provide the majority of antiretroviral treatment (ART) to the 1.5 million patients currently on treatment in Mozambique. During an EPTS DQA, IP evaluation teams visit sites and sample patient charts, abstracting a set of variables and comparing them to data in electronic patient files. Teams calculate concordance as a measure of timeliness of data entry, accuracy of information, and completeness of both paper and electronic records. For the exact steps involved in this activity, review the project protocol. PEPFAR strategic information partner, the University of California San Francisco (UCSF) manages the national EPTS DQA dataset and produces summary reports.

This report presents findings from EPTS DQAs that PEPFAR IPs conducted from October to December 2017. In total, IPs conducted EPTS DQAs at 278 sites, reviewing data from the Q4 quarter. IPs shared site level results with UCSF via Excel for analysis. UCSF looked at global concordance between patient charts and EPTS data for select variables and presents trends by type of error: documentation error in patient charts; lack of data entry in EPTS; or error in EPTS data entry. To be considered good quality data, we are aiming for concordance of 90% or higher.

1.2 Objectives

1. Evaluate the agreement of the existing data in the Clinical Process and the data in the electronic database (SESP).
2. Evaluate the completeness of the data entered into the electronic database.
3. Improve the quality of data in SESP.

2.1 Patient Chart Selection for EPTS DQA

For the first EPTS DQA, IPs selected a representative sample of patient charts, using the guidance from Table 1. For each subsequent EPTS DQA at the same site, IPs selected a convenience sample, as described below.

2.2 Electronic Patient Tracking System DQA Indicators

Electronic patient tracking system DQA assesses the quality of variables (see table 2) in the database, compared with the information in the clinical files (primary source of the data).

Table 2: Variables assessed during the EPTS DQA activities

2.3 EPTS DQA Data Collection

Using an Excel book (not shown here), teams pull the charts identified in the patent sampling and compare the data in the charts with the data in the EPTS. Possible results are:

1. X: Errors in documentation in patient file: No data in the patient file
2. 0: Concordance: Data in the patent file and the data in the EPTS match
3. 1: Missing Data Entry: Data in the patient file has not been entered into the EPTS
4. 2: Error in Data Entry: Data in the patient file and data in the ePTS do not match
5. Error in documentation: data are in the EPTS but not in the patient file (similar to X)
   1. 3a) Possible to confirm with another primary source document
   2. 3b) Data entry error: Not possible to confirm with another primary source document
6. Not Applicable: When the data are not applicable, such as pregnancy for men

2.4 EPTS DQA Data Analysis

3.1 Health Facility Sample

IPs conducted EPTS DQA in 278 health facilities, from a total of 470¹ health facilities in Mozambique with EPTS. The majority of health facilities were assessed in Maputo province (58), followed by Zambezia (47), Gaza (32), Maputo City (27) and 26 in Inhambane province. The USAID IP for Tete, Manica, Sofala and Niassa assessed fewer than 20 health facilities per province, with teams assessing the fewest sites in Niassa (5) and Sofala (7). This is due in part to the low coverage of EPTS in these provinces. Figure 3.1 shows the interactive maps with clustered number of health facilities assessed together with the volume of patients currently on ART² by facility.

3.2 Patient Charts Sample

The number of health facilities assessed per province ranged from 5-47, as partners used both sampling methodologies to select charts for assessment. The maximum number of 47 health facilities was assessed in Zambézia province, while the minimum number assessed of 5 was assessed in Niassa province. The mean number of charts assessed was 48.9, while the median number was 50. In two provinces- Maputo City, Manica and Gaza - partners sampled more than 50 patient charts per facility, suggesting that they used the sampling approach for first time assessments (Table 3.1).

Most partners adhered to the representative sampling outlined in Table 1, while some partners assessed a larger sample of patient charts. At two health facilities in Maputo City- HG José Macamo (n=100) and CS Chamanculo - (n=150) the IP selected a sample greater than 50. In Manica province, the IP assessed a large sample at CS Inchope (n=146), while the IP for Gaza province assessed a large sample at CS Zongoene (n=100) and CS Manjangue (n=146). There were some health facilities with a sample of patient files less than 50 and out of scope to be considered as part of the first round of assessment or convenience sample for subsequent assessments. More details on sample of patient chart per health facility can be found in Table 3.2.

3.3 Global Concordance Results

From Table 3.3, results suggest that the majority of health facilities had concordance of good quality, i.e., 90% or greater of data abstracted from patient files matched the electronic information in the database. There were some health facilities clustered in Gaza and Tete province with global concordance less than the target of 90%, meaning that the information from EPTS differed from original source, on average, greater than 10%. We also see this trend in some health facilities in Inhambane and Maputo City (in one health facility), Manica (in two health facilities), Nampula and Sofala (in three health facilities) and Zambezia (in five health facilities) with concordance less than 90%, showing mismatch of data into the electronic systems when compared with patient files.

3.4 Quality Issues by Variable

We analyzed each issue in the assessment of EPTS related to the documentation error, data entry error and lack of data entry, looking specifically to the variables from the primary sources of data, used for concordance assessment by province.

3.4.1 Quality Issues related to each variable by province

We analyzed each issue identified via EPTS separately (documentation error, data entry error and lack of data entry) for each variable, by province. The patient chart is considered the primary source, as data is retrospectively entered from the chart into the EPTS database. Thus, documentation error is defined as a lack of data filled in the patient charts (missing data elements in the charts). Data entry error is when the data entered into the database differs from what is found in the patient chart. Lack of data entry is when there is data in the patient chart that is absent from the database.

We looked at issues related to quality for each variable, by province. In Niassa province (Figure 2.1), a majority of the variables had percentage of quality issues less than 10%, with only two variables (estimated date of delivery and date of last menstruation) with percentage greater than 10 for documentation error.

Cabo Delgado (Figure 2.2) had issues in five variables: Isoniazid treatment and Cotrimoxazole variables had issues related with documentation error, height and CD4 date had issues on data entry error and documentation error, and pregnancy at entrance had issue related with lack of data entry.

In Nampula province (Figure 2.3), the majority of quality issues were related to lack of data entry in most variables: height, date of last menstruation, estimated date of delivery, pregnancy on period, date of next drug pick up, date of viral load, date of viral load delivery, result of viral load, Isoniazid treatment, IMC, TB treatment, weight, Lab CD4 date and Lab CD4 result. Also, STI screening and date of next consultation had issues related to data entry error; date of viral load, date of viral load delivery, results of viral load, Lab CD4 date and Lab CD4 result had issues related with documentation error.

In Zambezia province (Figure 2.4), the majority of issues were related to documentation error on CD4 result, CD4 date, pregnancy in follow-up, pregnancy at entrance, pregnancy in ART, GAAC. TB treatment had issues in all areas of quality (data entry, documentation error and lack of data entry) while GAAC and height had issues with lack of data entry too. It’s worth highlighting that the majority of variables assessed in Zambezia had good quality of data, with most of the issues at less than 10%.

Tete province (Figure 2.5) had issues with documentation error in Lab CD4 result and date, result of viral load, date of viral load and viral load delivery, date of last menstruation and estimated date of delivery. Lack of data entry was highlighted in the majority of variables: GAAC, STI screening, last consultation date, date of viral load and delivery, TB treatment, TB screening, weight, WHO staging, height, date of next consultation, IMC, Isoniazid treatment, pregnancy on period, date of last menstruation and estimated date of delivery. It’s worth highlighting that Tete province had good quality data related to data entry with the percentage of issues below 10% in most of variables.

Results from Manica province (Figure 2.6) highlighted quality issues related to documentation error and lack of data entry for most of the variables: date of sample collection, result and date of delivery of viral load, weight, height, IMC on issues related with documentation error, and date of last menstruation, estimated date of delivery and pregnancy on period on issues related with documentation error and lack of data entry. Also, we can highlight good quality of data related to lack of data entry and data entry error, with percentage of issues on those indicators less than 10%.

Sofala province (Figure 2.7) had few variables with good quality in all areas – last consultation date, weight, WHO staging and height. The majority of issues were related to documentation error on date of viral load delivery, TB screening, isoniazid treatment, STI screening, Lab CD4 result, Lab CD4 date, TB treatment, date and ART regime, date of next drug pick up, date and result of viral load, date of last menstruation, estimated date of delivery and pregnancy on period. Some of the variables had issues with lack of data entry – date and result of viral load, estimated date of delivery, pregnancy on period and GAAC.

Inhambane province (Figure 2.8) had issues with some of the variables related to documentation error – Lab CD4 date and result, date of viral load delivery, date and result of viral load. There were also issues with lack of data entry in pregnancy on period, GAAC. Date of last menstruation and estimated date of delivery had issues in both indicators: documentation error and lack of data entry, while TB treatment had issues with data entry error and lack of data entry.

Gaza province (Figure 2.9) had good quality data entry for most variables, with standout issues regarding documentation error and lack of data entry in weight, isoniazid treatment, CD4 date and result, TB treatment. STI and TB screening, pregnancy in ART, CTX, GAAC, height, pregnancy at entrance and pregnancy in follow-up had issues related to lack of data entry.

Maputo province (Figure 2.10) highlighted good quality in most of the variables with percentage of issues below 10%. Two variables had issues related with data entry error – IMC and TB treatment. Three variables had issues related to documentation error – date and result of viral load and date of viral load delivery.

The majority of issues found in Maputo City (Figure 2.11) were related to documentation error – lab CD4 date and result, IMC, date and result of viral load, date of viral load delivery, isoniazid treatment, WHO staging and pregnancy on period. Three variables had issues related to both documentation error and lack of data entry - estimated date of delivery, TB treatment and date of last menstruation - while GAAC highlighted issues related to lack of data entry.