Data Review Tool (DRT) – User’s Guide
- What Data Is Available In This Tool? When Can I See My Data?
- How to Download the DRT
- Quick Start Guide for How to Use This Tool
- General Types of Data Quality Checks in the DRT
This Data Review Tool (DRT) contains checks to assess several aspects of data quality that can help identify potential issues and inconsistencies in the data. Each data quality check occurs at the site-by-implementing mechanism (IM) level (in other words, it occurs at the level that data entry takes place).
What Data Is Available In This Tool? When Can I See My Data?
In general, this tool follows similar data viewing permissions as the DATIM pivot tables. So if your account type can see data in a pivot table, you should see that data reflected in this tool. However, there is a delay before data that is entered into DATIM appears in this tool. When you downloaded the DRT, there a date and time in the Genie app (see example below). Data entered into DATIM before the time shown in Genie will not appear in this tool yet - check back later. See the Additional Information tab of the DRT extract for more details about account permissions and the timing of when data becomes available.
How to Download the DRT
1. Login into datim.org and select the DATIM Genie application
2. Select the Data Review Tool Extract
3. Apply any filters you would like (optional) and then “Run Report”
4. Select “Export All as XLSX”
You must click Enable Editing when you first open this tool! If you do not enable editing then you will not be able to see your data.
The "Main Site IM Checks" tab shows cases where an IM reporting at a site violated one a reporting rule. Importantly, the only checks that appear in the pivot tables are those that have one or more Site/IMs that violated that check given the (a) filters selected in the DATIM Genie and (b) combination of filters/slicers that are selected. If a particular data quality check does not appear in the pivot table, it is because either 1) no site by IM combinations have violated that check or 2) any site/IM combinations that violated that check were excluded from analysis based on filters in DATIM Genie or in this tool's slicers/filters.
The numbers in the "Number of Cases Violating the Check" column on the "Main Site IM Checks" tab represent the number of instances where an IM at a site violated each data check.
There are drill-down buttons (indicated by a '+') on each row next to each check that allows the user to drill-down to the PSNU, site name, Agency and then Implementing Mechanism(s) ID that violated each check at each site. Tip: Right click on any data quality check, PSNU, Agency, or IM in the pivot table and utilize the various expand & collapse options to quickly expand/collapse the entire pivot table to the level of your choice (see above).
Additionally, if the user is interested in seeing the raw data for particular row in the pivot table, the user can do so by double clicking the cell in the "Number of Cases Violating a Check' column. This will open a new excel sheet with only the row(s) that pertain to that geographic level and check. For instance, double clicking on the cell in the image below to the left will open a new sheet with only data from the PSNU Dorne that violated the check shown below. The user can do this for any level that is shown in the pivot table.
General Types of Data Quality Checks in the DRT
This section provides general information about the four different types of checks that are included in the DRT. Detailed information about each individual check is available on the Full List of Checks tab once you download the DRT extract from Genie.
1) MER Logic Checks: These check the relationships a) within an indicator or b) between two related indicators to see if logic rules from the MER Guidance were followed.
Example: The numerator is greater than the denominator (usually you expect a denominator to be greater than or equal a numerator).
Action to take if a MER Logic Check is violated: Refer to MER Reporting Guidance for additional information for how the indicator(s) should be reported. Then check to see if the data reported is accurate or if a reporting error was made.
2) Disaggregate Completeness: These find cases where the sum of a disaggregate is not equal to the total numerator or total denominator. Ideally, partners should be reporting complete disaggregations of their totals.
Action to take: Check the accuracy of the reported disaggregated data and the reported totals. Is there additional disaggregated information that could be reported? Were there any data entry errors?
3) Checks Across Time Periods: these assess the consistency of reporting data across time periods. There are checks that show when:
- A Site/IM has targets but not results (this may indicate a reporting error or it may indicate that previously planned activities stopped at this site)
- A Site/IM has results but not targets (this may indicate a reporting error or it may indicate that a new activity began in this site)
- A Site/IM reported results last period but not this period (this may indicate a reporting error or it may indicate that activities stopped at this site)
- A Site/IM reported results this period but not last period (this may indicate a reporting error or it may indicate that a new activity began in this site)
Action to take: Check to see if this period's results are accidently missing or was reported in error. Please note that cases identified here may not be a problem – especially if there were intentional program shifts!
4) Contextual Site by IM Information: Checks in this category do not imply a data quality error! This set of checks provides contextual information about the number of sites/IMs combinations that:
- Reported results this quarter (regardless of whether they have targets)
- Have targets (regardless of whether they reported results this quarter)
- Reported both results this quarter and had targets
This information is provided as a reference. It can be useful to help prioritize data cleaning efforts - this can be done by comparing values from the Contextual Site by IM tab to the values of the data quality checks. By doing this comparison, you can see if data quality violations represent a large or small proportion of the overall data for that indicator.
Action to take: None, this information is provided as a reference.