DATIM

DATIM Data Import and Exchange Resources

Ping Huang -

  •  

    Purpose

    This online guide provides resources and describes procedures necessary for importing and submitting data into DATIM.

    DATIM Concepts

    Data intended for import into DATIM must satisfy strict requirements with respect to the format of the data as well the relationship of the data to the current metadata within of the system.

    • Current Code List: The code lists provides the identifier to be used in the data exchange files i.e. the Codes and UIDs code for MER (Results and Targets/COP), SIMS data elements, Disaggregates/Category Option Combos, Mechanisms and Organization Unit. In addition they contain common names and codes for data elements (Indicators & Disaggregates) and data sets (forms).
    * These code lists are open to the public i.e. you do not need a DATIM log in to access them
    • Data Exchange Guidance: This section provides instructions and useful resources necessary for performing MER and SIMS data exchange with DATIM.
    • Other Resources: Other useful DATIM and DHIS2 links and resources are provided in this section.

    Metadata dimensions

    Data must be valid with respect to the destination sites, periods, indicators, disaggregation and mechanisms, therefore all data values must be associated with four metadata dimensions which describe different aspects of the data namely:

    • Where - Organization Unit: This dimension describes the location of the data i.e. where the data is captured.
      • Example: Clinical Facility, Community Site, or OU Level
    • What - Data element (Indicator) + Disaggregation: This dimension describes the phenomena which the data value is attached to, such as the Number of HTC tests for Females 1-4. In this case the data element has a disaggregation of Female 1-4. Data elements in DATIM typically have many different disaggregations with respect to age and gender. In addition to that, different classes of data elements such as Targets and Results and DSD and TA exist.
    • When - Period: This dimension describes the time period of the data being reported.
      • Example: January through March 2016 or April through June 2016
    • Who - Funding mechanism: Also known as the “attribute option combination” this is an extra custom dimension in DATIM, which describes which implementing mechanism the data is being reported for and refers to the FACTS Info funding mechanism ID.

    Back to the top.

    Format

    Data intended for import into DATIM must also satisfy strict requirements with respect to the format i.e. valid types of data (integers, options, etc.)

    The data formats are generic DHIS2 formats which have been mapped to PEPFAR specific concepts in DATIM. The elements in the formats are described in the following.

    DHIS2
    concept
    PEPFAR
    concept
    Description Format
    Data element Indicator Data item being captured. In PEPFAR terms referred to as indicator. Example is "10.1 Post-Violence Care Score". Data elements can be reported either using Codes or UIDs (refer to Current Code List contained in this online guide for specific Codes and UIDs)
    Period Period The time period for which the data is captured.
    DATIM uses the DHIS2 internal date format for designating reporting periods. All periods in DATIM data exchange files must provide a valid DHIS2 period identifier. In particular, DATIM uses Financial Year October for COP targets, PEPFAR Quarters for MER Results, and Days for SIMS reporting.
    • Days: use yyyyMMdd e.g. 20160101 for January 01, 2016
    • PEPFAR Quarters: use yyyyQn e.g. 2016Q1 for PEPFAR FY16Q2 i.e.  January - March 2016
    • Financial Year October: use yyyyOct e.g.2016Oct for October 2016 - September 2017
    Org unit Facility / Site / Community / OU Organizational unit for the data. Examples are site, district and country. Not to be confused with PEPFAR OU. Would typically be the site where the data is captured. Facility, Site, Community and OU codes and UIDs are available in the Current Code List contained in this online guide.
    Category option combo Disaggregation The disaggregation of an indicator/data element. Example is "Female, Under 5". For the Disaggregation/ Category option combo Codes or UIDs refer to the Current Code List contained in this online guide for specific Codes and UIDs)
    Attribute option combo Funding mechanism Funding mechanism for which the data applies. Example is "25147". Refers to the FACTS info mechanism ID. The appropriate funding mechanism Code or UID should be reported (refer to Current Code List contained in this online guide for specific Codes and UIDs)
    Value Data value This is the data value being reported. An example data value is "45".
    Comment Data comment This is a comment associated with the data value. An example comment is "Needs follow-up".

    Back to the top.

    Current Code List

    Data Elements *

    * These code lists are open to the public i.e. you do not need a DATIM log in to access them
    Year/
    Version
    Type Data Set HTML JSON CSV XML
    COP16
    (FY17Q4)
    Results Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    Narratives (IM) HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Host Country Results: Narratives (USG) HTML JSON CSV XML
    Host Country Results: Operating Unit Level (USG) HTML JSON CSV XML
    Host Country Results: COP Prioritization SNU (USG) HTML JSON CSV XML
    COP16
    (FY17Q2, FY17Q3)
    Results Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    Narratives (IM) HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Host Country Results: Narratives (USG) HTML JSON CSV XML
    COP16
    (FY17Q1)
    Results Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    Narratives (IM) HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Host Country Results: Narratives (USG) HTML JSON CSV XML
    Host Country Results: Operating Unit Level (USG) HTML JSON CSV XML
    Host Country Results: COP Prioritization SNU (USG) HTML JSON CSV XML
    COP17
    (FY18)
    Target Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Narratives (IM) Code List HTML JSON CSV XML
    Host Country COP Prioritization SNU (USG) HTML JSON CSV XML
    Operating Unit Level (IM) Code List HTML JSON CSV XML
    Host Country Operating Unit Level (USG) Code HTML JSON CSV XML
    Planning Attributes: COP Prioritization National HTML JSON CSV XML
    Planning Attributes: COP Prioritization SNU HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    SIMS 3.0* SIMS Facility Based Code List HTML  JSON CSV XML  
    Community Based Code List HTML JSON   CSV XML  
    Above Site Based Code List HTML  JSON  CSV XML 
    SIMS v2 Option Sets HTML JSON CSV XML
      Tiered Site Support Tiered Site Support Data Elements HTML JSON CSV XML
    Tiered Site Support Option Set List HTML JSON CSV XML

    Back to the top.

    Mechanism *

    * These code lists are open to the public i.e. you do not need a DATIM log in to access them
    Concepts HTML JSON CSV XML
    Mechanism Attribute Combo Option UIDs HTML JSON CSV XML

    Back to the top.

    Data Exchange Guidance

    Date Type Document Description
    November 23, 2016 SIMS
    SIMS 2.0 Data Exchange Exercise Guidance_V2.pdf
    (700 KB)

    Agencies are required to submit SIMS data to S/GAC via the DATIM system on a quarterly basis. This guide serves as a resource for participating agencies to facilitate the process of submitting their SIMS 2.0 data to S/GAC for import into DATIM. Data intended for import into DATIM must satisfy strict requirements with respect to the format of the data as well the relationship of the data to the current metadata of the system. Data must be valid with respect to the organization units, data elements, and mechanisms.

    The purpose of this exercise is for participating agencies (CDC, DoD, HRSA, PC, and USAID) to verify and validate the script developed by the DATIM team to correct records containing the same OU/IM/Period combination. This will enforce adherence to the org unit/mechanism/period uniqueness constraint required by DATIM.
    Note: The script does not alter the assessment date, the integrity of which is important for subsequent analyses.

    April 18, 2017 SIMS
    SIMS 2 0 Data Import Reference Guide_v6.pdf

    The attached SIMS 2.0 Data Import reference guide serves as a resource outlining the steps involved with importing SIMS 2.0 data into DATIM. Reference this guide for all the requirements before Import.

    August 31, 2017 SIMS

    SIMS 2.0 Validation and Import Process guide serves as a resource outlining the steps to check the import files for DATIM system rules.

    December 21, 2016 SIMS
    PEPFAR Data Exchange Implementer Community_SIMS Data Exchange Exercise_v6.3.docx (1 MB)

    For Agencies looking to import SIMS data into DATIM, please download and complete the attached Data Exchange exercise.

    NOTE: Please ensure you are using the latest version, 6.3, for your data exchange exercises.*  

    July 28, 2016 MER

    DATIM for Operating Units (DATIM4U) is a standalone version of the Global DATIM web-based application that can be implemented exclusively for your PEPFAR Country Team (OU) and Implementing Partners (IPs). With DATIM4U, OUs can collect MER Results on a monthly basis and submit the quarterly aggregations to DATIM Global at the press of a button. Access this Self-Assessment guide to determine if DATIM4U is right for you  

    *NOTE: Downloadable resources last updated on July 25. Please reference the latest documentation posted here.  

    September 29, 2017 MER
    Guide_DataImport_Exchange_Last_Updated_September_2017.pdf (600 KB)

    This document outlines steps involved in importing data into DATIM and steps towards eventually establishing a data exchange between DATIM and another system.

    If your team is unable to meet the requirements for constructing files described in this resource, your team is probably not staffed with skills sufficient to recommend you as a candidate for importing data into DATIM.

    This updated data import reference guide also includes a section (2.2.7) on data cleaning via data import.

    ***NEW*** Listen to and watch an overview about Data Import and Exchange from the 2016 DATIM Field Training here  

    November 16, 2016 MER
    DATIM Data Import Hands-on Activity - DATIM Training 2016

    Follow along with Jason as he walks you through the data import & exchange steps and processes. If you've missed it, or are new to Data Import / Exchange, be sure to watch Jason's overview first.

    November 16, 2016 MER
    DATIM Data Import Overview - DATIM Training 2016

    At a high-level, Jason Pickering answers your questions surrounding the mystical Data Import. Listen and watch as he explains the requirements, guidance, policies, processes, and procedures, and shares available resources to assist in successful data import and exchange!

    April 01, 2016 MER
    DataSetAssignments_April_1_2016.xlsx (10 KB)

    Dataset assignments by country-, community-, and facility-level by OU/country. These dataset assignments are subject to change. Please note the date at which the file was last updated: April 1, 2016

    March 07, 2016 SIMS
    SIMS Planner Reports

    This page contains the SIMS planner reports for all operating units. One can retrieve the reports as CSV files or HTML pages.


    Back to the top.

  • Archives (old code lists) *

    * These code lists are open to the public i.e. you do not need a DATIM log in to access them
    Year/
    Version
    Type Data Set HTML JSON CSV XML
    COP16
    (FY17)
    Target Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Narratives (IM) Code List HTML JSON CSV XML
    Narratives (USG) Code List HTML JSON CSV XML
    Operating Unit Level (IM) Code List HTML JSON CSV XML
    Operating Unit Level (USG) Code HTML JSON CSV XML
    COP15
    (FY16)
    Target Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Narratives (IM) Code List HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Narratives (USG) HTML JSON CSV XML
    Operating Unit Level (USG) HTML JSON CSV XML
    COP Prioritization SNU (USG) HTML JSON CSV XML
    COP15
    (FY16 Q4)
    Results Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    Narratives (IM) HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Host Country Results: Narratives (USG) HTML JSON CSV XML
    Operating Unit Level (USG) HTML JSON CSV XML
    COP Prioritization SNU (USG) HTML JSON CSV XML
    COP15
    (FY16 Q1Q2Q3)
    Results Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Facility Based - DoD ONLY Code List HTML JSON CSV XML
    Community Based - DoD ONLY Code List HTML JSON CSV XML
    Medical Store HTML JSON CSV XML
    Narratives (IM) HTML JSON CSV XML
    Operating Unit Level (IM) HTML JSON CSV XML
    Host Country Results: Narratives (USG) HTML JSON CSV XML
    Host Country Results: Operating Unit Level (USG) HTML JSON CSV XML
    SIMS 2.0* SIMS Facility Based Code List HTML JSON CSV XML
    Community Based Code List HTML JSON CSV XML
    Above Site Based Code List HTML JSON CSV XML
    SIMS v2 Option Sets HTML JSON CSV XML

    Back to top

  • Other Resources

    DATIM instances

    Multiple instances of DATIM with various purposes are deployed.

    Data exchange formats

    DATIM supports three formats for data exchange:

    • CSV (Comma separated values)
    • XML (DXF)
    • JSON
    CSV

    The CSV format is documented in general here. A sample CSV data file looks like this:

      "data_element","period","org_unit","category_option_combo", /
      "attribute_option_combo","value","stored_by","last_updated","comment"
      "deCode","201202","ouCode",,"aocID","1",,,"comment1"
      "deCode","201202","ouCode",,"aocID","2",,,"comment2"
      "deCode","201202","ouCode",,"aocID","3",,,"comment3"
    XML (DXF)

    The XML (DXF or DHIS eXchange Format) is documented in general here. A sample DXF data file looks like this:

       <dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" idScheme="code" 
    period="peCode" orgUnit="ouCode" attributeOptionCombo="aocCode">
    <dataValue dataElement="deCode" value="1" comment="com1"/>
    <dataValue dataElement="deCode" value="2" comment="com2"/>
    <dataValue dataElement="deCode" value="3" comment="com3"/>
    </dataValueSet>

    Note that the "period" and "orgUnit" elements can be defined on the individual "dataValue" elements instead of on the outer "dataValueSet" element if you would like to submit data for multiple periods and organisation units / sites.

    JSON

    The JSON format is documented in general here. A sample JSON data file looks like this:

          {
              "idScheme": "code",
              "period": "peCode",
              "orgUnit": "ouCode",
              "attributeOptionCombo", "aocCode",
              "dataValues": [
                { "dataElement": "deCode", "value": "1", "comment": "com1" },
                { "dataElement": "deCode", "value": "2", "comment": "com2" },
                { "dataElement": "deCode", "value": "3", "comment": "com3" }
               ]
          }        

    Back to the top.

    Reporting periods

    When exporting data from DATIM you might want to retrieve data according to the SAPR and APR reporting periods. S/APR refers to Semi/Annual Performance Review.

    SAPR runs from October to March. APR runs from October to September (the following year).

    An example analytics URL snippet for the SAPR reporting period from Oct 2015 to March 2016 looks like this:

     api/analytics?pe=2015AprilS2

    An example analytics URL snippet for the APR reporting period from Oct 2015 to Sept 2016 looks like this:

     api/analytics?pe=2015Oct

    The time and period format of the DATIM Web API is documented here.

    Back to top.

    Data import process

    Please refer to the DATIM Data Import/Exchange Reference Guide for the process of testing and submitting data import files:

    https://datim.zendesk.com/hc/en-us/articles/205981785-Data-Import-Exchange-Reference-Guide

    Note: use of the DATIM Web API for import is not currently allowed.

    DATIM Web API technical notes

    In order to submit data electronically through the Web API using XML (DXF) you should interact with the “dataValueSets” Web API resource. At the DATIM test instance this Web API resource is located at:

    https://test.datim.org/api/dataValueSets

    If you are testing against another DATIM instance you need to replace the base part of the URL. One example could be:

    https://dev.datim.org/api/dataValueSets

    The DATIM Web API uses basic authentication. The XML data file or stream must be assembled according to the format described above. An example using the curl tool which assumes the XML file is saved as data.xml looks like this:

    
    curl -d @data.xml "https://test.datim.org/api/dataValueSets" -H "Content-Type:application/xml" -u user:pword -v
    

    You can find documentation on how to send and read data values through the dataValueSets resource here.

    Support

    For questions or issues related to data exchange please submit a help desk ticket at the help desk.

     

    Back to top.

  • Org Units (these codes are not public; a valid DATIM log in with appropriate access rights is needed to access them)

    Org unit / IPSL HTML CSV
    Angola HTML CSV
    Asia Regional Program HTML CSV
    Botswana HTML CSV
    Burma HTML CSV
    Burundi HTML CSV
    Cambodia HTML CSV
    Cameroon HTML CSV
    Caribbean Region HTML CSV
    Central America Region HTML CSV
    Central Asia Region HTML CSV
    Cote d'Ivoire HTML CSV
    Democratic Republic of the Congo HTML CSV
    Dominican Republic HTML CSV
    Ethiopia HTML CSV
    Ghana HTML CSV
    Guyana HTML CSV
    Haiti HTML CSV
    India HTML CSV
    Indonesia HTML CSV
    Kenya HTML CSV
    Lesotho HTML CSV
    Malawi HTML CSV
    Mozambique HTML CSV
    Namibia HTML CSV
    Nigeria HTML CSV
    Papua New Guinea HTML CSV
    Rwanda HTML CSV
    South Africa HTML CSV
    South Sudan HTML CSV
    Swaziland HTML CSV
    Tanzania HTML CSV
    Uganda HTML CSV
    Ukraine HTML CSV
    Vietnam HTML CSV
    Zambia HTML CSV
    Zimbabwe HTML CSV
    Countries / OUs HTML CSV

 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk