Attachment 12: Overview of DFC Me system

Attachment 12: Overview of DFC Me system for data collection, reporting and monitoring/compliance

The DFC Me system has been designed expressly for the use of coalition members, project officers, the evaluation team and program administrators in reporting, monitoring progress and compliance, and managing and evaluating data related to the DFC program. The system has tabs for a Dashboard, Reporting, Sector Representatives, viewing E-Blast communications from the evaluation team and program administrators, a Learning Center featuring Success Stories, webinars, tip sheets and other resources, and Calendar of Events.

The DFC Me system is secure; individuals must login with username and password, and complete a basic math question to show they are not a “robot.”

Each coalition has up to three key user roles for individuals who may enter data for each funded coalition: Program Director, Program Coordinator, and Grant Award Recipient Representative. Staff members in these roles are defined in the DFC Request for Applications (RFA) and any changes to them must be confirmed and approved by assigned SAMHSA project officer.

Required data captured in the DFC Me system includes: Progress Report, Sustainability Plan, Core Measures, Coalition Classification Tool, and Sector Representatives. Depending on the type of data required, coalitions may either be asked to enter data directly into the system or to upload supporting documentation. Only aggregated data is reported, i.e., no individual level data or individual identifiers can be entered other than names and contacts of individual coalition staff users and names of sector representatives for each coalition. Individual level data is not collected for any sensitive data such as core measures.

Reporting and Submissions

Coalitions provide their data in the Reporting and Sector Representatives tabs.

Surveys: Coalitions must submit core measures surveys for review by the DFC National Evaluation Team to ensure the items meet the standards set by the DFC-approved core measures. Surveys are submitted in the DFC Me system by uploading documentation that is then accessed and reviewed by evaluation staff. The status of this review is reflected in the coalition’s Reporting tab.

Core Measures: Coalitions can only submit data for survey items that have been approved. Core Measures are also submitted through the Reporting tab. Data is submitted by manually entering aggregated values for each Core Measure and substance for each grade level available. Individual data cannot be entered. The data can be saved for completion in a later session. When the Core Measures data is complete, the user can mark it as Ready for Submission; it can then be attached to a Progress Report for submission.

Progress Report and Coalition Information: These elements are updated through the Reporting tab. Some elements of the Progress Report are pre-populated to reduce entry burden on coalition staff.

If the coalition has Core Measures that have been marked as “Ready for Submission,” the user can attach them to this Progress Report in the Community & Population Level Outcomes section. Coalitions also submit data on needs, budget, membership, processes, planning, strategy implementation in their Progress Reports.

Success Stories: Coalitions may submit Success Stories to share with other coalitions, through the Learning Center. This is optional.

Monitoring/Compliance

Coalition users can view the status of their own current compliance for required elements. The Grant Requirement Status section gives a quick overview of the items the coalition is required to submit. It updates in real time to reflect the beginning of each reporting period, and each time the coalition submits materials that complete a requirement. The compliance window is re-set biannually (February and August).

Users can also view their status in more detail over the last several reporting periods in a Report Card, which shows status (Requirement Met, Requirement Not Met, etc.) for the separate elements of Progress Report, Sustainability Plan, Core Measures, Coalition Classification Tool and Sector Representatives. Sample screenshots are shown below:

Government Project Officers (GPO) can also see a dashboard showing the status of their assigned coalitions, and may view individual coalition report cards.

The compliance feature of DFC Me consists of two tools: (1) Program Overview and (2) My Portfolio Status. The Program Overview and My Portfolio tools allow the GPO to easily look up information such as how many coalitions have not submitted the most recent progress report, how many progress reports the GPO still needs to review, who is missing a sustainability plan, and more. They may also look at a coalition’s Report Card to better understand the status of requirements for that particular grant.

The Program Overview section provides GPOs with a summary of compliance for evaluation-related grant requirements of all of their assigned coalitions.

Government Project Officers can also see the status of their assigned portfolio of coalitions through the Portfolio Status page, which shows information about compliance for both group and individual levels. This information includes requirements that are met or not met by their assigned coalitions. The Portfolio at a Glance table at the top of the page shows the Status of Compliance Items graph with the six requirements tracked by DFC Me and the count of coalitions in each compliance status. Portfolio at a Glance also contains an Outstanding Progress Reports graph, which provides a quick summary of how many of the GPO’s assigned coalitions (if any) have not met the progress report requirement for the four most recent reporting periods. This graph is updated throughout the year as statuses for these coalitions change.

Searching for and Downloading Compliance Items

GPOs may also filter their portfolio for compliance items. For example, they can easily search for which of their FY 2017 coalitions have not met the Core Measures requirement by selecting “2017” for Fiscal Year and “Requirement Not Met” for Core Measures Status.

GPOs can download this information into an Excel file for sorting and follow-up. To see more detailed information about a specific coalition, GPOs can scroll to the bottom of the GPO Portfolio Summary to see a table with all of their assigned coalitions listed. They can either use the filters to find the coalition of interest or scroll through the table to find them. For a particular coalition of interest, the GPO can click View to open the Coalition Report Card in a new tab:

The Coalition Report Card page shows details for Coalition Information as well as the grant requirements tracked by DFC Me: Progress Report, Sustainability Plan, Core Measures, the Coalition Classification Tool (CCT), and Sector Representatives and date the information was last updated. An example is shown below for the Sector Representatives component.

Note: For some requirements, like the sustainability plan, the system makes assumptions. In this case, if a coalition is in Year 3 or Year 7 and did not submit a sustainability plan with their progress report, the Report Card will show a red X. However, if a coalition is not in either of those years, a gray bar will appear under “Status” next to the required year, indicating that this requirement is “not applicable” for the coalition.

The GPO can see an explanation for when the system marks a coalition as having completed a particular requirement, with help text, if they hover over the blue circle containing a question mark, located next to each main category, to see an associated tool tip.

The system updates in real time and follows the reporting period timeline. For example, the compliance status of Progress Reports will reset on the first day that coalitions can begin to submit their new report (typically February 1 and August 1). Some items without submission deadlines, like Survey Review, will reset once the requirement is met.