(2) The purpose of the results-oriented accountability program is to monitor and measure the use of resources, the quality and amount of services provided, and child and family outcomes. The program includes data analysis, research review, and evaluation. The program shall produce an assessment of individual entities’ performance, as well as the performance of groups of entities working together on a local, judicial circuit, regional, and statewide basis to provide an integrated system of care. Data analyzed and communicated through the accountability program shall inform the department’s development and maintenance of an inclusive, interactive, and evidence-supported program of quality improvement which promotes individual skill building as well as organizational learning. The department may use data generated by the program regarding performance drivers, process improvements, short-term and long-term outcomes, and quality improvement efforts to determine contract compliance and as the basis for payment of performance incentives if funds for such payments are made available through the General Appropriations Act. The information compiled and utilized in the accountability program must incorporate, at a minimum:(a) Valid and reliable outcome measures for each of the goals specified in this subsection. The outcome data set must consist of a limited number of understandable measures using available data to quantify outcomes as children move through the system of care. Such measures may aggregate multiple variables that affect the overall achievement of the outcome goals. Valid and reliable measures must be based on adequate sample sizes, be gathered over suitable time periods, and reflect authentic rather than spurious results, and may not be susceptible to manipulation.
(b) Regular and periodic monitoring activities that track the identified outcome measures on a statewide, regional, and provider-specific basis. Monitoring reports must identify trends and chart progress toward achievement of the goals specified in this subsection. The accountability program may not rank or compare performance among community-based care regions unless adequate and specific adjustments are adopted which account for the diversity in regions’ demographics, resources, and other relevant characteristics. The requirements of the monitoring program may be incorporated into the department’s quality assurance and contract management programs.
(c) An analytical framework that builds on the results of the outcomes monitoring procedures and assesses the statistical validity of observed associations between child welfare interventions and the measured outcomes. The analysis must use quantitative methods to adjust for variations in demographic or other conditions. The analysis must include longitudinal studies to evaluate longer term outcomes, such as continued safety, family permanence, and transition to self-sufficiency. The analysis may also include qualitative research methods to provide insight into statistical patterns.
(d) A program of research review to identify interventions that are supported by evidence as causally linked to improved outcomes.
(e) An ongoing process of evaluation to determine the efficacy and effectiveness of various interventions. Efficacy evaluation is intended to determine the validity of a causal relationship between an intervention and an outcome. Effectiveness evaluation is intended to determine the extent to which the results can be generalized.
(f) Procedures for making the results of the accountability program transparent for all parties involved in the child welfare system as well as policymakers and the public, which shall be updated at least quarterly and published on the department’s website in a manner that allows custom searches of the performance data. The presentation of the data shall provide a comprehensible, visual report card for the state and each community-based care region, indicating the current status of the outcomes relative to each goal and trends in that status over time. The presentation shall identify and report outcome measures that assess the performance of the department, the community-based care lead agencies, and their subcontractors working together to provide an integrated system of care.
(g) An annual performance report that is provided to interested parties including the dependency judge or judges in the community-based care service area. The report shall be submitted to the Governor, the President of the Senate, and the Speaker of the House of Representatives by November 15 of each year.
