Within any program, the goal of continuous quality improvement is to examine the content and quality of ECMHC services within the program, to determine if the MHC services are achieving the expected outcomes of the consultation services, and to make improvements to the program based on the findings. While counting such things as number of children served by the consultant, number of hours spent coaching teachers in classrooms, or number of screenings/referrals made is important, and relatively straightforward to document and monitor, these types of service or process indicators provide little information about whether the consultation service are effective.

There is a growing body of evidence supporting the efficacy of ECMHC in producing positive outcomes for children, families, and ECE providers and programs (Brennan at al., 2008, Perry et al., 2010). More specifically, the research suggests that consultation is effective in:

• Reducing problematic behavior in young children;
• Increasing young children’s social skills;
• Decreasing expulsions from ECE settings;
• Building behavior and classroom management skills of ECE providers;
• Enhancing parent ability to manage problem behaviors; and
• Reducing turnover in ECE staff.

Below we outline several outcome areas for mental health consultation that may be important to monitor as part of your program’s ongoing quality improvement efforts, including: child outcomes, family outcomes, staff/classroom outcomes, and program outcomes. The outcomes that you determine to be most important for your program to monitor should be based on your strategic plan and the specific goals of consultation services.

It is important to note that the approach that we are describing here for outcome-based continuous program improvement does not strive to achieve a high level of scientific rigor that would involve the use of comparison groups or other strategies to determine whether the mental health services and consultation “caused” specific outcomes. Instead, this strategy relies on collecting and using various types of information to determine whether there is evidence that children, families, staff and programs show outcomes consistent with those that the mental health approach and services are intended to achieve, and to inform ongoing program practice.

Example outcomes for quality improvement efforts

Example outcomes for quality improvement efforts are described below. The Early Childhood Mental Health Consultation Evaluation Toolkit is an excellent source of tools that measure each of these four domains.

Staff/Classroom Outcomes

Because consultation is fundamentally rooted in the quality of consultant-staff relationships, and designed to build teacher capacity for working with mental health and social-emotional concerns, collecting data from staff as part of quality improvement efforts is extremely important. Quality improvement measures for determining the degree to which mental health consultation results in improved teacher or staff outcomes might examine:

• staff’s satisfaction with the MHC,
• the quality of the staff and consultant relationship,
• reduced staff stress,
• improved emotional climate of the classroom, and
• improved teacher’s classroom management skills.

See an example of a Mental Health Services Satisfaction Survey for Staff that measures staff-consultant relationships and perceived value of consultation. A brief staff survey conducted at identified points in the program year would provide useful data for continuous quality improvement.

Observational measures of classroom environment and the quality of teacher-child interactions are increasingly used in Head Start/Early Head Start programs:

• The Classroom Assessment Scoring System (CLASS) ) is being widely used in Head Start programs to assess classroom quality;
• The Pyramid Infant Toddler Observation Scale (TPITOS) is being used observe adult behaviors for supporting infants and toddlers; and
• The Teaching Pyramid Observation Tool (TPOT) is used to assess the quality of teacher / child interactions in preschool classrooms.

Such tools can serve multiple purposes, by providing structured and valid tools for mental health consultants to use for conducting classroom observations that can be used to inform their work with staff, and to monitor the ways that classroom environments change over time. Depending on your program, you could ask consultants to complete follow-up observations, or have the consultant’s administrative supervisor or another staff trained to utilize the tool take responsibility for follow-up assessments to monitor outcomes. Either way, these tools can provide useful information for continuous quality improvement.

Child Outcomes

Most ECMHC strategies ultimately hope to improve children’s social-emotional and behavioral outcomes. Quality improvement efforts related to working with the consultant might include monitoring children experiencing:

• fewer internalizing and externalizing behaviors,
• improved pro-social behaviors, or
• fewer days absent from Head Start.

Programs might also want to track such things are whether children who are identified as needing outside supports or services for mental health-related needs are successfully linked to these services, and that parents feel that their needs in these areas are being successfully addressed.

Family Outcomes

To determine if mental health consultation services are helping families, programs might examine:

• family satisfaction with the MHC,
• the quality of the family’s relationship with the MHC,
• the degree to which families have reduced levels of parenting stress, or
• increased parenting confidence after receiving consultation services.

Brief surveys that identify what parents would like to learn from the consultant, and whether the consultation met their expectations could also be useful in knowing if services meet families’ needs.

For an example mental health service satisfaction survey for parents, see Mental Health Services Satisfaction Survey for Parents.

Program Outcomes

Finally, quality improvement measures might examine changes that happen at the overall program level after receiving consultation services. Program level measures might include collecting information on things like:

• the number of times children are removed from classrooms for safety reasons,
• the amount of time teachers spend attending to behavioral concerns in a period of time,
• the frequency and number of successful mental health referrals for adults and children, and
• increased staff retention.