State of South Carolina
Pilot Program: Year 2

Summary

The South Carolina Department of Education recently completed year two of a three-year pilot program to address the needs of their K-12 students with Autism Spectrum Disorder (ASD).  The primary focus of the pilot was to identify ways these students could successfully improve their human-to-human interactions and executive functioning. Specifically, the pilot targeted social and emotional understanding, communication and self-regulation, and how best to achieve generalization of those behaviors with peers and family members.

The robots4autism program (from RoboKind LLC) was selected by the South Carolina Department of Education as the program that would best fit the objectives of the pilot.  robots4autism was created specifically to help children with ASD in the development of social and emotional understanding, conversational dynamics and self-regulation (calm down).

The first phase of the pilot included 15 South Carolina school districts with a variety of sizes, geography, and demographics (i.e., ethnicity, socioeconomic status, etc.).  A total of 47 educators facilitated the program with 305 students from October 2017 through June 2018. The facilitators included special education teachers, specialists, and paraprofessionals.

The second phase of the pilot included the same 15 South Carolina school districts plus one additional school district.  A total of 79 educators facilitated over 11,000 lessons to 314 students including 77 students from the first phase.

Objectives

The objective of the pilot was to achieve generalizing or transference to human-to-human interaction of targeted behaviors in 1) social and emotional understanding, 2) communicational dynamics, and 3) self-regulation or executive functioning.  To measure these outcomes, both quantitative and qualitative data were collected and analyzed.

The quantitative data consisted of 1) lessons attempted and completed, 2) modules attempted and completed, 3) accuracy of response, and 4) time to respond.  Lesson mastery was achieved when a student gave no more than one incorrect response during an individual lesson.  Correlating accuracy with time to respond demonstrates a generalizing relationship and was deemed an important consideration. 

Process

A robust process guided the identification and selection of pilot districts.  Program criteria, including expectations for commitment to program fidelity, progress monitoring, training, etc., were provided to all South Carolina school districts.  Interested districts were asked to submit an application in order to participate.  Applications were screened against criteria and selected districts were notified of their acceptance into the pilot program.  District staff participated in training facilitated by the State Department of Education.  Weekly, monthly and quarterly monitoring was conducted during the pilot period and included data reviews and on-site visits.

Results

The overall effect of the second-year pilot were similar to year one.  The results show that the second-year pilot met or exceeded the objectives of the program.  The collected data confirmed the successful impact on the two focus areas with respect to utilization, lessons attempted and completed, and the level of mastery achieved by participating students. Though the total lessons completed were slightly lower in year two, as was initial mastery achieved, students demonstrated consistency with sustaining mastery with less repetition of lessons attributed directly to an increase in the use of structured activities and naturalistic practice.

Returning students (77) from year one accounted for over 40% of the lessons attempted and completed.  Specifically, fifty three percent (53%) of returning students demonstrated mastery resulting in observable generalizing, transference to human-to-human interaction.  As anticipated, the initial and sustained mastery attainment levels were overall lower than in year one.  However, this is attributed to students engaged in more complex and difficult lessons. 

The observational and subjective analysis affirmed that students were, in fact, generalizing at a consistent rate across lessons. To support consistency, staff and students utilized repetition of lessons to ensure and validate targeted behavior mastery. The data showed that approximately 30% of students demonstrating mastery achieved that level by repeating a particular lesson multiple times, a featured benefit of the robots4autism program.

Consistent with the primary focus of the pilot, similar lesson distribution was achieved in year two. Forty-four percent (44%) of the total lessons completed were from the Social and Emotional Modules and 25% of total lessons completed were from the Conversational Dynamics Module. Consistent with year one, the number of lessons in the area of self-regulation was 13%. Similarly, self-regulation lessons were repeated significantly lower than those in either the social and emotional modules or conversational dynamics modules. These findings are consistent with other studies showing that students achieve mastery and quickly generalize these self-regulation behaviors without having to repeat calm down lessons.