Birth to Three Early Intervention System As Part of the Interagency Resource Management Committee Early Intervention Outcome Evaluation Projects Child Outcome Evaluation Report 2005 January 2006 Michael Gamel-McCormick, Ph.D. Denise Hartranft, M.Ed. Deborah Amsden, M.S. Center for Disabilities Studies 166 Graham Hall College of Human Services, Education, and Public Policy University of Delaware Newark, DE 19716 (302) 831-6974 (voice) (302) 831-4690 (FAX) (302) 831-4689 (TTD) http://www.udel.edu/chep/cds/ (Web Site) About the Center for Disabilities Studies The Center for Disabilities Studies at the University of Delaware is one of the 61 university affiliated program Centers for Excellence in Developmental Disability Research Education and Service (UCEDD) in the United States. The Center was established in 1992 and works in conjunction with individuals with disabilities to better their lives. The Center staff and affiliated faculty teach both pre-service and in-service courses for teachers, social service workers, and other service providers working with individuals with disabilities and their families. The Center operates state-of-the-art programs and assists both public and private organizations in adopting the procedures developed to operate those programs. Center staff and affiliated faculty also serve on state and national policy boards and commissions that address housing, transportation, education, advocacy, child care, health care, and other service areas. Center staff also conducts evaluations of programs serving individuals with disabilities and assists in policy development at both the local and state levels. The Center for Disabilities Studies is located in 166 Graham Hall at the University of Delaware in Newark. The Director of the Center is Dr. Michael Gamel-McCormick. About the Interagency Resource Management Committee The Interagency Resource Management Committee (IRMC) is a Delaware state level governmental committee that includes the Secretaries of Education, Health and Social Services and Services for Children, Youth and Their Families as well as the state Budget Director and Controller General. The Committee makes both policy and budgetary decisions for three major early intervention programs: the Birth to Three Early Intervention System of Part C of the Individuals with Disabilities Education Act; the state Early Childhood Assistance Programs, programs for four-year-olds and their families; and the Preschool Disabilities Program, programs for three- and four-year-olds with mild disabilities and speech and language delays. The Committee also oversees a statewide data management system for child and family support services. The Chair of the IRMC is Ms. Valerie Woodruff, Secretary of Education. The IRMC Coordinators during this project were Rhonda Tsoi-A-Fatt and Janet Carter. This document was prepared with the support of the Delaware Public Assistantship Program through the College of Human Services, Education and Public Policy. The Public Service Assistantship Program is designed to provide both graduate and undergraduate students with experiences that will aid the citizens of the state of Delaware. We greatly appreciate the funding provided to support this program that allows for work such as this to continue. Acknowledgements The Child Outcomes Evaluation has involved many people who have given of their time and energy to support this evaluation process. First and foremost, thank you to the parents who allowed their children to participate in this process over several years. They gave of their time and were willing to coordinate schedules so that the children could be observed to gather the information that you see reported here. Thanks go to the Ongoing Program Evaluation Committee of the Birth to Three Early Intervention System who had the vision to do this evaluation of the outcomes for children as a result of early intervention services. This project has taken the collaborative efforts of many staff members in the Birth to Three Early Intervention System. From those who identify the children to be involved in the evaluation to the service coordinators who have discussed this with families and scheduled the assessments. It is important detailed work. Finally, we have greatly appreciated working with the Rosanne Griff-Cabelli, Lora Lewis, Cathie Frost, and most recently Barbara Akenhead, director and coordinators of the Child Development Watch offices, who oversaw the evaluation activities. This year, Kathleen Devine has been responsible for the random selection of the children to be involved with the Outcomes Evaluation. Everyone’s overall support of the project has been greatly appreciated. Table of Contents Acknowledgement i Evaluation Design and Methods 1 Child Change Results for 2005 3 Conclusions 6 Appendix A: Membership of the Advisory Committee Planning the Child Outcomes Evaluation 7 Appendix B: Membership of the Ongoing Program Evaluation Committee 11 Birth to Three Early Intervention System Child Outcome Evaluation Report 2005 This report is a summary of the results of the on-going evaluation child outcomes for children enrolled in Delaware’s early intervention system. Since 1997 when a pilot of this process was initiated, a sample of children enrolled in Delaware’s Part C early intervention services have been tracked to determine their developmental outcome. During 1997-98 and 1998-99 two pilot projects were conducted to determine the feasibility of tracking children using a variety of assessment instruments, family outcome measures and demographic forms. It was determined by an advisory group (see Appendix A) at the end of these pilots that it was feasible to track a sample of the children enrolled in Part C services and to focus on their developmental outcomes. At that point in time, it was determined by the advisory group that tracking family outcomes was not feasible or cost effective. Evaluation Design and Methods The current child change outcomes evaluation project is designed to randomly sample a group of children enrolled in Delaware’s Part C early intervention system and track their developmental outcomes from the time of their enrollment to the time of their exit. The children are assessed annually using two different assessment instruments. Children are included in the annual report of the Child Change Outcomes if they have at least two assessments completed. Sample Children included in the evaluation must be under two years of age and must be found eligible for Part C services as defined by Delaware’s Interagency Coordinating Council. Children are randomly selected for inclusion in the Child Change Outcome evaluation. The family of every tenth child enrolled for services at the Milford Child Development Watch office and the family of every 15th child enrolled for services at the Limestone Child Development Watch office are asked permission to include their child in the evaluation. If the family grants permission, informed consent forms are signed and the child is enrolled. If the family of a randomly selected child does not grant permission, the family of the next child enrolled is asked permission to enroll their child in the evaluation. This process is followed until a family grants permission, then the selection reverts back to every 10th or 15th child enrolled, depending upon the location of the Child Development Watch office. Children remain enrolled in the evaluation project until they are either too old for Part C services, they are no longer eligible for services due to their improved developmental skills, or their families choose to remove them from the evaluation project. Children are included in the analysis of the project once assessments have been completed at two different periods of time (usually one year apart). Children remain in the analysis for one year after they have stopped receiving Part C services. Measurements Three measurements are collected for each child enrolled in the project. Two of the measurements are development assessments, the Bayley Scales of Infant Development II (1992) and the Play Assessment Scale (Fewell, 1985). The third measurement is a family and child demographic form designed to collect child and family background information. The Bayley Scales of Infant Development II (Bayley II) is a developmental assessment instrument designed to measure the motor and mental development of children aged four months to four years. The assessment is conducted by developmental specialists who have been trained in the use of the Bayley II. For this evaluation project the individual motor and mental scores are used. The Play Assessment Scale (PAS) is a cognitive assessment instrument developed by Rebecca Fewell during the late 1970s and 1980s and is designed for use with children aged four months to four years. It is designed to measure children’s cognitive abilities, from a Piagetian and Vygostskian perspective, based on their play behaviors. The results of the PAS are a play age score that equates to a cognitive age score. This assessment is conducted by developmental specialists who have been trained in the use of the PAS. The demographic form used for the Child Change Outcomes project includes information about the child’s age, the primary language spoken in the family household, parents’ education levels, number of members in the household, annual household income, participation in care outside of the household, gender and ethnicity. Data Collection The family members of randomly selected children are asked to participate in the Child Change Outcomes evaluation by the family’s Child Development Watch Service Coordinator. When a family agrees to participate, the demographic form is completed with the Service Coordinator and the Bayley II assessment results are sent to a secure file at the Center for Disabilities Studies. An appointment is also made for a PAS assessment to take place. These three pieces of data are collected within a two month window. At or around the one year anniversary of the child being enrolled in Part C services, the Bayley II and PAS assessments are again conducted. This information is also sent to the Center for Disabilities Studies. This process continues on an annual basis until the child leaves Part C services. Children enrolled in the Child Change Outcomes project may have two points of data and, in some cases, three points of data at the time of their exiting the project. Analysis Analysis of the child change data occurs annually in December. Analysis occurs with all children currently enrolled in the evaluation project who have at least two data points collected and all children who have left Part C services within the previous two calendar years who had at least two data points. All children with two data points as of November 30th are included in the analysis. Each year, the sample to be reported on is determined based on the definition above and a demographic summary of the sample is completed. The developmental outcomes of the children are then analyzed in two ways: 1) the overall change in mental, motor, and play skills are reported using the results of the Bayley II and PAS scores and 2) a ratio of actual development compared to an expected rate of development is also calculated and reported. All analyses are reported in the aggregate and no children or families are identified in any way. Child Change Outcomes Results for 2005 Included in this section are the results of the Child Change Outcome analysis. The first portion of this section includes information about the cohort of children included in the sample and the demographic profiles of the families and the children. The second portion of this section reports the developmental changes and outcomes of the children included in the sample. Sample and Demographics For the 2005 report all children who were active in the evaluation project as of November 30, 2005 or had exited Part C services between December 1, 2003 and November 30, 2005 were included in the cohort for this report. This resulted in 39 children who had at least two data points for two of the three assessments (motor, mental, and play skills) being included in the sample for the 2005 analysis. The demographic profile of the families of the children in the sample is as follows: * 35 of 39 used English as their primary language in their home * 35 of the children were cared for by their mother and/or father (the other four were cared for by grandparents) * the education level of the 39 mothers were: o 11 college graduates o 12 with some college o 11 with a high school diploma or GED o 5 with less than a high school diploma * the education level of the 32 fathers for whom data was available were: o 14 college graduates o 6 with some college o 6 with a high school diploma or GED o 6 with less than a high school diploma * 25 of the 39 families lived in New Castle County; the other 14 lived in either Kent or Sussex counties * the average number of people living in the household of these children was 4.46 * the annual income of the households was diverse and included: o 13 families reporting less than $17,000 o 4 families reporting between $17,000 and $24,999 o 12 families reporting between $25,000 and $50,000 o 10 families reporting over $50,000 The demographic profile of the children included in this sample is as follows: * 21 of the children were male and 18 were female * the ethnic background of the children was diverse and included o 13 families reporting their children’s ethnicity as African American o 23 reporting their children’s ethnicity as Caucasian o 2 reporting their children’s ethnicity as Hispanic o 1 reporting their child’s ethnicity as mixed * 17 of the 39 children were reported to have been born premature o the average prematurity was 8 weeks o the range of prematurity was 5 to 17 weeks * 12 of the children received care outside of their home o 4 of the children attended a child care center o 4 attended a family child care home o 4 received care at a relative’s home Developmental Changes All of the 39 children included in the sample for the 2005 analysis had two assessment data points for at least two of the three assessment measures (motor, mental, and play skills). The average time between assessments for the Bayley II motor assessments and mental assessments for the 39 children was 12.43 months. The average time between assessments for the PAS was 12.64 months. At the time of the 39 children’s initial assessments, they were showing delays of six to twelve months in the areas of motor, mental and play skills. The children’s rates of development were 0.65 for their motors skills (approximately eight months of development for the passage of every twelve chronological months), 0.67 for their mental skills (again, approximately eight months of development for the passage of every twelve chronological months), and 0.83 for their play skills (approximately ten months of development for every twelve chronological months). These rates of development indicate that the children had average delays between 20% and 40% in these developmental domains. At the time of the children’s second assessment, the developmental changes were substantial. The average developmental change in the Bayley II motor scores between the two assessments was 12.84 months of development. The average developmental change in Bayley II mental scores between the two assessments was 12.73 months of development. The average developmental change in the PAS play skills scores between the two assessments was 12.33 months of development. These average scores indicate that the children were developing at approximately a rate of one developmental month for the passage of each chronological month; essentially, at a typical rate of development (see Table 1 for a summary of these results). Table 1. Summary of child change assessment results for 39 children enrolled in Delaware Part C services. Develop mental Domain Rate of Development* based on Initial Age and Developmental Score Average Time Between Assessments (months) Expected Annual Rate of Development* (months) Average Change between Assessments (months) Actual Rate of Development* Per month Interv ention Effica cy Index + Motor 0.65 12.43 8.08 12.84 1.03 1.58 Mental 0.67 12.43 8.33 12.73 1.02 1.52 Play 0.83 12.64 10.49 12.33 0.97 1.17 *Rate of Development indicates how many months of development to expect with the passage of one month of time. A rate of 1.0 indicates one month of developmental gain for the passage of every chronological month. +The Intervention Efficacy Index is reported as a ratio. Values of 1.0 are the expected rate of development for the children enrolled in Part C services. Values above 1.0 indicate developmental gains beyond what would be expected without intervention. Values below 1.0 indicate developmental gains at a rate slower than expected. To systematically analyze the developmental outcomes of the 39 children in this sample, a rate of development was calculated for each child. This rate was determined by dividing the developmental level of the child by his or her age at the time of the developmental assessment. Because this group of children had delays that made them eligible for Part C services, this rate of development was below 1.0 for the motor, mental, and play skills measured (see the column labeled “Rate of Development based on Age and Developmental Score” in Table 1). Using this rate of development at the time of the first assessment, an expected rate of development at the time of the second assessment was calculated. This was determined by multiplying each child’s rate of development at the time of the first assessment by the number of months between the two assessments. This calculation resulted in the expected rate of development for the period of time between the two assessments (see the column labeled “Expected Rate of Development” in Table 1). Since the expected rate of development for the children was significantly lower than the typical rate of development, these results are especially encouraging. While the expected rate of development value is not a precise predictor of children’s developmental progress over time, it does acknowledge that children with delays and disabilities do not develop at the typical developmental rate. Finally, to determine the impact that early intervention services and other variables such as family interactions, medical and health supports and other services had on the children’s development, an Intervention Efficacy Index ratio was calculated. This ratio is the child’s actual rate of development at the time of the second assessment divided by the expected rate of development from the first assessment. Conclusions As in past years, the rate of development observed for the children in the sample was greater than expected. This year, in contrast to previous years, the intervention efficacy index was far greater for two of the three developmental domains assessed (see Table 2). While the play skills of children receiving early intervention services continue to advance at a rate greater than expected and consistent with findings from previous years, the rate of advancement for motor and mental skills for this cohort was significantly greater than in the past. Table 2. Comparison of Intervention Efficacy Index Domain 2004 Index 2005 Index Motor 1.22 1.58 Mental 1.29 1.52 Play 1.21 1.17 In addition to the high rate of development in the domains of motor and mental skills, the children in this sample showed advancement of their skills in all three developmental domains at a rate that was comparable to children who were developing typically. For this cohort, once they were receiving early intervention services their developmental progress was similar to children without disabilities. The high rate of motor and mental development and the close to typical rate of development for all three skill domains is significant in the context that this sample was one that included families from very diverse backgrounds in terms of annual income and parental education. Likewise, the children in this sample were diverse and included almost 50% children who were born prematurely by an average of eight weeks. Given the environmental and health contexts of this sample of children, these findings indicate that the early intervention services being provided through Delaware’s Part C early intervention program, Child Development Watch, and the other supports that these children receive are having a positive impact on their developmental outcome. Appendix A Membership of the Advisory Committee Planning the Child Outcomes Evaluation Advisory Committee Developing Child Development Watch’s Child Outcomes Evaluation Birth to Three Early Intervention System Rosanne Griff-Cabelli, Administrator Coordinating Council for Children with Disabilities Bob Piech, Coordinator Department of Education Jim Lesko, Early Childhood Programs Betty Richardson, Head Start Martha Toomey, Programs for Preschoolers with Disabilities Nancy Wilson, IRMC Policy Coordinator University of Delaware, College of Human Services, Education and Public Policy Leslie Cooksy Michael Gamel-McCormick Appendix B 2004-2005 Membership of the Ongoing Program Evaluation Committee 2005 Child Development Watch Ongoing Program Evaluation Committee Members Rosanne Griff-Cabelli Birth to Three Early Intervention System Division of Management Services Department of Health and Social Services Anne Marie Higley Child Development Watch, Northern Health Services Meri Jo Montague Child Development Watch, Southern Health Services Division of Developmental Disabilities Services Department of Health and Social Services Richard Michele Christiana Care Health Services, Inc. Angie Sipple Parent and Interagency Coordinating Council member Cathie Frost Barbara Akenhead Child Development Watch Northern Health Services, Division of Public Health, Department of Health and Social Services Lora Lewis Carol Owens Child Development Watch Southern Health Services, Division of Public Health, Department of Health and Social Services Dennis Rubino Director of Children with Special Health Care Needs, Community Health Care Access, Division of Public Health Department of Health and Social Services Michael Gamel-McCormick Deborah Amsden Denise Hartranft Center for Disabilities Studies, University of Delaware Equal Opportunity Statement AN EQUAL OPPORTUNITY/AFFIRMATIVE ACTION EMPLOYER The University of Delaware is committed to assuring equal opportunity to all persons and does not discriminate on the basis of race, color, gender, religion, ancestry, national origin, sexual orientation, veteran status, age, or disability in its educational programs, activities, admissions, or employment practices as required by Title IX of the Education Amendments of 1972, Title VI of the Civil Rights Act of 1964, the Rehabilitation Act of 1973, the Americans with Disabilities Act, other applicable statutes, and University policy. Inquiries concerning these statutes and information regarding campus accessibility should be referred to the Affirmative Action officer, 305 Hullihen Hall, 302/831-2835 (voice), 302/831-4552 (TDD) i iii Center for Disabilities Studies, University of Delaware 7 Delaware Birth to Three Early Intervention System Child Outcome Evaluation Report – 2005 Delaware Birth to Three Early Intervention System Child Outcome Evaluation Report – 2005 Delaware Birth to Three Early Intervention System Child Outcome Evaluation Report – 2005 Center for Disabilities Studies, University of Delaware 9 Delaware Birth to Three Early Intervention System Child Outcome Evaluation Report – 2005 Center for Disabilities Studies, University of Delaware 11 Delaware Birth to Three Early Intervention System Child Outcome Evaluation Report – 2005 Center for Disabilities Studies, University of Delaware 13