Arizona State Schools for the Deaf and the Blind |
Previous | 1 of 4 | Next |
|
|
Small
Medium
Large
Extra Large
Full-size
Full-size archival image
|
This page
All
|
A REPORT TO THE ARIZONA LEGISLATURE Debra K. Davenport Auditor General Performance Audit and Sunset Review Arizona State Schools for the Deaf and the Blind Performance Audit Division September • 2012 REPORT NO. 12-05 The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the opera-tions of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Audit Staff Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov Dale Chapman, Director Shan Hays, Manager and Contact Person Kori Minckler, Team Leader Jay Rasband Beth Rensvold Richie Rinaldi Representative Carl Seel, Chair Representative Tom Chabin Representative Justin Olson Representative David Stevens Representative Anna Tovar Representative Andy Tobin (ex officio) Senator Rick Murphy, Vice Chair Senator Andy Biggs Senator Rich Crandall Senator Linda Lopez Senator David Lujan Senator Steve Pierce (ex officio) ASDB uses standardized tests to assess student progress— These tests include the state-wide Arizona’s Instrument to Measure Standards (AIMS) test as well as another test similar to the AIMS called Measures of Academic Progress (MAP). Most ASDB students did not meet standards on AIMS— ASDB students generally scored lower on AIMS than did Arizona students as a whole. Although some ASDB students passed and even exceeded standards on the spring 2011 AIMS, most did not. MAP results showed students made some progress but still testing well below grade level— ASDB students’ MAP test scores showed that they start at a much lower level than national norms. Although limited growth occurred after 5th and 6th grades and continued into high school, it was not sufficient to bring students’ scores within reach of national norms. For example, by the 11th grade, ASDB students scored the same as the average 3rd-grade student would in reading and scored slightly below the average 4th-grade student in math. Standardized test scores are limited indicators of program success—Sensory-impaired students may have difficulty taking standardized tests because of natural disadvantages these students may face in taking these tests. In addition, there is little comparative information about scores attained by similar students in other Arizona schools or other states’ schools. ASDB uses common practices, but can strengthen its practices—To help ensure success in educating its students, ASDB 2012 September • Report No. 12-05 Arizona State Schools for the Deaf and the Blind Our Conclusion The Arizona State Schools for the Deaf and the Blind (ASDB) serves children and students from birth through 21 years of age who have sensory impairments such as deafness and blindness. As of April 2012, ASDB had 2,240 students who either attended one of its three on-campus schools, participated in its birth to age three or preschool programs, or received ASDB services at their local schools. ASDB can do more to promote student success by ensuring that children with sensory impairments receive services early in their lives, increasing students’ access to highly qualified teachers, and researching and addressing differences in students’ test scores. ASDB should also ensure its regional cooperative program, which serves students at their local schools, more fully recovers its costs. It should also strengthen oversight and management of the program. Finally, ASDB needs to address critical information technology weaknesses. REPORT HIGHLIGHTS PERFORMANCE AUDIT ASDB can do more to promote student success follows commonly used practices, including using technology to help students learn. In addition, ASDB groups students according to their academic ability rather than age. However, ASDB can strengthen the following practices: • Early Intervention—Early interven-tion services are a critical element in a student’s success. ASDB provides early intervention services through a program called Arizona Early Interven-tion Program. ASDB has identified areas where it can do more to improve this program such as coordinating with the other agencies involved in the pro-gram, better organizing and training staff, and reaching out to families with sensory-impaired infants and children. • Highly qualified teachers—Federal law requires that disabled students be taught by teachers who are highly qualified in special education and in core academic subjects. According Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 0 10 20 30 40 50 60 70 Falls Far Below Approaches Meets Exceeds Percent of Students Reading ASDB State-wide 0 10 20 30 40 50 60 Falls Far Below Approaches Meets Exceeds Percent of Students Math ASDB State-wide Arizona State Schools for the Deaf and the Blind REPORT HIGHLIGHTS PERFORMANCE AUDIT September 2012 • Report No. 12-05 A copy of the full report is available at: www.azauditor.gov Contact person: Shan Hays (602) 553-0333 to an Arizona Department of Education report, about 90 percent of ASDB teachers were highly qualified during the 2011-2012 school year. According to ASDB officials, it is difficult to hire and retain highly qualified teachers for many reasons, including lack of competitive pay and the challenges of obtaining the required certifi-cations. ASDB is working with the University of Arizona and the Arizona Department of Educa-tion to increase the pool of qualified teachers. • Studying differences in test scores—Federal law requires that all public schools work to nar-row achievements gaps in their states. Study-ing differences in AIMS passing rates across the various ASDB locations may help improve ASDB’s student achievement. Recommendations—ASDB should: • Improve its early intervention program to ensure that children receive needed services. • Increase students’ access to highly qualified teachers. • Study differences in test scores and implement potential solutions to improve student achieve-ment. ASDB provides services at local school districts through its regional cooperative program—ASDB established a regional cooperative program in 1988 to help local school districts serve sensory impaired students. There are five regional cooperatives, and each has a staff of teachers, interpreters, and other specialists. About one-half of state school districts participate in the program. Funding comes from various sources—Funding for the regional cooperatives comes from special education vouchers through the Arizona Department of Education for assessments, counseling, and direct teaching services; from the State General Fund for program administrative costs; and from school district membership fees to screen children and train school staff. ASDB should examine fees—Each regional cooperative has established its own membership fees it charges to school districts. Districts also pay ASDB for direct educational services provided to their sensory impaired students. ASDB does not have policies for determining appropriate membership and other fees. Further, ASDB has not evaluated whether such fees cover the costs of the services provided. Recommendation—ASDB should ensure that fees cover the cost of all services provided by regional cooperatives by assessing operations, tracking costs, and developing appropriate fees. ASDB should review regional cooperative service fees and strengthen its operations ASDB needs to improve its information technology practices In 2009, the Arizona Department of Administration’s Information Security group assessed ASDB’s infor-mation technology (IT) environment. This assess-ment identified deficiencies and made several recommendations. Although ASDB tried to address the deficiencies, this audit identified the following IT security control weaknesses: • No risk assessment or security reviews of the IT environment have been done. • The IT network does not have adequate controls to secure it from hackers. • There is no effective process for updating com-puters and servers, which are running critical systems on outdated software. ASDB also does not have a comprehensive disaster recovery plan and does not properly test or backup data. As a result, ASDB may not be able to meet federal and state requirements to protect the privacy of it students and staff. Recommendations—ASDB should: • Strengthen IT security controls. • Address disaster recovery deficiencies. • Improve data backup. TABLE OF CONTENTS continued Introduction 1 Finding 1: ASDB can do more to promote student success 9 ASDB aims to prepare students for future 9 ASDB uses several assessments to measure student progress 10 Test scores showed some academic progress, but comparative information for assessing program success is limited 11 ASDB uses commonly used practices but can work to improve 15 Recommendations 24 Finding 2: ASDB should examine regional cooperative program service fees and strengthen program operations 27 ASDB provides services to students at local school districts through its regional cooperative program 27 ASDB should examine fees for regional cooperative programs services 29 Regional councils’ role in fee-setting needs review 34 Regional cooperatives are not consistent in tracking resources 37 Recommendations 38 Finding 3: ASDB needs to improve its information technology practices 41 IT systems used extensively and contain sensitive data 41 page i Office of the Auditor General TABLE OF CONTENTS continued ASDB has not addressed critical IT weaknesses 41 Recommendations 47 Sunset Factors 49 Appendix A: Test results a-i Appendix B: Methodology b-i Agency Response Tables 1 Student Enrollment by Type of Disability School Year 2011-2012 2 2 Schedule of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2009 through 2012 (Unaudited) 7 3 Services Provided by Regional Cooperative Staff School Year 2011-2012 30 4 Regional Cooperative Membership Fee Structure School Year 2011-2012 31 5 Regional Cooperative Reimbursements to Districts School Year 2011-2012 32 6 Example of Differences in Regional Cooperative Service Costs School Year 2011-2012 33 page ii State of Arizona TABLE OF CONTENTS concluded Figures 1 Regional Cooperatives, School Districts, and Students Served As of April 2012 4 2 Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 12 3 Mississippi Joint Legislative PEER Committee’s Structured Fee-Setting Process Developed for State Government 35 4 Spring 2008, 2009, 2010, and 2011 AIMS Math Test Results for ASDB Students and State-wide Students a-i 5 Spring 2008, 2009, 2010, and 2011 AIMS Reading Test Results for ASDB Students and State-wide Students a-i page iii Office of the Auditor General ASDB educates students with sensory impairments Programs and responsibilities ASDB was established in 1912 to provide education to students with sensory impairments such as deafness or blindness. ASDB enrolled 478 students at its on-campus schools in the 2011-2012 school year (see Table 1, page 2). In addition, ASDB provided services to 1,190 additional students in school districts and 299 preschool students throughout the state during the 2011- 2012 school year. In accordance with state law, ASDB provides educational programs for chil-dren and students from birth through 21 years who have a vision and/or hear-ing loss (see textbox for ASDB’s mission). According to statute, school districts must arrange for a placement and evaluation team for students with a vision and/or hearing impairment. The team includes the child’s parent or guardian as well as school district and ASDB rep-resentatives. This team deter-mines the appropriate educa-tional placement for the stu-dent based on his or her indi-vidualized education program. Students can attend a local school district, charter school, or one of the ASDB campuses. As of April 2012, the 1,668 students ASDB served were enrolled in the following schools and/or programs: • Arizona Schools for the Deaf and Blind (110 students at the Arizona School for the Deaf and 85 students at the Arizona School for the Blind)—Two schools, the Arizona School for the Deaf and the Arizona School for the Blind, make up the ASDB Tucson campus. According to ASDB officials, students can live on campus as residential students or attend as day students if they live within 60 miles of the campus. As of April 2012, 70 students lived on campus. Both schools are accredited by page 1 Scope and Objectives INTRODUCTION The Office of the Auditor General has conducted a performance audit and sunset review of the Arizona State Schools for the Deaf and the Blind (ASDB) pursuant to an October 26, 2010, resolution of the Joint Legislative Audit Committee. This audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This performance audit and sunset review addresses (1) how ASDB can increase its use of best practices to promote its students’ success, (2) improvements that ASDB should make to the operations of its regional cooperatives program, and (3) the need to better protect student data and make other improvements to ASDB’s information systems management. The report also includes responses to the statutory sunset factors. Office of the Auditor General ASDB Mission: To work together with parents, school districts, advocacy organizations, and business and community members to create nurturing environments in which children with a vision or hearing loss feel valued, develop their abilities, strive to achieve academic excellence, and develop skills to help them become productive and responsible members of society. Source: Fiscal Years 2010-2013 master list of state government programs. AdvancED.1 The Arizona School for the Deaf offers educational and support services, including instruction in a direct communication environment and counseling (see textbox), as well as training that teaches students with cochlear implants to hear and understand speech and other sounds. At the Arizona School for the Blind, services are provided based on an expanded core curriculum that includes state standard curriculum and special services such as orientation and mobility training, braille, visual efficiency training, assistive technology, and compensatory skills and tools. Both schools also offer other services based on students’ needs, including activities of daily living; social skills; occupational, physical, and speech therapy; and residential services. • Phoenix Day School for the Deaf (283 students)—According to ASDB officials, the Phoenix school educates students who are deaf or hard-of-hearing and live within 60 miles of the metropolitan Phoenix area. It provides educational and support services similar to the Arizona School for the Deaf. The school supports students developing two languages— 1 AdvancED is an organization of public and private schools in the United States and other countries world-wide that accredits schools based on adherence to quality standards in several areas, including curriculum, instructional design, and assessment practices. The organization was formed in 2006 from the North Central Association Commission on Accreditation and School Improvement, the Southern Association of Colleges and Schools Council on Accreditation and School Improvement, and the National Study of School Evaluation. page 2 State of Arizona Direct Communication Environment— Direct communication and access are a critical part of education and language development. Students who are deaf or hard-of-hearing interact directly with teachers, staff, and peers every day in the classroom, cafeteria, after-school program, and residential program. Direct access to instruction and peer interaction leads to enhanced educational, social, and emotional development; and allows a greater degree of independence, self-confidence, and self-advocacy. Source: Arizona School for the Deaf Web site. Table 1: Student Enrollment by Type of Disability School Year 2011-2012 1 According to ASDB officials, “other” refers to a student with a sensory impairment who also has a medical condition or a physical impairment. 2 According to ASDB officials, multiple disabilities are not identified in preschool children but are identified at a later age. Source: Auditor General staff analysis of ASDB’s April 2012 report, Agency enrollment: school year 2011-2012. Program Deaf or Hard-of- Hearing Visually- Impaired Multiple Disabilities with Sensory Impairment Multiple Disabilities with Severe Sensory Impairment Other with Sensory Impairment 1 Total Students ASDB Campuses Arizona School for the Blind 0 35 22 28 0 85 Arizona School for the Deaf 79 0 16 15 0 110 Phoenix Day School for the Deaf 211 0 40 32 0 283 Regional Cooperatives Desert Valleys 160 51 80 92 1 384 Eastern Highlands 34 17 42 19 3 115 North Central 99 28 69 58 9 263 Southeast 130 36 88 39 15 308 Southwest 57 10 29 23 1 120 Preschool2 154 107 0 0 38 299 Total 924 284 386 306 67 1,967 American Sign Language (ASL) and English—and its curriculum parallels the curriculum provided by public schools with modifications to meet students’ communication needs. According to ASDB officials, the school also offers intensive instruction in functional and daily living skills for students who need it. The school is accredited by AdvancED. • Regional cooperatives (1,190 students)—ASDB provides educational and support services in students’ local schools through five regional cooperatives located in various parts of the State (see Figure 1, page 4). The Legislature first authorized ASDB to establish regional cooperatives in 1988 to provide various services that would enable districts to serve sensory-impaired students in a cost-effective way and enable these students to reach their individual potentials by providing access to the general education curriculum in district schools. According to ASDB officials, the regional cooperatives provide access to a pool of specially trained personnel and equipment, such as sign language interpreters and braille-writer machines. In addition to enrolling students at its campuses or in regional cooperatives, ASDB provides services through the following programs: • Birth to 3 program (273 children)—In cooperation with the Arizona Early Intervention Program (AzEIP, see textbox on page 18), ASDB provides federally mandated services for families of/and children ages birth to 3 who have hearing or vision loss or who are deaf-blind. The program provides a comprehensive and coordinated interagency system of early intervention services in the child’s home and is designed to work in cooperation with other agencies and programs to support the child and family’s needs. In addition to assessments that determine if an infant or toddler is eligible for the program and to plan appropriate services, the AzEIP program offers service coordination, assistive technology devices, family training, physical and occupational therapy, and speech-language pathology services. • Preschool program (299 students)—ASDB provides preschool education to eligible children, ages 3 to 5, at its campuses in Tucson and Phoenix and in classrooms in public schools throughout Tucson and Phoenix. The services offered to the children include preschool education, educational assessment, vision and audiological assessments, orientation and mobility services, and speech therapy. ASDB’s campuses and other programs serve students with vision or hearing loss, including students with multiple disabilities if at least one of the disabilities is a visual or hearing impairment (see textbox). In the 2011- 2012 school year, most students ASDB served state-page 3 Office of the Auditor General Multiple disabilities—Learning and developmental problems that require special education and related services and that result from two or more of (1) hearing impairment, (2) orthopedic impairment, (3) moderate intellectual disability, and (4) visual impairment, or any of those conditions concurrently with mild intellectual disability, emotional disability, or specific learning disability. Multiple disabilities with severe sensory impairment—Multiple disabilities that include (1) severe visual or hearing impairment with another severe disability or (2) severe visual and severe hearing impairments. Source: A.R.S. §15-761 page 4 State of Arizona Source: Auditor General staff analysis of data received from ASDB’s regional cooperatives as of April 2012. Figure 1: Regional Cooperatives, School Districts, and Students Served As of April 2012 Mohave Coconino Yavapai La Paz Yuma Pinal Graham Pima Cochise Santa Cruz Maricopa Gila Navajo Apache North Central Regional Cooperative Eastern Highlands Regional Cooperative Southeast Regional Cooperative Desert Valleys Regional Cooperative Southwest Regional Cooperative page 5 Office of the Auditor General wide had a hearing loss, with smaller numbers of students with a vision loss or multiple disabilities (see Table 1, page 2). Board and staff ASDB’s Board of Directors (Board) is responsible for the governance of ASDB, including setting policy and appointing, evaluating, and determining whether or not to offer a new contract to ASDB’s Superintendent. It has ten members, including the Governor and the State’s Superintendent of Public Instruction who serve as ex officio members. The Governor appoints the remaining members, who serve 3-year terms. The appointed members include: • One member from the Commission for the Deaf and the Hard-of-hearing; • One member from the Governor’s Council on Blindness and Visual Impairment; • One school district employee who works with his or her district’s program for sensory impaired pupils; and • Five additional members. A.R.S. §15-1321 requires the Governor to give preference to people with experience and knowledge of sensory-impaired education for three of these members. ASDB has staff at its three school campuses and at its five regional cooperatives that are located in various parts of the State. As of May 2012, ASDB reported that it had 803 full-time equivalent (FTE) positions, with 256 vacancies. The vacancies included 78 teachers; 38 instructional assistants; 47 educational interpreters, audiologists, and transcribers; 9 drivers; and 84 various other positions. ASDB’s staffing includes the following: • Administration—The administrative staff include a superintendent who serves as the executive officer for ASDB, an assistant superintendent, and principals for each school. In addition, ASDB has directors over operations, regional cooperatives, information technology, and human resources; a staff development specialist; and an accountability specialist. • Teaching and educational support—As of May 2012, ASDB employed 227.75 teachers and 20 supervisory teachers. Educational support includes 11.25 audiologists, 4.25 braillists, 1 ASL specialist, 3.25 speech/language pathologists, and 65.75 educational interpreters. The educational interpreters work at the regional cooperatives. ASDB’s teachers and other educational staff generally have special training and certification in communicating with and teaching students with sensory impairments. page 6 State of Arizona • Other—ASDB’s on-campus staff in Tucson also include night supervisors, residential hall team leaders, and live-in staff for the student dormitories. These staff also include teaching parents, who are educators responsible for working with students on goals in areas such as personal care, recreation and leisure, and domestic life. In addition, ASDB’s operations staff include custodial, maintenance, security, food service, and equipment repair staff as well as full-time and part-time drivers. Budget and finances As shown in Table 2 (see page 7), ASDB received approximately $55.5 million, $56.5 million, $54.7 million, and $53.8 million in annual net revenues in fiscal years 2009 through 2012, respectively. These revenues included: • Special education voucher monies from the Arizona Department of Education (ADE) to reimburse ASDB for educational costs based on its enrollment—These monies are provided for students at ASDB’s three campus schools and some students enrolled in ASDB’s regional cooperative program. In the 2011-2012 school year, the voucher amount was $19,122.39 for each visually-impaired student and $19,008.02 for each hearing-impaired student. For each student who receives services through the regional cooperative program, ASDB and the student’s district decide whether ASDB or the district will collect the voucher monies. • Payments from school districts for services provided by the regional cooperative program—These include membership fees to participate in the regional cooperative program and fee-for-service payments for students whose voucher monies are retained by the district. Fee-for-service payments vary depending on the level of service provided by ASDB. In the 2011-2012 school year, the payments ranged from $2,800 annually for a student receiving 1 hour per month of services to $21,500 annually for a student receiving 10 to 15 hours per week of services (See Finding 2, pages 27 through 39, for additional information on the fees charged for services provided through ASDB’s regional cooperative program). • State General Fund appropriations made directly to ASDB—State General Fund appropriations help bridge the gap between the cost of educating ASDB students and the amounts provided by special education vouchers and fees paid by school districts. In fiscal year 2012, ASDB received a $20.8 million State General Fund appropriation. As shown in Table 2, ASDB’s total expenditures ranged between nearly $55.2 million in fiscal year 2009 to approximately $54.5 million in fiscal year 2012. Most of ASDB’s expenditures are for personal services and related benefits. These monies pay for teachers and other staff at the three on-campus schools, teachers in the regional page 7 Office of the Auditor General 1 Amount is primarily composed of educational reimbursements from the Arizona Department of Education based on enrollment and is determined by statutory formula. The amount also includes federal grant revenues that are either received directly or passed through from the Arizona Departments of Economic Security or Education. 2 Amount consists of revenues received from Arizona school districts for services rendered by ASDB’s regional cooperatives on a fee-for-service basis. 3 Amount primarily consists of interest and rental income. 4 Amount consists of transfers to the State General Fund in accordance with Laws 2008, Ch. 285, §46; Laws 2010, Ch. 1, §148 and 7th S.S., Ch. 3, §8; and Laws 2011, Ch. 24, §§108, 129, and 138 to provide support for state agencies. 5 Amount primarily consists of transfers to the Arizona Department of Economic Security for required matching to obtain federal monies. According to ASDB, the fiscal year 2009 required match was not made until fiscal year 2010; therefore, the transfer is reported in fiscal year 2010. Similarly, the fiscal year 2012 required match was not made until August 2012; therefore, it will be reported in fiscal year 2013. 6 According to ASDB, end-of-year fund balances consist of revenues from several dedicated funding sources such as federal grants, Proposition 301 (2000) sales taxes, and educational reimbursements from the Arizona Department of Education. The balances are generally needed for cash flow at the beginning of the next fiscal year or are restricted for specific purposes. For example, in order to pay the payroll costs for current 12-month employees and purchase equipment, supplies, and materials needed for the start of the school year, the balances are needed in July and August since educational reimbursements from the Arizona Department of Education, fees for services, or membership fees are not received until late September. Source: Auditor General staff analysis of the Arizona Financial Information System (AFIS) Accounting Event Transaction File for fiscal years 2009 through 2012 and the AFIS Management Information System Status of General Ledger-Trial Balance screen for fiscal years 2010 through 2012. Table 2: Schedule of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2009 through 2012 (Unaudited) 2009 2010 2011 2012 Revenues: State General Fund appropriations $ 21,351,663 $ 21,580,479 $ 21,511,083 $ 20,802,910 Intergovernmental1 31,373,770 31,646,493 29,759,177 29,544,022 Educational and support service fees2 2 ,216,073 2 ,949,195 2 ,913,044 2 ,885,938 Other3 515,669 336,512 484,947 574,821 Gross revenues 55,457,175 56,512,679 54,668,251 53,807,691 Remittances to the State General Fund (12,034) (1,292) Net revenues 55,445,141 56,511,387 54,668,251 53,807,691 Expenditures and transfers: Personal services and related benefits 44,215,881 43,421,888 40,744,344 42,882,990 Professional and outside services 3,563,179 3,215,689 3,943,356 3,550,435 Travel 310,034 300,216 289,665 283,635 Food 255,407 231,084 232,703 237,015 Aid to individuals 1,110 Other operating 5,798,761 5,783,749 6,582,648 4,833,542 Equipment 1,034,733 2,100,684 2,451,489 2,715,179 Total expenditures 55,179,105 55,053,310 54,244,205 54,502,796 Transfers to the State General Fund4 157,200 18,200 531,100 616,400 Transfers to other agencies5 1,063,064 461,882 Total expenditures and transfers 55,336,305 56,134,574 55,237,187 55,119,196 Net change in fund balance 108,836 376,813 (568,936) (1,311,505) Fund balance, beginning of year 6,506,384 6,615,220 6,992,033 6,423,097 Fund balance, end of year6 $ 6,615,220 $ 6,992,033 $ 6,423,097 $ 5,111,592 cooperative program who travel to the surrounding school districts in their respective regions throughout the State, and administrative staff. These expenditures also include salaries and benefits for the staff who provide student transportation to and from the day school programs, which totaled approximately $2.9 million in fiscal year 2011, according to the fiscal year 2011 Annual Report of the Arizona Superintendent of Public Instruction and other financial information obtained from ASDB. page 8 State of Arizona ASDB can do more to promote student success FINDING 1 page 9 ASDB aims to prepare students for future ASDB’s mission is to work with parents, school districts, and others to enable children with a vision or hearing loss to develop their abilities, strive to achieve academic excellence, and develop skills to help them become productive and responsible members of society. This mission aligns with ASDB’s statutory responsibility to provide educational services, including instruction in a direct communication environment, so that deaf, hard-of-hearing, and visually-impaired students may become self-sustaining and useful citizens. ASDB’s surveys of recent graduates show that many of its on-campus students either continue their education or begin working, either independently or with supports, such as a job coach. In 2011, ASDB reported that the Arizona School for the Blind had 13 graduates, the Arizona School for the Deaf had 14 graduates, and the Phoenix Day School for the Deaf had 15 graduates. Surveys conducted approximately 1 year after the students graduated indicated that the students became involved in a variety of activities. For example: • Out of 13 survey respondents who graduated from the Arizona School for the Blind in 2011, 9 reported that they were participating in work enclave programs through the Department of Economic Security. Work enclaves are competitive work site jobs where workers with disabilities are supervised by program staff and receive a special wage commensurate with their abilities. • All 6 2011 graduates of the Phoenix Day School for the Deaf who responded to the survey and 10 of the 14 respondents from the Arizona School for the Deaf reported that they were in college or community college, working, or volunteering. For example, one graduate worked part-time as a physical therapist assistant after graduation and then resigned to attend Gallaudet University where he is studying physical therapy. Another graduate volunteers at the public library for 6 hours per week, a third works in a supported-employment program sponsored by the Department of Economic Security, and three graduates were attending a community college for the deaf in Texas. The Arizona State Schools for the Deaf and the Blind (ASDB) can make improvements in its use of best practices to help ensure that its students make good educational progress and are prepared for their future. Although ASDB uses standardized tests to assess individual student progress, these tests are somewhat limited in their ability to assess ASDB’s students’ progress and provide relatively limited information for comparing ASDB to other schools that educate similar students. Thus, attention to best practices recommended by experts and officials at other schools is particularly important. ASDB follows many of these best practices, including using technology and modifying instruction and services to meet students’ particular needs. In four respects, however, ASDB can strengthen its efforts. These are: • Improving its early inter-vention program; • Seeking ways to increase students’ access to highly qualified teachers; • Compiling and using information about its graduates to improve its programs; and • Studying differences in its students’ test scores and establishing expectations regarding practices to improve students’ test scores. Office of the Auditor General ASDB uses several assessments to measure student progress To monitor students’ progress, ASDB relies on several different assessment mechanisms. These include the same standardized test all Arizona students take, as well as additional standardized tests, including the Measures of Academic Progress (MAP), that provide more information on individual student progress. In addition, at least once a year, ASDB staff will review each student’s progress toward the individual goals established in his or her Individualized Education Program (IEP) (see textbox, page 11). Each student’s IEP determines what assessments will be used to measure the student’s progress, and results are reviewed when the IEP is updated. ASDB frequently uses the following tests:1 • Arizona’s Instrument to Measure Standards (AIMS)—This is the standardized assessment taken by all Arizona students in all school districts. The federal No Child Left Behind Act requires that each state adopt standardized assessments that align with state standards to demonstrate adequate yearly progress toward the goal of narrowing achievement gaps. The State of Arizona uses AIMS (see textbox, page 11) as its standards-based assessment.2 Arizona does not require special education students to pass the test to meet the requirements for a high school diploma unless their IEP requires it. • Measures of Academic Progress (MAP)—This test, created by the Northwest Evaluation Association (NWEA), assesses the same content as the AIMS, according to an NWEA official. However, it is an interactive form, meaning that it does not test a student on more advanced concepts if he/she did not answer the initial basic concept question correctly. According to ASDB, this type of test is particularly useful for students who are not performing at grade level because it provides more information about students’ actual performance levels and their progress over time than the AIMS does. ASDB administers the test to only some on-campus students. ASDB officials reported that ASDB encourages all students to take the MAP if they are able to do so. In spring 2011, approximately 59 percent of ASDB on-campus students took a MAP reading and/or math test. 1 In addition to AIMS and MAP, ASDB reported that it uses other assessments to measure student progress. For example, it uses the Stanford Achievement Test (SAT-10) in second and ninth grades, as required by the Arizona Department of Education. In addition, ASDB uses Teaching Strategies GOLD, another Arizona Department of Education required test for special education children in preschool and kindergarten. For children aged birth to 3 years, ASDB reported that it also uses the Developmental Assessment of Young Children test for deaf children and the Oregon Project for blind children to assess the children’s cognitive skill level. ASDB also reported that it uses the 6-Traits Writing Rubric to evaluate student writing samples, the Fairview assessment focusing on American Sign Language development, the STAR Reading assessment, Work Sampling—an observational assessment containing grade-level samples of what students should know and how they would demonstrate their knowledge—and classroom-type tests. 2 This report uses AIMS to refer to both AIMS and AIMS A, the alternate assessment used for some students. Both tests report student results in the same way, indicating that the student exceeds, meets, approaches, or falls far below standards. page 10 State of Arizona Arizona does not require special education students to pass the AIMS test to obtain a high school diploma unless their IEP requires it. Tests scores showed some academic progress, but comparative information for assessing program success is limited ASDB students, on average, underperformed on standardized tests, although they made some progress throughout their school years. For several reasons, however, standardized tests such as AIMS and MAP are limited in their ability to assess the progress of students with these types of impairments, and state-wide information that would allow comparisons between ASDB and other Arizona schools educating similar students is also limited. Most ASDB students who took AIMS did not meet standards—Students attending ASDB, either on one of its campuses or through the regional cooperatives, generally scored lower on AIMS than did Arizona students as a whole. Some ASDB students passed and even exceeded standards on the AIMS, but most did not. As shown in Figure 2 (see page 12), in spring 2011, only 5 percent of ASDB students exceeded standards, and only 25 percent met standards when they took the AIMS math test in 3rd through 12th grades. By comparison, 23 percent of all students state-wide who took AIMS exceeded standards, and 36 percent met standards for math in spring 2011. Similarly, only 2 percent of ASDB students exceeded, and 34 page 11 Office of the Auditor General Assessment terms Individualized Education Program (IEP)—Every child with a disability must have an IEP that shows the child’s level of achievement, measurable annual goals, how progress toward the goals will be measured, and what services will be provided to the child. The goals may include passing the AIMS if appropriate for the child, but Arizona Revised Statutes (A.R.S.) §15-763 states that a special education student does not need to pass the AIMS to graduate from high school unless this is required by the student’s IEP. A team that includes the child’s parent or guardian develops the IEP and updates it at least annually. The federal Individuals with Disabilities Education Act, Arizona statutes, and Arizona Department of Education regulations specify the requirements the IEP must meet. Arizona’s Instrument to Measure Standards (AIMS) and AIMS Alternate (AIMS A)—Tests students’ grade-level mastery of state academic standards in writing, reading, math, and science. AIMS A is an alternate assessment based on alternate achievement standards for students with significant cognitive disabilities. Both AIMS and AIMS A test students in 3rd through 8th grades and in high school beginning in 10th grade (or in 9th grade if the student repeats that grade) until they pass the test because it is a graduation requirement for most Arizona students. Special education students are not required to pass the test to receive a high school diploma unless stipulated in their IEP. Test scores are classified into four levels: exceeds, meets, approaches, and falls far below standards, with exceeds and meets defined as passing scores. Measures of Academic Progress (MAP)—Nationally normed test developed by the NWEA of language usage, reading, math, and science that uses an equal interval scale to allow charting progress from year to year. The test adapts to a student’s performance by offering questions based on whether the student answered previous questions correctly. ASDB tests on-campus students in 2nd through 12th grades. Source: Auditor General staff analysis of information about AIMS, AIMS A, and the IEP obtained from the Arizona Department of Education Web site, information about MAP from the Northwest Evaluation Association (test publisher) Web site, and A.R.S. §15-763. In spring 2011, 5 percent of ASDB students exceeded standards, and 25 percent met standards on the AIMS math test. percent met standards in reading, compared to 9 percent and 65 percent, respectively, for all students state-wide who took the AIMS reading test. Approximately two-thirds of ASDB students who took the AIMS math or reading tests did not meet the standards. AIMS test results for 2008 through 2010 show that, in those years, ASDB students also had much lower rates for meeting and exceeding the standards than students state-wide (see Appendix A, page a-i). page 12 State of Arizona Source: Auditor General staff analysis of AIMS and AIMS A spring 2011 test scores for all ASDB students enrolled in the 2011-2012 school year, including on-campus students and regional cooperative students for whom ASDB receives voucher funding, and state-wide AIMS and AIMS A results for spring 2011 published October 2011 by the Arizona Department of Education in its 2011 Technical Report. Figure 2: Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 0 10 20 30 40 50 60 70 Falls Far Below Approaches Meets Exceeds Percent of Students Reading ASDB State-wide 0 10 20 30 40 50 60 Falls Far Below Approaches Meets Exceeds Percent of Students Math ASDB State-wide MAP results show students made some progress but were still testing well below grade level—ASDB students’ test scores for MAP were well below national norms, which are based on a stratified sampling of students throughout the United States. ASDB students showed less improvement from one year to the next during the elementary school years as compared to national norms for all students.1 However, the test is not separately normed for deaf or blind students. Beginning in 5th and 6th grades, ASDB students improved their performance in reading and math, respectively, compared to national norms. Even so, ASDB students started at a much lower level than national norms, so while limited growth occurred after 5th and 6th grades and continued in high school, it was not sufficient to bring students’ scores within reach of national norms (see discussion of early intervention programs, pages 18 through 21, for reasons ASDB students started at a lower level). Specifically: • ASDB on-campus students’ average MAP scores in 2nd grade were in the first to second percentile in reading and the third to fourth percentile in math. This result indicates that nearly all students nationally who took the MAP received higher scores than ASDB students who took the test. • By the 11th grade, ASDB students who took the MAP scored on average the same as 3rd-grade students would in reading and scored slightly below the average 4th-grade student in math. This reflects a slight gain in performance, with results at the fifth to sixth percentile in reading and the seventh to eighth percentile in math. On average, 11th-grade students nationally who took the MAP scored at approximately the 50th percentile and scored the same as a typical-11th grade student would. Standardized test scores are limited as indicators of program success—For several reasons, great care must be taken in inferring conclusions from these results. Key limitations include natural disadvantages ASDB students may face in taking standardized tests. Additionally, there is little comparative information about scores attained by similar students in other settings in Arizona or in schools elsewhere in the United States. Specifically: • Students who are deaf or hard-of-hearing and students who are blind or visually-impaired may have difficulty taking standardized tests—Students attending ASDB have hearing and/or visual impairments or multiple disabilities (see textbox, page 3, for information on multiple disabilities). According to experts, language deficits and testing barriers make it more difficult for these students to perform at national or state norm levels on any standardized tests. Despite accommodations such as braille versions of tests and having test directions read aloud or signed, barriers that affect student performance still exist. For example, deaf students do not learn English in infancy, but the tests are administered in English so deaf students are taking the test in their second 1 Auditors analyzed ASDB students’ spring test results for the MAP test for 2006 through 2011 and compared them with the spring 2011 national results published by NWEA. page 13 Office of the Auditor General According to experts, language deficits and testing barriers make it more difficult for sensory-impaired students to perform at national or state norm levels on any standardized test. language. Additionally, blind students are unable to see illustrations, such as graphs, used in standardized math tests. • ASDB students’ scores cannot be reliably compared with results for other sensory-impaired students in Arizona or nation-wide—Within Arizona, the available information regarding student test scores makes it difficult to compare ASDB students’ performance with the performance of similar students throughout the State. A key reason is the way in which school districts must report special education students’ test scores. Specifically, to comply with Family Educational Rights and Privacy Act regulations, when school districts submit test scores to the Arizona Department of Education, they report all students with disabilities under the broad category of special education. They do not report students’ specific disabilities because such categorization could enable students to be individually identified in published reports. Therefore, sensory-impaired students’ test scores are reported in the same category as students with learning disabilities, developmental delays, and other disabilities. Further, although the Arizona Department of Education has data regarding all Arizona students, including their specific disabilities that could be matched with test scores, Arizona Department of Education officials reported that this specific data may not be complete. As a result, the available information does not provide an accurate way to compare the State’s entire population of sensory-impaired students by disability. Similar problems exist when attempting to compare ASDB students to sensory-impaired students in other states. First, each state administers different tests to measure students against their state’s standards. This makes it difficult to compare students’ progress across states. Second, states differ in the degree to which they require sensory-impaired students to take standardized tests required of other students. Some deaf and blind schools must participate in their states’ standardized testing. For example, students at Perkins School for the Blind in Massachusetts are required to pass the state test in order to obtain a high school diploma. Additionally, Alabama requires all high school students to take the state test to graduate with a diploma, and eligible students with disabilities who do not pass the test can obtain an occupational diploma instead of the regular high school diploma. However, similar to Arizona, at least two other states do not make passing the standardized test a graduation requirement for all deaf and blind students. Specifically, Florida School for the Deaf and the Blind students and Texas students who are deaf or blind can receive waivers from testing requirements for a high school diploma as long as the students can demonstrate in some other way that they have met state standards. Additionally, at least two states’ deaf and blind schools have admission policies that restrict or deny admission of severely impaired students. This would greatly impact the comparability of those schools’ test results with ASDB’s test results as students with severe or multiple impairments often struggle with passing standardized tests. page 14 State of Arizona When school districts submit test scores to the Arizona Department of Education, they report all students with disabilities under the broad category of special education. page 15 Office of the Auditor General Although there is a lack of comparison information regarding blind and multiply disabled students’ test scores, some limited studies have been conducted for deaf students. For example, a 2005 study found that the SAT-9 reading comprehension score for 17- and 18-year-old deaf and hard-of-hearing students was approximately equivalent to that of 4th-grade hearing students, and a 2012 longitudinal study reported that deaf 17-year-olds’ SAT-9 math scores approached the 6th-grade level.1,2 In addition, test results for a small number of students at a small number of schools for the deaf nation-wide suggest that ASDB deaf and hard-of-hearing students’ scores are similar to those of their peers at other schools.3 ASDB uses commonly used practices but can work to improve Given the limitations of test results as a way to gauge ASDB’s success, auditors looked for other ways to assess ASDB’s efforts. One commonly used assessment is the degree to which an organization employs “best practices”—that is, those approaches and techniques that experts, researchers, and practitioners have identified as likely characteristics of a well-performing program. Although scientific research is limited regarding what works in educating deaf and visually-impaired students, experts have identified several commonly used practices. ASDB already follows a number of these practices. However, it should focus additional effort in four key areas in order to promote its students’ success: early intervention, increasing its students’ access to highly qualified teachers, using surveys of its graduates to identify ways to improve its programs, and developing ways to improve students’ scores on standardized tests. Research is limited—Experts in the fields of deaf and blind education have conducted extensive reviews of existing research and concluded that there is not a large body of empirical research to draw upon to establish evidence-based practices.4 This contrasts with research in educating children with more common characteristics such as learning disabilities. However, experts at Gallaudet University, a prominent university for deaf students in Washington D.C., and Perkins School for the Blind in Massachusetts, as well as officials at deaf and blind schools in Texas 1 Gallaudet Research Institute. (n.d.). Literacy & deaf students. Retrieved, July 10, 2012, from http://www.gallaudet.edu/ gallaudet_research_institute/publications_and_presentations/literacy.html 2 Qi, S. & Mitchell, R.E. (2012). Large-scale academic achievement testing of deaf and hard-of-hearing students: past, present, and future. Journal of Deaf Studies and Education, 17(1), 1-18. 3 The results from the other schools are preliminary. The test publisher cautions that because of the small number of test scores included and the lack of stratification, the results should not be used for comparison as part of a decision-making process. The data does represent a set of grade-level averages of test results of students identified as being deaf over a number of years. 4 Luckner, J.L., & Handley, C.M. (2008). A summary of the reading comprehension research undertaken with students who are deaf or hard-of-hearing. American Annals of the Deaf, 153(1), 6-36; and Douglas, G., McCall, S., McLinden, M., & Pavey, S. (2009). International review of the literature of the evidence of best practice models and outcomes in the education of blind and visually-impaired children. Trim, Co. Meath, Ireland: National Council For Special Education. Test results for a small number of schools for the deaf nation-wide suggest that ASDB deaf and hard-of- hearing students’ scores are similar to those of their peers at other schools. page 16 State of Arizona and Florida, shared information with auditors regarding practices they have found to be effective in educating deaf, hard-of-hearing, and visually-impaired students. ASDB follows some commonly used practices—ASDB uses some commonly used practices that are consistent with those identified by the experts and practitioners auditors interviewed although there is a shortage of direct evidence demonstrating practices are effective with deaf and blind students. These include using technology to help students learn and adjusting instruction to meet students’ needs. Specifically: • Technology innovations help students learn—Experts and national deaf and blind advocacy organizations indicated that innovations in technology are vital to student advancement. These technological innovations help improve deaf and blind students’ ability to access information. For example, an official with the National Council of State Agencies for the Blind reported that portable devices with refreshable braille displays enable blind students to keep up with note taking and reading requirements in high school and beyond. Braille-reading devices can allow students to access content as braille text or through conversion to spoken words. Additionally, these portable devices allow blind students’ access to Global Positioning Systems, which can enhance their orientation and mobility skills and increase their independence. Another assistive technology used by other states’ schools is distributed video in every classroom. Distributed video uses smart whiteboards that bring the board content to a screen on the student’s desk, and a swivel hinge allows visually-impaired students to bring the screen as close as the student needs. Other states’ schools also use technology to assist with their outreach programs by providing webinars, online information, and training as well as publications and other resources to professionals and families. ASDB uses technology extensively and has developed a technology plan for fiscal years 2011 through 2014, which identifies several technological innovations to further make student learning a priority. ASDB already uses technology such as computers and smart whiteboards in some classrooms, computers in the library, video telephone booths, and refreshable braille readers. The plan addresses several areas, including Reading/Language Arts, Math, Highly Qualified Teachers, American Sign Language (ASL) and Braille, Parent Involvement, and Technology Literacy. For each area, the plan identifies a technology strategy and an action step with measurable goals. For example, by the 2011-2012 school year, ASDB’s goal was to have all teachers use digital video and other multimedia tools to promote language development, and according to an ASDB official, this has been implemented. Another 2011- 2012 school year goal was that all students in the Arizona School for the Blind would have accounts that would allow them to download library books in braille. According to ASDB, this has also been implemented. Additionally, ASDB plans to use interactive whiteboards for teaching math during the 2012- 2013 school year. ASDB also plans to build an accessible Web site for parents ASDB uses technology such as computers and smart whiteboards in some classrooms, computers in the library, video telephone, and refreshable braille readers. page 17 Office of the Auditor General that will include trainings and tutorials as well as access to student events, grades, and achievements for the 2012-2013 school year. • ASDB modifies instruction and services to student needs—Other states’ deaf and blind school officials indicated that providing expanded curriculum and individualized instruction, and grouping students based on ability level are commonly used practices for educating deaf and/or blind students. At least two other states’ deaf and blind schools have separate programs for students based on their educational progress and needs. Specifically, the Florida School for the Deaf and the Blind has three separate high school programs, including one program that is specifically for students with additional conditions or needs. Similarly, according to officials at the Texas School for the Blind and Visually-impaired, their school has four separate academic programs with curriculum designed to meet students’ needs. These programs include an academic program designed for students functioning within 2 years of their grade level, an academic program designed for students who are not yet reading at the first-grade level, an academic program designed for students functioning more than 2 years below their chronological age, and a basic skills program for students with multiple disabilities who learn best with consistent routines and meaningful functional activities. ASDB serves students with a wide range of needs, including students with multiple disabilities and severe sensory impairments. Similar to public schools, ASDB structures some of its classrooms by age and grade. However, ASDB policy calls for placing students based on the student’s needs, and according to ASDB staff, ASDB avoids assigning students with a wide range of academic needs to the same classroom. Thus, an ASDB class may include students grouped by academic ability level rather than age, and according to ASDB staff, teachers must adapt instruction to meet each student’s individual needs. By policy, ASDB classroom sizes are small, with student-teacher ratios ranging from six to one for multiply disabled students with severe sensory impairments to ten to one for deaf and hard-of-hearing students in elementary through middle school grades. Typical classroom sizes range from five to nine students.1 ASDB also offers separate classes for students with additional disabilities or higher need levels. According to ASDB staff, these classes cover the same content as required by Arizona state standards, but the way in which the teacher provides instruction may be different. For example, teachers may provide reading materials at a lower level, e.g. fifth grade instead of tenth grade, for those who have difficulty with reading comprehension or reading fluency. According to ASDB staff, these classes usually also include a full-time teacher’s aide. 1 Classroom sizes are based on 2011-2012 rosters for the Arizona School for the Blind and the Phoenix Day School for the Deaf and 2012-2013 rosters for the Arizona School for the Deaf. ASDB classroom sizes are small, with student-teacher ratios ranging from six to one for multiply disabled students with severe sensory impairments to ten to one for deaf and hard-of-hearing students. Four areas require attention—Although ASDB follows some commonly used practices, it can strengthen its practices in the following four ways: • Early intervention and education critical to student success—The earlier a child is identified as having a sensory impairment and receives services for the impairment, the faster the child can begin learning the appropriate skills to adjust to his/her environment. Research shows that the critical intervention time for a baby who is deaf or hard-of-hearing is the first 3 years after birth, and that the first 6 months after birth are crucial for the development of communication and language skills.1 According to two other states’ deaf and blind school officials, students who begin receiving specialized educational services late may have missed the critical window when the brain is most able to develop language, and these students may never catch up. Experts indicated that early intervention services along with effective and meaningful parent-child interactions help ensure successful outcomes in areas such as vocabulary development, verbal reasoning skills, and social interaction.2 The story of one ASDB student provided by ASDB staff illustrates the positive outcome of early intervention and parent involvement: ◦ The student began receiving services almost immediately after birth. She transitioned into preschool at Phoenix Day School for the Deaf and is now in seventh grade, where she participates in sports, other activities, and after-school events. In addition to early intervention, ASDB staff reported that the mother’s involvement has played a large role in the student’s success. While her daughter was young, the Spanish-speaking mother began learning sign language. Although the mother is not fluent in ASL, she can communicate with her daughter and is very involved in her daughter’s education by helping with homework and communicating with her daughter’s teachers. The student is progressing academically and scored above the average ASDB student on MAP reading and math tests in second through sixth grades. ASDB provides early intervention services to sensory-impaired students through the AzEIP program (see textbox). The AzEIP program is responsible for providing early intervention services to referred children and their families to support children’s development. ASDB is one of the five 1 The National Agenda (2005). Moving forward on achieving educational equality for deaf and hard-of-hearing students. Retrieved July 10, 2012, from http://www.ceasd.org/agenda/downloads/natl-agenda-2005-04.pdf 2 American Speech-Language Hearing Association. (2008). Service provision to students who are deaf and hard-of-hearing: Birth to 36 months [Technical report]. Rockville, MD: Author; and Douglas, McCall, McLinden, & Pavey, 2009. page 18 State of Arizona Arizona Early Intervention Program (AzEIP) The AzEIP is Arizona’s state-wide, interagency system of supports and services for infants and toddlers with developmental delays or disabilities and their families. AzEIP services include assistive technology devices and services, family training, physical and occupational therapy, vision and audiological assessments, orientation and mobility services, and speech language pathology services. The federal Individuals with Disabilities Education Act, Part C, establishes AzEIP to help provide eligible children and their families access to services to enhance the child’s development. Source: Arizona Department of Economic Security AzEIP Web site. Experts indicated that early intervention services along with effective parent-child interactions help ensure successful outcomes in vocabulary development, verbal reasoning skills, and social interaction. agencies participating in the AzEIP program.1 ASDB’s AzEIP program responsibilities for sensory-impaired children include evaluating and assessing the child’s needs within 45 days of a referral; ensuring that the child has an appropriate Individualized Family Service Plan (IFSP); providing or arranging for the services on the IFSP in the appropriate intensity, frequency, and duration; and participating in the transition and review of the child before the child enters school. In addition, ASDB is responsible for various other tasks such as coordinating with other AzEIP agencies, providing information for a state-wide resource directory, and collecting and reporting data to the Department of Economic Security. Although ASDB complies with the 45-day requirement to assess children’s sensory impairment needs and develop a service plan, it can improve in regards to its other responsibilities within the AzEIP program. Specifically, as part of the referral process, after a child has been identified as having a hearing or visual impairment, ASDB is responsible for assessing the child’s sensory impairment service needs and developing a service plan that will address these needs within 45 days of a referral. According to an AzEIP official, ASDB meets this 45-day time frame requirement. Adhering to this time frame for assessing and then developing a plan to address service needs is important because experts indicate that almost half of newborns who are referred from a hearing screening do not have appropriate followup to determine the presence of a hearing loss and to initiate appropriate early intervention services.2 However, ASDB should continue its efforts to improve in several other aspects of its early intervention program. As of June 2012, ASDB had identified the following areas for improvement: ◦ Coordination with other agencies—According to ASDB staff, it is working with other state agencies that are part of the AzEIP program to collectively establish contract services, such as therapists, that will allow referred children and families to begin receiving services more efficiently. According to ASDB staff, ASDB also plans to work with the Arizona Department of Education and the Department of Economic Security to establish uniform guidelines for the use of services provided to children when they transition out of the AzEIP program and into schools. ◦ Staff training and certification—According to an ASDB official, ASDB plans to provide training to its early intervention program staff to ensure that all staff are using a consistent curriculum for early intervention services when they work with children and families. In addition, ASDB reported it is taking steps to ensure that its staff have a Standards of Practice certification as required 1 The five AzEIP participating agencies identified in A.R.S. §8-652 are the Department of Economic Security, ASDB, the Arizona Department of Health Services, the Arizona Health Care Cost Containment System, and the Arizona Department of Education. The Department of Economic Security oversees the program. 2 American Speech-Language Hearing Association, 2008 page 19 Office of the Auditor General As part of the AZEIP program, ASDB is responsible for assessing the child’s sensory impairment service needs and developing a service plan within 45 days of a referral. by the AzEIP program, through which staff demonstrate the knowledge and skills required to provide early intervention services. ◦ Organizational structure and staffing—According to ASDB staff, in June 2011, ASDB appointed a Tucson official to oversee its early intervention program to help ensure the consistent operation of this program state-wide. Prior to this change, ASDB’s regional cooperatives were responsible for coordinating and managing early intervention services to children and their families, which led to some inconsistencies within ASDB’s early intervention program. In addition, ASDB is planning to revise staff job descriptions to more accurately reflect the services it provides to children and their families within the early intervention program and ensure staff clearly understand their responsibilities. ◦ Outreach to families—ASDB has developed a new Web site that will provide more detailed information to the public about resources available through the AzEIP program as well as descriptions of all the programs provided by ASDB, including early intervention, preschool, and school-age programs. In addition, ASDB reported that it plans to use other outreach mechanisms, including social networking sites such as Facebook, YouTube, and Twitter, to reach more families that might benefit from its services. ◦ Delays in starting services—ASDB plans to work with the Arizona Department of Health Services’ Early Hearing Detection and Intervention program to develop a new system that will allow ASDB staff access to infant screening records so they can help ensure any child who fails a screening receives appropriate assessments and other services in a timely manner. In addition, ASDB is providing training to its staff to ensure any delays in services are accurately reported to the Department of Economic Security. The Department of Economic Security uses this information for federal reporting requirements, to assess the reasons for the delay in services, and to take appropriate action, if necessary. ◦ Compliance with state-wide AzEIP program requirements—The Department of Economic Security’s review of ASDB’s early intervention program for the period July 2010 through June 2011 found that ASDB did not provide the required monthly service data to the Department of Economic Security in a timely manner and included inaccurate information in its database, which generates the monthly service data. The Department of Economic Security uses the monthly service data to satisfy federal government reporting requirements for the AzEIP program. The Department of Economic Security also reported that ASDB’s IFSPs do not always contain measurable goals, which are required by federal regulations. Additionally, the review found that ASDB did not obtain sufficient parent survey responses from one region of the State. This survey is used to help page 20 State of Arizona The Department of Economic Security’s review of ASDB’s early intervention program for the period July 2010 through June 2011 found that ASDB did not provide accurate service data in a timely manner. ensure that parents understand their rights regarding early intervention services and that early intervention services have helped their children learn and develop. ASDB has developed performance improvement plans to address areas of noncompliance with AzEIP requirements. Additionally, ASDB Information Technology department has developed a plan to address issues with the current database, and the official responsible for ASDB’s early intervention program provided training to staff in December 2011 to help address data entry practices. • Highly qualified teaching staff important in subject matter as well as in teaching techniques— The federal Individuals with Disabilities Education Act requires schools to hire highly qualified teachers to teach students categorized as having disabilities (see textbox). In addition, the No Child Left Behind Act requires 100 percent of a school’s teachers of core academic subjects, such as English, mathematics, and reading or language arts, to be highly qualified in the subjects they teach. Therefore, all ASDB teachers must be highly qualified in special education, and those who teach core academic subjects must also be highly qualified in the subjects they teach. Experts agree that students make better progress when their teachers are highly qualified in the subjects they teach, and students with disabilities make better progress when their teachers have specialized skills in educating these students. Most, although not all, ASDB teachers are highly qualified in the subjects they teach, according to ASDB. An Arizona Department of Education report shows that approximately 10 percent of ASDB’s teachers were not highly qualified in hearing-impaired or visually-impaired special education and/or in one or more of the subjects they taught during at least part of the 2011-2012 school year.1 ASDB officials reported that the teachers who do not meet the requirements are working toward 1 ASDB reports its teachers’ status to the Arizona Department of Education on October 1 each year, and according to ASDB, updates the database occasionally throughout the year. As of May 7, 2012, the database showed that ASDB had 20 teachers out of the 213 listed who were not highly qualified in hearing-impaired or visually-impaired special education and/ or one or more of the subjects they were teaching. Because ASDB does not review and completely update the database until closer to October 1, the number of teachers who were not highly qualified as of May 7, 2012, is likely higher or lower than 20. page 21 Office of the Auditor General Highly Qualified Teachers—To be deemed highly qualified, teachers must have a bachelor’s degree and full state certification or licensure, and demonstrate competency in each subject taught. Demonstration of Competency—Teachers can demonstrate subject competency with a major in the subject, credits equivalent to a major in the subject, passage of a state-developed test; a combination of teaching experience, professional development, and knowledge in the subject garnered over time in the profession; an advanced certification from the state; or a graduate degree. No Child Left Behind Act Requirements—The No Child Left Behind Act requires that teachers teaching in core academic subjects in public elementary and secondary school are highly qualified. Individuals with Disabilities Education Act Requirements—The Individuals with Disabilities Education Act requires each person employed as a public school special education teacher to be highly qualified as a special education teacher. Core academic subjects—English, reading or language arts, mathematics, science, foreign languages, civics and government, economics, arts, history, and geography. Source: U.S. Department of Education’s Individuals with Disabilities Education Act Web site. certification through professional development plans. According to ASDB, many of these teachers are responsible for teaching multiple subjects, but may be highly qualified in only one core subject. According to ASDB staff, acquiring and retaining highly qualified staff is difficult for many reasons, including a lack of competitive pay and the challenges associated with obtaining the required certifications. Officials at other deaf and blind schools and ASDB officials identified three possible ways that deaf and blind schools can improve their ability to ensure that its students are taught by highly qualified staff. The first way, used by a private school for the blind in Massachusetts—offering tuition reimbursement and free room and board to teachers while they work to become highly qualified—may not be available to ASDB. The second way, increasing the pool of qualified teachers in the State, which was suggested by a practice in Missouri, is already under way in Arizona. Specifically, ASDB is working with the University of Arizona and reported that it is also working with the Arizona Department of Education to increase the number of hearing-impaired and visually-impaired special education teachers in the State. ASDB is working with a University of Arizona masters-level program that is intended to address the shortage of highly qualified teachers needed to serve students who are deaf or hard-of-hearing in Arizona, the Southwest, and the nation. The federally funded program offers online, full- or part-time courses, and its objective is to graduate 30 teachers who will be eligible for certification in the State of Arizona. A third approach used in Florida may offer additional potential for ASDB to better ensure that its students take classes from highly qualified teachers. Specifically, the Florida School for the Deaf and the Blind offers its students the opportunity to take off-campus classes online. Experts have noted that Web-based courses can make courses available to students without having to employ highly qualified full-time teachers in each subject in each school, and that states are choosing to implement online programs for special education students in order to provide highly qualified teachers in subjects where highly qualified teachers may be lacking.1 According to ASDB officials, they are considering using online education classes to offer ASDB students an opportunity to learn from highly qualified subject matter experts, with local assistance from teachers who are highly qualified to teach sensory impaired students. In fact, ASDB tentatively plans to enroll two Arizona School for the Deaf students in math classes at an Arizona online charter school in the 2012- 2013 school year. ASDB should continue to seek opportunities, such as alternative delivery classes, to increase its students’ access to highly qualified teachers. 1 Thomas, W.R. (2002). Funding web-based courses for K-12 students to meet state educational goals. Atlanta, GA: Southern Regional Education Board, and Muller, E. (2009). Serving students with disabilities in a state-level virtual K-12 public school programs. Alexandria, VA: National Association of State Directors of Special Education, Inc. page 22 State of Arizona ASDB is working with a University of Arizona masters-level program intended to address the shortage of highly qualified teachers needed to serve students who are deaf or hard-of-hearing. • Post-graduation tracking can help determine student progress and identify areas that need improvement—Although ASDB has information about students’ status 1 year after graduation, according to ASDB staff, it does not compile or analyze the information and use it to improve its programs. Officials at one state’s deaf and blind schools and a private school stated that tracking students after graduation is necessary to identify areas in the curriculum that could use improvement. For example, a state school for the deaf collected information from its high school graduates to help determine why so many students were dropping out of post-secondary education programs. The information gathered indicated students did not feel prepared for dormitory life. This enabled the school to restructure its high school program so seniors now live in a dormitory setting that allows students to focus on living skills that are required to succeed in college. Similarly, ASDB could use the information collected from recent graduates to improve its programs. Some comments received from seven students and families who responded to auditors’ survey of 2011 ASDB graduates suggest ways ASDB can consider modifying its programs. Specifically: ◦ Providing additional homework help; ◦ Offering group living to facilitate life skills development; ◦ Increasing opportunities to gain work and volunteer experiences; and ◦ Introducing vocational programs earlier. ASDB should use the results of its surveys of students after graduation to measure student progress and to identify and implement enhancements to its students’ educational programs. • Studying differences in test scores can help narrow achievement gaps— The No Child left Behind Act requires that all public schools in every state work towards narrowing achievement gaps in the state as measured by each state’s standardized test, which in Arizona is AIMS. Studying differences in AIMS passing rates across ASDB locations may hold some immediate assessment benefits for ASDB. AIMS test results varied between ASDB’s on-campus students and students receiving services from ASDB’s regional cooperatives and among the regional cooperatives. At ASDB’s three campus schools, 74 to 94 percent of students’ spring math and reading scores fell below standards. According to ASDB officials, on-campus students have higher needs because of their disabilities, which may contribute to lower test scores. For example, some students have secondary physical disabilities that cause absences from school or learning disabilities that cause additional delays in development. However, failure rates also varied between the five regional cooperatives. Students at four of the five regional cooperatives passed AIMS at a higher rate page 23 Office of the Auditor General AIMS test results varied between ASDB’s on-campus students and students receiving services from ASDB’s regional cooperatives and among the regional cooperatives. than the on-campus students. For example, only 49 percent of the Eastern Highlands regional cooperative students failed the reading test, compared to 77 percent of students in the Southwest regional cooperative where results were more similar to results for the on-campus students. Examining these variances more closely may help ASDB officials determine if teaching practices vary among locations and whether higher scores in some regions reflect more effective practices that should be implemented across all locations. To increase student achievement at all the regional cooperatives and on-site campuses, ASDB should determine the reasons for variations in test scores and identify potential solutions to improve test results, and establish expectations that each campus and regional cooperative will implement best practices to improve test performance. Recommendations: 1.1 To help ensure that children with sensory impairments receive needed services, ASDB should improve its early intervention program by continuing to take the following steps: a. Coordinating with other state agencies that are part of the AzEIP program to establish contract early intervention services throughout the State; b. Working with the Arizona Department of Education and the Department of Economic Security to establish guidelines for the use of services provided to children when they transition out of the AzEIP program and into schools; c. Providing staff training to ensure that all staff use a consistent curriculum for early intervention services and that staff accurately reflect delays in starting services in ASDB’s database; d. Ensuring that its early intervention program staff have Standards of Practice certifications as required by the AzEIP program; e. Revising the early intervention staff job descriptions to ensure they accurately reflect the services ASDB provides to children and their families; f. Providing more detailed information to the public about resources available through the AzEIP program, as well as descriptions of all the programs provided by ASDB, including early intervention, preschool, and school-age programs through its newly developed Web site; page 24 State of Arizona page 25 Office of the Auditor General g. Using other outreach mechanisms, including social networking sites such as Facebook, YouTube, and Twitter, to reach more families who might benefit from its services; h. Working with the Arizona Department of Health Services’ Early Hearing Detection and Intervention program to develop a new system that will allow ASDB staff access to infant screening records so they can help ensure any child who fails a screening receives appropriate assessments and other services in a timely manner; and i. Addressing areas identified in the Department of Economic Security’s review of ASDB’s compliance with state-wide AzEIP requirements. Specifically, ASDB should: ◦ Provide accurate and timely monthly service data to the Department of Economic Security; ◦ Improve staff data entry practices and establish procedures that will help ensure the accuracy of data in ASDB’s database and the monthly service reports that are generated from the database; ◦ Ensure that Individualized Family Service Plans contain appropriate and measurable goals as required by the AzEIP program; and ◦ Encourage families to complete and return early intervention surveys. 1.2 ASDB should continue to seek opportunities, such as alternative delivery classes, to increase its students’ access to highly qualified teachers. 1.3 ASDB should establish a process for compiling, analyzing, and using information obtained from surveys about its students after graduation to measure student progress, and to identify and implement enhancements to its students’ educational programs. 1.4 To narrow the achievement gaps and increase the AIMS passing rate among its students, ASDB should: a. Determine the reasons for variations in test scores and identify potential ways to improve test results at the campuses and the regional cooperatives; and b. Establish expectations that each campus and regional cooperative will implement best practices to improve test performance. page 26 State of Arizona ASDB should examine regional cooperative program service fees and strengthen program operations FINDING 2 page 27 ASDB provides services to students at local school districts through its regional cooperative program As authorized by the Legislature, ASDB established a regional cooperative pilot program and started providing services in 1988, and continues to operate five regional cooperatives throughout the State (see Figure 1, page 4). The intent of this program was to enable local school districts to serve deaf, hard-of- hearing, and visually-impaired students in a cost-effective way by providing access to a pool of specially trained personnel and equipment provided by ASDB (see textbox). Regional cooperative staff travel to the participating school districts and provide services to students as they remain at the schools in their districts. Each regional cooperative has a director who supervises operations and various staff, including teachers, interpreters, other specialists, and administrative staff who work with the member districts’ sensory-impaired students. Regional cooperative teachers consult with school staff, classroom teachers, parents, and students; provide specialized direct instruction to students in their areas of need; prepare curricular modifications and demonstrate techniques for working with individual students; and assist in student placement by serving as members of the team that develops each The Arizona State Schools for the Deaf and the Blind (ASDB) should take several steps to improve the operation of its regional cooperative program, including more fully recovering program costs and strengthening the oversight and management of the program. ASDB’s regional cooperative program serves as a resource for school districts by providing staff and equipment to sensory-impaired students in member districts. However, some fees charged by the five individual regional cooperatives are inconsistent, and even for the fees that are consistently applied across all cooperatives, ASDB has no method to determine whether the fees cover the cost of the services provided. In addition to developing a structured approach to review and establish appropriate fees, ASDB needs to review the regional cooperatives’ advisory councils’ role in fee-setting. Additionally, ASDB should develop a consistent way for regional cooperatives to keep track of student needs and staff availability, as well as equipment provided for students’ use. Office of the Auditor General Overview of the regional cooperative pilot program • Provide educational programs and related services to all sensory impaired students if the school participates in cooperative program and cannot provide an appropriate placement for the student within the district. • Provide supplemental services such as audiological and vision assessments, specialized curriculum materials and equipment, and district staff development assistance. • Costs to be paid by participating schools. • Locations decided by ASDB in consultation with districts. • Each cooperative advised by committee, including parents, districts, and local private service organizations. Source: Auditor General staff summary of Laws 1987, Ch. 363, §19. student’s Individualized Education Program (IEP) (see textbox, page 11). Each regional cooperative also has an advisory council that provides advice on the administration of the cooperative. About half of Arizona’s school districts participate in the regional cooperative program. School districts and other public schools, such as charter schools and Bureau of Indian Affairs (BIA) community schools, must sign agreements with ASDB if they elect to receive educational services for their deaf, hard-of-hearing, and visually-impaired students through the regional cooperatives. In the 2011-2012 school year, 114 school districts and 33 other entities participated in the regional cooperatives. In the 2011-2012 school year, the regional cooperatives served 1,190 students in these districts. According to a representative from a participating school district, it benefits the district to participate in the regional cooperative because of the costs the district would incur to obtain the services for its students elsewhere. Many of the 116 districts in the State that did not participate in the regional cooperatives fell into two groups: (1) large districts such as the Mesa, Chandler, and Tucson districts, which use their own staff to provide these services, and (2) very small districts that do not have any sensory-impaired students. The regional cooperatives receive monies from various funding sources to pay for the services provided to the students in the program. Funding sources for services provided to students by ASDB’s regional cooperatives include the following: • State General Fund appropriations—ASDB receives an annual appropriation from the State General Fund for the regional cooperative program. In fiscal year 2013, the appropriation for the program was $803,500. According to ASDB staff, ASDB uses these appropriations to cover its administrative costs for the regional cooperatives, such as office rent, utilities, and administrative staff salaries and employee-related expenses. • School district membership fees—Each participating school district pays a per-district membership fee to belong to a regional cooperative. The membership fees are used to provide services that are not tied to a particular child’s IEP and include assistance with screening and identification of children, evaluation of children, and training of school staff. • Special education vouchers—ASDB receives voucher monies from the Arizona Department of Education for many of the regional cooperative students it serves. Generally, ASDB receives voucher monies for students whose primary disability is a hearing or vision impairment, while districts receive voucher monies for students with multiple disabilities (see textbox, page 3, for the definition of multiple disabilities). ASDB and districts work together to decide which entity will receive the voucher monies. The amount of the educational voucher is determined by statute, and as of the 2011-2012 school year, the education voucher was $19,122 for visually-impaired students and $19,008 for deaf and hard-of-hearing students. If ASDB receives voucher monies for a page 28 State of Arizona In the 2011-2012 school year, 114 school districts and 33 other entities participated in the regional cooperatives. student, ASDB pays an amount to the district to reimburse the district for services the district provides to the student such as transportation and any educational services not provided by the regional cooperative. • Fees for services—When a district rather than ASDB receives the Arizona Department of Education voucher payment, it pays ASDB a fee for the services ASDB provides. These services include ongoing assessment, instruction, provision of equipment, and vocation counseling. The costs for services under the fee-for-service structure depend on the number of hours of service provided. As shown in Table 3 (see page 30) the levels of service, types of service, and annual fee-for-service charges are uniform among all the cooperatives. For example, districts pay $5,245 per year for a student who receives 1.5 hours of services per week, and $21,500 per year for a student who receives 10 to 15 hours of services on a weekly basis. • Additional fees—ASDB receives additional monies from districts for services that exceed what is outlined in the membership fees or fee-for-service amount. For example, one regional cooperative’s membership fee for districts with less than 200 students covers 26 audiological assessments. If the number of assessments exceeds 26, the cooperative charges $20 for each additional audiological assessment. ASDB should examine fees for regional cooperative program services ASDB should systematically examine the various fees that regional cooperatives charge, make them more consistent, and ensure that they cover the costs of the services provided. Although fee-for-service amounts and services are consistent state-wide, many other fees or payments, such as membership fees paid by participating districts and reimbursements paid to districts for transportation and other services, are not consistent across regions. These inconsistencies create inequities in which some districts’ payments subsidize other districts’ expenses. Further, ASDB has not determined if the fees paid by districts for services provided by regional cooperative staff cover the costs of the services. To resolve these issues, ASDB needs to develop and implement a structured approach for determining appropriate fees and payments. Membership and supplemental fees charged to districts and payments to districts are inconsistent—ASDB has not established policies or procedures for determining appropriate membership and supplemental fees for school districts that participate in regional cooperatives or for determining the amounts that the regional cooperatives should pay to reimburse district schools for services provided to students for whom ASDB receives voucher monies. Instead, each regional cooperative has developed its own membership fee and list of services included in the membership fee. In addition, some have established page 29 Office of the Auditor General Each regional cooperative has developed its own membership fee and list of services included in the membership fee. page 30 State of Arizona 1 The rates for BIA community schools are set by a separate process. 2 According to a regional cooperative official, a teacher typically provides an itinerant level service a couple of times a week and the student receives visually-impaired or hearing-impaired services that complement the district teacher’s teaching curriculum. 3 According to a regional cooperative official, a teacher provides a resource level service on a daily basis. The student might receive some services from the district teacher. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Table 3: Services Provided by Regional Cooperative Staff School Year 2011-2012 Service Level Type of Service Hours of Service Service per Year1 Consult indirect Limited direct service 1 hour a month and no equipment provided $2,800 Consult direct Ongoing assessment and limited service 1 hour a month and no equipment provided 2,800 Level I itinerant2 Ongoing assessment, specialized and direct instruction, provision of equipment, vocational counseling, and direct service to student in amplification use and listening skill training. 1.5 hours a week 5,245 Level II itinerant2 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary, speech development, math, science, and social studies. Direct service to student in amplification use, listening skill training, and sign language instruction. 1.5-3 hours a week 7,335 Level III resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary, speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, and sign language instruction. 3-5 hours a week 11,000 Level IV resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary and speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, sign language instruction, sign language interpreting and note taking, and classroom tutor or instructional aide. 5-10 hours a week 16,260 Level V resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary and speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, sign language instruction, sign language interpreting and note taking, and classroom tutor or instructional aide. 10-15 hours a week 21,500 additional supplemental fees. Further, each regional cooperative has determined an amount it reimburses to districts for services the districts provide to students for whom ASDB receives Arizona Department of Education voucher monies even though the voucher amount is the same regardless of which district or cooperative the student comes from. The resulting amounts vary, and the regional cooperatives explained that the differences have historically existed. Specifically: • Membership fees differ among regions—The regional cooperatives use different tiers for grouping districts and setting membership fees based on student population within the district (see Table 4). For example, the Southwest regional cooperative has three tiers—districts with less than 500 students, 501 to 2,000 students, and more than 2,000 students, with different fees for each tier. However, the Eastern Highlands regional cooperative has five tiers ranging from under 200 students to 5,000 or more students, again with different fees for each tier. Additionally, as shown in Table 4, the range of membership fees charged by each regional cooperative for their various tiers differs for each regional cooperative. For example, the fees for the tier comprising the regional cooperative’s smallest districts ranged from $240 to $500 in 2011- 2012. • Services covered by membership fees differ among cooperatives, and some cooperatives charge supplemental fees for services, while others do not—The services covered by the membership fees also differ from region to region. As a result, some regional cooperatives charge additional fees for supplemental services that some regional cooperatives include in the set of services covered by the membership fees. For example, although most regional cooperatives include district staff training in the membership fees, one regional cooperative charges a separate fee of $25 per person to train district staff in conducting hearing screenings. Another regional cooperative has additional charges for certain services if they are provided to fee-for-service students—$71 per hour for mobility and orientation training and $45 per hour for interpreter services. The other three cooperatives have not established similar additional charges and instead provide these types of services with no additional charges. • Reimbursements to districts inconsistent and not based on actual services provided by districts—As previously mentioned, ASDB receives voucher monies for students whose primary disability is a hearing or vision impairment and for whom ASDB assumes the primary educational responsibility within page 31 Office of the Auditor General Table 4: Regional Cooperative Membership Fee Structure School Year 2011-2012 1 Bureau of Indian Affairs community schools. Source: Auditor General staff analysis of data received from ASDB’s regional cooperatives. Membership fees and supplemental fees differ among regions— Regional Cooperative Number of Tiers Membership Fee Southeast 4 $260 up to $820 Desert Valleys 5 $280 up to $1,000 North Central BIA Schools1 4 5 $240 up to $720 $280 up to $1,000 Eastern Highlands 5 $280 up to $1,000 Southwest 3 $500 up to $1,250 these students’ home school districts. If ASDB receives voucher monies for a student, ASDB reimburses the district for services the district provides to the student such as transportation and any educational services not provided by the regional cooperative. However, the regional cooperatives have set differing reimbursement amounts, even though the voucher amount is the same regardless of which district or cooperative the student comes from. As shown in Table 5, the per-student amounts range from no reimbursement for students in the southeast region who are receiving specified services from the regional cooperative to $3,500 annually for all students in two regions and for some students in a third region. The cumulative result of these inconsistencies can be substantial. Table 6 (see page 33) shows examples of the differences that can result, both for voucher students—those students for whom ASDB receives the Arizona Department of Education voucher— and for those students for whom the district receives the voucher. For a visually-impaired voucher student, regional cooperative 1 receives a membership fee of $260 and reimburses the district $2,000, while regional cooperative 2 receives a membership fee of $500 and reimburses the district $3,500. As a result, regional cooperative 1 has $17,382 to cover the students’ costs, while regional cooperative 2 has $16,122 to cover the students’ costs, which is a difference of $1,260. For fee-for-service students, the difference is less—$240—because it relates only to the membership fee. As a result, the regional cooperatives can have a different amount of monies per student to cover the same services. ASDB unable to determine if fees are adequate to cover costs— ASDB policy requires its regional cooperatives’ advisory councils to recommend a fee structure that will cover all costs of services that are not covered by legislative appropriations. However, ASDB has not evaluated whether the membership fees and fee-for-service amounts cover the costs of the services provided to regional cooperative students. Although ASDB does not have records of past fees, officials at three regional cooperatives reported that fees had not been changed since at least 2007, and a fourth regional cooperative official said fees had not been changed since before 1997. The fifth regional cooperative official reported that fees had not changed since before 1992. Additionally, in fiscal year 2011, the ASDB Cooperative Services Fund’s fund balance decreased, suggesting the fee amounts may be inadequate to cover costs. Two regional cooperatives prepare a report that shows some of the revenues and expenses associated with a district’s students if the district requests the report. However, because this report does not include all revenues and expenses and is prepared by only two of the five regional cooperatives, ASDB cannot rely on it to determine whether its fees are adequate. page 32 State of Arizona Table 5: Regional Cooperative Reimbursements to Districts School Year 2011-2012 1 Without educational interpreter, instructional assistant, orientation and mobility, and rehabilitation therapy. 2 With educational interpreter, instructional assistance, orientation and mobility, rehabilitation therapy, and educational transcriber. 3 If the student receives services involving two or more staff. 4 If a full-time sign interpreter or instructional assistant is assigned to the regular education classroom. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Regional Cooperative Reimbursement Amounts Per Voucher Student Southeast $2,0001 or $02 Desert Valleys $3,500 North Central $2,700 or $1,5003 Eastern Highlands $3,500 Southwest $3,500 or $1,5004 ASDB has not evaluated whether the membership fees and fee-for-service amounts cover the costs of the services provided to regional cooperative students. page 33 Office of the Auditor General Although regional cooperatives independently establish their fees, ASDB combines the revenues to pay regional cooperatives’ costs throughout the State. For example, at the end of fiscal year 2011, the North Central regional cooperative’s portion of the ASDB Cooperative Services Fund’s fund balance was negative $129,118, while the Desert Valleys regional cooperative’s portion was more than $2.5 million. ASDB used monies from the Desert Valleys regional cooperative to purchase 29 cars for all of the regional cooperatives and to pay for a conference attended by all regional cooperative staff. In effect, Desert Valleys regional cooperative member school districts were subsidizing other regional cooperatives’ expenses. ASDB needs to adopt structured approach to establishing fees—To address the problems regarding inconsistencies in some fees and to determine whether its fees cover the cost of services provided, ASDB needs to adopt a structured fee-setting approach. A structured approach would allow ASDB to evaluate current fees and propose new fees that would (1) fully cover costs related to the services provided by the regional cooperatives and (2) ensure that specific fees are appropriate for the services. Mississippi’s Joint Legislative Committee on Performance Evaluation and Expenditure Review (PEER) developed an approach for evaluating and setting fees that may assist ASDB.1 PEER’s approach consists of a decision model for establishing or increasing government fees, called the Theory of Fee Setting in Government, as well 1 Joint Legislative Committee on Performance Evaluation and Expenditure Review. (2002). State agency fees: FY 2001 collections and potential new fee revenues. Jackson, MS: Author. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Table 6: Example of Differences in Regional Cooperative Service Costs School Year 2011-2012 Voucher Students Regional Cooperative 1 Regional Cooperative 2 Difference Membership fee $ 260 $ 500 $ 240 Voucher 19,122 19,122 0 Reimbursement to district (2,000) (3,500) (1,500) Total per student $17,382 $16,122 $1,260 Fee-For-Service Students Regional Cooperative 1 Regional Cooperative 2 Difference Membership fee $ 260 $ 500 $240 Level II itinerant 5,245 5,245 0 Total per student $ 5,505 $5,745 $ 240 Although regional cooperatives independently establish their fees, ASDB combines the revenues to pay regional cooperatives’ costs throughout the State. page 34 State of Arizona as guidance on implementing new fees.1 Figure 3 (see page 35) summarizes key concepts from PEER’s approach. ASDB’s approach should include the following: • Assessing efficiency of operations—ASDB should assess the efficiency of its operations to ensure costs are as low as possible and document the results of its assessment. As ASDB assesses the efficiency of its operations, it should seek to minimize costs where possible. • Developing a cost-accounting method—ASDB should determine whether to consider costs independently for each regional cooperative or in combination on a state-wide basis, and then develop and finalize a method for tracking and allocating relevant service and equipment costs. • Developing fees based on relevant costs—To help ensure fees are appropriate and equitable, ASDB should identify the actual costs for specific services for which fees are charged. In addition, fees should take into account factors that affect the cost of the specific service. For example, some regional cooperatives may incur more costs for transportation because these regional cooperatives must transport staff and/or equipment greater distances to reach the districts they serve. Once developed, ASDB should use this approach to assess its fees and establish new fees. If the new fees are substantially higher than existing fees, ASDB should consider phasing in the increases over time. In addition, ASDB should develop and implement policies and procedures for using the method to establish new fees. ASDB should also develop a systematic way to determine whether and how much to pay school districts for services provided to students for whom ASDB receives Arizona Department of Education voucher monies. ASDB is not statutorily required to pay the districts, but it has historically chosen to reimburse the districts for transportation and other services. If ASDB continues its practice of reimbursing districts, it should develop a process to determine the appropriate amount of these reimbursements. Regional councils’ role in fee-setting needs review As ASDB develops a structured approach to setting fees, it also needs to examine the fee-setting role played by the regional cooperatives’ advisory councils. Each regional cooperative has an advisory council comprising representatives from 1 According to PEER, the approach was based on a review of academic literature, economics theory, and policies and procedures from various states and the United States and Canadian governments. If ASDB continues its practice of reimbursing school districts, it should develop a process to determine the appropriate amount of these reimbursements. page 35 Office of the Auditor General Figure 3: Mississippi Joint Legislative PEER Committee’s Structured Fee-Setting Process Developed for State Government Source: Auditor General staff analysis of fee-setting model. Determine whether fees or taxes should fund the service Who benefits from the service: individuals, the public, or both? Fees should finance services that benefit individuals. Taxes should finance services that benefit the public. When both individuals and the public benefit from a service, financing can come from both fees and taxes. Identify and analyze legal issues Are fees limited by statute? If so, is legislation required to change them? Should administrative rules be revised? Identify the fees’ purpose Should fees cover the cost of providing the service? Should fees be set to influence behavior? Should fees be set to encourage compliance with program regulation and goals? Assess factors influencing fee amount What effect will fees have on those who pay them? What effect will fees have on annual revenue? What do similar states charge for the service? Will the public accept the fees’ necessity? Is the Department subsidizing other government operations? Determine appropriate methodology for setting fees Determine if there is a comprehensive cost accounting system. Seek to reduce costs as much as possible. Measure direct and indirect costs of the time staff spends in service activities. Determine economic impact on regulated entities. Implement fees Obtain amended legislation and regulation as needed. Prepare those who pay fees for changes by providing advanced notice and explaining the purpose and reasoning of new fees. Train staff to answer questions regarding the new fees. Periodically assess revenue, costs, and program outcomes to update fee amounts page 36 State of Arizona participating public schools and ASDB staff. These councils have taken on the responsibility of approving membership fees and reimbursement amounts. Although neither statute nor ASDB policy requires an advisory council vote to increase a regional cooperative’s membership fee or propose other changes in reimbursement amounts, at least four of the regional cooperatives rely on the council members’ vote to do so. The regional cooperatives hold annual or more frequent meetings with their respective advisory councils. During these meetings, regional cooperative staff inform council members of the regional cooperative budget, the number of participating member districts, and any other important information affecting the operations of the regional cooperative. For two reasons, setting fees appears to be an inappropriate role for the advisory councils. First, it goes beyond the role for these councils established in ASDB policy. Under ASDB policy, the councils’ duties include making recommendations about membership fees, but not actually setting them. By allowing the councils to determine fees instead of providing recommendations for ASDB management to consider, regional cooperative staff appear to have misinterpreted the advisory council’s responsibilities. Second, in the kind of structured fee-setting approach discussed earlier, the information needed to set appropriate fees would be most fully available to management, and not the advisory councils. ASDB should modify its policy to remove the provision that advisory councils will recommend a membership fee structure. Additionally, ASDB should determine whether to continue with advisory councils for each regional cooperative or if it would be more appropriate to establish a single advisory council for all of the regional cooperatives. There is no statutory or regulatory requirement for the regional cooperatives to have advisory councils. Instead, the advisory councils have evolved over time from earlier requirements outlined in the regional cooperative pilot program legislation, which is no longer in place. Specifically, when the regional cooperative pilot program was introduced in 1987, legislation required ASDB to establish an advisory committee for each cooperative to provide advice on the administration of the cooperative. The advisory committee included parents, representatives from local private organizations that provide services to the sensory impaired, and representatives of participating school districts. Current regional advisory councils do not include parents or representatives of local private service organizations. The administrative duties outlined in ASDB policy regarding advisory councils could be addressed by a state-wide council that includes representatives of participating districts throughout the State and could also include parents and representatives of private service organizations. Alternatively, ASDB could consider retaining the regional advisory councils but include parents and service organization representatives, similar to the way the councils were envisioned in the pilot program legislation. ASDB should determine and implement the appropriate structure for the advisory councils. Regardless of how the advisory councils are structured, ASDB management needs ASDB should determine whether to continue with advisory councils for each regional cooperative or if it would be more appropriate to establish a single advisory council for all of the regional cooperatives. page 37 Office of the Auditor General to provide oversight to ensure council authority in practice does not exceed the advisory role outlined in ASDB policy. Regional cooperatives are not consistent in tracking resources Regional cooperatives do not use the same systems to track resources, meaning that students potentially may not receive the resources they need. Specifically: • Education staff services—The regional cooperatives use different systems to keep track of student needs, such as the need for an educational interpreter, and the availability of educational staff such as interpreters, teachers, and therapists. Each regional cooperative uses either a computer program such as Microsoft Access or hard copy forms to manage its student caseload. • Equipment inventory—The regional cooperatives use different systems to track and manage the educational equipment provided to the students in local school districts, such as braille writers and educational books. Some regional cooperatives use a hard copy inventory tracking system, and others do not use an inventory tracking system at all. Additionally, ASDB lacks a state-wide inventory system to track equipment owned by all of its regional cooperatives. Consistent tracking could help ensure that students have access to all the educational services and equipment they need. For example: • Providing resources when students move across cooperatives—Regional cooperative officials reported that students move from one cooperative to another, and because the regional cooperatives do not use the same computer programs to track educational services, it can be hard to transfer the services to the student’s new school. • Identifying underused resources—Because there is no state-wide inventory of equipment resources, some regional cooperative officials reported that they informally call one another if they need equipment that might not be in use by another regional cooperative. Without a state-wide inventory, equipment could be going unused in one region that could benefit students in other regions. • Training staff—Regional cooperative officials also stated that a single computer program used by all regional cooperatives would make it easier to train staff and IT support staff on how to use the program to track these services. ASDB could use systems it has developed or is developing for other programs to better track and manage educational services and equipment in the regional cooperatives. Specifically: ASDB lacks a state-wide inventory system to track equipment owned by all of its regional cooperatives. page 38 State of Arizona • ASDB is developing an in-house computer program to track and manage the schedules of the education services in the birth to 3 program discussed in Finding 1 (see pages 9 though 25). This program administers these services throughout the State. Once developed and tested, the regional cooperatives should use this program to track and manage the educational services that are provided to students by the regional cooperatives. • As of January 2012, ASDB purchased and implemented an inventory system to track on-campus assets. The regional cooperatives should use this same system to track and manage the inventory of the equipment provided to the students in the regional cooperatives. Recommendations: 2.1 To ensure its fees more fully reflect its costs, ASDB should develop a structured approach to evaluate current fees and implement new fees that would cover all costs related to the services provided by the regional cooperatives that are not covered by legislative appropriations, and ensure that specific fees are appropriate for the services. In developing this approach, ASDB should do the following: a. Assess the efficiency of its operations to ensure costs are as low as possible and document the results of its assessment. As ASDB assesses the efficiency of its operations, it should continue seeking to minimize costs where possible; b. Determine whether to consider costs independently for each regional cooperative or in combination on a state-wide basis, and develop and implement a method for tracking and allocating relevant ASDB costs; c. Identify the actual costs for specific fees, including membership fees, fee-for- service costs, and additional supplemental service costs to help ensure fees are appropriate and equitable. In addition, fees should take into account factors that affect the cost of the specific service; and d. Develop and implement policies and procedures for using the method to develop appropriate fees. 2.2 ASDB should develop a systematic way to determine whether and how much to pay school districts for services the districts provide to students for whom ASDB receives Arizona Department of Education voucher monies. page 39 Office of the Auditor General 2.3 ASDB should provide more oversight to ensure that advisory councils play an appropriate role in the regional cooperative program by: a. Modifying its policy to remove the provision that advisory councils will recommend a fee structure; b. Determining and implementing the appropriate structure for the advisory councils, such as a single state-wide advisory council composed of participating school district representatives and parents and representatives of local private service organizations, or regional advisory councils that include parents and local private service organization representatives; and c. Ensuring that its councils adhere to the advisory role and responsibilities outlined in ASDB policy. 2.4 ASDB should establish a single, consistent system for managing and tracking regional cooperative resources by: a. Expanding the use of ASDB’s in-house computer program for tracking and managing birth to 3 program educational services, once the program has been developed and tested, to track and manage educational services provided to students in the regional cooperatives; and b. Using ASDB’s inventory system for on-campus assets to track and manage the inventory of equipment provided to students in the regional cooperatives. page 40 State of Arizona ASDB needs to improve its information technology practices FINDING 3 page 41 IT systems used extensively and contain sensitive data ASDB uses IT systems extensively. In addition to administrative functions such as payroll and accounting, ASDB’s IT systems track and monitor student information, including personal and medical information regarding students’ hearing or vision impairments and other disabilities, as well as information about their educational performance. This type of information is private, and two federal laws—the Health Insurance Portability and Accountability Act (HIPAA) and the Family Educational Rights and Privacy Act (FERPA)—require that such information be protected against unauthorized disclosure. ASDB also uses IT in the classroom as part of a variety of adaptive teaching and learning techniques for its students with hearing and vision impairments. ASDB’s IT department, which consists of nine full-time employees, supports technology for staff and students across its three campuses and five regional cooperatives. This support includes administering and supporting user workstations, servers, assistive and educational technologies, critical business applications and services, and various other network devices. ASDB has not addressed critical IT weaknesses State information security experts and auditors have identified critical weaknesses in ASDB’s IT environment. These weaknesses have persisted for several years. In 2009, ASDB engaged ADOA’s Information Security group to assess ASDB’s information technology environment. The resulting report identified several gaps between the existing state of ASDB’s IT environment and best practices and made recommendations for how ASDB should address the deficiencies. The report also found that ASDB’s IT department had received little strategic direction or oversight from ASDB management. Although ASDB has made some efforts to address the weaknesses ADOA identified, auditors found that critical IT weaknesses still exist. Specifically, auditors found that ASDB’s IT security controls are weak, its disaster recovery planning is inadequate, its data backup strategy is flawed, and it has no data classification process to help ensure that the information it maintains is sufficiently protected. ASDB officials stated that the IT department lacks the Although the Arizona State Schools for the Deaf and the Blind (ASDB) has made some improvements to its information technology (IT) practices since the Arizona Department of Administration (ADOA) assessed those practices in 2009, several additional improvements are needed to help ensure that student and school information is properly safeguarded. Specifically, auditors found that continued weaknesses have led to critical vulnerabilities in several IT areas. These include IT security management, disaster recovery, and data backup. ASDB should first prioritize and then correct these IT weaknesses to minimize the impact these vulnerabilities and security threats could have on its operations. Office of the Auditor General page 42 State of Arizona staff and resources necessary to address all of ASDB’s IT issues and needs. As a consequence, it has not prioritized addressing the recommendations made in ADOA’s report, which has contributed to the lack of progress in addressing and resolving weaknesses. In October 2011, the department had turnover in a key position when ASDB’s IT Director left. At that time, the existing IT security specialist was made the interim IT Director but also retained his responsibilities for security. As of June 2012, ASDB had still not appointed a permanent IT Director. IT security controls are weak—ASDB’s controls over IT security are weak, and its systems are susceptible to attack. According to IT standards and best practices, effective security management helps protect IT assets and minimizes the impact that security vulnerabilities and incidents could have on IT operations. Security monitoring is also essential to help ASDB comply with federal laws and regulations designed to protect sensitive information, financial aid records, and health information. Despite the critical importance of effective IT security management, auditors found weaknesses in several key areas, including ASDB’s efforts to assess risk and monitor its systems, secure its net
Object Description
TITLE | Performance Audit & Sunset Review Arizona State Schools for the Deaf and the Blind |
CREATOR | Office of the Auditor General |
SUBJECT | Arizona State School for the Deaf and the Blind--Auditing; Arizona State School for the Deaf and the Blind--Evaluation; Deaf--Education--Arizona; Blind--Education--Arizona; |
Browse Topic |
Government and politics Education |
DESCRIPTION | This title contains one or more publications |
Language | English |
Publisher | Office of the Auditor General |
Material Collection | State Documents |
Acquisition Note | Report No. 12-05 |
Source Identifier | LG 6.2:R 36/ 2012-05 |
Location | o847765923 |
REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library |
Description
TITLE | Arizona State Schools for the Deaf and the Blind |
DESCRIPTION | 76 pages (PDF version). File size: 1932 KB |
TYPE |
Text |
Acquisition Note | Report No. 12-05 |
RIGHTS MANAGEMENT | Copyright to this resource is held by the creating agency and is provided here for educational purposes only. It may not be downloaded, reproduced or distributed in any format without written permission of the creating agency. Any attempt to circumvent the access controls placed on this file is a violation of United States and international copyright laws, and is subject to criminal prosecution. |
DATE ORIGINAL | 2012-09 |
Time Period |
2010s (2010-2019) |
ORIGINAL FORMAT | Born Digital |
Source Identifier | LG 6.2:R 36 |
Location | o811354564 |
DIGITAL IDENTIFIER | 12-05_Report.pdf |
DIGITAL FORMAT | PDF (Portable Document Format) |
REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library. |
File Size | 1978283 Bytes |
Full Text | A REPORT TO THE ARIZONA LEGISLATURE Debra K. Davenport Auditor General Performance Audit and Sunset Review Arizona State Schools for the Deaf and the Blind Performance Audit Division September • 2012 REPORT NO. 12-05 The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the opera-tions of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Audit Staff Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov Dale Chapman, Director Shan Hays, Manager and Contact Person Kori Minckler, Team Leader Jay Rasband Beth Rensvold Richie Rinaldi Representative Carl Seel, Chair Representative Tom Chabin Representative Justin Olson Representative David Stevens Representative Anna Tovar Representative Andy Tobin (ex officio) Senator Rick Murphy, Vice Chair Senator Andy Biggs Senator Rich Crandall Senator Linda Lopez Senator David Lujan Senator Steve Pierce (ex officio) ASDB uses standardized tests to assess student progress— These tests include the state-wide Arizona’s Instrument to Measure Standards (AIMS) test as well as another test similar to the AIMS called Measures of Academic Progress (MAP). Most ASDB students did not meet standards on AIMS— ASDB students generally scored lower on AIMS than did Arizona students as a whole. Although some ASDB students passed and even exceeded standards on the spring 2011 AIMS, most did not. MAP results showed students made some progress but still testing well below grade level— ASDB students’ MAP test scores showed that they start at a much lower level than national norms. Although limited growth occurred after 5th and 6th grades and continued into high school, it was not sufficient to bring students’ scores within reach of national norms. For example, by the 11th grade, ASDB students scored the same as the average 3rd-grade student would in reading and scored slightly below the average 4th-grade student in math. Standardized test scores are limited indicators of program success—Sensory-impaired students may have difficulty taking standardized tests because of natural disadvantages these students may face in taking these tests. In addition, there is little comparative information about scores attained by similar students in other Arizona schools or other states’ schools. ASDB uses common practices, but can strengthen its practices—To help ensure success in educating its students, ASDB 2012 September • Report No. 12-05 Arizona State Schools for the Deaf and the Blind Our Conclusion The Arizona State Schools for the Deaf and the Blind (ASDB) serves children and students from birth through 21 years of age who have sensory impairments such as deafness and blindness. As of April 2012, ASDB had 2,240 students who either attended one of its three on-campus schools, participated in its birth to age three or preschool programs, or received ASDB services at their local schools. ASDB can do more to promote student success by ensuring that children with sensory impairments receive services early in their lives, increasing students’ access to highly qualified teachers, and researching and addressing differences in students’ test scores. ASDB should also ensure its regional cooperative program, which serves students at their local schools, more fully recovers its costs. It should also strengthen oversight and management of the program. Finally, ASDB needs to address critical information technology weaknesses. REPORT HIGHLIGHTS PERFORMANCE AUDIT ASDB can do more to promote student success follows commonly used practices, including using technology to help students learn. In addition, ASDB groups students according to their academic ability rather than age. However, ASDB can strengthen the following practices: • Early Intervention—Early interven-tion services are a critical element in a student’s success. ASDB provides early intervention services through a program called Arizona Early Interven-tion Program. ASDB has identified areas where it can do more to improve this program such as coordinating with the other agencies involved in the pro-gram, better organizing and training staff, and reaching out to families with sensory-impaired infants and children. • Highly qualified teachers—Federal law requires that disabled students be taught by teachers who are highly qualified in special education and in core academic subjects. According Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 0 10 20 30 40 50 60 70 Falls Far Below Approaches Meets Exceeds Percent of Students Reading ASDB State-wide 0 10 20 30 40 50 60 Falls Far Below Approaches Meets Exceeds Percent of Students Math ASDB State-wide Arizona State Schools for the Deaf and the Blind REPORT HIGHLIGHTS PERFORMANCE AUDIT September 2012 • Report No. 12-05 A copy of the full report is available at: www.azauditor.gov Contact person: Shan Hays (602) 553-0333 to an Arizona Department of Education report, about 90 percent of ASDB teachers were highly qualified during the 2011-2012 school year. According to ASDB officials, it is difficult to hire and retain highly qualified teachers for many reasons, including lack of competitive pay and the challenges of obtaining the required certifi-cations. ASDB is working with the University of Arizona and the Arizona Department of Educa-tion to increase the pool of qualified teachers. • Studying differences in test scores—Federal law requires that all public schools work to nar-row achievements gaps in their states. Study-ing differences in AIMS passing rates across the various ASDB locations may help improve ASDB’s student achievement. Recommendations—ASDB should: • Improve its early intervention program to ensure that children receive needed services. • Increase students’ access to highly qualified teachers. • Study differences in test scores and implement potential solutions to improve student achieve-ment. ASDB provides services at local school districts through its regional cooperative program—ASDB established a regional cooperative program in 1988 to help local school districts serve sensory impaired students. There are five regional cooperatives, and each has a staff of teachers, interpreters, and other specialists. About one-half of state school districts participate in the program. Funding comes from various sources—Funding for the regional cooperatives comes from special education vouchers through the Arizona Department of Education for assessments, counseling, and direct teaching services; from the State General Fund for program administrative costs; and from school district membership fees to screen children and train school staff. ASDB should examine fees—Each regional cooperative has established its own membership fees it charges to school districts. Districts also pay ASDB for direct educational services provided to their sensory impaired students. ASDB does not have policies for determining appropriate membership and other fees. Further, ASDB has not evaluated whether such fees cover the costs of the services provided. Recommendation—ASDB should ensure that fees cover the cost of all services provided by regional cooperatives by assessing operations, tracking costs, and developing appropriate fees. ASDB should review regional cooperative service fees and strengthen its operations ASDB needs to improve its information technology practices In 2009, the Arizona Department of Administration’s Information Security group assessed ASDB’s infor-mation technology (IT) environment. This assess-ment identified deficiencies and made several recommendations. Although ASDB tried to address the deficiencies, this audit identified the following IT security control weaknesses: • No risk assessment or security reviews of the IT environment have been done. • The IT network does not have adequate controls to secure it from hackers. • There is no effective process for updating com-puters and servers, which are running critical systems on outdated software. ASDB also does not have a comprehensive disaster recovery plan and does not properly test or backup data. As a result, ASDB may not be able to meet federal and state requirements to protect the privacy of it students and staff. Recommendations—ASDB should: • Strengthen IT security controls. • Address disaster recovery deficiencies. • Improve data backup. TABLE OF CONTENTS continued Introduction 1 Finding 1: ASDB can do more to promote student success 9 ASDB aims to prepare students for future 9 ASDB uses several assessments to measure student progress 10 Test scores showed some academic progress, but comparative information for assessing program success is limited 11 ASDB uses commonly used practices but can work to improve 15 Recommendations 24 Finding 2: ASDB should examine regional cooperative program service fees and strengthen program operations 27 ASDB provides services to students at local school districts through its regional cooperative program 27 ASDB should examine fees for regional cooperative programs services 29 Regional councils’ role in fee-setting needs review 34 Regional cooperatives are not consistent in tracking resources 37 Recommendations 38 Finding 3: ASDB needs to improve its information technology practices 41 IT systems used extensively and contain sensitive data 41 page i Office of the Auditor General TABLE OF CONTENTS continued ASDB has not addressed critical IT weaknesses 41 Recommendations 47 Sunset Factors 49 Appendix A: Test results a-i Appendix B: Methodology b-i Agency Response Tables 1 Student Enrollment by Type of Disability School Year 2011-2012 2 2 Schedule of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2009 through 2012 (Unaudited) 7 3 Services Provided by Regional Cooperative Staff School Year 2011-2012 30 4 Regional Cooperative Membership Fee Structure School Year 2011-2012 31 5 Regional Cooperative Reimbursements to Districts School Year 2011-2012 32 6 Example of Differences in Regional Cooperative Service Costs School Year 2011-2012 33 page ii State of Arizona TABLE OF CONTENTS concluded Figures 1 Regional Cooperatives, School Districts, and Students Served As of April 2012 4 2 Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 12 3 Mississippi Joint Legislative PEER Committee’s Structured Fee-Setting Process Developed for State Government 35 4 Spring 2008, 2009, 2010, and 2011 AIMS Math Test Results for ASDB Students and State-wide Students a-i 5 Spring 2008, 2009, 2010, and 2011 AIMS Reading Test Results for ASDB Students and State-wide Students a-i page iii Office of the Auditor General ASDB educates students with sensory impairments Programs and responsibilities ASDB was established in 1912 to provide education to students with sensory impairments such as deafness or blindness. ASDB enrolled 478 students at its on-campus schools in the 2011-2012 school year (see Table 1, page 2). In addition, ASDB provided services to 1,190 additional students in school districts and 299 preschool students throughout the state during the 2011- 2012 school year. In accordance with state law, ASDB provides educational programs for chil-dren and students from birth through 21 years who have a vision and/or hear-ing loss (see textbox for ASDB’s mission). According to statute, school districts must arrange for a placement and evaluation team for students with a vision and/or hearing impairment. The team includes the child’s parent or guardian as well as school district and ASDB rep-resentatives. This team deter-mines the appropriate educa-tional placement for the stu-dent based on his or her indi-vidualized education program. Students can attend a local school district, charter school, or one of the ASDB campuses. As of April 2012, the 1,668 students ASDB served were enrolled in the following schools and/or programs: • Arizona Schools for the Deaf and Blind (110 students at the Arizona School for the Deaf and 85 students at the Arizona School for the Blind)—Two schools, the Arizona School for the Deaf and the Arizona School for the Blind, make up the ASDB Tucson campus. According to ASDB officials, students can live on campus as residential students or attend as day students if they live within 60 miles of the campus. As of April 2012, 70 students lived on campus. Both schools are accredited by page 1 Scope and Objectives INTRODUCTION The Office of the Auditor General has conducted a performance audit and sunset review of the Arizona State Schools for the Deaf and the Blind (ASDB) pursuant to an October 26, 2010, resolution of the Joint Legislative Audit Committee. This audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This performance audit and sunset review addresses (1) how ASDB can increase its use of best practices to promote its students’ success, (2) improvements that ASDB should make to the operations of its regional cooperatives program, and (3) the need to better protect student data and make other improvements to ASDB’s information systems management. The report also includes responses to the statutory sunset factors. Office of the Auditor General ASDB Mission: To work together with parents, school districts, advocacy organizations, and business and community members to create nurturing environments in which children with a vision or hearing loss feel valued, develop their abilities, strive to achieve academic excellence, and develop skills to help them become productive and responsible members of society. Source: Fiscal Years 2010-2013 master list of state government programs. AdvancED.1 The Arizona School for the Deaf offers educational and support services, including instruction in a direct communication environment and counseling (see textbox), as well as training that teaches students with cochlear implants to hear and understand speech and other sounds. At the Arizona School for the Blind, services are provided based on an expanded core curriculum that includes state standard curriculum and special services such as orientation and mobility training, braille, visual efficiency training, assistive technology, and compensatory skills and tools. Both schools also offer other services based on students’ needs, including activities of daily living; social skills; occupational, physical, and speech therapy; and residential services. • Phoenix Day School for the Deaf (283 students)—According to ASDB officials, the Phoenix school educates students who are deaf or hard-of-hearing and live within 60 miles of the metropolitan Phoenix area. It provides educational and support services similar to the Arizona School for the Deaf. The school supports students developing two languages— 1 AdvancED is an organization of public and private schools in the United States and other countries world-wide that accredits schools based on adherence to quality standards in several areas, including curriculum, instructional design, and assessment practices. The organization was formed in 2006 from the North Central Association Commission on Accreditation and School Improvement, the Southern Association of Colleges and Schools Council on Accreditation and School Improvement, and the National Study of School Evaluation. page 2 State of Arizona Direct Communication Environment— Direct communication and access are a critical part of education and language development. Students who are deaf or hard-of-hearing interact directly with teachers, staff, and peers every day in the classroom, cafeteria, after-school program, and residential program. Direct access to instruction and peer interaction leads to enhanced educational, social, and emotional development; and allows a greater degree of independence, self-confidence, and self-advocacy. Source: Arizona School for the Deaf Web site. Table 1: Student Enrollment by Type of Disability School Year 2011-2012 1 According to ASDB officials, “other” refers to a student with a sensory impairment who also has a medical condition or a physical impairment. 2 According to ASDB officials, multiple disabilities are not identified in preschool children but are identified at a later age. Source: Auditor General staff analysis of ASDB’s April 2012 report, Agency enrollment: school year 2011-2012. Program Deaf or Hard-of- Hearing Visually- Impaired Multiple Disabilities with Sensory Impairment Multiple Disabilities with Severe Sensory Impairment Other with Sensory Impairment 1 Total Students ASDB Campuses Arizona School for the Blind 0 35 22 28 0 85 Arizona School for the Deaf 79 0 16 15 0 110 Phoenix Day School for the Deaf 211 0 40 32 0 283 Regional Cooperatives Desert Valleys 160 51 80 92 1 384 Eastern Highlands 34 17 42 19 3 115 North Central 99 28 69 58 9 263 Southeast 130 36 88 39 15 308 Southwest 57 10 29 23 1 120 Preschool2 154 107 0 0 38 299 Total 924 284 386 306 67 1,967 American Sign Language (ASL) and English—and its curriculum parallels the curriculum provided by public schools with modifications to meet students’ communication needs. According to ASDB officials, the school also offers intensive instruction in functional and daily living skills for students who need it. The school is accredited by AdvancED. • Regional cooperatives (1,190 students)—ASDB provides educational and support services in students’ local schools through five regional cooperatives located in various parts of the State (see Figure 1, page 4). The Legislature first authorized ASDB to establish regional cooperatives in 1988 to provide various services that would enable districts to serve sensory-impaired students in a cost-effective way and enable these students to reach their individual potentials by providing access to the general education curriculum in district schools. According to ASDB officials, the regional cooperatives provide access to a pool of specially trained personnel and equipment, such as sign language interpreters and braille-writer machines. In addition to enrolling students at its campuses or in regional cooperatives, ASDB provides services through the following programs: • Birth to 3 program (273 children)—In cooperation with the Arizona Early Intervention Program (AzEIP, see textbox on page 18), ASDB provides federally mandated services for families of/and children ages birth to 3 who have hearing or vision loss or who are deaf-blind. The program provides a comprehensive and coordinated interagency system of early intervention services in the child’s home and is designed to work in cooperation with other agencies and programs to support the child and family’s needs. In addition to assessments that determine if an infant or toddler is eligible for the program and to plan appropriate services, the AzEIP program offers service coordination, assistive technology devices, family training, physical and occupational therapy, and speech-language pathology services. • Preschool program (299 students)—ASDB provides preschool education to eligible children, ages 3 to 5, at its campuses in Tucson and Phoenix and in classrooms in public schools throughout Tucson and Phoenix. The services offered to the children include preschool education, educational assessment, vision and audiological assessments, orientation and mobility services, and speech therapy. ASDB’s campuses and other programs serve students with vision or hearing loss, including students with multiple disabilities if at least one of the disabilities is a visual or hearing impairment (see textbox). In the 2011- 2012 school year, most students ASDB served state-page 3 Office of the Auditor General Multiple disabilities—Learning and developmental problems that require special education and related services and that result from two or more of (1) hearing impairment, (2) orthopedic impairment, (3) moderate intellectual disability, and (4) visual impairment, or any of those conditions concurrently with mild intellectual disability, emotional disability, or specific learning disability. Multiple disabilities with severe sensory impairment—Multiple disabilities that include (1) severe visual or hearing impairment with another severe disability or (2) severe visual and severe hearing impairments. Source: A.R.S. §15-761 page 4 State of Arizona Source: Auditor General staff analysis of data received from ASDB’s regional cooperatives as of April 2012. Figure 1: Regional Cooperatives, School Districts, and Students Served As of April 2012 Mohave Coconino Yavapai La Paz Yuma Pinal Graham Pima Cochise Santa Cruz Maricopa Gila Navajo Apache North Central Regional Cooperative Eastern Highlands Regional Cooperative Southeast Regional Cooperative Desert Valleys Regional Cooperative Southwest Regional Cooperative page 5 Office of the Auditor General wide had a hearing loss, with smaller numbers of students with a vision loss or multiple disabilities (see Table 1, page 2). Board and staff ASDB’s Board of Directors (Board) is responsible for the governance of ASDB, including setting policy and appointing, evaluating, and determining whether or not to offer a new contract to ASDB’s Superintendent. It has ten members, including the Governor and the State’s Superintendent of Public Instruction who serve as ex officio members. The Governor appoints the remaining members, who serve 3-year terms. The appointed members include: • One member from the Commission for the Deaf and the Hard-of-hearing; • One member from the Governor’s Council on Blindness and Visual Impairment; • One school district employee who works with his or her district’s program for sensory impaired pupils; and • Five additional members. A.R.S. §15-1321 requires the Governor to give preference to people with experience and knowledge of sensory-impaired education for three of these members. ASDB has staff at its three school campuses and at its five regional cooperatives that are located in various parts of the State. As of May 2012, ASDB reported that it had 803 full-time equivalent (FTE) positions, with 256 vacancies. The vacancies included 78 teachers; 38 instructional assistants; 47 educational interpreters, audiologists, and transcribers; 9 drivers; and 84 various other positions. ASDB’s staffing includes the following: • Administration—The administrative staff include a superintendent who serves as the executive officer for ASDB, an assistant superintendent, and principals for each school. In addition, ASDB has directors over operations, regional cooperatives, information technology, and human resources; a staff development specialist; and an accountability specialist. • Teaching and educational support—As of May 2012, ASDB employed 227.75 teachers and 20 supervisory teachers. Educational support includes 11.25 audiologists, 4.25 braillists, 1 ASL specialist, 3.25 speech/language pathologists, and 65.75 educational interpreters. The educational interpreters work at the regional cooperatives. ASDB’s teachers and other educational staff generally have special training and certification in communicating with and teaching students with sensory impairments. page 6 State of Arizona • Other—ASDB’s on-campus staff in Tucson also include night supervisors, residential hall team leaders, and live-in staff for the student dormitories. These staff also include teaching parents, who are educators responsible for working with students on goals in areas such as personal care, recreation and leisure, and domestic life. In addition, ASDB’s operations staff include custodial, maintenance, security, food service, and equipment repair staff as well as full-time and part-time drivers. Budget and finances As shown in Table 2 (see page 7), ASDB received approximately $55.5 million, $56.5 million, $54.7 million, and $53.8 million in annual net revenues in fiscal years 2009 through 2012, respectively. These revenues included: • Special education voucher monies from the Arizona Department of Education (ADE) to reimburse ASDB for educational costs based on its enrollment—These monies are provided for students at ASDB’s three campus schools and some students enrolled in ASDB’s regional cooperative program. In the 2011-2012 school year, the voucher amount was $19,122.39 for each visually-impaired student and $19,008.02 for each hearing-impaired student. For each student who receives services through the regional cooperative program, ASDB and the student’s district decide whether ASDB or the district will collect the voucher monies. • Payments from school districts for services provided by the regional cooperative program—These include membership fees to participate in the regional cooperative program and fee-for-service payments for students whose voucher monies are retained by the district. Fee-for-service payments vary depending on the level of service provided by ASDB. In the 2011-2012 school year, the payments ranged from $2,800 annually for a student receiving 1 hour per month of services to $21,500 annually for a student receiving 10 to 15 hours per week of services (See Finding 2, pages 27 through 39, for additional information on the fees charged for services provided through ASDB’s regional cooperative program). • State General Fund appropriations made directly to ASDB—State General Fund appropriations help bridge the gap between the cost of educating ASDB students and the amounts provided by special education vouchers and fees paid by school districts. In fiscal year 2012, ASDB received a $20.8 million State General Fund appropriation. As shown in Table 2, ASDB’s total expenditures ranged between nearly $55.2 million in fiscal year 2009 to approximately $54.5 million in fiscal year 2012. Most of ASDB’s expenditures are for personal services and related benefits. These monies pay for teachers and other staff at the three on-campus schools, teachers in the regional page 7 Office of the Auditor General 1 Amount is primarily composed of educational reimbursements from the Arizona Department of Education based on enrollment and is determined by statutory formula. The amount also includes federal grant revenues that are either received directly or passed through from the Arizona Departments of Economic Security or Education. 2 Amount consists of revenues received from Arizona school districts for services rendered by ASDB’s regional cooperatives on a fee-for-service basis. 3 Amount primarily consists of interest and rental income. 4 Amount consists of transfers to the State General Fund in accordance with Laws 2008, Ch. 285, §46; Laws 2010, Ch. 1, §148 and 7th S.S., Ch. 3, §8; and Laws 2011, Ch. 24, §§108, 129, and 138 to provide support for state agencies. 5 Amount primarily consists of transfers to the Arizona Department of Economic Security for required matching to obtain federal monies. According to ASDB, the fiscal year 2009 required match was not made until fiscal year 2010; therefore, the transfer is reported in fiscal year 2010. Similarly, the fiscal year 2012 required match was not made until August 2012; therefore, it will be reported in fiscal year 2013. 6 According to ASDB, end-of-year fund balances consist of revenues from several dedicated funding sources such as federal grants, Proposition 301 (2000) sales taxes, and educational reimbursements from the Arizona Department of Education. The balances are generally needed for cash flow at the beginning of the next fiscal year or are restricted for specific purposes. For example, in order to pay the payroll costs for current 12-month employees and purchase equipment, supplies, and materials needed for the start of the school year, the balances are needed in July and August since educational reimbursements from the Arizona Department of Education, fees for services, or membership fees are not received until late September. Source: Auditor General staff analysis of the Arizona Financial Information System (AFIS) Accounting Event Transaction File for fiscal years 2009 through 2012 and the AFIS Management Information System Status of General Ledger-Trial Balance screen for fiscal years 2010 through 2012. Table 2: Schedule of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2009 through 2012 (Unaudited) 2009 2010 2011 2012 Revenues: State General Fund appropriations $ 21,351,663 $ 21,580,479 $ 21,511,083 $ 20,802,910 Intergovernmental1 31,373,770 31,646,493 29,759,177 29,544,022 Educational and support service fees2 2 ,216,073 2 ,949,195 2 ,913,044 2 ,885,938 Other3 515,669 336,512 484,947 574,821 Gross revenues 55,457,175 56,512,679 54,668,251 53,807,691 Remittances to the State General Fund (12,034) (1,292) Net revenues 55,445,141 56,511,387 54,668,251 53,807,691 Expenditures and transfers: Personal services and related benefits 44,215,881 43,421,888 40,744,344 42,882,990 Professional and outside services 3,563,179 3,215,689 3,943,356 3,550,435 Travel 310,034 300,216 289,665 283,635 Food 255,407 231,084 232,703 237,015 Aid to individuals 1,110 Other operating 5,798,761 5,783,749 6,582,648 4,833,542 Equipment 1,034,733 2,100,684 2,451,489 2,715,179 Total expenditures 55,179,105 55,053,310 54,244,205 54,502,796 Transfers to the State General Fund4 157,200 18,200 531,100 616,400 Transfers to other agencies5 1,063,064 461,882 Total expenditures and transfers 55,336,305 56,134,574 55,237,187 55,119,196 Net change in fund balance 108,836 376,813 (568,936) (1,311,505) Fund balance, beginning of year 6,506,384 6,615,220 6,992,033 6,423,097 Fund balance, end of year6 $ 6,615,220 $ 6,992,033 $ 6,423,097 $ 5,111,592 cooperative program who travel to the surrounding school districts in their respective regions throughout the State, and administrative staff. These expenditures also include salaries and benefits for the staff who provide student transportation to and from the day school programs, which totaled approximately $2.9 million in fiscal year 2011, according to the fiscal year 2011 Annual Report of the Arizona Superintendent of Public Instruction and other financial information obtained from ASDB. page 8 State of Arizona ASDB can do more to promote student success FINDING 1 page 9 ASDB aims to prepare students for future ASDB’s mission is to work with parents, school districts, and others to enable children with a vision or hearing loss to develop their abilities, strive to achieve academic excellence, and develop skills to help them become productive and responsible members of society. This mission aligns with ASDB’s statutory responsibility to provide educational services, including instruction in a direct communication environment, so that deaf, hard-of-hearing, and visually-impaired students may become self-sustaining and useful citizens. ASDB’s surveys of recent graduates show that many of its on-campus students either continue their education or begin working, either independently or with supports, such as a job coach. In 2011, ASDB reported that the Arizona School for the Blind had 13 graduates, the Arizona School for the Deaf had 14 graduates, and the Phoenix Day School for the Deaf had 15 graduates. Surveys conducted approximately 1 year after the students graduated indicated that the students became involved in a variety of activities. For example: • Out of 13 survey respondents who graduated from the Arizona School for the Blind in 2011, 9 reported that they were participating in work enclave programs through the Department of Economic Security. Work enclaves are competitive work site jobs where workers with disabilities are supervised by program staff and receive a special wage commensurate with their abilities. • All 6 2011 graduates of the Phoenix Day School for the Deaf who responded to the survey and 10 of the 14 respondents from the Arizona School for the Deaf reported that they were in college or community college, working, or volunteering. For example, one graduate worked part-time as a physical therapist assistant after graduation and then resigned to attend Gallaudet University where he is studying physical therapy. Another graduate volunteers at the public library for 6 hours per week, a third works in a supported-employment program sponsored by the Department of Economic Security, and three graduates were attending a community college for the deaf in Texas. The Arizona State Schools for the Deaf and the Blind (ASDB) can make improvements in its use of best practices to help ensure that its students make good educational progress and are prepared for their future. Although ASDB uses standardized tests to assess individual student progress, these tests are somewhat limited in their ability to assess ASDB’s students’ progress and provide relatively limited information for comparing ASDB to other schools that educate similar students. Thus, attention to best practices recommended by experts and officials at other schools is particularly important. ASDB follows many of these best practices, including using technology and modifying instruction and services to meet students’ particular needs. In four respects, however, ASDB can strengthen its efforts. These are: • Improving its early inter-vention program; • Seeking ways to increase students’ access to highly qualified teachers; • Compiling and using information about its graduates to improve its programs; and • Studying differences in its students’ test scores and establishing expectations regarding practices to improve students’ test scores. Office of the Auditor General ASDB uses several assessments to measure student progress To monitor students’ progress, ASDB relies on several different assessment mechanisms. These include the same standardized test all Arizona students take, as well as additional standardized tests, including the Measures of Academic Progress (MAP), that provide more information on individual student progress. In addition, at least once a year, ASDB staff will review each student’s progress toward the individual goals established in his or her Individualized Education Program (IEP) (see textbox, page 11). Each student’s IEP determines what assessments will be used to measure the student’s progress, and results are reviewed when the IEP is updated. ASDB frequently uses the following tests:1 • Arizona’s Instrument to Measure Standards (AIMS)—This is the standardized assessment taken by all Arizona students in all school districts. The federal No Child Left Behind Act requires that each state adopt standardized assessments that align with state standards to demonstrate adequate yearly progress toward the goal of narrowing achievement gaps. The State of Arizona uses AIMS (see textbox, page 11) as its standards-based assessment.2 Arizona does not require special education students to pass the test to meet the requirements for a high school diploma unless their IEP requires it. • Measures of Academic Progress (MAP)—This test, created by the Northwest Evaluation Association (NWEA), assesses the same content as the AIMS, according to an NWEA official. However, it is an interactive form, meaning that it does not test a student on more advanced concepts if he/she did not answer the initial basic concept question correctly. According to ASDB, this type of test is particularly useful for students who are not performing at grade level because it provides more information about students’ actual performance levels and their progress over time than the AIMS does. ASDB administers the test to only some on-campus students. ASDB officials reported that ASDB encourages all students to take the MAP if they are able to do so. In spring 2011, approximately 59 percent of ASDB on-campus students took a MAP reading and/or math test. 1 In addition to AIMS and MAP, ASDB reported that it uses other assessments to measure student progress. For example, it uses the Stanford Achievement Test (SAT-10) in second and ninth grades, as required by the Arizona Department of Education. In addition, ASDB uses Teaching Strategies GOLD, another Arizona Department of Education required test for special education children in preschool and kindergarten. For children aged birth to 3 years, ASDB reported that it also uses the Developmental Assessment of Young Children test for deaf children and the Oregon Project for blind children to assess the children’s cognitive skill level. ASDB also reported that it uses the 6-Traits Writing Rubric to evaluate student writing samples, the Fairview assessment focusing on American Sign Language development, the STAR Reading assessment, Work Sampling—an observational assessment containing grade-level samples of what students should know and how they would demonstrate their knowledge—and classroom-type tests. 2 This report uses AIMS to refer to both AIMS and AIMS A, the alternate assessment used for some students. Both tests report student results in the same way, indicating that the student exceeds, meets, approaches, or falls far below standards. page 10 State of Arizona Arizona does not require special education students to pass the AIMS test to obtain a high school diploma unless their IEP requires it. Tests scores showed some academic progress, but comparative information for assessing program success is limited ASDB students, on average, underperformed on standardized tests, although they made some progress throughout their school years. For several reasons, however, standardized tests such as AIMS and MAP are limited in their ability to assess the progress of students with these types of impairments, and state-wide information that would allow comparisons between ASDB and other Arizona schools educating similar students is also limited. Most ASDB students who took AIMS did not meet standards—Students attending ASDB, either on one of its campuses or through the regional cooperatives, generally scored lower on AIMS than did Arizona students as a whole. Some ASDB students passed and even exceeded standards on the AIMS, but most did not. As shown in Figure 2 (see page 12), in spring 2011, only 5 percent of ASDB students exceeded standards, and only 25 percent met standards when they took the AIMS math test in 3rd through 12th grades. By comparison, 23 percent of all students state-wide who took AIMS exceeded standards, and 36 percent met standards for math in spring 2011. Similarly, only 2 percent of ASDB students exceeded, and 34 page 11 Office of the Auditor General Assessment terms Individualized Education Program (IEP)—Every child with a disability must have an IEP that shows the child’s level of achievement, measurable annual goals, how progress toward the goals will be measured, and what services will be provided to the child. The goals may include passing the AIMS if appropriate for the child, but Arizona Revised Statutes (A.R.S.) §15-763 states that a special education student does not need to pass the AIMS to graduate from high school unless this is required by the student’s IEP. A team that includes the child’s parent or guardian develops the IEP and updates it at least annually. The federal Individuals with Disabilities Education Act, Arizona statutes, and Arizona Department of Education regulations specify the requirements the IEP must meet. Arizona’s Instrument to Measure Standards (AIMS) and AIMS Alternate (AIMS A)—Tests students’ grade-level mastery of state academic standards in writing, reading, math, and science. AIMS A is an alternate assessment based on alternate achievement standards for students with significant cognitive disabilities. Both AIMS and AIMS A test students in 3rd through 8th grades and in high school beginning in 10th grade (or in 9th grade if the student repeats that grade) until they pass the test because it is a graduation requirement for most Arizona students. Special education students are not required to pass the test to receive a high school diploma unless stipulated in their IEP. Test scores are classified into four levels: exceeds, meets, approaches, and falls far below standards, with exceeds and meets defined as passing scores. Measures of Academic Progress (MAP)—Nationally normed test developed by the NWEA of language usage, reading, math, and science that uses an equal interval scale to allow charting progress from year to year. The test adapts to a student’s performance by offering questions based on whether the student answered previous questions correctly. ASDB tests on-campus students in 2nd through 12th grades. Source: Auditor General staff analysis of information about AIMS, AIMS A, and the IEP obtained from the Arizona Department of Education Web site, information about MAP from the Northwest Evaluation Association (test publisher) Web site, and A.R.S. §15-763. In spring 2011, 5 percent of ASDB students exceeded standards, and 25 percent met standards on the AIMS math test. percent met standards in reading, compared to 9 percent and 65 percent, respectively, for all students state-wide who took the AIMS reading test. Approximately two-thirds of ASDB students who took the AIMS math or reading tests did not meet the standards. AIMS test results for 2008 through 2010 show that, in those years, ASDB students also had much lower rates for meeting and exceeding the standards than students state-wide (see Appendix A, page a-i). page 12 State of Arizona Source: Auditor General staff analysis of AIMS and AIMS A spring 2011 test scores for all ASDB students enrolled in the 2011-2012 school year, including on-campus students and regional cooperative students for whom ASDB receives voucher funding, and state-wide AIMS and AIMS A results for spring 2011 published October 2011 by the Arizona Department of Education in its 2011 Technical Report. Figure 2: Spring 2011 AIMS Test Results for ASDB Students and State-wide Students 0 10 20 30 40 50 60 70 Falls Far Below Approaches Meets Exceeds Percent of Students Reading ASDB State-wide 0 10 20 30 40 50 60 Falls Far Below Approaches Meets Exceeds Percent of Students Math ASDB State-wide MAP results show students made some progress but were still testing well below grade level—ASDB students’ test scores for MAP were well below national norms, which are based on a stratified sampling of students throughout the United States. ASDB students showed less improvement from one year to the next during the elementary school years as compared to national norms for all students.1 However, the test is not separately normed for deaf or blind students. Beginning in 5th and 6th grades, ASDB students improved their performance in reading and math, respectively, compared to national norms. Even so, ASDB students started at a much lower level than national norms, so while limited growth occurred after 5th and 6th grades and continued in high school, it was not sufficient to bring students’ scores within reach of national norms (see discussion of early intervention programs, pages 18 through 21, for reasons ASDB students started at a lower level). Specifically: • ASDB on-campus students’ average MAP scores in 2nd grade were in the first to second percentile in reading and the third to fourth percentile in math. This result indicates that nearly all students nationally who took the MAP received higher scores than ASDB students who took the test. • By the 11th grade, ASDB students who took the MAP scored on average the same as 3rd-grade students would in reading and scored slightly below the average 4th-grade student in math. This reflects a slight gain in performance, with results at the fifth to sixth percentile in reading and the seventh to eighth percentile in math. On average, 11th-grade students nationally who took the MAP scored at approximately the 50th percentile and scored the same as a typical-11th grade student would. Standardized test scores are limited as indicators of program success—For several reasons, great care must be taken in inferring conclusions from these results. Key limitations include natural disadvantages ASDB students may face in taking standardized tests. Additionally, there is little comparative information about scores attained by similar students in other settings in Arizona or in schools elsewhere in the United States. Specifically: • Students who are deaf or hard-of-hearing and students who are blind or visually-impaired may have difficulty taking standardized tests—Students attending ASDB have hearing and/or visual impairments or multiple disabilities (see textbox, page 3, for information on multiple disabilities). According to experts, language deficits and testing barriers make it more difficult for these students to perform at national or state norm levels on any standardized tests. Despite accommodations such as braille versions of tests and having test directions read aloud or signed, barriers that affect student performance still exist. For example, deaf students do not learn English in infancy, but the tests are administered in English so deaf students are taking the test in their second 1 Auditors analyzed ASDB students’ spring test results for the MAP test for 2006 through 2011 and compared them with the spring 2011 national results published by NWEA. page 13 Office of the Auditor General According to experts, language deficits and testing barriers make it more difficult for sensory-impaired students to perform at national or state norm levels on any standardized test. language. Additionally, blind students are unable to see illustrations, such as graphs, used in standardized math tests. • ASDB students’ scores cannot be reliably compared with results for other sensory-impaired students in Arizona or nation-wide—Within Arizona, the available information regarding student test scores makes it difficult to compare ASDB students’ performance with the performance of similar students throughout the State. A key reason is the way in which school districts must report special education students’ test scores. Specifically, to comply with Family Educational Rights and Privacy Act regulations, when school districts submit test scores to the Arizona Department of Education, they report all students with disabilities under the broad category of special education. They do not report students’ specific disabilities because such categorization could enable students to be individually identified in published reports. Therefore, sensory-impaired students’ test scores are reported in the same category as students with learning disabilities, developmental delays, and other disabilities. Further, although the Arizona Department of Education has data regarding all Arizona students, including their specific disabilities that could be matched with test scores, Arizona Department of Education officials reported that this specific data may not be complete. As a result, the available information does not provide an accurate way to compare the State’s entire population of sensory-impaired students by disability. Similar problems exist when attempting to compare ASDB students to sensory-impaired students in other states. First, each state administers different tests to measure students against their state’s standards. This makes it difficult to compare students’ progress across states. Second, states differ in the degree to which they require sensory-impaired students to take standardized tests required of other students. Some deaf and blind schools must participate in their states’ standardized testing. For example, students at Perkins School for the Blind in Massachusetts are required to pass the state test in order to obtain a high school diploma. Additionally, Alabama requires all high school students to take the state test to graduate with a diploma, and eligible students with disabilities who do not pass the test can obtain an occupational diploma instead of the regular high school diploma. However, similar to Arizona, at least two other states do not make passing the standardized test a graduation requirement for all deaf and blind students. Specifically, Florida School for the Deaf and the Blind students and Texas students who are deaf or blind can receive waivers from testing requirements for a high school diploma as long as the students can demonstrate in some other way that they have met state standards. Additionally, at least two states’ deaf and blind schools have admission policies that restrict or deny admission of severely impaired students. This would greatly impact the comparability of those schools’ test results with ASDB’s test results as students with severe or multiple impairments often struggle with passing standardized tests. page 14 State of Arizona When school districts submit test scores to the Arizona Department of Education, they report all students with disabilities under the broad category of special education. page 15 Office of the Auditor General Although there is a lack of comparison information regarding blind and multiply disabled students’ test scores, some limited studies have been conducted for deaf students. For example, a 2005 study found that the SAT-9 reading comprehension score for 17- and 18-year-old deaf and hard-of-hearing students was approximately equivalent to that of 4th-grade hearing students, and a 2012 longitudinal study reported that deaf 17-year-olds’ SAT-9 math scores approached the 6th-grade level.1,2 In addition, test results for a small number of students at a small number of schools for the deaf nation-wide suggest that ASDB deaf and hard-of-hearing students’ scores are similar to those of their peers at other schools.3 ASDB uses commonly used practices but can work to improve Given the limitations of test results as a way to gauge ASDB’s success, auditors looked for other ways to assess ASDB’s efforts. One commonly used assessment is the degree to which an organization employs “best practices”—that is, those approaches and techniques that experts, researchers, and practitioners have identified as likely characteristics of a well-performing program. Although scientific research is limited regarding what works in educating deaf and visually-impaired students, experts have identified several commonly used practices. ASDB already follows a number of these practices. However, it should focus additional effort in four key areas in order to promote its students’ success: early intervention, increasing its students’ access to highly qualified teachers, using surveys of its graduates to identify ways to improve its programs, and developing ways to improve students’ scores on standardized tests. Research is limited—Experts in the fields of deaf and blind education have conducted extensive reviews of existing research and concluded that there is not a large body of empirical research to draw upon to establish evidence-based practices.4 This contrasts with research in educating children with more common characteristics such as learning disabilities. However, experts at Gallaudet University, a prominent university for deaf students in Washington D.C., and Perkins School for the Blind in Massachusetts, as well as officials at deaf and blind schools in Texas 1 Gallaudet Research Institute. (n.d.). Literacy & deaf students. Retrieved, July 10, 2012, from http://www.gallaudet.edu/ gallaudet_research_institute/publications_and_presentations/literacy.html 2 Qi, S. & Mitchell, R.E. (2012). Large-scale academic achievement testing of deaf and hard-of-hearing students: past, present, and future. Journal of Deaf Studies and Education, 17(1), 1-18. 3 The results from the other schools are preliminary. The test publisher cautions that because of the small number of test scores included and the lack of stratification, the results should not be used for comparison as part of a decision-making process. The data does represent a set of grade-level averages of test results of students identified as being deaf over a number of years. 4 Luckner, J.L., & Handley, C.M. (2008). A summary of the reading comprehension research undertaken with students who are deaf or hard-of-hearing. American Annals of the Deaf, 153(1), 6-36; and Douglas, G., McCall, S., McLinden, M., & Pavey, S. (2009). International review of the literature of the evidence of best practice models and outcomes in the education of blind and visually-impaired children. Trim, Co. Meath, Ireland: National Council For Special Education. Test results for a small number of schools for the deaf nation-wide suggest that ASDB deaf and hard-of- hearing students’ scores are similar to those of their peers at other schools. page 16 State of Arizona and Florida, shared information with auditors regarding practices they have found to be effective in educating deaf, hard-of-hearing, and visually-impaired students. ASDB follows some commonly used practices—ASDB uses some commonly used practices that are consistent with those identified by the experts and practitioners auditors interviewed although there is a shortage of direct evidence demonstrating practices are effective with deaf and blind students. These include using technology to help students learn and adjusting instruction to meet students’ needs. Specifically: • Technology innovations help students learn—Experts and national deaf and blind advocacy organizations indicated that innovations in technology are vital to student advancement. These technological innovations help improve deaf and blind students’ ability to access information. For example, an official with the National Council of State Agencies for the Blind reported that portable devices with refreshable braille displays enable blind students to keep up with note taking and reading requirements in high school and beyond. Braille-reading devices can allow students to access content as braille text or through conversion to spoken words. Additionally, these portable devices allow blind students’ access to Global Positioning Systems, which can enhance their orientation and mobility skills and increase their independence. Another assistive technology used by other states’ schools is distributed video in every classroom. Distributed video uses smart whiteboards that bring the board content to a screen on the student’s desk, and a swivel hinge allows visually-impaired students to bring the screen as close as the student needs. Other states’ schools also use technology to assist with their outreach programs by providing webinars, online information, and training as well as publications and other resources to professionals and families. ASDB uses technology extensively and has developed a technology plan for fiscal years 2011 through 2014, which identifies several technological innovations to further make student learning a priority. ASDB already uses technology such as computers and smart whiteboards in some classrooms, computers in the library, video telephone booths, and refreshable braille readers. The plan addresses several areas, including Reading/Language Arts, Math, Highly Qualified Teachers, American Sign Language (ASL) and Braille, Parent Involvement, and Technology Literacy. For each area, the plan identifies a technology strategy and an action step with measurable goals. For example, by the 2011-2012 school year, ASDB’s goal was to have all teachers use digital video and other multimedia tools to promote language development, and according to an ASDB official, this has been implemented. Another 2011- 2012 school year goal was that all students in the Arizona School for the Blind would have accounts that would allow them to download library books in braille. According to ASDB, this has also been implemented. Additionally, ASDB plans to use interactive whiteboards for teaching math during the 2012- 2013 school year. ASDB also plans to build an accessible Web site for parents ASDB uses technology such as computers and smart whiteboards in some classrooms, computers in the library, video telephone, and refreshable braille readers. page 17 Office of the Auditor General that will include trainings and tutorials as well as access to student events, grades, and achievements for the 2012-2013 school year. • ASDB modifies instruction and services to student needs—Other states’ deaf and blind school officials indicated that providing expanded curriculum and individualized instruction, and grouping students based on ability level are commonly used practices for educating deaf and/or blind students. At least two other states’ deaf and blind schools have separate programs for students based on their educational progress and needs. Specifically, the Florida School for the Deaf and the Blind has three separate high school programs, including one program that is specifically for students with additional conditions or needs. Similarly, according to officials at the Texas School for the Blind and Visually-impaired, their school has four separate academic programs with curriculum designed to meet students’ needs. These programs include an academic program designed for students functioning within 2 years of their grade level, an academic program designed for students who are not yet reading at the first-grade level, an academic program designed for students functioning more than 2 years below their chronological age, and a basic skills program for students with multiple disabilities who learn best with consistent routines and meaningful functional activities. ASDB serves students with a wide range of needs, including students with multiple disabilities and severe sensory impairments. Similar to public schools, ASDB structures some of its classrooms by age and grade. However, ASDB policy calls for placing students based on the student’s needs, and according to ASDB staff, ASDB avoids assigning students with a wide range of academic needs to the same classroom. Thus, an ASDB class may include students grouped by academic ability level rather than age, and according to ASDB staff, teachers must adapt instruction to meet each student’s individual needs. By policy, ASDB classroom sizes are small, with student-teacher ratios ranging from six to one for multiply disabled students with severe sensory impairments to ten to one for deaf and hard-of-hearing students in elementary through middle school grades. Typical classroom sizes range from five to nine students.1 ASDB also offers separate classes for students with additional disabilities or higher need levels. According to ASDB staff, these classes cover the same content as required by Arizona state standards, but the way in which the teacher provides instruction may be different. For example, teachers may provide reading materials at a lower level, e.g. fifth grade instead of tenth grade, for those who have difficulty with reading comprehension or reading fluency. According to ASDB staff, these classes usually also include a full-time teacher’s aide. 1 Classroom sizes are based on 2011-2012 rosters for the Arizona School for the Blind and the Phoenix Day School for the Deaf and 2012-2013 rosters for the Arizona School for the Deaf. ASDB classroom sizes are small, with student-teacher ratios ranging from six to one for multiply disabled students with severe sensory impairments to ten to one for deaf and hard-of-hearing students. Four areas require attention—Although ASDB follows some commonly used practices, it can strengthen its practices in the following four ways: • Early intervention and education critical to student success—The earlier a child is identified as having a sensory impairment and receives services for the impairment, the faster the child can begin learning the appropriate skills to adjust to his/her environment. Research shows that the critical intervention time for a baby who is deaf or hard-of-hearing is the first 3 years after birth, and that the first 6 months after birth are crucial for the development of communication and language skills.1 According to two other states’ deaf and blind school officials, students who begin receiving specialized educational services late may have missed the critical window when the brain is most able to develop language, and these students may never catch up. Experts indicated that early intervention services along with effective and meaningful parent-child interactions help ensure successful outcomes in areas such as vocabulary development, verbal reasoning skills, and social interaction.2 The story of one ASDB student provided by ASDB staff illustrates the positive outcome of early intervention and parent involvement: ◦ The student began receiving services almost immediately after birth. She transitioned into preschool at Phoenix Day School for the Deaf and is now in seventh grade, where she participates in sports, other activities, and after-school events. In addition to early intervention, ASDB staff reported that the mother’s involvement has played a large role in the student’s success. While her daughter was young, the Spanish-speaking mother began learning sign language. Although the mother is not fluent in ASL, she can communicate with her daughter and is very involved in her daughter’s education by helping with homework and communicating with her daughter’s teachers. The student is progressing academically and scored above the average ASDB student on MAP reading and math tests in second through sixth grades. ASDB provides early intervention services to sensory-impaired students through the AzEIP program (see textbox). The AzEIP program is responsible for providing early intervention services to referred children and their families to support children’s development. ASDB is one of the five 1 The National Agenda (2005). Moving forward on achieving educational equality for deaf and hard-of-hearing students. Retrieved July 10, 2012, from http://www.ceasd.org/agenda/downloads/natl-agenda-2005-04.pdf 2 American Speech-Language Hearing Association. (2008). Service provision to students who are deaf and hard-of-hearing: Birth to 36 months [Technical report]. Rockville, MD: Author; and Douglas, McCall, McLinden, & Pavey, 2009. page 18 State of Arizona Arizona Early Intervention Program (AzEIP) The AzEIP is Arizona’s state-wide, interagency system of supports and services for infants and toddlers with developmental delays or disabilities and their families. AzEIP services include assistive technology devices and services, family training, physical and occupational therapy, vision and audiological assessments, orientation and mobility services, and speech language pathology services. The federal Individuals with Disabilities Education Act, Part C, establishes AzEIP to help provide eligible children and their families access to services to enhance the child’s development. Source: Arizona Department of Economic Security AzEIP Web site. Experts indicated that early intervention services along with effective parent-child interactions help ensure successful outcomes in vocabulary development, verbal reasoning skills, and social interaction. agencies participating in the AzEIP program.1 ASDB’s AzEIP program responsibilities for sensory-impaired children include evaluating and assessing the child’s needs within 45 days of a referral; ensuring that the child has an appropriate Individualized Family Service Plan (IFSP); providing or arranging for the services on the IFSP in the appropriate intensity, frequency, and duration; and participating in the transition and review of the child before the child enters school. In addition, ASDB is responsible for various other tasks such as coordinating with other AzEIP agencies, providing information for a state-wide resource directory, and collecting and reporting data to the Department of Economic Security. Although ASDB complies with the 45-day requirement to assess children’s sensory impairment needs and develop a service plan, it can improve in regards to its other responsibilities within the AzEIP program. Specifically, as part of the referral process, after a child has been identified as having a hearing or visual impairment, ASDB is responsible for assessing the child’s sensory impairment service needs and developing a service plan that will address these needs within 45 days of a referral. According to an AzEIP official, ASDB meets this 45-day time frame requirement. Adhering to this time frame for assessing and then developing a plan to address service needs is important because experts indicate that almost half of newborns who are referred from a hearing screening do not have appropriate followup to determine the presence of a hearing loss and to initiate appropriate early intervention services.2 However, ASDB should continue its efforts to improve in several other aspects of its early intervention program. As of June 2012, ASDB had identified the following areas for improvement: ◦ Coordination with other agencies—According to ASDB staff, it is working with other state agencies that are part of the AzEIP program to collectively establish contract services, such as therapists, that will allow referred children and families to begin receiving services more efficiently. According to ASDB staff, ASDB also plans to work with the Arizona Department of Education and the Department of Economic Security to establish uniform guidelines for the use of services provided to children when they transition out of the AzEIP program and into schools. ◦ Staff training and certification—According to an ASDB official, ASDB plans to provide training to its early intervention program staff to ensure that all staff are using a consistent curriculum for early intervention services when they work with children and families. In addition, ASDB reported it is taking steps to ensure that its staff have a Standards of Practice certification as required 1 The five AzEIP participating agencies identified in A.R.S. §8-652 are the Department of Economic Security, ASDB, the Arizona Department of Health Services, the Arizona Health Care Cost Containment System, and the Arizona Department of Education. The Department of Economic Security oversees the program. 2 American Speech-Language Hearing Association, 2008 page 19 Office of the Auditor General As part of the AZEIP program, ASDB is responsible for assessing the child’s sensory impairment service needs and developing a service plan within 45 days of a referral. by the AzEIP program, through which staff demonstrate the knowledge and skills required to provide early intervention services. ◦ Organizational structure and staffing—According to ASDB staff, in June 2011, ASDB appointed a Tucson official to oversee its early intervention program to help ensure the consistent operation of this program state-wide. Prior to this change, ASDB’s regional cooperatives were responsible for coordinating and managing early intervention services to children and their families, which led to some inconsistencies within ASDB’s early intervention program. In addition, ASDB is planning to revise staff job descriptions to more accurately reflect the services it provides to children and their families within the early intervention program and ensure staff clearly understand their responsibilities. ◦ Outreach to families—ASDB has developed a new Web site that will provide more detailed information to the public about resources available through the AzEIP program as well as descriptions of all the programs provided by ASDB, including early intervention, preschool, and school-age programs. In addition, ASDB reported that it plans to use other outreach mechanisms, including social networking sites such as Facebook, YouTube, and Twitter, to reach more families that might benefit from its services. ◦ Delays in starting services—ASDB plans to work with the Arizona Department of Health Services’ Early Hearing Detection and Intervention program to develop a new system that will allow ASDB staff access to infant screening records so they can help ensure any child who fails a screening receives appropriate assessments and other services in a timely manner. In addition, ASDB is providing training to its staff to ensure any delays in services are accurately reported to the Department of Economic Security. The Department of Economic Security uses this information for federal reporting requirements, to assess the reasons for the delay in services, and to take appropriate action, if necessary. ◦ Compliance with state-wide AzEIP program requirements—The Department of Economic Security’s review of ASDB’s early intervention program for the period July 2010 through June 2011 found that ASDB did not provide the required monthly service data to the Department of Economic Security in a timely manner and included inaccurate information in its database, which generates the monthly service data. The Department of Economic Security uses the monthly service data to satisfy federal government reporting requirements for the AzEIP program. The Department of Economic Security also reported that ASDB’s IFSPs do not always contain measurable goals, which are required by federal regulations. Additionally, the review found that ASDB did not obtain sufficient parent survey responses from one region of the State. This survey is used to help page 20 State of Arizona The Department of Economic Security’s review of ASDB’s early intervention program for the period July 2010 through June 2011 found that ASDB did not provide accurate service data in a timely manner. ensure that parents understand their rights regarding early intervention services and that early intervention services have helped their children learn and develop. ASDB has developed performance improvement plans to address areas of noncompliance with AzEIP requirements. Additionally, ASDB Information Technology department has developed a plan to address issues with the current database, and the official responsible for ASDB’s early intervention program provided training to staff in December 2011 to help address data entry practices. • Highly qualified teaching staff important in subject matter as well as in teaching techniques— The federal Individuals with Disabilities Education Act requires schools to hire highly qualified teachers to teach students categorized as having disabilities (see textbox). In addition, the No Child Left Behind Act requires 100 percent of a school’s teachers of core academic subjects, such as English, mathematics, and reading or language arts, to be highly qualified in the subjects they teach. Therefore, all ASDB teachers must be highly qualified in special education, and those who teach core academic subjects must also be highly qualified in the subjects they teach. Experts agree that students make better progress when their teachers are highly qualified in the subjects they teach, and students with disabilities make better progress when their teachers have specialized skills in educating these students. Most, although not all, ASDB teachers are highly qualified in the subjects they teach, according to ASDB. An Arizona Department of Education report shows that approximately 10 percent of ASDB’s teachers were not highly qualified in hearing-impaired or visually-impaired special education and/or in one or more of the subjects they taught during at least part of the 2011-2012 school year.1 ASDB officials reported that the teachers who do not meet the requirements are working toward 1 ASDB reports its teachers’ status to the Arizona Department of Education on October 1 each year, and according to ASDB, updates the database occasionally throughout the year. As of May 7, 2012, the database showed that ASDB had 20 teachers out of the 213 listed who were not highly qualified in hearing-impaired or visually-impaired special education and/ or one or more of the subjects they were teaching. Because ASDB does not review and completely update the database until closer to October 1, the number of teachers who were not highly qualified as of May 7, 2012, is likely higher or lower than 20. page 21 Office of the Auditor General Highly Qualified Teachers—To be deemed highly qualified, teachers must have a bachelor’s degree and full state certification or licensure, and demonstrate competency in each subject taught. Demonstration of Competency—Teachers can demonstrate subject competency with a major in the subject, credits equivalent to a major in the subject, passage of a state-developed test; a combination of teaching experience, professional development, and knowledge in the subject garnered over time in the profession; an advanced certification from the state; or a graduate degree. No Child Left Behind Act Requirements—The No Child Left Behind Act requires that teachers teaching in core academic subjects in public elementary and secondary school are highly qualified. Individuals with Disabilities Education Act Requirements—The Individuals with Disabilities Education Act requires each person employed as a public school special education teacher to be highly qualified as a special education teacher. Core academic subjects—English, reading or language arts, mathematics, science, foreign languages, civics and government, economics, arts, history, and geography. Source: U.S. Department of Education’s Individuals with Disabilities Education Act Web site. certification through professional development plans. According to ASDB, many of these teachers are responsible for teaching multiple subjects, but may be highly qualified in only one core subject. According to ASDB staff, acquiring and retaining highly qualified staff is difficult for many reasons, including a lack of competitive pay and the challenges associated with obtaining the required certifications. Officials at other deaf and blind schools and ASDB officials identified three possible ways that deaf and blind schools can improve their ability to ensure that its students are taught by highly qualified staff. The first way, used by a private school for the blind in Massachusetts—offering tuition reimbursement and free room and board to teachers while they work to become highly qualified—may not be available to ASDB. The second way, increasing the pool of qualified teachers in the State, which was suggested by a practice in Missouri, is already under way in Arizona. Specifically, ASDB is working with the University of Arizona and reported that it is also working with the Arizona Department of Education to increase the number of hearing-impaired and visually-impaired special education teachers in the State. ASDB is working with a University of Arizona masters-level program that is intended to address the shortage of highly qualified teachers needed to serve students who are deaf or hard-of-hearing in Arizona, the Southwest, and the nation. The federally funded program offers online, full- or part-time courses, and its objective is to graduate 30 teachers who will be eligible for certification in the State of Arizona. A third approach used in Florida may offer additional potential for ASDB to better ensure that its students take classes from highly qualified teachers. Specifically, the Florida School for the Deaf and the Blind offers its students the opportunity to take off-campus classes online. Experts have noted that Web-based courses can make courses available to students without having to employ highly qualified full-time teachers in each subject in each school, and that states are choosing to implement online programs for special education students in order to provide highly qualified teachers in subjects where highly qualified teachers may be lacking.1 According to ASDB officials, they are considering using online education classes to offer ASDB students an opportunity to learn from highly qualified subject matter experts, with local assistance from teachers who are highly qualified to teach sensory impaired students. In fact, ASDB tentatively plans to enroll two Arizona School for the Deaf students in math classes at an Arizona online charter school in the 2012- 2013 school year. ASDB should continue to seek opportunities, such as alternative delivery classes, to increase its students’ access to highly qualified teachers. 1 Thomas, W.R. (2002). Funding web-based courses for K-12 students to meet state educational goals. Atlanta, GA: Southern Regional Education Board, and Muller, E. (2009). Serving students with disabilities in a state-level virtual K-12 public school programs. Alexandria, VA: National Association of State Directors of Special Education, Inc. page 22 State of Arizona ASDB is working with a University of Arizona masters-level program intended to address the shortage of highly qualified teachers needed to serve students who are deaf or hard-of-hearing. • Post-graduation tracking can help determine student progress and identify areas that need improvement—Although ASDB has information about students’ status 1 year after graduation, according to ASDB staff, it does not compile or analyze the information and use it to improve its programs. Officials at one state’s deaf and blind schools and a private school stated that tracking students after graduation is necessary to identify areas in the curriculum that could use improvement. For example, a state school for the deaf collected information from its high school graduates to help determine why so many students were dropping out of post-secondary education programs. The information gathered indicated students did not feel prepared for dormitory life. This enabled the school to restructure its high school program so seniors now live in a dormitory setting that allows students to focus on living skills that are required to succeed in college. Similarly, ASDB could use the information collected from recent graduates to improve its programs. Some comments received from seven students and families who responded to auditors’ survey of 2011 ASDB graduates suggest ways ASDB can consider modifying its programs. Specifically: ◦ Providing additional homework help; ◦ Offering group living to facilitate life skills development; ◦ Increasing opportunities to gain work and volunteer experiences; and ◦ Introducing vocational programs earlier. ASDB should use the results of its surveys of students after graduation to measure student progress and to identify and implement enhancements to its students’ educational programs. • Studying differences in test scores can help narrow achievement gaps— The No Child left Behind Act requires that all public schools in every state work towards narrowing achievement gaps in the state as measured by each state’s standardized test, which in Arizona is AIMS. Studying differences in AIMS passing rates across ASDB locations may hold some immediate assessment benefits for ASDB. AIMS test results varied between ASDB’s on-campus students and students receiving services from ASDB’s regional cooperatives and among the regional cooperatives. At ASDB’s three campus schools, 74 to 94 percent of students’ spring math and reading scores fell below standards. According to ASDB officials, on-campus students have higher needs because of their disabilities, which may contribute to lower test scores. For example, some students have secondary physical disabilities that cause absences from school or learning disabilities that cause additional delays in development. However, failure rates also varied between the five regional cooperatives. Students at four of the five regional cooperatives passed AIMS at a higher rate page 23 Office of the Auditor General AIMS test results varied between ASDB’s on-campus students and students receiving services from ASDB’s regional cooperatives and among the regional cooperatives. than the on-campus students. For example, only 49 percent of the Eastern Highlands regional cooperative students failed the reading test, compared to 77 percent of students in the Southwest regional cooperative where results were more similar to results for the on-campus students. Examining these variances more closely may help ASDB officials determine if teaching practices vary among locations and whether higher scores in some regions reflect more effective practices that should be implemented across all locations. To increase student achievement at all the regional cooperatives and on-site campuses, ASDB should determine the reasons for variations in test scores and identify potential solutions to improve test results, and establish expectations that each campus and regional cooperative will implement best practices to improve test performance. Recommendations: 1.1 To help ensure that children with sensory impairments receive needed services, ASDB should improve its early intervention program by continuing to take the following steps: a. Coordinating with other state agencies that are part of the AzEIP program to establish contract early intervention services throughout the State; b. Working with the Arizona Department of Education and the Department of Economic Security to establish guidelines for the use of services provided to children when they transition out of the AzEIP program and into schools; c. Providing staff training to ensure that all staff use a consistent curriculum for early intervention services and that staff accurately reflect delays in starting services in ASDB’s database; d. Ensuring that its early intervention program staff have Standards of Practice certifications as required by the AzEIP program; e. Revising the early intervention staff job descriptions to ensure they accurately reflect the services ASDB provides to children and their families; f. Providing more detailed information to the public about resources available through the AzEIP program, as well as descriptions of all the programs provided by ASDB, including early intervention, preschool, and school-age programs through its newly developed Web site; page 24 State of Arizona page 25 Office of the Auditor General g. Using other outreach mechanisms, including social networking sites such as Facebook, YouTube, and Twitter, to reach more families who might benefit from its services; h. Working with the Arizona Department of Health Services’ Early Hearing Detection and Intervention program to develop a new system that will allow ASDB staff access to infant screening records so they can help ensure any child who fails a screening receives appropriate assessments and other services in a timely manner; and i. Addressing areas identified in the Department of Economic Security’s review of ASDB’s compliance with state-wide AzEIP requirements. Specifically, ASDB should: ◦ Provide accurate and timely monthly service data to the Department of Economic Security; ◦ Improve staff data entry practices and establish procedures that will help ensure the accuracy of data in ASDB’s database and the monthly service reports that are generated from the database; ◦ Ensure that Individualized Family Service Plans contain appropriate and measurable goals as required by the AzEIP program; and ◦ Encourage families to complete and return early intervention surveys. 1.2 ASDB should continue to seek opportunities, such as alternative delivery classes, to increase its students’ access to highly qualified teachers. 1.3 ASDB should establish a process for compiling, analyzing, and using information obtained from surveys about its students after graduation to measure student progress, and to identify and implement enhancements to its students’ educational programs. 1.4 To narrow the achievement gaps and increase the AIMS passing rate among its students, ASDB should: a. Determine the reasons for variations in test scores and identify potential ways to improve test results at the campuses and the regional cooperatives; and b. Establish expectations that each campus and regional cooperative will implement best practices to improve test performance. page 26 State of Arizona ASDB should examine regional cooperative program service fees and strengthen program operations FINDING 2 page 27 ASDB provides services to students at local school districts through its regional cooperative program As authorized by the Legislature, ASDB established a regional cooperative pilot program and started providing services in 1988, and continues to operate five regional cooperatives throughout the State (see Figure 1, page 4). The intent of this program was to enable local school districts to serve deaf, hard-of- hearing, and visually-impaired students in a cost-effective way by providing access to a pool of specially trained personnel and equipment provided by ASDB (see textbox). Regional cooperative staff travel to the participating school districts and provide services to students as they remain at the schools in their districts. Each regional cooperative has a director who supervises operations and various staff, including teachers, interpreters, other specialists, and administrative staff who work with the member districts’ sensory-impaired students. Regional cooperative teachers consult with school staff, classroom teachers, parents, and students; provide specialized direct instruction to students in their areas of need; prepare curricular modifications and demonstrate techniques for working with individual students; and assist in student placement by serving as members of the team that develops each The Arizona State Schools for the Deaf and the Blind (ASDB) should take several steps to improve the operation of its regional cooperative program, including more fully recovering program costs and strengthening the oversight and management of the program. ASDB’s regional cooperative program serves as a resource for school districts by providing staff and equipment to sensory-impaired students in member districts. However, some fees charged by the five individual regional cooperatives are inconsistent, and even for the fees that are consistently applied across all cooperatives, ASDB has no method to determine whether the fees cover the cost of the services provided. In addition to developing a structured approach to review and establish appropriate fees, ASDB needs to review the regional cooperatives’ advisory councils’ role in fee-setting. Additionally, ASDB should develop a consistent way for regional cooperatives to keep track of student needs and staff availability, as well as equipment provided for students’ use. Office of the Auditor General Overview of the regional cooperative pilot program • Provide educational programs and related services to all sensory impaired students if the school participates in cooperative program and cannot provide an appropriate placement for the student within the district. • Provide supplemental services such as audiological and vision assessments, specialized curriculum materials and equipment, and district staff development assistance. • Costs to be paid by participating schools. • Locations decided by ASDB in consultation with districts. • Each cooperative advised by committee, including parents, districts, and local private service organizations. Source: Auditor General staff summary of Laws 1987, Ch. 363, §19. student’s Individualized Education Program (IEP) (see textbox, page 11). Each regional cooperative also has an advisory council that provides advice on the administration of the cooperative. About half of Arizona’s school districts participate in the regional cooperative program. School districts and other public schools, such as charter schools and Bureau of Indian Affairs (BIA) community schools, must sign agreements with ASDB if they elect to receive educational services for their deaf, hard-of-hearing, and visually-impaired students through the regional cooperatives. In the 2011-2012 school year, 114 school districts and 33 other entities participated in the regional cooperatives. In the 2011-2012 school year, the regional cooperatives served 1,190 students in these districts. According to a representative from a participating school district, it benefits the district to participate in the regional cooperative because of the costs the district would incur to obtain the services for its students elsewhere. Many of the 116 districts in the State that did not participate in the regional cooperatives fell into two groups: (1) large districts such as the Mesa, Chandler, and Tucson districts, which use their own staff to provide these services, and (2) very small districts that do not have any sensory-impaired students. The regional cooperatives receive monies from various funding sources to pay for the services provided to the students in the program. Funding sources for services provided to students by ASDB’s regional cooperatives include the following: • State General Fund appropriations—ASDB receives an annual appropriation from the State General Fund for the regional cooperative program. In fiscal year 2013, the appropriation for the program was $803,500. According to ASDB staff, ASDB uses these appropriations to cover its administrative costs for the regional cooperatives, such as office rent, utilities, and administrative staff salaries and employee-related expenses. • School district membership fees—Each participating school district pays a per-district membership fee to belong to a regional cooperative. The membership fees are used to provide services that are not tied to a particular child’s IEP and include assistance with screening and identification of children, evaluation of children, and training of school staff. • Special education vouchers—ASDB receives voucher monies from the Arizona Department of Education for many of the regional cooperative students it serves. Generally, ASDB receives voucher monies for students whose primary disability is a hearing or vision impairment, while districts receive voucher monies for students with multiple disabilities (see textbox, page 3, for the definition of multiple disabilities). ASDB and districts work together to decide which entity will receive the voucher monies. The amount of the educational voucher is determined by statute, and as of the 2011-2012 school year, the education voucher was $19,122 for visually-impaired students and $19,008 for deaf and hard-of-hearing students. If ASDB receives voucher monies for a page 28 State of Arizona In the 2011-2012 school year, 114 school districts and 33 other entities participated in the regional cooperatives. student, ASDB pays an amount to the district to reimburse the district for services the district provides to the student such as transportation and any educational services not provided by the regional cooperative. • Fees for services—When a district rather than ASDB receives the Arizona Department of Education voucher payment, it pays ASDB a fee for the services ASDB provides. These services include ongoing assessment, instruction, provision of equipment, and vocation counseling. The costs for services under the fee-for-service structure depend on the number of hours of service provided. As shown in Table 3 (see page 30) the levels of service, types of service, and annual fee-for-service charges are uniform among all the cooperatives. For example, districts pay $5,245 per year for a student who receives 1.5 hours of services per week, and $21,500 per year for a student who receives 10 to 15 hours of services on a weekly basis. • Additional fees—ASDB receives additional monies from districts for services that exceed what is outlined in the membership fees or fee-for-service amount. For example, one regional cooperative’s membership fee for districts with less than 200 students covers 26 audiological assessments. If the number of assessments exceeds 26, the cooperative charges $20 for each additional audiological assessment. ASDB should examine fees for regional cooperative program services ASDB should systematically examine the various fees that regional cooperatives charge, make them more consistent, and ensure that they cover the costs of the services provided. Although fee-for-service amounts and services are consistent state-wide, many other fees or payments, such as membership fees paid by participating districts and reimbursements paid to districts for transportation and other services, are not consistent across regions. These inconsistencies create inequities in which some districts’ payments subsidize other districts’ expenses. Further, ASDB has not determined if the fees paid by districts for services provided by regional cooperative staff cover the costs of the services. To resolve these issues, ASDB needs to develop and implement a structured approach for determining appropriate fees and payments. Membership and supplemental fees charged to districts and payments to districts are inconsistent—ASDB has not established policies or procedures for determining appropriate membership and supplemental fees for school districts that participate in regional cooperatives or for determining the amounts that the regional cooperatives should pay to reimburse district schools for services provided to students for whom ASDB receives voucher monies. Instead, each regional cooperative has developed its own membership fee and list of services included in the membership fee. In addition, some have established page 29 Office of the Auditor General Each regional cooperative has developed its own membership fee and list of services included in the membership fee. page 30 State of Arizona 1 The rates for BIA community schools are set by a separate process. 2 According to a regional cooperative official, a teacher typically provides an itinerant level service a couple of times a week and the student receives visually-impaired or hearing-impaired services that complement the district teacher’s teaching curriculum. 3 According to a regional cooperative official, a teacher provides a resource level service on a daily basis. The student might receive some services from the district teacher. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Table 3: Services Provided by Regional Cooperative Staff School Year 2011-2012 Service Level Type of Service Hours of Service Service per Year1 Consult indirect Limited direct service 1 hour a month and no equipment provided $2,800 Consult direct Ongoing assessment and limited service 1 hour a month and no equipment provided 2,800 Level I itinerant2 Ongoing assessment, specialized and direct instruction, provision of equipment, vocational counseling, and direct service to student in amplification use and listening skill training. 1.5 hours a week 5,245 Level II itinerant2 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary, speech development, math, science, and social studies. Direct service to student in amplification use, listening skill training, and sign language instruction. 1.5-3 hours a week 7,335 Level III resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary, speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, and sign language instruction. 3-5 hours a week 11,000 Level IV resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary and speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, sign language instruction, sign language interpreting and note taking, and classroom tutor or instructional aide. 5-10 hours a week 16,260 Level V resource3 Ongoing assessment, provision of equipment, and vocational counseling. Direct services to student in language, reading, vocabulary and speech development, math, science, and social studies. Direct services to student in amplification use, listening skill training, sign language instruction, sign language interpreting and note taking, and classroom tutor or instructional aide. 10-15 hours a week 21,500 additional supplemental fees. Further, each regional cooperative has determined an amount it reimburses to districts for services the districts provide to students for whom ASDB receives Arizona Department of Education voucher monies even though the voucher amount is the same regardless of which district or cooperative the student comes from. The resulting amounts vary, and the regional cooperatives explained that the differences have historically existed. Specifically: • Membership fees differ among regions—The regional cooperatives use different tiers for grouping districts and setting membership fees based on student population within the district (see Table 4). For example, the Southwest regional cooperative has three tiers—districts with less than 500 students, 501 to 2,000 students, and more than 2,000 students, with different fees for each tier. However, the Eastern Highlands regional cooperative has five tiers ranging from under 200 students to 5,000 or more students, again with different fees for each tier. Additionally, as shown in Table 4, the range of membership fees charged by each regional cooperative for their various tiers differs for each regional cooperative. For example, the fees for the tier comprising the regional cooperative’s smallest districts ranged from $240 to $500 in 2011- 2012. • Services covered by membership fees differ among cooperatives, and some cooperatives charge supplemental fees for services, while others do not—The services covered by the membership fees also differ from region to region. As a result, some regional cooperatives charge additional fees for supplemental services that some regional cooperatives include in the set of services covered by the membership fees. For example, although most regional cooperatives include district staff training in the membership fees, one regional cooperative charges a separate fee of $25 per person to train district staff in conducting hearing screenings. Another regional cooperative has additional charges for certain services if they are provided to fee-for-service students—$71 per hour for mobility and orientation training and $45 per hour for interpreter services. The other three cooperatives have not established similar additional charges and instead provide these types of services with no additional charges. • Reimbursements to districts inconsistent and not based on actual services provided by districts—As previously mentioned, ASDB receives voucher monies for students whose primary disability is a hearing or vision impairment and for whom ASDB assumes the primary educational responsibility within page 31 Office of the Auditor General Table 4: Regional Cooperative Membership Fee Structure School Year 2011-2012 1 Bureau of Indian Affairs community schools. Source: Auditor General staff analysis of data received from ASDB’s regional cooperatives. Membership fees and supplemental fees differ among regions— Regional Cooperative Number of Tiers Membership Fee Southeast 4 $260 up to $820 Desert Valleys 5 $280 up to $1,000 North Central BIA Schools1 4 5 $240 up to $720 $280 up to $1,000 Eastern Highlands 5 $280 up to $1,000 Southwest 3 $500 up to $1,250 these students’ home school districts. If ASDB receives voucher monies for a student, ASDB reimburses the district for services the district provides to the student such as transportation and any educational services not provided by the regional cooperative. However, the regional cooperatives have set differing reimbursement amounts, even though the voucher amount is the same regardless of which district or cooperative the student comes from. As shown in Table 5, the per-student amounts range from no reimbursement for students in the southeast region who are receiving specified services from the regional cooperative to $3,500 annually for all students in two regions and for some students in a third region. The cumulative result of these inconsistencies can be substantial. Table 6 (see page 33) shows examples of the differences that can result, both for voucher students—those students for whom ASDB receives the Arizona Department of Education voucher— and for those students for whom the district receives the voucher. For a visually-impaired voucher student, regional cooperative 1 receives a membership fee of $260 and reimburses the district $2,000, while regional cooperative 2 receives a membership fee of $500 and reimburses the district $3,500. As a result, regional cooperative 1 has $17,382 to cover the students’ costs, while regional cooperative 2 has $16,122 to cover the students’ costs, which is a difference of $1,260. For fee-for-service students, the difference is less—$240—because it relates only to the membership fee. As a result, the regional cooperatives can have a different amount of monies per student to cover the same services. ASDB unable to determine if fees are adequate to cover costs— ASDB policy requires its regional cooperatives’ advisory councils to recommend a fee structure that will cover all costs of services that are not covered by legislative appropriations. However, ASDB has not evaluated whether the membership fees and fee-for-service amounts cover the costs of the services provided to regional cooperative students. Although ASDB does not have records of past fees, officials at three regional cooperatives reported that fees had not been changed since at least 2007, and a fourth regional cooperative official said fees had not been changed since before 1997. The fifth regional cooperative official reported that fees had not changed since before 1992. Additionally, in fiscal year 2011, the ASDB Cooperative Services Fund’s fund balance decreased, suggesting the fee amounts may be inadequate to cover costs. Two regional cooperatives prepare a report that shows some of the revenues and expenses associated with a district’s students if the district requests the report. However, because this report does not include all revenues and expenses and is prepared by only two of the five regional cooperatives, ASDB cannot rely on it to determine whether its fees are adequate. page 32 State of Arizona Table 5: Regional Cooperative Reimbursements to Districts School Year 2011-2012 1 Without educational interpreter, instructional assistant, orientation and mobility, and rehabilitation therapy. 2 With educational interpreter, instructional assistance, orientation and mobility, rehabilitation therapy, and educational transcriber. 3 If the student receives services involving two or more staff. 4 If a full-time sign interpreter or instructional assistant is assigned to the regular education classroom. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Regional Cooperative Reimbursement Amounts Per Voucher Student Southeast $2,0001 or $02 Desert Valleys $3,500 North Central $2,700 or $1,5003 Eastern Highlands $3,500 Southwest $3,500 or $1,5004 ASDB has not evaluated whether the membership fees and fee-for-service amounts cover the costs of the services provided to regional cooperative students. page 33 Office of the Auditor General Although regional cooperatives independently establish their fees, ASDB combines the revenues to pay regional cooperatives’ costs throughout the State. For example, at the end of fiscal year 2011, the North Central regional cooperative’s portion of the ASDB Cooperative Services Fund’s fund balance was negative $129,118, while the Desert Valleys regional cooperative’s portion was more than $2.5 million. ASDB used monies from the Desert Valleys regional cooperative to purchase 29 cars for all of the regional cooperatives and to pay for a conference attended by all regional cooperative staff. In effect, Desert Valleys regional cooperative member school districts were subsidizing other regional cooperatives’ expenses. ASDB needs to adopt structured approach to establishing fees—To address the problems regarding inconsistencies in some fees and to determine whether its fees cover the cost of services provided, ASDB needs to adopt a structured fee-setting approach. A structured approach would allow ASDB to evaluate current fees and propose new fees that would (1) fully cover costs related to the services provided by the regional cooperatives and (2) ensure that specific fees are appropriate for the services. Mississippi’s Joint Legislative Committee on Performance Evaluation and Expenditure Review (PEER) developed an approach for evaluating and setting fees that may assist ASDB.1 PEER’s approach consists of a decision model for establishing or increasing government fees, called the Theory of Fee Setting in Government, as well 1 Joint Legislative Committee on Performance Evaluation and Expenditure Review. (2002). State agency fees: FY 2001 collections and potential new fee revenues. Jackson, MS: Author. Source: Auditor General staff compiled information from ASDB’s regional cooperatives. Table 6: Example of Differences in Regional Cooperative Service Costs School Year 2011-2012 Voucher Students Regional Cooperative 1 Regional Cooperative 2 Difference Membership fee $ 260 $ 500 $ 240 Voucher 19,122 19,122 0 Reimbursement to district (2,000) (3,500) (1,500) Total per student $17,382 $16,122 $1,260 Fee-For-Service Students Regional Cooperative 1 Regional Cooperative 2 Difference Membership fee $ 260 $ 500 $240 Level II itinerant 5,245 5,245 0 Total per student $ 5,505 $5,745 $ 240 Although regional cooperatives independently establish their fees, ASDB combines the revenues to pay regional cooperatives’ costs throughout the State. page 34 State of Arizona as guidance on implementing new fees.1 Figure 3 (see page 35) summarizes key concepts from PEER’s approach. ASDB’s approach should include the following: • Assessing efficiency of operations—ASDB should assess the efficiency of its operations to ensure costs are as low as possible and document the results of its assessment. As ASDB assesses the efficiency of its operations, it should seek to minimize costs where possible. • Developing a cost-accounting method—ASDB should determine whether to consider costs independently for each regional cooperative or in combination on a state-wide basis, and then develop and finalize a method for tracking and allocating relevant service and equipment costs. • Developing fees based on relevant costs—To help ensure fees are appropriate and equitable, ASDB should identify the actual costs for specific services for which fees are charged. In addition, fees should take into account factors that affect the cost of the specific service. For example, some regional cooperatives may incur more costs for transportation because these regional cooperatives must transport staff and/or equipment greater distances to reach the districts they serve. Once developed, ASDB should use this approach to assess its fees and establish new fees. If the new fees are substantially higher than existing fees, ASDB should consider phasing in the increases over time. In addition, ASDB should develop and implement policies and procedures for using the method to establish new fees. ASDB should also develop a systematic way to determine whether and how much to pay school districts for services provided to students for whom ASDB receives Arizona Department of Education voucher monies. ASDB is not statutorily required to pay the districts, but it has historically chosen to reimburse the districts for transportation and other services. If ASDB continues its practice of reimbursing districts, it should develop a process to determine the appropriate amount of these reimbursements. Regional councils’ role in fee-setting needs review As ASDB develops a structured approach to setting fees, it also needs to examine the fee-setting role played by the regional cooperatives’ advisory councils. Each regional cooperative has an advisory council comprising representatives from 1 According to PEER, the approach was based on a review of academic literature, economics theory, and policies and procedures from various states and the United States and Canadian governments. If ASDB continues its practice of reimbursing school districts, it should develop a process to determine the appropriate amount of these reimbursements. page 35 Office of the Auditor General Figure 3: Mississippi Joint Legislative PEER Committee’s Structured Fee-Setting Process Developed for State Government Source: Auditor General staff analysis of fee-setting model. Determine whether fees or taxes should fund the service Who benefits from the service: individuals, the public, or both? Fees should finance services that benefit individuals. Taxes should finance services that benefit the public. When both individuals and the public benefit from a service, financing can come from both fees and taxes. Identify and analyze legal issues Are fees limited by statute? If so, is legislation required to change them? Should administrative rules be revised? Identify the fees’ purpose Should fees cover the cost of providing the service? Should fees be set to influence behavior? Should fees be set to encourage compliance with program regulation and goals? Assess factors influencing fee amount What effect will fees have on those who pay them? What effect will fees have on annual revenue? What do similar states charge for the service? Will the public accept the fees’ necessity? Is the Department subsidizing other government operations? Determine appropriate methodology for setting fees Determine if there is a comprehensive cost accounting system. Seek to reduce costs as much as possible. Measure direct and indirect costs of the time staff spends in service activities. Determine economic impact on regulated entities. Implement fees Obtain amended legislation and regulation as needed. Prepare those who pay fees for changes by providing advanced notice and explaining the purpose and reasoning of new fees. Train staff to answer questions regarding the new fees. Periodically assess revenue, costs, and program outcomes to update fee amounts page 36 State of Arizona participating public schools and ASDB staff. These councils have taken on the responsibility of approving membership fees and reimbursement amounts. Although neither statute nor ASDB policy requires an advisory council vote to increase a regional cooperative’s membership fee or propose other changes in reimbursement amounts, at least four of the regional cooperatives rely on the council members’ vote to do so. The regional cooperatives hold annual or more frequent meetings with their respective advisory councils. During these meetings, regional cooperative staff inform council members of the regional cooperative budget, the number of participating member districts, and any other important information affecting the operations of the regional cooperative. For two reasons, setting fees appears to be an inappropriate role for the advisory councils. First, it goes beyond the role for these councils established in ASDB policy. Under ASDB policy, the councils’ duties include making recommendations about membership fees, but not actually setting them. By allowing the councils to determine fees instead of providing recommendations for ASDB management to consider, regional cooperative staff appear to have misinterpreted the advisory council’s responsibilities. Second, in the kind of structured fee-setting approach discussed earlier, the information needed to set appropriate fees would be most fully available to management, and not the advisory councils. ASDB should modify its policy to remove the provision that advisory councils will recommend a membership fee structure. Additionally, ASDB should determine whether to continue with advisory councils for each regional cooperative or if it would be more appropriate to establish a single advisory council for all of the regional cooperatives. There is no statutory or regulatory requirement for the regional cooperatives to have advisory councils. Instead, the advisory councils have evolved over time from earlier requirements outlined in the regional cooperative pilot program legislation, which is no longer in place. Specifically, when the regional cooperative pilot program was introduced in 1987, legislation required ASDB to establish an advisory committee for each cooperative to provide advice on the administration of the cooperative. The advisory committee included parents, representatives from local private organizations that provide services to the sensory impaired, and representatives of participating school districts. Current regional advisory councils do not include parents or representatives of local private service organizations. The administrative duties outlined in ASDB policy regarding advisory councils could be addressed by a state-wide council that includes representatives of participating districts throughout the State and could also include parents and representatives of private service organizations. Alternatively, ASDB could consider retaining the regional advisory councils but include parents and service organization representatives, similar to the way the councils were envisioned in the pilot program legislation. ASDB should determine and implement the appropriate structure for the advisory councils. Regardless of how the advisory councils are structured, ASDB management needs ASDB should determine whether to continue with advisory councils for each regional cooperative or if it would be more appropriate to establish a single advisory council for all of the regional cooperatives. page 37 Office of the Auditor General to provide oversight to ensure council authority in practice does not exceed the advisory role outlined in ASDB policy. Regional cooperatives are not consistent in tracking resources Regional cooperatives do not use the same systems to track resources, meaning that students potentially may not receive the resources they need. Specifically: • Education staff services—The regional cooperatives use different systems to keep track of student needs, such as the need for an educational interpreter, and the availability of educational staff such as interpreters, teachers, and therapists. Each regional cooperative uses either a computer program such as Microsoft Access or hard copy forms to manage its student caseload. • Equipment inventory—The regional cooperatives use different systems to track and manage the educational equipment provided to the students in local school districts, such as braille writers and educational books. Some regional cooperatives use a hard copy inventory tracking system, and others do not use an inventory tracking system at all. Additionally, ASDB lacks a state-wide inventory system to track equipment owned by all of its regional cooperatives. Consistent tracking could help ensure that students have access to all the educational services and equipment they need. For example: • Providing resources when students move across cooperatives—Regional cooperative officials reported that students move from one cooperative to another, and because the regional cooperatives do not use the same computer programs to track educational services, it can be hard to transfer the services to the student’s new school. • Identifying underused resources—Because there is no state-wide inventory of equipment resources, some regional cooperative officials reported that they informally call one another if they need equipment that might not be in use by another regional cooperative. Without a state-wide inventory, equipment could be going unused in one region that could benefit students in other regions. • Training staff—Regional cooperative officials also stated that a single computer program used by all regional cooperatives would make it easier to train staff and IT support staff on how to use the program to track these services. ASDB could use systems it has developed or is developing for other programs to better track and manage educational services and equipment in the regional cooperatives. Specifically: ASDB lacks a state-wide inventory system to track equipment owned by all of its regional cooperatives. page 38 State of Arizona • ASDB is developing an in-house computer program to track and manage the schedules of the education services in the birth to 3 program discussed in Finding 1 (see pages 9 though 25). This program administers these services throughout the State. Once developed and tested, the regional cooperatives should use this program to track and manage the educational services that are provided to students by the regional cooperatives. • As of January 2012, ASDB purchased and implemented an inventory system to track on-campus assets. The regional cooperatives should use this same system to track and manage the inventory of the equipment provided to the students in the regional cooperatives. Recommendations: 2.1 To ensure its fees more fully reflect its costs, ASDB should develop a structured approach to evaluate current fees and implement new fees that would cover all costs related to the services provided by the regional cooperatives that are not covered by legislative appropriations, and ensure that specific fees are appropriate for the services. In developing this approach, ASDB should do the following: a. Assess the efficiency of its operations to ensure costs are as low as possible and document the results of its assessment. As ASDB assesses the efficiency of its operations, it should continue seeking to minimize costs where possible; b. Determine whether to consider costs independently for each regional cooperative or in combination on a state-wide basis, and develop and implement a method for tracking and allocating relevant ASDB costs; c. Identify the actual costs for specific fees, including membership fees, fee-for- service costs, and additional supplemental service costs to help ensure fees are appropriate and equitable. In addition, fees should take into account factors that affect the cost of the specific service; and d. Develop and implement policies and procedures for using the method to develop appropriate fees. 2.2 ASDB should develop a systematic way to determine whether and how much to pay school districts for services the districts provide to students for whom ASDB receives Arizona Department of Education voucher monies. page 39 Office of the Auditor General 2.3 ASDB should provide more oversight to ensure that advisory councils play an appropriate role in the regional cooperative program by: a. Modifying its policy to remove the provision that advisory councils will recommend a fee structure; b. Determining and implementing the appropriate structure for the advisory councils, such as a single state-wide advisory council composed of participating school district representatives and parents and representatives of local private service organizations, or regional advisory councils that include parents and local private service organization representatives; and c. Ensuring that its councils adhere to the advisory role and responsibilities outlined in ASDB policy. 2.4 ASDB should establish a single, consistent system for managing and tracking regional cooperative resources by: a. Expanding the use of ASDB’s in-house computer program for tracking and managing birth to 3 program educational services, once the program has been developed and tested, to track and manage educational services provided to students in the regional cooperatives; and b. Using ASDB’s inventory system for on-campus assets to track and manage the inventory of equipment provided to students in the regional cooperatives. page 40 State of Arizona ASDB needs to improve its information technology practices FINDING 3 page 41 IT systems used extensively and contain sensitive data ASDB uses IT systems extensively. In addition to administrative functions such as payroll and accounting, ASDB’s IT systems track and monitor student information, including personal and medical information regarding students’ hearing or vision impairments and other disabilities, as well as information about their educational performance. This type of information is private, and two federal laws—the Health Insurance Portability and Accountability Act (HIPAA) and the Family Educational Rights and Privacy Act (FERPA)—require that such information be protected against unauthorized disclosure. ASDB also uses IT in the classroom as part of a variety of adaptive teaching and learning techniques for its students with hearing and vision impairments. ASDB’s IT department, which consists of nine full-time employees, supports technology for staff and students across its three campuses and five regional cooperatives. This support includes administering and supporting user workstations, servers, assistive and educational technologies, critical business applications and services, and various other network devices. ASDB has not addressed critical IT weaknesses State information security experts and auditors have identified critical weaknesses in ASDB’s IT environment. These weaknesses have persisted for several years. In 2009, ASDB engaged ADOA’s Information Security group to assess ASDB’s information technology environment. The resulting report identified several gaps between the existing state of ASDB’s IT environment and best practices and made recommendations for how ASDB should address the deficiencies. The report also found that ASDB’s IT department had received little strategic direction or oversight from ASDB management. Although ASDB has made some efforts to address the weaknesses ADOA identified, auditors found that critical IT weaknesses still exist. Specifically, auditors found that ASDB’s IT security controls are weak, its disaster recovery planning is inadequate, its data backup strategy is flawed, and it has no data classification process to help ensure that the information it maintains is sufficiently protected. ASDB officials stated that the IT department lacks the Although the Arizona State Schools for the Deaf and the Blind (ASDB) has made some improvements to its information technology (IT) practices since the Arizona Department of Administration (ADOA) assessed those practices in 2009, several additional improvements are needed to help ensure that student and school information is properly safeguarded. Specifically, auditors found that continued weaknesses have led to critical vulnerabilities in several IT areas. These include IT security management, disaster recovery, and data backup. ASDB should first prioritize and then correct these IT weaknesses to minimize the impact these vulnerabilities and security threats could have on its operations. Office of the Auditor General page 42 State of Arizona staff and resources necessary to address all of ASDB’s IT issues and needs. As a consequence, it has not prioritized addressing the recommendations made in ADOA’s report, which has contributed to the lack of progress in addressing and resolving weaknesses. In October 2011, the department had turnover in a key position when ASDB’s IT Director left. At that time, the existing IT security specialist was made the interim IT Director but also retained his responsibilities for security. As of June 2012, ASDB had still not appointed a permanent IT Director. IT security controls are weak—ASDB’s controls over IT security are weak, and its systems are susceptible to attack. According to IT standards and best practices, effective security management helps protect IT assets and minimizes the impact that security vulnerabilities and incidents could have on IT operations. Security monitoring is also essential to help ASDB comply with federal laws and regulations designed to protect sensitive information, financial aid records, and health information. Despite the critical importance of effective IT security management, auditors found weaknesses in several key areas, including ASDB’s efforts to assess risk and monitor its systems, secure its net |