Performance audit, Department of Economic Security Division of Children, Youth and Families Child Protective Services on-the-job training and continuing education

              Debra K. Davenport
Auditor General
Performance Audit
Department of Economic
Security–Division of Children, Youth
and Families—Child Protective Services—
On-the-Job Training and Continuing Education
Performance Audit Division
APRIL • 2006
REPORT NO. CPS-0601
A REPORT
TO THE
ARIZONA LEGISLATURE
The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Representative Laura Knaperek, Chair Senator Robert Blendu, Vice Chair
Representative Tom Boone Senator Carolyn Allen
Representative Ted Downing Senator John Huppenthal
Representative Pete Rios Senator Richard Miranda
Representative Steve Yarbrough Senator Harry Mitchell
Representative Jim Weiers (ex-officio) Senator Ken Bennett (ex-officio)
Audit Staff
Melanie Chesney, Director
Dot Reinhard, Manager and Contact Person
Paul Christiansen, Team Leader
Sandra Tulloss
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.azauditor.gov
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
WILLIAM THOMSON
DEPUTY AUDITOR GENERAL
April 27, 2006
Members of the Arizona Legislature
The Honorable Janet Napolitano, Governor
Mr. David Berns, Director
Department of Economic Security
Transmitted herewith is a report of the Auditor General, A Performance Audit of the
Department of Economic Security, Division of Children, Youth and Families—Child
Protective Services—On-the-Job Training and Continuing Education. This report was
conducted under the authority vested in the Auditor General by Arizona Revised
Statutes §41-1966.
The report addresses the Division’s need to further ensure that all new case managers
receive consistent and effective on-the-job (i.e., field) training, and to finalize a formal
continuing-education program for CPS staff. National child welfare organizations
identify training as essential in providing quality services and achieving better
outcomes for children and their families, and in retaining competent staff. Further, they
recommend that agencies provide or require staff to receive continuing education to
improve their skills and effectiveness.
Although the Division reports placing more emphasis on field training consistency, field
training continues to differ between trainees. For example, case managers who
completed training in 2005 found differences in the quantity and type of field training
they received. A similar issue was reported in a 2003 performance audit (see Auditor
General Report No. 03-09). To ensure that new case managers receive consistent and
effective field training, the Division needs to adopt comprehensive training policies,
finalize its draft field training manual, formally train all supervisors who provide field
training, and better monitor whether trainees receive required and effective field
training.
The report also found that the Division provides continuing education opportunities for
its CPS staff and is developing a formal continuing-education program. In doing so, the
Division should establish a timeline for completing the program, formalize its
continuing education requirements in policy, ensure that staff receive the required
continuing education, and monitor the program’s effectiveness.
As outlined in its response, the Department of Economic Security agrees with both
findings and plans to implement all of the recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on April 28, 2006.
Sincerely,
Debbie Davenport
Auditor General
DD:Acm
Enclosure
Office of the Auditor General
TABLE OF CONTENTS
page i
1
7
7
9
13
15
17
17
18
19
19
22
23
2
Introduction & Background
Finding 1: Division needs to do more to ensure
all new case managers receive consistent
on-the-job training
Case managers’ field training experiences continue to vary
Division needs to expand guidance and oversight
Districts and training supervisors using unique and potentially
helpful training tools
Recommendations
Finding 2: Division needs to continue efforts to
implement continuing education program
Continuing education important
Division provides some continuing education opportunities
Division developing a formal continuing education program
Division needs to address several areas while developing its
formal continuing education program
Recommendations
Agency Response
Figure:
1 Number of Offices by District
Fiscal Year 2006
State of Arizona
Office of the Auditor General
INTRODUCTION
& BACKGROUND
page 1
The Office of the Auditor General has conducted a performance audit of the
Department of Economic Security, Division of Children, Youth and Families’ (Division)
efforts to ensure that new Child Protective Services (CPS) case managers state-wide
receive consistent field training and to develop a continuing education program for
CPS staff. This audit was conducted under the authority vested in the Auditor General
by Arizona Revised Statutes §41-1966.
CPS’ Training Institute provides most CPS caseworker
training
The Division established the Child Welfare Training Institute in 2002 to
administer state-wide training for CPS management and staff.1 The
Training Institute oversees the development and delivery of training for
CPS staff, including case managers, supervisors, and management. The
Child Welfare League of America (CLWA) standards identify staff training
as an essential element in the prevention of child abuse and neglect.
Preparing staff to work with families in a child protective services context
is a critical component of providing quality services and retaining
competent staff.2 Further, the National Child Welfare Resource Center for
Organizational Improvement reports that training is an important part of
achieving better outcomes for children and their families.3
To help provide quality services to Arizona’s children and families, the
Training Institute provides classroom training to new case managers, new
supervisors, and other CPS staff, and provides ongoing training for all CPS staff on
specialized CPS topics and on the Division’s automated case management system,
CHILDS. In addition to the Training Institute courses, CPS staff also receive training
1 The Division reports that before the Training Institute was established, a unit within the Division provided training.
2 Child Welfare League of America. CWLA Standards of Excellence for Services for Abused or Neglected Children and
Their Families. Washington, D.C.: Child Welfare League of America, 1999.
3 Kanak, Susan, Susan Maciolek, and Mary O’Brien. The Training System Assessment Guide for Child Welfare Agencies.
Portland, Maine: University of Southern Maine, Muskie School of Public Policy, Institute for Child and Family Policy,
2005.
Training Institute’s Mission
. . . to provide timely, effective,
and culturally sensitive
professional development for the
Division’s staff and establish
partnerships with others involved
in child welfare to enhance the
safety, permanency and well-being
of children and families.
Source: Division of Children, Youth and Families.
State of Arizona
page 2
from the Department and from the Arizona Government University, such as training
on performance evaluations, workplace violence, and general supervision.1
Case manager training key focus of Training Institute
Most of the Training Institute’s time and effort is spent training new case managers.
Training Institute staff report that the purpose of this new staff training is to provide
new case managers with a good foundation to perform their job responsibilities. The
training is designed to provide practical experiences and covers the basics of case
management, such as investigating a report of abuse or neglect, removing a child
from the home, and developing and managing the case outcome goals and the
family’s progress toward meeting these goals. New case managers are to be
provided approximately 22 weeks of training in the following three phases:
􀁺􇩐 Pre-core training involves administrative items and job
exposure—During the first 2 weeks of employment, new
case managers are provided general employment
information and may also have an opportunity to observe
some of the specific tasks they will be doing as a case
manager. CPS case managers are assigned to work in one
of 68 offices in 6 regional districts across the State (see
Figure 1).2 A trainer from the Training Institute who is
assigned to the new case managers’ district meets with the
new case managers in a small group setting or one-on-one
in their offices to provide this training. New case managers
are provided with an overview of the district and office
within the district to which they have been assigned. In
addition, new case managers receive information about the
Division’s automated case management system and other
basic district- and state-specific material relating to their
new job, such as employment policies, timesheet and
travel procedures, and the performance evaluation
process. New case managers may also have the
opportunity to shadow experienced CPS staff to observe
firsthand some of the duties they will be required to
perform, such as accompanying a case manager on a
home visit or an investigation.
1 Arizona Government University administers the State’s centralized employee training activities. It develops standardized
curriculum, provides agencies with information on outside training vendors, and partners with the State’s community
colleges and universities to provide college credit for some courses. Its courses include ethics, management
development, health and safety, and time and project management.
2 In addition to the 68 CPS offices, the Division has case management staff assigned to 7 nondepartment locations, such
as the Mesa Center Against Family Violence.
District IV District III
District I
District V
District II District VI
12 Offices
10 Offices
8 Offices
19 Offices
9 Offices
10 Offices
Figure 1: Number of Offices by District
Fiscal Year 2006
Source: Auditor General staff analysis of the Division of
Children, Youth and Families' Directory of Child
Protective Services Offices.
Office of the Auditor General
page 3
􀁺􇩃 Core training consists of classroom training—Following pre-core training,
new case managers receive 6 weeks of core classroom training. The 177 hours
of classroom training are primarily provided by Training Institute staff and cover
the basics of case management, such as investigating a report of abuse or
neglect, removing a child from the home, and developing and managing a case
plan.1 In January 2005, the Training Institute extended its case manager core
training from 4 weeks to 6 weeks to add more training on CHILDS and more
applicable, hands-on exercises, such as an exercise to identify the community
and agency services that a family might need and an exercise on completing
referrals for services to be provided to the child and/or the child’s family. The
Training Institute also modified the presentation of these topics to follow the “Life
of a Case” model. In this model, the case management topics are presented
following the development of a case, from the initial child abuse and neglect
report to CPS’ state-wide, toll-free, 24-hour hotline, through the CPS
investigation, provision of services, and removal, to case closure.
A new segment of this classroom training is begun approximately every 8 weeks
in Phoenix, Tucson, and Prescott Valley.2 The Training Institute attempts to
assign new case managers to the training facility closest to their home office.
However, this may not always be possible depending on the date when the case
manager is hired and whether there is any space available in the facility closest
to his/her home office.
Beginning in January 2006, case managers attending classroom training return
to their home office after the third week for a 1-week field break. The Training
Institute reports that this break was put in place to produce a better learning
environment and to improve learning retention. This break week occurs after the
new case managers have received training on topics such as interviewing, and
building and maintaining relationships with clients. During the break week, new
case managers have the opportunity to practice the information presented in the
classroom training with experienced workers and to participate in actual case
visits. The new case managers then return to the classroom to complete the
case development training, which includes such topics as child removal,
permanency planning, and working with the courts.
􀁺􇩆 Field training involves providing on-the-job training—The last and longest
segment of the new case manager training program is the field training phase.
Field training, which is to last approximately 13 weeks, is intended to allow new
case managers to integrate and apply the knowledge they gained in classroom
training while experiencing the day-to-day activities of work in the field. Each
1 The Training Institute reports that it purchased its case manager training package from the Institute for Human Services
in Ohio more than 10 years ago and has modified it for Arizona’s specific needs.
2 The Prescott Valley location opened in October 2005.
State of Arizona
page 4
CPS case manager, whether new or experienced, works
under the direct supervision of a CPS unit supervisor,
and this supervisor often provides the field training or
may have an experienced case manager assist the new
case manager. In 2002, the Division established field
training supervisors in each district, and these positions
are responsible for overseeing the new case managers’
development and coordinating with the case manager’s
unit supervisor to provide field training.1
During field training, division policy limits the number of
cases the new case managers may be assigned. Also,
during the field training period, the new case managers
are supposed to observe experienced case managers
performing a variety of activities that are designed to
help the new case managers understand their job
responsibilities. The Division has developed a one-page
field training checklist form that lists nine activities that
new case managers are to perform or observe, such as
accompanying an experienced case manager on two
investigations (see text box at left).
In addition to the checklist activities, the Division has
developed forms for the new case managers to
complete as they observe experienced case managers
performing tasks such as conducting an investigation or
testifying at a court hearing. Although not required to be
completed, these forms allow the new case managers to
document their understanding of the activity’s process,
the techniques used by the case manager, and the
reasons for the outcome.
Staffing and budget
To accomplish its mission, the Training Institute has several staff and three training
facilities located around the State. The Training Institute, which as of April 2006 has a
total of 26 full-time and 3 part-time positions with 7 vacancies, is led by an
administrator who reports directly to the Division’s Assistant Deputy Director. The
remaining staff are divided into three main functional groups:
􀁺􇩃 Classroom trainers—The Training Institute has 14 full-time and 3 part-time
positions, with 5 vacancies, to manage and provide classroom training. These
Division’s Field Training Checklist
1. Log on to the automated case management system while
in the field.
2. Accompany an experienced case manager on two
investigations and review their assessments of risk factors
and child safety.
3. Conduct a joint interview with an experienced worker.
4. Observe a removal review staffing, i.e., a meeting of CPS
staff and Foster Care Review Board representatives, to
review the decision to remove a child from his/her home
(as it applies to the worker’s job function).
5. Observe a case manager on two home visits/placement
visits.
6. Observe a case plan staffing, i.e., a meeting of CPS staff,
the family, and other professionals as necessary, to
develop or reassess the case outcome goals and the
family’s progress toward achieving them (as it applies to
the worker’s job function).
7. Observe a preliminary protective conference and hearing,
i.e., dependency hearing.
8. Observe a report and review hearing, i.e., court review of
the progress being made on the child’s case (as it applies
to the worker’s job function).
9. Observe a Foster Care Review Board hearing.
Source: Division of Children, Youth and Families’ CPS Pre/Post Core Exercise Checklist.
1 The Training Institute coordinates the field training for new case managers in five of the six CPS districts. District 1
(Maricopa County) offers its own field training program. However, District 1 staff report that the District coordinates with
the Training Institute on a regular basis and that its field training program is similar to the Training Institute’s field training
program.
trainers are divided into two units, with one covering the classroom training in
Phoenix and the other covering the classroom training in Tucson and Prescott
Valley.
􀁺􇩆 Field trainers—The Training Institute has 6 full-time positions to coordinate the
field, or on-the-job, training provided to new case managers across the State.1
􀁺􇩓 Support staff—The Training Institute has 5 full-time positions, with 2 vacancies,
to provide training at the classroom sites and some centralized functions, such
as scheduling and recordkeeping. The Training Institute is also making use of 1
temporary position to assist in this area.
The Division reported that in fiscal year 2005, the Training Institute had $2.8 million of
training costs. These costs include personal services and employee-related costs for
both the Training Institute staff and for the new case managers during their 22-week
training period. The Division indicates that these costs are primarily funded by the
State’s General Fund appropriations and federal foster care grant (Title IV-E) monies.
Specifically, it reports that 54 percent of these training costs were funded by state
appropriations, 45 percent were funded by federal monies, and the remaining 1
percent were funded from other sources, such as court and other fees deposited in
the CPS Training Fund.
Office of the Auditor General
page 5
1 As of March 2006, District 1 had 3 FTEs, with 1 vacancy, to provide field training to new case managers.
State of Arizona
page 6
Division needs to do more to ensure all new case
managers receive consistent on-the-job training
The Division needs to take additional steps to ensure that new case manager field or
on-the-job training does not continue to differ from trainee to trainee. Consistent with
the Auditor General’s 2003 performance audit finding, interviews with case managers
who completed field training in 2005 found differences in both the quantity and type
of field training they received. To ensure that new case managers receive consistent
and effective training, the Division needs to adopt comprehensive training policies,
finalize its draft field training manual, formally train all supervisors who provide field
training, and better monitor whether trainees receive required and effective field
training. Finally, the Division is considering whether unique training practices being
piloted or used within some of the districts can be expanded to other districts.
Case managers’ field training experiences continue to
vary
The field training provided to new case managers continues to differ from trainee to
trainee. Auditors first identified this problem in 2003. Despite some division efforts to
address this concern, auditors’ interviews with case managers found that
inconsistencies continue to remain in both the quantity and type of field training
provided.
Prior audit identified field training inconsistencies—A 2003 performance
audit (see Auditor General Report No. 03-09) identified several concerns with
training provided to new case managers, including that the field training provided
differed from trainee to trainee. Specifically, interviews and a letter from a group of
case managers indicated that trainees were not consistently given the opportunity
to participate in the same type of on-the-job training activities, such as conducting
an interview with clients. As a result, auditors recommended that the Division take
Office of the Auditor General
page 7
FINDING 1
steps to ensure that the field portion of new case manager training is consistently
implemented state-wide.
Case managers report receiving inconsistent field training—The
Division needs to take additional steps to ensure that field training provided to
trainees is consistent. Although the Division reports placing an increased
emphasis on field training consistency by taking such steps as standardizing a
nine-item checklist of activities that trainees are to experience during field training
(as shown on page 4), these efforts have not been sufficient to resolve the
problem. Rather, when 24 case managers who enrolled in new case manager
training in calendar year 2005 were asked by auditors to describe the type of field
training they received, their responses indicated that new case manager field
training continues to differ from trainee to trainee. Specifically:
􀁺􇩖 Variations in the quantity of training—New case managers reported
receiving different amounts of field training. For example, 10 of the 24 case
managers’ descriptions about the field training they received indicated that
they received either a minimal amount or no field training. In contrast, 3
other case managers indicated that they received a large amount of field
training. Additionally, case managers volunteered a variety of overall
opinions about their field training, ranging from “worthless” to a “great
experience.”
􀁺􇩖 Variations in field training activities—Most case managers reported
participating in certain important activities, but other important activities
were not as consistently reported. For example, 21 of the 24 case
managers indicated that they shadowed co-workers. Similarly, 15 of the 24
case managers indicated that they had met with their unit supervisor to
discuss their progress on assigned cases, although sometimes it was at
the case manager’s request, rather than being initiated by the supervisor.
However, other important opportunities were not as consistently
mentioned. For example, only 9 of the 24 case managers stated that a co-worker
or supervisor had accompanied them on job-related activities. In
addition, only 5 of the 24 case managers reported being assigned a
mentor or coach during field training, and 3 of 24 reported receiving hands-on
instruction and practice with CHILDS.
Because field training is an important aspect of new case manager training, a lack
of consistency may leave some case managers unprepared to perform some
aspects of their jobs. In-line with child welfare literature and other states’ training
materials, the Division acknowledges that field training is one of the most important
elements in preparing new staff who are responsible, in conjunction with their CPS
unit supervisor, for determining whether child abuse has occurred, ensuring
children’s safety, and providing services to at–risk abused and neglected children
State of Arizona
page 8
and their families. Without adequate training, new case managers’ ability to meet
these critical needs may be limited.
Division needs to expand guidance and oversight
The Division needs to do more to ensure that new case managers receive consistent
and effective field training. First, because the quality and quantity of field training is
dependent on the field training and unit supervisors who provide the training, the
Division needs to develop comprehensive training policies and other written
guidance, as well as provide formal training on this guidance to appropriate
supervisors. Second, the Division needs to improve its monitoring method to help
ensure that all new case managers receive consistent and effective field training.
Develop comprehensive guidance—The Division provides only limited written
instructions for the unit and field training supervisors who provide the field training
and needs to develop comprehensive training policies and finalize its draft field
training manual. Although the focus, structure, and sequence of training is
described in a training plan the Division submits to the federal government, the
Division has not outlined these items in policy for its staff. Rather, the Division has
only one policy related to training.1 As a result, some case managers who auditors
interviewed indicated that they only received field training or received almost all of
their field training during their approximate 2-week, pre-core orientation period
before attending the classroom core training. While it may be appropriate for case
managers to receive some field training prior to the classroom training, receiving
little or no field training after the classroom core training can result in trainees’
being denied an opportunity to practice or apply the theoretical knowledge learned
during the classroom training. Thus, trainees may not have as thorough an
understanding of their jobs as possible. To ensure that staff are informed of training
requirements, the Division should implement comprehensive training policies,
including ones related to field training.
In addition, the Division needs to take steps to finalize its field training manual. In
response to the Auditor General’s 2003 recommendation to ensure the field
training was implemented consistently state-wide, the Division indicated that it
developed a draft of a field training manual that was being field-tested in two
districts. However, the Division has been in the process of revising this manual for
about 2 years and it remains substantially incomplete. For example, five of six
chapters planned for the manual have not been finished. The Division reported that
the delay is because the field training coordinator has limited time to complete the
manual since he is also responsible for coordinating CPS supervisor training and
an annual state-wide conference. As of January 2006, the Division reported that it
plans to have the manual completed by June 2006. The timely completion of this
Office of the Auditor General
page 9
1 This policy indicates that staff shall not be assigned cases until they have completed classroom training, but may be
assigned a limited number of cases to facilitate learning during field training.
manual is especially important because field training can be provided by more
than 100 unit supervisors located across the State. Without clear written guidance,
the risk for inconsistency and ineffective field training is heightened.
Based on field training resource manuals developed for use in California,
Colorado, and Nevada, the Division should ensure that its field training manual
includes information on the following topics:1
􀁺􇩏 Overview of entire new case manager training program—The Division’s
manual should provide basic information about the entire new case manager
training program, including an overview of the case manager core classroom
training content. Because one of the purposes of field training is to reinforce or
show how theoretical information taught in the classroom should be
implemented in the field, supervisors need to be aware of what is taught in the
classroom. This allows supervisors to use the field training experience to
reinforce the classroom lessons.
􀁺􇩆 Field training and unit supervisors’ roles and responsibilities—The
Division’s manual should clearly describe the various roles of those involved in
providing new case manager field training. While the draft table of contents for
the field training manual as of December 2005 suggests that the Division plans
to include information on the field training supervisors’ roles and responsibilities,
there is no mention of unit supervisors’ roles and responsibilities. Although it is
important to provide information regarding the field training supervisors’ duties,
it is also important to clearly define the training roles and responsibilities for the
unit supervisors because they are primary field training providers. Seventeen of
the 24 case managers auditors interviewed indicated that their unit supervisors
were responsible for providing field training.
􀁺􇩔 Training requirements and expectations—The Division’s manual should
include additional details describing the specific field training activities that
should occur, the objectives that should be met, and the supervisors’
responsibilities related to ensuring that activities occur and objectives are met.
Although the Division has developed a standardized checklist of nine activities
trainees are to perform or observe during field training (as shown on page 4),
additional detail would help strengthen and ensure field training is consistently
provided. For example, one of the nine activities on the checklist calls for the
trainee to “conduct a joint interview with an experienced worker,” but it does not
detail what should be learned from this activity or what tasks the supervisor
should perform. In comparison, California’s training manual’s steps related to
conducting interviews provide more detail for both the trainee and the supervisor
State of Arizona
page 10
1 California, Colorado, and Nevada rely on local training units or trainees’ supervisors to provide field training and do not
employ field training supervisors. In contrast, Arizona uses both field training supervisors and trainees’ unit supervisors
to provide field training. Therefore, California, Colorado, and Nevada’s manuals are not directed to field training
supervisors. Additionally, California and Nevada have published separate resource manuals for trainees. However, the
supervisor/trainer and trainee manuals closely mirror each other.
such as “discuss your observations of the interview with the worker and/or your
supervisor,” and “review the skills used by the worker and give feedback.” As the
Division develops this additional detail, it will need to assess whether any
corresponding changes will be needed on the field training checklist. For
example, as the Division adds more activities that new case managers are to
experience during field training, these additional activities will need to be added
to the checklist.
􀁺􇩓 Shadow forms—The Division should ensure it includes some existing tools
within its finalized field training manual. Specifically, the Division has created
several forms that are already available for trainees to fill out as they attend
certain meetings, hearings, and other events as a part of their field training. The
forms help trainees to take notes and focus on important lessons to take away
from their experience. The forms should be included in the field training manual
in order to ensure that they are consistently used. In addition, as the Division
develops additional detailed field training requirements and expectations, it will
need to review these shadow forms and determine whether corresponding
changes or additional shadow forms are needed. For example, if additional
trainee tasks are developed, the Division should consider whether developing
additional corresponding shadow forms would be helpful for trainees.
Provide formal training to field trainers—Once the training manual is
finalized, the Division should provide formal training on how the manual should be
implemented to all staff responsible for field training. Currently, neither the unit nor
the field training supervisors receive formal instruction on how to provide field
training to new case managers. Such training is especially important because field
training is provided in division offices throughout the State by many different
people.
The Division has several options for providing formal training to unit supervisors.
First, the Division is in the process of revising the supervisor core training program
that all new supervisors are encouraged to complete. While this training does not
include information on field training, the Division reports that it could be added.
However, because supervisor core training is provided only to new supervisors, the
Division would also need to take steps to formally train existing supervisors.
Alternatively, the Division could develop specialized training for unit supervisors
dedicated to field training. For example, Pennsylvania requires supervisors to
attend a specialized training session about field training.
Additionally, the Division will need to ensure that all field training supervisors are
formally trained on how to implement the manual. There are fewer than ten field
training supervisors state-wide, so it may not be cost-effective to develop a
Office of the Auditor General
page 11
State of Arizona
page 12
classroom training program specifically for them. However, the Division should
ensure the field training supervisors receive formal training in either a one-on-one
or a group setting.
Improve monitoring of field training consistency and effectiveness—
While the Division has taken steps to improve its oversight of field training activities,
it needs to take the following additional steps to monitor whether field training is
consistently provided and effective:
􀁺􇩂 Better monitor consistency by ensuring checklists are submitted and
reviewed—Although the Division requires field training supervisors to
submit copies of the completed field training files, including the checklists,
to the Institute, this has not occurred. In April 2005, the Division mandated
that field training supervisors send copies of completed field training files
to the Institute. However, as of January 2006, only 12 files have been
provided to the Institute. While the Division does not track the number of
field training graduates, more than 180 trainees enrolled in case manager
core training between January and July 2005. If these trainees did not leave
division employment, they should have completed field training between
July and December 2005, and thus their checklists should have been
provided to the Institute.
Similarly, although the Division mandated the use of the checklist in May
2005 and sent an e-mail to all field training supervisors in September 2005
reminding them of the importance of using the checklist, there is still some
confusion as to whether the checklist is mandatory. For example, two of the
five field training supervisors auditors interviewed did not understand that
trainees were required to complete the standardized checklist. Likewise,
some case managers have only recently been asked to fill in the checklist
although they had already completed their field training. As a result, they
have to complete the checklist from memory, which may not be reliable.
Therefore, as the Division develops its formal training policies, as
recommended on page 9, it should include a statement that completing
the checklist is a required component of field training.
Finally, the Division does not have a process to review field training
documentation, such as the completed checklists, to ensure that the
specified content has been provided. Once the Division begins to receive
all of the checklists, it will need to review them on a routine basis and take
appropriate actions, based on the results of the monitoring, such as
providing feedback to the field training and unit supervisors on what
additional training is necessary to ensure that required topics are covered.
Office of the Auditor General
page 13
􀁺􇩕 Use forthcoming evaluation results to improve field training—The Division
contracted with Arizona State University in September 2005 to conduct an
ongoing evaluation of its entire new case manager training program.
Specifically, the University has developed a series of self-assessment
forms for trainees and their supervisors to complete at multiple times
throughout the entire new case manager training process, including core
classroom and field training. The evaluation involves comparing these self-assessments,
which measure trainees’ knowledge and competencies,
across time and analyzing the results to develop recommendations to
improve the training program and case manager retention. The Division
expects the results to be released in a series of reports beginning no later
than June 2006. Therefore, the Division should ensure that it uses the
evaluation’s results to identify and then implement improvements to its new
case manager training program, including field training.
Districts and training supervisors using unique and
potentially helpful training tools
Several of the districts and field training supervisors
have developed unique training tools that have the
potential to help provide a more thorough, helpful field
training experience. The Division is already aware of
many of these practices and is considering whether
they can be expanded to other districts. These unique
district practices include:
􀁺􇩔 Training unit—District 2 operates a specialized
field training unit dedicated to training and
coaching new case managers. According to the
Division, this unit was developed in cooperation
with the Training Institute as a pilot for the Division
as a whole. As space is available, new case
manager trainees are placed into this specialized
unit for 5 weeks of field training, rather than being
assigned to a regular field unit. The new case
manager trainees are assigned to an experienced
case manager who serves as their coach. The
coach, with the trainee’s assistance, is assigned
primary responsibility for working cases,
sometimes from the CPS unit the trainee is permanently assigned to. An official
from the training unit reports that initially the trainee primarily observes the coach
working two to three cases. Over time, the trainee is expected to take on more
District 2 Field Training Unit Goals
􀁺􇩁 Apply classroom instruction to field work.
􀁺􇩐 Provide additional instruction in a real-time,
hands-on setting.
􀁺􇩁 Assist the new worker to successfully integrate
into the role of a CPS case manager.
􀁺􇩆 Further assess the new case manager’s
sustainability within CPS.
􀁺􇩉 Identify areas of need that must be addressed prior
to assignment in the field unit.
􀁺􇩒 Reduce District 2 caseloads by completing
investigations and managing ongoing cases
during field training.
􀁺􇩃 Centralize, organize, and standardize CPS case
manager training within District 2.
Source: Division of Children, Youth and Families’ District 2 Field
Training Unit Proposal.
State of Arizona
page 14
responsibility for the cases, and work up to handling five to six cases. After 5
weeks, the trainees move to their permanently assigned CPS field units where
they would be expected to continue to receive regular field training for the
balance of their approximately 22-week training period. An evaluation of this
training unit, which is being completed as a part of the larger Arizona State
University evaluation, includes a comparison of the self-reported and supervisor
assessments of the knowledge and skills of case managers who were assigned
to this unit to similar assessments of case managers who were assigned to a
regular field unit. According to the Division, the resulting report is expected to be
issued no later than June 2006 and is expected to be used to help decide
whether to expand the practice to other districts.
􀁺􇩏 Other practices—Some of the districts and field training supervisors have also
developed other unique training tools. For example, District 3 provides
organized group training to new case managers who will be working in on-call
positions where they must respond to after-hour emergency calls. Such training
may be of limited use in the urban areas where after-hour emergencies are often
handled by a dedicated group of case managers. However, case managers in
rural areas of the State may find it helpful. The field training supervisors and the
field training coordinator have discussed this training as a part of their regular
meetings, which occur approximately monthly. As a part of these discussions,
another district’s field training supervisor expressed an interest in providing this
same training, and arrangements were made for the field training supervisors to
cooperate.
Similarly, in order to provide more thorough training, District 6’s field training
supervisor developed an open-book test that she administers to all her trainees.
It provides an opportunity for trainees to practice their skills in looking up
reference material, such as district policies and procedures, as well as providing
an opportunity for the field training supervisor to identify any topics on which the
trainee needs additional instruction. Similarly, this field training supervisor has
also developed a list of key forms used by case managers. She provides this list
to all her trainees and expects them to create a reference file containing these
forms.
Recommendations:
1. To ensure that staff are informed of training requirements, the Division should
implement comprehensive policies regarding case manager training, including
ones related to field training.
2. The Division should finalize its field training manual by June 2006.
3. The Division’s finalized field training manual should include, but not be limited
to, the following elements:
a. Overview of the entire new case manager training program.
b. Field training and unit supervisors’ roles and responsibilities.
c. Training requirements and expectations.
d. Shadow forms.
4. After the detailed requirements and expectations are developed, the Division
should review its field training checklist and shadow forms and make any
needed corresponding changes.
5. After the field training manual is finalized, the Division should ensure it is
consistently implemented by:
a. Providing the manual to all staff who provide field training, including field
training and unit supervisors.
b. Providing formal training to all staff who provide field training, including field
training and unit supervisors, on their roles and responsibilities, including
implementing the field training manual.
6. The Division should better monitor field training consistency by:
a. Enforcing the existing requirement that completed field training
documentation, including field training checklists, be submitted to the
Training Institute.
b. Developing a process to review whether the checklists have been
completed and providing feedback to the field training and unit supervisors
on what additional training is necessary to ensure that required topics have
been covered.
7. The Division should use the results of the ongoing Arizona State University
evaluation of the new case manager training program to identify and implement
appropriate training program improvements, including improvements specific to
field training.
Office of the Auditor General
page 15
State of Arizona
page 16
Division needs to continue efforts to implement
continuing education program
The Division is working to develop a formal continuing education program to provide
CPS staff with additional skills and knowledge so that they can more effectively
perform their functions, including ensuring children’s safety. The Division provides
some continuing education opportunities and is working to develop a more formal
continuing education program. While it does this, the Division needs to take a
number of steps, including establishing a timeline for completing the program,
formalizing its continuing education requirements in policy, establishing a
mechanism for monitoring whether staff are receiving the required continuing
education, and monitoring the program’s effectiveness.
Continuing education important
Continuing education is important for CPS staff to acquire the additional skills and
knowledge needed to continually enhance and improve their skills, abilities, and
effectiveness in performing their duties. To ensure that staff members have the
specialized skills and knowledge necessary to provide quality services, the Child
Welfare League of America recommends that child protection agencies routinely
provide in-service training and continuing education opportunities. Also, the National
Association of Social Workers views continuing education as an essential activity for
ensuring quality social work services for clients. Its continuing professional education
standards recommend that social workers complete 48 hours of continuing
education every 2 years. Further, auditors identified at least 5 states and the District
of Columbia that have requirements for child welfare continuing education ranging
from 6 to 40 hours annually, with 4 of these jurisdictions requiring at least 20 hours.1
For instance, North Carolina requires that its child welfare services staff receive a
minimum of 24 hours of training annually, while Pennsylvania requires child welfare
staff who perform case management services to complete 20 hours of training
annually.
Office of the Auditor General
page 17
1 The five states auditors identified were Colorado, Minnesota, North Carolina, Pennsylvania, and Wyoming.
FINDING 2
In late 2003, the Division received two recommendations for developing continuing
education programs. Specifically, the Office of the Auditor General (see Auditor
General Report No. 03-09) recommended that the Division develop a centralized
continuing education program specifically focused on the CPS supervisors’
professional development needs, such as how to mentor new employees, ensure
adherence to division policy and procedure, make case-planning recommendations,
and provide assistance to the staff they supervise. In addition, to revise and expand
training opportunities for case managers, supervisors, and related staff, the
Governor’s Action Plan for Reform of Arizona’s Child Protection System directed CPS
to require all CPS staff to receive at least 24 hours of training annually.
Division provides some continuing education
opportunities
Although the Division does not have a formal continuing education program, it does
provide some continuing education opportunities for its case managers and
supervisors. For example, the Division provides advanced training courses that are
designed to give workers further information in areas they are involved with or many
encounter in the field. These advanced training courses, primarily for case managers,
are generally taught by community experts. To identify topics for advanced training,
Training Institute staff reported that they receive input from the district program
managers. The Training Institute reports that in 2005 it offered eight courses, such as
a workshop on working with chemically dependent families, at several times and
locations. In some cases, multiple sessions of these courses were offered. In 2005,
the Training Institute’s 21 advanced training sessions totaled approximately 107
hours of instruction. Although attendance is not required, according to Institute staff,
more than 700 division staff attended these trainings.
In addition to the advanced training courses, case managers and supervisors have
other opportunities to obtain continuing education. For example, CPS staff may
obtain continuing education by attending the Division’s annual supervisory
leadership conference, selected community workshops and conferences, and local
CPS office training, and by taking refresher training courses provided by the Training
Institute on CHILDS and its safety and risk assessment tools and some courses
provided through the Arizona Government University.
State of Arizona
page 18
Division developing a formal continuing education
program
The Division is working on developing a more formal continuing education program
for case managers and has begun planning a supervisors’ continuing education
program. In July 2005, Training Institute staff surveyed case managers, supervisors,
and management to identify potential training topics for case managers and
supervisors. Based on this survey, Training Institute staff identified topics for case
manager continuing education courses and organized them into job specialties,
such as investigations, ongoing case management, and adoptions. Further, Training
Institute staff report that case managers will be required to attend some of the
continuing education courses, while others will be elective, and some of these
courses will be developed internally, while others will be obtained through community
providers. The Training Institute has developed course descriptions for some of these
courses and is arranging for their development. The Training Institute expects some
of these courses to be offered in late 2006, but has not set a firm timetable. In
addition, the Training Institute has identified topics for its supervisor continuing
education courses, but has not developed course descriptions or a timetable.
Division needs to address several areas while developing
its formal continuing education program
In designing its formal continuing education program, the Division needs to address
the following areas to ensure that the program is established and that CPS case
managers and supervisors receive the required training.
􀁺􇩅 Establish a timeline for developing the continuing education program—
The Division needs to develop a timeline for completing its case manager and
supervisor continuing education program. The Division indicates that
development of this program was delayed as the Division was focused on
revisions to its case manager core training. The Division began working on a
needs assessment and course identification around July 2005. Although there
are estimates for completing different tasks, such as developing the internally
taught courses by the fall of 2006, there is no overall time frame for completing
and implementing the case manager and supervisor continuing education
program.
􀁺􇩄 Develop written policy regarding continuing education requirements—The
Division should develop a written policy regarding its continuing education
requirements and communicate this policy to staff. The National Association for
Social Workers’ standards for continuing professional education recommend
Office of the Auditor General
page 19
that agencies implement policies supporting continuing professional education.
Although the Department reports in its 2005-2009 strategic plan that it plans to
implement the Governor’s 2003 action plan, including the recommendation for
all CPS staff to receive 24 hours of training each year, the Division has not
developed a written policy detailing these requirements.
􀁺􇩅 Ensure courses are available to CPS staff state-wide—The Division should
ensure that all case managers and supervisors have adequate access to
continuing education courses. Of the eight advanced training courses offered in
2005, five were provided only in Phoenix or Tucson, and only one advanced
training course was provided in each of the four rural districts. As a result, case
managers and supervisors in the rural areas had to travel to Phoenix or Tucson
to attend many of the advanced training courses. Training Institute staff report
that the community instructors are often not willing to travel outside of Phoenix
or Tucson. To ensure that all staff are able to meet the continuing education
requirements the Division establishes, the Division should take steps to ensure
that its new continuing education courses are provided throughout the State,
including considering interactive distance learning using such technology as
interactive audio or video, computer-based instruction, or other electronic
technologies.
􀁺􇩅 Establish requirements for accepting locally provided training—Some CPS
staff receive training provided by their local offices, so the Division needs to
develop a formal process for determining which local office trainings qualify as
continuing education. Specifically, the Division should establish requirements on
the types of training that will qualify as continuing education as well as a process
for reviewing requests to receive continuing education credit for local office
trainings. The International Association for Continuing Education and Training
recommends that continuing education courses have clear learning objectives
that meet a specific training need, be planned and conducted by qualified
personnel, and have appropriate content and instructional methods. The
Division should establish similar requirements and ensure it grants continuing
education credit only for courses that meet these requirements.
􀁺􇩍 Maintain adequate training records—To ensure that CPS staff receive the
required continuing education, the Division will need an effective tracking and
monitoring mechanism. Training records help ensure that caseworkers have
been provided with the skills and knowledge needed to fulfill their
responsibilities. The National Association of Social Workers’ standards for
continuing professional education recommend that organizations establish a
system for maintaining and verifying records on each continuing education
event. Although the Training Institute uses the Division’s automated case
management system, CHILDS, to track training attendance, auditors found that
staff were up to 6 months behind in recording training attendance. In addition,
although the attendance tracking system can produce reports on an individual
State of Arizona
page 20
employee’s training history, it does not produce summary reports that would
allow the Training Institute to easily monitor whether CPS staff are meeting the
Division’s continuing education requirements. The Division reports that it has
already begun developing these reports.
􀁺􇩍 Monitor program effectiveness—To ensure that its continuing education
program is effective, the Division should establish a process for assessing its
program and for making adjustments as appropriate. Training Institute staff
report that they plan to administer course-level evaluations that would be
reviewed by both the trainer and the training supervisor to identify content and
presentation improvements. The Training Institute should ensure that these
course evaluations assess the quality of the instructors and courses as well as
the extent to which the learning outcomes were achieved. The Division should
then establish a process to review these evaluations and to make appropriate
adjustments based on the results. Further, as these course-level evaluations
may not provide the data needed for an adequate evaluation of learning
experiences, the Division should consider the need for a comprehensive
program evaluation to assess how well the continuing education training
program satisfied employees’ needs and expectations.
Office of the Auditor General
page 21
State of Arizona
Recommendations:
1. The Division should establish a timeline for completing its case manager and
supervisor continuing education program.
2. The Division should develop a written policy on its continuing education
requirements and communicate this policy to its staff.
3. The Division should take steps to ensure that its new continuing education
courses are provided throughout the State so that all staff are able to meet the
continuing education requirements.
4. The Division should establish requirements on the types of training courses that
will qualify as continuing education and establish and implement review
procedures to ensure it grants continuing education credit only for courses that
meet these requirements.
5. To ensure that caseworkers receive the required amount of training, the Division
should:
a. Ensure that training attendance records are entered into CHILDS in a
timely manner, and
b. Develop summary reports that will allow Training Institute staff to monitor
compliance with continuing education policy requirements.
6. To ensure that its continuing education program is effective, the Division should
establish a process for assessing the program and for making adjustments as
appropriate.
page 22
Office of the Auditor General
AGENCY RESPONSE
State of Arizona
___________________ ARIZONA DEPARTMENT OF ECONOMIC SECURITY ___________________
1717 W. Jefferson - P.O. Box 6123 - Phoenix, AZ 85005
Janet Napolitano David A. Berns
Governor Director
Ms. Debra K. Davenport, CPA
Auditor General
Office of the Auditor General
2910 North 44th Street, Suite 410
Phoenix, Arizona 85018
Dear Ms. Davenport:
The Department of Economic Security is providing the enclosed response to the Auditor General’s Child
Protective Services Training performance audit.
We are pleased that your office recognizes the tremendous improvements that have been made to the content
of the Child Protective Services core training curriculum in the past two years. Enhancement of the training to
include more subject areas – including forensic interviewing techniques taught by community experts – as well
as increased, hands-on experiences in areas such as our electronic database, safety and risk assessments
and case planning, have given our new case managers a much stronger foundation to work from.
We agree that CPS needs to enhance and standardize the field training experiences of our new workers; this
complements the improved classroom instruction they have received. Thank you for acknowledging the
progress that has been made with the addition of field training supervisors to help unit supervisors work with
new case managers. We are very excited about the field training pilot program in Pima County, as well as the
lessons we are learning from District-level efforts to support new employees. The most promising strategies
from those efforts will become the best practices used statewide.
We appreciate your work on this report, your consideration of our previous comments and suggestions, and
your acknowledgement of the quality and variety of training already provided by this Division. We look forward
to reporting to you the additional improvements that will be made in the coming months.
Please feel free to call me at (602) 542-5678 if any additional information is needed.
Sincerely,
David A. Berns
Enclosure
ARIZONA DEPARTMENT OF ECONOMIC SECURITY’S RESPONSE TO THE
OFFICE OF THE AUDITOR GENERAL’S REPORT ON
CHILD PROTECTIVE SERVICES TRAINING
APRIL 2006
The Department of Economic Security (DES) is pleased to provide the following
comments and responses to the findings and recommendations of the Office of the
Auditor General’s performance audit of the Division of Children, Youth, and Families
Child Protective Services Training.
Comments to Introduction and Background
Working to implement the Recommendations of the Governor’s Commission on CPS
Reform, the Division of Children, Youth, and Families has undergone tremendous
change, in policy, practice, and responsibilities. As acknowledged in the Auditor
General’s report, the training for Child Protective Services case managers has undergone
dramatic change in that time as well. As noted in the audit, the basic case manager core
has been extended from 4 to 6 weeks, and now includes extensive hands-on training in
the CHILDS case management database, as well as practice in case-planning, service
referrals, interviewing and court testimony. Additional time is now allotted for more
practice scenarios in assessing child safety and creating safety plans. There is a full day
of Forensic Interviewing in this core, taught by a community professional. The training
now incorporates a field break week that allows for transfer of learning to occur.
At the same time, field training supervisors have been added in each district to assist unit
supervisors with the field portion of the case managers’ training. However, these
positions have been fully staffed and operational state-wide for just over one year. As
with any new model, the guidelines, roles, and responsibilities of each of the players have
been evolving and solidifying over the course of this year. The Child Welfare Training
Institute staff have researched other models, learned from experience, and now have a
greater understanding of what is needed for an optimal field training experience for new
staff. The guidelines reflected in the newly revised field training manual will reflect that
understanding, will provide a more formal outline for that field training to follow, and
will aid the coordination between Training Institute staff and CPS unit supervisors.
As noted in this audit as well, the Training Institute partnered with District 2 in an
important Mentoring/Coaching pilot for new case managers, the results of which are very
promising. The best elements of this and other innovative practices will be incorporated
into best practice across our entire system.
In addition to specific field training experiences provided to new case managers by field
training supervisors and unit supervisors, other training and support opportunities also
exist for new case managers. New case managers are encouraged to call the Training
Institute with questions and for assistance; the Training Institute provides refreshers in
the CHILDS case management database, as well as the Child Safety Assessment, and
Safety and Risk Assessment tools. Although less quantifiable and not mentioned by the
Agency Response to Draft Audit
April 2006
2
auditors, a vital source of support is the informal field training that occurs as new case
managers build relationships with more experienced colleagues and community partners.
The Division acknowledges the importance of providing a field training manual and
accompanying instruction to all staff who provide field training, including unit
supervisors. The Division is also engaged in a number of other activities to provide
education and support to those supervisors in the performance of their duties. These
include supervision circles, critical decision-making seminars, a revision of the
Supervisor Core to better meet their needs, and the creation of continuing education
courses that will assist them in their growth as supervisors and in their support of new
case managers.
The Division agrees with the recommendations in this report, and had in fact already
begun implementation of these recommendations as an outgrowth of the changes in
practice and in training over the last two years.
DES COMMENTS REGARDING FINDING 1: PAGES 7 – 16
Finding 1, Recommendation 1
To ensure that staff are informed of training requirements, the Division should implement
comprehensive policies regarding case manager training, including ones relating to field
training.
DES Response
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
While the outline and structure of both the core training and the field training component
are extensively described in the Child and Family Services Plan (IVB Report) Section V,
page 150-164, the Division does acknowledge the need for creating more comprehensive
training policies and communicating those to all staff. These policies will address not
only classroom core training, but also the field component of case manager training, and
the policy regarding continuing education as outlined by the CPS Reform Commission’s
Recommendations.
Finding 1, Recommendation 2
The Division should finalize its field training manual by June 2006.
DES Response
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
Agency Response to Draft Audit
April 2006
3
The Division agrees that the provision of consistent guidelines regarding implementation
of field training are vital to a well-trained workforce, and that the revision of its Field
Training Manual needs to be completed. The Division will complete this Revised Field
Training Manual, as outlined in Recommendation 3 below, by June 30, 2006.
Finding 1, Recommendation 3
The Division’s finalized field training manual should include, but not be limited to, the
following elements:
a. Overview of the entire new case manager training program.
b. Field training and unit supervisor’s roles and responsibilities.
c. Training requirements and expectations.
d. Shadow forms.
DES Response
The finding of the Auditor General is agreed to, and the audit recommendation is being
implemented.
a) The Division agrees that the revised field training manual should include basic
information about the entire new case manager training program, including a
description of the case manager classroom core training content.
b) The Division’s revised field training manual will also include the roles and
responsibilities for both the field training supervisors and unit supervisors.
c) The Division agrees to include fundamental guidelines for training
requirements and expectations to ensure basic quality and consistency state-wide.
Since the needs of each District in the state vary (e.g. urban vs. rural
populations), it is understood that additional training may occur in each
District that specifically meets the needs of case managers in their geographic
area. This would be in addition to those requirements set forth in the state-wide
field training manual.
d) The Division agrees that the revised field training manual will include the
field training checklist and shadow forms, as well as other information and
guidelines to prepare those supervisors to provide appropriate field
experiences.
Finding 1, Recommendation 4
After the detailed requirements and expectations are developed, the Division should
review its field training checklist and shadow forms and make any needed corresponding
changes.
Agency Response to Draft Audit
April 2006
4
DES Response
The finding of the Auditor General is agreed to and the audit recommendation is being
implemented.
The Division agrees that as the requirements and expectations are more fully developed,
it will be necessary to review the field training checklist and shadow forms, and alter
them as needed to correlate with the content. Because the Districts vary in their
composition and job functioning, the guidelines and content for the revised field training
manual will be designed to ensure fundamental quality and consistency, while allowing
flexibility and enhancements from each district.
Finding 1, Recommendation 5
After the field training manual is finalized, the Division should ensure it is consistently
implemented by:
a. Providing the manual to all staff who provide field training, including field
training and unit supervisors.
b. Providing formal training to all staff who provide field training, including
field training and unit supervisors, on their roles and responsibilities,
including implementing the field training manual.
DES Response
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented
The Division acknowledges the importance of training all staff who provide field training
to new case managers, including providing them with a copy of the field training manual.
a) The Division agrees to provide the revised field training manual to all staff
who provide field training, including field training and unit supervisors.
b) The Division agrees to provide formal training to all staff who provide field
training, including field training and unit supervisors, on their roles and
responsibilities, including implementing the field training manual. Currently,
the field training supervisors do receive informal training and supervision
from their field training coordinator. However, the Division plans to
implement formal training of both field training supervisors and CPS unit
supervisors in their roles and responsibilities, as well as implementation of the
manual, both on an initial and an ongoing basis.
Finding 1, Recommendation 6
The Division should better monitor field training consistency by:
Agency Response to Draft Audit
April 2006
5
a. Enforcing the existing requirement that completed field training
documentation, including field training checklists, be submitted to the
Training Institute.
b. Developing a process to review whether the checklists have been completed
and providing feedback to the field training and unit supervisors on what
additional training is necessary to ensure that required topics have been
covered.
DES Response
The finding of the Auditor General is agreed to and will be implemented.
In the interest of better monitoring consistency in the field portion of the training, the
Division agrees to:
a) Enforce the existing requirement that completed field training documentation,
including field training checklists, be submitted to the Training Institute; and
b) Develop a process to review whether the checklists have been completed, and
provide feedback to the field training and unit supervisors on what additional
training is necessary to ensure that required topics have been covered.
Finding 1, Recommendation 7:
The Division should use the results of the ongoing Arizona State University evaluation of
the new case manager training program to identify and implement appropriate training
program improvements, including improvements specific to field training.
DES Response:
The finding of the Auditor General is agreed to, and the recommendation will be
implemented.
The Division has extended and enhanced its collaboration with the Arizona State
University School of Social Work:
Over the course of 2005, ASU and CWTI developed a “Self-Assessment of Knowledge,
Skills, and Abilities.” This self-assessment survey is now provided to all new case
manager trainees at four points during their first year of employment. It is completed (1)
as a Pre-Core Test; (2) Post-Core; (3) after the end of their training period (approximately
22 weeks); and (4) at the end of their first year of employment. This survey is currently
available on-line for ease and consistency of completion, and it is then tabulated by ASU
Social Work faculty. As of March 1, this survey has now been in use for one year. ASU
School of Social Work expects to have its report on the results no later than June 30,
2006. The Training Institute intends to use the results of this survey to assess the efficacy
of its new case manager training program and to implement improvements as indicated
Agency Response to Draft Audit
April 2006
6
by the results. Because the surveys are done over the course of one year, the results
should allow for evaluation of both the Core training and the field portion of the case
manager training. In addition to this, the Training Institute hopes to differentially assess
the results from those trainees who participated in the Mentoring Pilot in District 2, so
that the Division may assess the success of that pilot.
Finding 2, Recommendation 1
The Division should establish a timeline for completing its case manager and supervisor
continuing education program.
DES Response
The finding of the Auditor General is agreed to, and the recommendation will be
implemented.
The Division acknowledges the need to set a timeline for enhancement and formalization
of its continuing education program. As outlined in the Auditor General’s report, the
Division provides many opportunities for continuing education at present. Work began
on formalizing and enhancing the continuing education program, including needs
assessment and course identification, during July of 2005. Course descriptions and
competencies are currently being finalized for case managers, and offering of those
courses will proceed, dependent upon allocation of resources to provide them. The
Division plans to continue, to the best of its ability, and contingent upon resources, to
expand its efforts to provide a variety of training opportunities that will meet the CPS
Reform Commission’s recommended 24 hours of annual training for CPS staff.
Finding 2, Recommendation 2
The Division should develop a written policy on its continuing education requirements
and communicate this policy to its staff.
DES Response
The finding of the Auditor General is agreed to, and the recommendation will be
implemented.
Finding 2, Recommendation 3
The Division should take steps to ensure that its new continuing education courses are
provided throughout the State so that all staff are able to meet the continuing education
requirements.
Agency Response to Draft Audit
April 2006
7
DES Response:
The Division agrees with the Auditor General’s finding, and the recommendation will be
implemented, contingent upon appropriate resources.
Finding 2, Recommendation 4
The Division should establish requirements on the types of training courses that will
qualify as continuing education, and establish and implement review procedures to ensure
that it grants continuing education credit only for courses that meet these requirements.
DES Response:
The Division agrees with the Auditor General’s finding and the recommendation is being
implemented.
The Division recently implemented a review process to assess which conferences would
be appropriate to enhance CPS staff’s knowledge, skills, and abilities. The Division will
further work to create guidelines, and to only grant credit for those trainings, courses, or
seminars which qualify as continuing education by meeting those guidelines.
Finding 2, Recommendation 5
To ensure that caseworkers receive the required amount of training, the Division should:
a) Ensure that training attendance records are entered into CHILDS in a timely
manner, and
b) Develop summary reports that will allow Training Institute staff to monitor
compliance with continuing education policy requirements.
DES Response:
The Division agrees with the Auditor General’s finding, and the recommendation is being
implemented.
a) The Division is currently able to enter and track its training records for
individual caseworkers through its case management system, known as
CHILDS. The Division will ensure that these training records are entered into
CHILDS in a timely manner.
b) The Division is currently working with the CHILDS department to develop
summary reports that will allow the Training Institute to monitor compliance
with continuing education policy requirements.
Agency Response to Draft Audit
April 2006
8
Finding 2, Recommendation 6
To ensure that its continuing education program is effective, the Division should establish
a process for assessing the program and for making adjustments as appropriate.
DES Response:
The Division agrees with the Auditor General’s finding, and the recommendation is being
implemented.
a) The Division currently administers course evaluations to attendees of
currently offered Advanced trainings, as well as the CHILDS refresher
trainings, and monitors the responses to those training courses. These
evaluations assess the quality of both the instructors and of the learning
outcomes achieved.
b) As the continuing education program is enhanced and other courses are
offered, the Division plans to administer similar course evaluations to the
attendees of these courses.
The Division plans to continue to review these evaluations and make
adjustments based on the results. For example, based on course evaluations,
changes were recently made to a District 1 CHILDS Investigations Refresher
course: That course was lengthened and adjusted to allow for a job-specific
question-and-answer period in the class.
c) In addition, the Division plans to conduct an overall assessment of the
continuing education program, determine its effectiveness, and identify further
needs, in order to make appropriate adjustments to the program overall.
CPS Performance Audits and Information Briefs Issued
Performance Audits
CPS-0501 CHILDS Data
Integrity Process
CPS-0502 Timeliness and
Thoroughness of
Investigations
Information Briefs
IB-0401 Federal Title IV-E Waiver
Demonstration Project
Proposal
IB-0501 Family Foster Homes
and Placements
IB-0502 Revenue Maximization
Future CPS Information Briefs
Types of Federal Monies Available