Performance audit, Department of Economic Security--Division of Children, Youth and Families - Child Protective Services - Complaint Management Process

              Debra K. Davenport
Auditor General
Performance Audit
Department of
Economic Security—
Division of Children, Youth and Families—
Child Protective Services—Complaint
Management Process
Performance Audit Division
JULY • 2008
REPORT NO. CPS-0801
A REPORT
TO THE
ARIZONA LEGISLATURE
The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Audit Staff
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.azauditor.gov
Representative John Nelson, Chair Senator Robert Blendu, Vice Chair
Representative Tom Boone Senator Carolyn Allen
Representative Jack Brown Senator Pamela Gorman
Representative Pete Rios Senator Richard Miranda
Representative Steve Yarbrough Senator Rebecca Rios
Representative Jim Weiers (ex-officio) Senator Tim Bee (ex efficio)
Melanie M. Chesney, Director
Catherine Dahlquist, Manager and Contact Person
Dan Hunt, Team Leader
Marc Owen
Additional actions needed to improve Division's
management of formal complaints (see pages 5
through 10)
Although the Division
has implemented two
important elements of
an effective
complaint-handling
process, it needs to
take additional actions
to improve the
management and
oversight of formal
complaints about
CPS. Specifically, the
Division should clarify
and expand complaint
policies and
procedures to ensure
complaints are
processed in a
consistent and timely
manner, centrally track
and analyze
information on all
formal CPS
complaints to identify
systemic problems
and improve services,
and periodically review
the complaint
management process
to ensure it remains
effective and efficient.
The Department
agrees with the finding
and will implement the
recommendations.
See gold pages for its
full response.
This audit was
conducted under the
authority vested in the
Auditor General by
Arizona Revised
Statutes §41-1966.
The Department of Economic Security's (Department) Division of Children, Youth
and Families (Division) uses several methods to process formal complaints
about Arizona's Child Protective Services (CPS) program. Most CPS complaints
are received and processed by the Division. However, other agencies, such as
the Arizona Ombudsman–Citizens' Aide, may also receive and investigate
complaints regarding CPS.
The Division's complaint management process includes two important elements.
First, the Department ensures that the public has easy access to information
related to the complaint process. Additionally, when the Division prepares a
written response letter to a CPS complaint, the complaint issues are generally
thoroughly addressed in the letter.
However, the Division should take additional steps to improve the management
and oversight of formal CPS complaints by:
 Clarifying and expanding complaint policies and procedures, including
defining the term "complaint" and providing comprehensive guidance on
timelines, documentation, and monitoring requirements to all staff handling
formal CPS complaints to ensure the complaints are processed in a
consistent and timely manner.
 Implementing a centralized tracking system that captures uniform CPS
complaint data to facilitate effective processing of formal complaints and
allows management to identify systemic problems and improve services.
 Periodically reviewing the Division's complaint management process to
ensure it is operating effectively and efficiently.
Office of the Auditor General
page i
Agency Comments
SUMMARY
Our Conclusion
State of Arizona
page ii
Several avenues exist for filing formal
complaints about CPS
Citizens have several
avenues available to
file formal complaints
about the State's CPS
program. Complaints
may be verbal or
written. It is the
Division's policy that
an individual first
attempt to resolve
his/her CPS complaint
informally through
discussion with CPS
case management
staff. However, if the
complaint cannot be
resolved informally,
the individual may file
a formal complaint
with the Division's
Family Advocacy
Office and Crisis
Response Unit, as well
as the Department
Director's Office.
Citizens may also file
formal complaints
regarding CPS with
other entities including
the Arizona
Ombudsman-Citizens'
Aide and the
Governor's Office of
Constituent Services.
BACKGROUND
Office of the Auditor General
Citizens have several avenues available to file complaints about the CPS
program, which the Division administers. Complaints may be received verbally
or in writing, and can range from a CPS specialist not returning a family's phone
calls to the program's failing to take adequate action to ensure required services
are provided to children and families through contracted service providers and
other entities, such as the regional behavioral health authorities.
The Division handles both informal and formal complaints. Informal complaints
are those issues received and addressed by CPS field unit staff as part of their
day-to-day case management responsibility. Although it is the Division's policy
that an individual first attempt to resolve his/her complaint informally through
discussion with CPS case management staff, if it cannot be resolved informally,
the complainant may file a formal complaint with the following internal and
external entities.
Department of Economic Security—The Division reviews and addresses formal
complaints regarding CPS received through the following internal department
units/offices:
 Family Advocacy Office—This Office was established in 2000 by Arizona
Revised Statutes (A.R.S.) §8-828 to respond to inquiries and complaints
about the child welfare system and to track complaint information, including
the number, type, and source of complaints, and the results of the
complaint investigation, among other responsibilities. Pursuant to the
legislation, the Office sunset on July 1, 2005. However, the Department
chose to continue this Office within the Division. This Office receives
inquiries and complaints regarding CPS through a dedicated telephone
number. The office phone number is listed on pamphlets and forms
provided by CPS staff to clients and is also available to the public. When a
complaint is received, office staff may contact field unit staff overseeing the
related CPS case to obtain clarification and, if appropriate, request that
action be taken to address the complaint. According to division personnel,
either office staff or a designated field unit staff person may contact the
complainant with a resolution. If a resolution cannot be achieved, office staff
may send a client grievance form to the complainant. According to office
records, in fiscal year 2007, the Office received more than 2,500 calls.
However, because office staff do not consistently record the nature of the
phone calls, it is not possible to determine how many of these calls were
complaints and how many were inquiries (see page 7 for additional
information).
page 1
 Crisis Response Unit—According to division management, this Unit was
established within the Division in 2003 to respond to media requests about high-profile
CPS cases and to manage written client grievances, which involves this
Unit's staff recording and responding to written complaints about CPS that are
submitted on the Division's client grievance form. Similar to the Family Advocacy
Office, this Unit may contact field unit staff overseeing the related CPS case to
obtain clarification and, if appropriate, request that action be taken to address
the complaint. According to unit records, the Unit received 41 written grievances
in fiscal year 2007.
 Director's Office—This Office receives written correspondence addressed to the
Department's director and deputy directors. The correspondence is first
reviewed by the Office's correspondence control staff to determine if some type
of action is needed by the Department—for example, responding to an inquiry
or investigating a complaint. Any correspondence concerning CPS is forwarded
to a designated division staff person who will assign it to the appropriate division
personnel for action. These items are to be completed within a specific time
frame established by the Office's correspondence control staff. Once the
necessary action is taken, the Division typically prepares and sends a written
response letter to the correspondent and provides a copy to the Director’s
Office. According to department records, the Office referred 272 items to the
Division for action in fiscal year 2007. However, because these items are not
categorized by type, it is not possible to determine how many items were
complaints.
External Entities—Formal complaints regarding CPS may also be filed with entities
external to the Department. For example, citizens may file complaints with the Arizona
Ombudsman–Citizens' Aide. Ombudsman staff have direct access to CPS hard-copy
and electronic case file information and will independently investigate any
serious complaint to determine if it is substantiated or unsubstantiated. Ombudsman
staff provide a preliminary written report with their investigation findings and any
recommendations to the Department for comment. If requested by the
Ombudsman–Citizens' Aide, the Department will provide a written response to the
investigative report, including its decision to accept or reject each recommendation.1
The Ombudsman–Citizens' Aide will provide its investigation findings to the
complainant verbally, or if requested, in writing. Additionally, it may provide the written
investigative report, including the Department's response, to the Governor's Office
and the Legislature. During calendar year 2007, the Ombudsman–Citizen's Aide
received 567 complaints related to CPS. The majority of these complaints were
addressed by educating complainants on CPS policies and procedures or assisting
the complainant through a phone call or meeting with CPS staff. Formal
investigations are typically reserved for addressing violations of policy or resolving
conflicts between policy and statutes.
State of Arizona
page 2
1 Although the Ombudsman–Citizens’ Aide can make recommendations, it does not have legal authority to compel the
Department to implement the recommendations.
In addition to the Ombudsman–Citizens' Aide, citizens may contact their legislators
or the Governor's Office of Constituent Services to file formal complaints about CPS.
These entities' staff must contact the Division in order to obtain the facts of the case.
The Division will respond back to external entities with information that the
complainant has been responded to and the issue resolved, or will explain in general
terms what would happen in a similar hypothetical situation. The external entities’
staff will assess the information provided by the complainant and the Division, and
may suggest recommendations to the Division's Crisis Response Unit. However,
similar to the Ombudsman–Citizens' Aide, the Department has the discretion to
implement or not implement these recommendations. It is unknown how many
complaints regarding CPS were received and addressed by these two external
entities in 2007 because not all the complaints were referred to the Division for action.
Further, the Division does not consistently track this information.
Office of the Auditor General
page 3
page 4
State of Arizona
page 5
Additional actions needed to improve Division's
management of formal complaints
Effective complaint management process is important
An effective process to manage complaints about CPS is important for several
reasons. By promptly and adequately addressing client complaints, the Division
may be able to mitigate children's exposure to unnecessary risks and better
ensure children and families receive appropriate and timely services. Further, an
effective complaint management process should facilitate the Division's ability to
rectify poor decisions quickly and efficiently, prevent complaints from escalating,
and identify areas needing improvement so that corrective action can be taken.
Potential economic benefits may also result from addressing problems internally,
close to the source, without the need for resources to be devoted to review by
external accountability bodies, and by ensuring complaints are handled
methodically and efficiently. An effective complaint management process should
also help reduce stress on staff by providing training and support to deal with
unhappy clients and a structured approach to addressing issues. Finally, it may
demonstrate to the public the seriousness with which the Division responds to
complaints.
Division includes two important elements in its
complaint management process
The Division's complaint management process includes two important
elements. Specifically, the Division:
 Ensures that the public has easy access to information on the complaint
process—International standards on proper complaint management
emphasize that information on the complaint process should be easily
accessible and communicated to the public.1 The Department's Web site
and various printed pamphlets and forms provided to CPS clients and
available to the public include information on a citizen's right to file a
complaint. In addition, these resources explain how to file a complaint by
providing information on the locations, phone numbers, and office hours of
those entities handling complaints about CPS.
1 American Society for Quality. ANSI/ISO/ASQ 10002-2004, Quality Management: Customer Satisfaction: Guidelines
for Complaints Handling in Organizations. Milwaukee, WI: ASQ Quality Press, 2006.
The Division should
take steps to improve
its process for
handling formal
complaints regarding
CPS. An adequate
complaint process
may help to mitigate
children's exposure to
unnecessary risks and
ensure timely services
for children and
families. The Division
employs two
important elements in
its complaint
management process:
ensuring the public
has easy access to
information on the
process and, when it
responds, adequately
addressing the
complaint concerns in
its written responses.
However, it should
improve its complaint
process by clarifying
and expanding
complaint policies and
procedures, including
establishing timelines
for responding to all
formal complaints and
ensuring that staff are
aware of and are
following the policies
and procedures,
centralizing its
complaint information,
and periodically
reviewing its com-plaint
management
process.
Office of the Auditor General
FINDING
 Prepares adequate written responses addressing complaint issues when it
responds—International standards also state that responding adequately to all
aspects of a complaint facilitates the improvement of an organization's services
and processes. The Division is required by policy to respond in writing to written
client grievances. Auditors identified several instances where the Division did not
provide a written response to the complainant (see page 8). However, when the
Division prepared written complaint responses, they were adequate. To assess
the adequacy of the Division's response to complainants, auditors reviewed 27
written replies to written client grievances received in fiscal year 2007 and found
that 26 adequately addressed all issues noted in the grievance. The one
exception failed to address concerns that the complainant's child was being
bitten by other children while at day care and that the complainant should be
receiving additional monies for child care. Auditors also reviewed the Division's
written replies to 30 correspondences expressing dissatisfaction with CPS that
were referred from the Director's Office for action in fiscal year 2007, and found
that 29 adequately addressed all stated concerns. In the one exception, the
Division's response stated that it would review the case and address any issues
that arise as part of a case review, but there was no follow-up letter or other
evidence that the Division performed these actions.
Auditors were unable to assess the adequacy of the Division's response to
complaints received through the Family Advocacy Office since division policy
does not require a written response be prepared for these complaints. Although
division policies and procedures indicate that staff should record information on
complaints and associated resolution activities in the automated CPS case
management system, auditors found the information to be inadequate (see
page 8).
Additional actions needed to improve the Division's
management of formal complaints
Although the Division has implemented two important elements of an effective
complaint management process, it needs to take additional actions to improve its
handling of formal complaints, including clarifying and expanding its complaint
policies and procedures, centralizing its complaint information, and periodically
evaluating its complaint management process.
Clarify and expand complaint policies and procedures—International
standards emphasize the need for complaint management policies and
procedures for all functions and personnel roles involved in the complaint-handling
process. Well-documented and up-to-date policies and procedures
can help prevent the inconsistent treatment of complaints and reduce the
incidence of complaints escalating unnecessarily. Although the Division has
page 6
State of Arizona
page 7
some complaint policies and procedures, they vary in the degree of guidance
provided and focus primarily on written client grievances handled through the
Crisis Response Unit. As such, they do not provide comprehensive guidance on
handling complaints received through the Family Advocacy Office or the
Director's Office. To better address the needs of all staff handling formal
complaints, the Division should clarify and expand its complaint-handling
policies and procedures and ensure staff are aware of and following them.
Specifically, it should:
 Establish definition of complaint—International standards indicate that a
definition of complaint will allow organizations to accurately differentiate
complaints from other forms of communication with the public, such as
inquiries and requests for information. Accurately identifying complaints
helps to ensure they are investigated and addressed, and that analyses
and reports of complaint trends are accurate. However, the Division's
complaint policies fail to define the term "complaint." During review of the
Family Advocacy Office records, auditors were unable to determine how
many of the calls the Office received were complaints because the term
was not clearly defined and understood by staff; therefore, it
was used sporadically and inconsistently. Complaints can
be defined in various ways. For example, international
standards for complaint handling define a complaint as an
expression of dissatisfaction requiring a response (see
textbox). Alternately, the Arizona Department of Health
Services' Division of Behavioral Health Services defines a
complaint as an expression of dissatisfaction with any
aspect of care. The Division should establish a clear
definition for a complaint regarding CPS and ensure staff
are aware of it so that they can ensure the complaint is
addressed using appropriate complaint-handling
procedures and that complaint analyses and reports are
based only upon complaint information.
 Provide comprehensive guidance for all staff handling formal CPS
complaints—International standards on complaint handling state that
complaint policies be supported by procedures for each function
addressing complaints and that personnel involved in the process be
aware of and trained in them. As stated before, the Division's policies and
procedures focus primarily on written client grievances handled through the
Crisis Response Unit and vary in degree of guidance provided. However, in
fiscal year 2007, approximately 2,800 calls and action items were received
by the Family Advocacy Office and/or referred from the Director's Office.
Therefore, the Division should expand its complaint procedures to provide
comprehensive guidance to all division functional areas and staff
responsible for receiving and responding to formal complaints about CPS.
Specific areas to address in the procedures include:
A is defined as an expression
of dissatisfaction made to an organization,
related to its products (services) or the
complaints-handling process itself, where
a response or resolution is explicitly or
implicitly expected.
Source: American Society for Quality. ANSI/ISO/ASQ
10002-2004, Quality Management:
Customer Satisfaction: Guidelines for
Complaints Handling in Organizations.
Milwaukee, WI: ASQ Quality Press, 2006.
Office of the Auditor General
page 8
 Timelines—Establishing and enforcing specific timelines for each
phase of the complaint management process will help ensure
complaints are addressed in a timely manner. Without timelines, the
complaint resolution process can become stagnant. Timelines also
need to be enforced or complainants may be left not knowing if and/or
when their complaints are going to be addressed. For example,
although procedures require the Division to provide a written response
to a written client grievance within 10 business days of receiving the
complaint, auditors found that 7 of the 27 response letters issued in
fiscal year 2007 were 16 to 53 days late and that the Division did not
prepare or could not locate response letters for another 13 of the
written client grievances. Additionally, the Division did not respond
within established time frames to 58 of 112 correspondences
expressing dissatisfaction with CPS, which were referred from the
Director's Office to the Division for action in fiscal year 2007. The
majority of the untimely responses ranged between 1 and 73 days late,
with a median of 10 days.
 Documentation—Establishing clear documentation requirements will
provide a means for verifying that appropriate and timely actions were
taken to address complaints. Division policies and procedures provide
general guidance on information that staff should record about
complaints—for example, meetings, contacts with the complainant,
and complaint resolution actions. This information is to be
documented within case notes in the Division's automated CPS case
management system, known as CHILDS (Children's Information
Library and Data Source). However, auditors' review of CHILDS case
notes for 14 written client grievances received in fiscal year 2007, as
well as interviews with division staff, revealed that the complaint details
and resolution information were not consistently being recorded.
Further, in a review of 10 written client grievance response letters
indicating specific resolution actions, auditors were unable to
determine if the prescribed actions actually occurred and/or were
monitored for 9 of the grievances based upon review of the CHILDS
case notes and the hard-copy complaint records.
 Monitoring—Establishing specific criteria and activities for use in
monitoring how formal complaints are being handled will better ensure
they are addressed in an appropriate and timely manner. International
standards identify examples of complaint monitoring criteria, including
complaint responsibility appropriately assigned, complaint responded
to within the expected time frame, and complaint addressed within the
expected time frame. Although division complaint procedures
designate staff to monitor and verify that resolution action on written
client grievances occurred, they do not contain guidance on how or
State of Arizona
page 9
when this monitoring activity is to occur, criteria to use in determining
if the complaint is being handled well or poorly, and how monitoring
activity is to be evidenced. Therefore, the Division is unable to ensure
that actions on written client grievances are consistently and
adequately monitored. In addition, the Division does not have
monitoring procedures for complaints received and addressed
through other avenues, such as the Family Advocacy Office.
Implement a centralized complaint information tracking system—
International standards and literature on effective complaint management
practices recommend that complaint information be recorded in an automated
centralized system in order to facilitate effective processing of complaints and
allow management to analyze and use complaint data to identify root causes of
dissatisfaction and determine future service improvements.1 As of April 2008,
the Division maintained three separate electronic logs to record complaint
information. However, the logs do not capture uniform and key complaint
information such as division personnel assigned to address and monitor the
complaint, complaint issue(s), requested remedies, actions taken, complaint
disposition, and critical dates necessary to ensure that complaints are
addressed in a timely manner. Further, the logs are not integrated into a uniform
system. Because of these limitations, the Division is unable to generate
management reports with a comprehensive overview of complaint data,
effectively preventing the Division from identifying operational or performance
issues on a systemic level. According to division management, a centralized
complaint database is in the process of being developed and will be
implemented by the end of 2008.
Periodically review complaint management process—International
standards indicate that it is important for an organization to periodically review
its complaint management process to ensure it is continuing to operate
efficiently and effectively. According to division management, the Division does
not conduct periodic reviews of its complaint-handling process. However, many
of the complaint issues and problems noted in this report, such as preparing
and providing timely response letters to written client grievances, and
adequately documenting and monitoring complaint processing, could have
been identified and addressed if division management regularly reviewed its
complaint-handling process. Therefore, the Division should establish a time
frame and procedures for conducting periodic reviews of its complaint
management process.
1 Office of the Vice President of the United States National Performance Review. Serving the American Public: Best Practices
in Resolving Customer Complaints, Federal Benchmarking Consortium Study Report. Washington, D.C.: National
Performance Review, March 1996.
Office of the Auditor General
Recommendations:
1. The Division should clarify and expand its complaint policies and procedures by:
a. Establishing a clear definition of "formal complaint."
b. Developing comprehensive and detailed guidance for all staff handling
formal CPS complaints, including timelines for completing all phases of the
complaint-handling process, documentation requirements, and complaint
monitoring activities.
2. The Division should ensure that staff are informed of and following division
complaint-handling policies and procedures.
3. The Division should complete its implementation of an automated, centralized
complaint-tracking system that will capture key information needed to ensure
complaints are effectively addressed and help management identify systemic
problems so that corrective action can be taken.
4. The Division should develop and routinely generate and review management
reports from the automated complaint-tracking system to identify systemic or
recurring problems and take corrective action as needed.
5. The Division should establish and implement time frames and procedures for
reviewing its complaint-management process to identify and promptly correct
any process deficiencies.
page 10
State of Arizona
AGENCY RESPONSE
Office of the Auditor General
CPS Reports Issued
-0 CHILDS Data
Integrity Process
-0 Timeliness and
Thoroughness of
Investigations
-0 On-the-Job Training and
Continuing Education
-0 Prevention Programs
-0 DES’ Federal Title IV-E
Waiver Demonstration
Project Proposal
-0 Family Foster Homes
and Placements
-0 Revenue Maximization
-0 In-Home Services
Program
-0 Federal Deficit Reduction
Act of 2005
-0 Federal Grant Monies
-0 Child Removal Process
Future CPS Reports
-0 Substance-Exposed
Newborns
-0 Child Abuse Hotline
-0 Confidentiality of CPS
Information
-0 Licensed Family Foster
Homes
-0 Child and Family
Advocacy Centers
Congregate Care
Family Demographics