Performance audit, Arizona Supreme Court, Administrative Office of the Courts, Information Technology and FARE Program: a report to the Arizona Legislature

              A REPORT

ARIZONA LEGISLATURE

TO THE



Performance Audit Division Performance Audit



Arizona Supreme Court

Administrative Office of the Courts--

Information Technology and FARE Program

AUGUST  2006 REPORT NO. 06  08



Debra K. Davenport Auditor General



The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer.



The Joint Legislative Audit Committee

Representative Laura Knaperek, Chair Representative Tom Boone Representative Ted Downing Representative Pete Rios Representative Steve Yarbrough Representative Jim Weiers (ex-officio) Senator Robert Blendu, Vice Chair Senator Ed Ableser Senator Carolyn Allen Senator John Huppenthal Senator Richard Miranda Senator Ken Bennett (ex-officio)



Audit Staff

Melanie M. Chesney, Director Dale Chapman, Manager and Contact Person Channin DeHaan, Team Leader Robin Hakes Michael Nickelsburg Dana Payne



Copies of the Auditor General's reports are free. You may request them by contacting us at:



Office of the Auditor General

2910 N. 44th Street, Suite 410  Phoenix, AZ 85018  (602) 553-0333

Additionally, many of our reports can be found in electronic format at:



www.azauditor.gov



STATE OF ARIZONA

DEBRA K. DAVENPORT, CPA AUDITOR GENERAL



OFFICE OF THE



AUDITOR GENERAL



WILLIAM THOMSON DEPUTY AUDITOR GENERAL



August 31, 2006



Members of the Arizona Legislature The Honorable Janet Napolitano, Governor Mr. David K. Byers, Director Administrative Office of the Courts Transmitted herewith is a report of the Auditor General, A Performance Audit of the Arizona Supreme Court, Administrative Office of the Courts--Information Technology and FARE Program. This report is in response to Laws 2004, Chapter 39, 2 and was conducted under the authority vested in the Auditor General by Arizona Revised Statutes 41-1279.03. I am also transmitting with this report a copy of the Report Highlights for this audit to provide a quick summary for your convenience. As outlined in its response, the Administrative Office of the Courts agrees with all of the findings and plans to implement all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. This report will be released to the public on September 1, 2006. Sincerely,



Debbie Davenport Auditor General Enclosure



2910 NORTH 44



th



STREET  SUITE 410  PHOENIX, ARIZONA



85018  (602) 553-0333  FAX (602) 553-0051



SUMMARY

The Office of the Auditor General has conducted a performance audit of the Arizona Supreme Court, Administrative Office of the Courts (AOC) that focused on information technology (IT) efforts pursuant to Laws 2004, Chapter 39, 2. This audit was conducted under the authority vested in the Auditor General by Arizona Revised Statutes (A.R.S.) 41-1279.03. Arizona's court system includes a supreme court, a court of appeals, and a superior court with a branch in each county, as well as 85 municipal courts and 85 justice courts as of March 2006. The Constitution establishes the Chief Justice of the Supreme Court as the administrative head of the judiciary branch and AOC assists the Chief Justice by providing a variety of services and programs to the courts, including information technology services. This audit focused primarily on two key aspects of AOC's information technology efforts:

Management



and oversight of new information technology systems and



projects; and Fines, Fees, and Restitution Enforcement (FARE) program, which provides a wide variety of automated collection services to participating courts.

The



This report also presents other pertinent information regarding the use of monies in the Judicial Collections Enhancement Fund (JCEF) and the Defensive Driving School Fund (DDSF) for information technology and operating purposes, and information on steps the Supreme Court has taken to secure its IT resources.



Fundamentally sound program for managing technology projects could be improved (see pages 11 through 21)

AOC's framework for managing new information technology projects compares favorably with industry practices for IT project management, but can still be improved in several important respects. The Commission plans to invest an estimated total of



Office of the Auditor General

page



i



$10.5 million in developing 11 new information technology systems between fiscal years 2006 and 2008, including new case management systems for both limited and general jurisdiction courts. A sound project management framework is important for ensuring that these projects are completed on time and within budget. Auditors' comparison of AOC's framework to 11 project management standards recommended by COBIT showed that the framework substantially or partially meets all of these standards.1 COBIT is an internationally recognized set of information technology guidelines. Areas that would benefit most from further attention: R Risk management--AOC's framework does not provide guidance on how project management teams should identify and manage risks related to the development of information technology systems. Limited guidance on risk management contributed to the delayed development of an update to the system for tracking juveniles on probation. S Stakeholder communication--AOC's framework does not require project managers to specify with whom to communicate and how to most effectively communicate. Industry standards speak to the importance of stakeholder participation to avoid disappointing results, and limited communication affected the development of the update to the juvenile probation tracking system. I Independent review--AOC's framework does not address the extent to which staff outside the project management team should independently review projects to ensure that the information provided to management is accurate and the project is on schedule. Without such reviews, project status and progress cannot be confirmed. This was the case for the juvenile probation system as the lack of independent reviews allowed project problems and the project's true status to go undetected by management for several months. AOC is taking action in several of these areas. It plans to add a formalized risk management process in August 2006, and it has added two positions to conduct independent reviews of project documentation and status. Beyond these steps, AOC should require each project management team to develop a communication plan that specifies the project's various stakeholders and how to most effectively communicate with these stakeholders. The team should also establish policies and procedures to guide the performance of the independent reviews. In addition, AOC has not always monitored whether project management teams adhere to its framework. In April 2006, a management team began to determine how each project should follow the framework. To ensure consistency, however, AOC needs to develop policies and procedures for applying the framework to developing IT systems. Separate from its own development of information technology projects, AOC sometimes relies on courts to develop state-wide technology applications and should improve its oversight of these projects. While AOC performs several activities to monitor the IT systems being developed by individual courts, it can do more to

1 IT Governance Institute. COBIT 4.0: Control Objectives, Management Guidelines, Maturity Models. Rolling Meadows, IL: IT Governance Institute, 2005.



State of Arizona

page



ii



better monitor the development of these systems. One of these steps includes entering into formal agreements with individual courts about such matters as the project's scope and the responsibilities of each party to help ensure that all parties are aware of these items. AOC should also develop and implement a process for identifying the risks in these projects and the actions that could be taken to manage these risks.



Additional collections contract oversight and program monitoring needed (see pages 23 through 30)

AOC can improve its management and oversight of the Fines, Fees, and Restitution Enforcement (FARE) program. The program, which is administered by a private vendor, provides participating courts with such services as collection notices, referrals for the interception of state tax refunds and lottery winnings to pay debts owed to the courts, and referrals for holds on motor vehicle registration until such debts are paid. As of July 2006, AOC reported that it had collected nearly $60.6 million through FARE. AOC has implemented a vendor contract that includes essential contract elements, a performance log that tracks the contract requirements, and data reconciliation processes to ensure that program data is reliable and complete. However, auditors' review identified needed improvements in the following areas: C Closing gaps in the contract--The contract lacks noncompliance penalties for 48 of the 84 contract requirements, which could make it more difficult for AOC to enforce contract provisions. Further, the contract lacks a requirement for a general assurance review or similar type of audit, which is a check to ensure that controls are adequate to safeguard data, monies, and other assets. I Improving oversight of vendor performance--The contract has 84 monitoring requirements--enough to require more formal processes for monitoring vendors' compliance. These processes should specify how frequently contract items will be monitored for compliance and include procedures AOC staff should perform to verify vendor reports of compliance with the contract requirements. I Implementing performance measures--While AOC program management has expressed a need for such measures, its focus has been on implementing the program. However, AOC collects seven of the eight types of information needed for collections performance measures recommended by the National Center for State Courts. AOC should use these recommended performance measures as a guide for developing FARE's performance measures.



Office of the Auditor General

page



iii



Other pertinent information (see pages 31 through 36)

In response to legislative inquiry, auditors collected other pertinent information related to (1) the use of monies in the Judicial Collections Enhancement Fund (JCEF) and the Defensive Driving School Fund (DDSF), and (2) the security of the Supreme Court's information technology resources. Specifically: and DDSF provide most of the monies for developing and maintaining technology for the court system. The Supreme Court uses these Funds to provide information technology services to the courts, develop new information technology systems and maintain its technology infrastructure, and to cover operational costs, such as rent payments for the Supreme Court. AOC projections show that current and continuing IT needs may deplete the balances of these funds sometime in fiscal year 2008. The Supreme Court has taken some steps to address this problem.

JCEF



review of technology security found that the Supreme Court has taken steps to secure its information technology resources, including controlling physical access to its facilities, installing network security devices and software, and controlling user access to and security of its systems and applications. AOC management also reports that the Supreme Court is taking additional steps to increase security, including removing signage outside its central computer center and dedicating an employee to monitoring systems and network security.

Auditors'



State of Arizona

page



iv



TABLE OF CONTENTS

Introduction & Background Finding 1: Fundamentally sound program for

managing technology projects could be improved

Project management framework covers most key areas AOC can improve framework guidelines and application AOC and the Commission monitor development of courts' IT systems, but can do more Recommendations



1 11 11 14 17 21 23 23 26 28 30 31 31 36



Finding 2: Additional collections contract oversight and

program monitoring needed

AOC provides collection assistance to the courts AOC can take additional steps to improve vendor oversight Performance measures needed to assess program effectiveness Recommendations



Other Pertinent Information:

Available money for technology expenses decreasing AOC has taken steps to secure IT resources



Agency Response



continued



Office of the Auditor General

page



v



TABLE OF CONTENTS

Tables:

1 Information Technology Projects in Development Fiscal Years 2006 through 2008 2 Supreme Court Schedule of Revenues and Expenditures And Changes in Fund Balance Fiscal Years 2004 through 2006 (Unaudited) 3 Project Management Framework Compared to Selected COBIT Project Management Detailed Control Objectives 4 Vendor Charges for FARE Services By Number of Items per Contract Year 5 Judicial Collection and Enhancement Fund and Defensive Driving School Fund Schedules of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2004 through 2006 (Unaudited)



4



6 13 25



32



Figure:

1 Judicial Collection Enhancement and Defensive Driving School Funds' Revenues and Expenditures Fiscal Years 2000 through 2005 (Unaudited)



35



concluded



State of Arizona

page



vi



INTRODUCTION & BACKGROUND

The Office of the Auditor General has conducted a performance audit of the Arizona Supreme Court, Administrative Office of the Courts (AOC) that focused on information technology (IT) efforts pursuant to Laws 2004, Chapter 39, 2. This audit was conducted under the authority vested in the Auditor General by Arizona Revised Statutes (A.R.S.) 41-1279.03.



The Arizona court system

The Arizona Constitution establishes an integrated judicial department headed by the Supreme Court. As such, all courts in the State are part of a single court network. The Constitution specifically establishes and defines some of the courts in this network, including the superior court and justice courts. However, the Constitution allows the Legislature to establish appellate courts and courts inferior to the superior court. In 1913, the Legislature established municipal courts for cities and towns, and in 1964 established the court of appeals. The Arizona court system includes the following three types of courts: Appellate courts--The Supreme Court and the court of appeals are appellate A courts and hear cases that have been tried previously in lower courts. For example, the Supreme Court hears appeals of decisions from the court of appeals, as well as death penalty cases, directly from the superior court. The court of appeals hears civil and criminal appeals from the superior court. General jurisdiction court--The superior court is a general jurisdiction court that G has jurisdiction over a wide variety of cases, including felonies, divorce, eviction of renters, and cases in which the value of the property in question is $1,000 or more. In addition, the superior court acts as the court of appeals for justice and municipal courts. Each county in the State has a superior court. Limited jurisdiction courts--Municipal and justice courts are more limited than L the superior court in the case variety that they hear. Municipal courts hear misdemeanor criminal and civil traffic violations issued in the city or town, and violations of city codes or ordinances. They also issue orders of protection and

Arizona's Constitution establishes an integrated judiciary.



Office of the Auditor General

page



1



search warrants. Arizona had 85 municipal courts as of March 2006. Arizona also had 85 justice courts as of March 2006 that represent precincts that do not necessarily correspond to the boundaries of cities and towns. Justice courts hear certain civil and criminal offenses such as domestic violence and harassment cases, and conduct preliminary hearings for felonies.



Administrative Office of the Courts

As part of the single network of courts, the Constitution establishes the Chief Justice of the Supreme Court with responsibility for the administrative supervision of all courts in the State. To assist with this responsibility, the Constitution requires the Chief Justice to hire an administrative director, who heads the Administrative Office of the Courts (AOC). AOC assists the Chief Justice by providing a variety of services and programs to assist the courts in fulfilling their responsibilities. In order to carry out its duties, AOC reports that it has established an executive office and eight divisions with a total of 412.4 (36.5 vacant) positions as of June 30, 2006.1 Executive Office (12.5 filled positions, 0 vacancies)--This division is composed E of the administrative director, deputy director, and staff who support strategic planning; internal auditing; and public, media, and government-relations activities. A Administrative Services Division (65 filled positions, 7 vacancies)--This division is responsible for financial services, legal services, facilities management, and human resources. A Adult Probation Services Division (23 filled positions, 2 vacancies)--This division oversees the state-wide administration of adult probation programs and services. C Certification and Licensing Division (27 filled positions, 2.5 vacancies)--This division certifies or licenses several professions, including fiduciaries, court reporters, legal document preparers, and confidential intermediaries. C Court Services Division (56.5 filled positions, 3 vacancies)--This division conducts research and reports statistics, and tracks and reports to the courts on legislation that affects the judiciary department. It also houses the Fines, Fees, and Restitution Enforcement (FARE) collections program, which provides automated collections services to individual courts. D Dependent Children's Services Division (46.6 filled positions, 1.0 vacancies)-- This division provides services for children going through dependency matters



AOC assists the Chief Justice with administration of the judiciary.



1



AOC reports that 138.6 of its FTEs are funded by the superior court, while 273.8 are funded by the Supreme Court.



State of Arizona

page



2



in the courts, such as those in the foster care system administered by the Department of Economic Security. E Education Services Division (26.5 filled positions, 4 vacancies)--This division maintains the state-wide system of education for the court system and oversees compliance with judicial education standards for court employees. I Information Technology Division (84.6 filled positions, 14 vacancies)--This division develops and supports judicial information technology and projects. J Juvenile Justice Services Division (34.2 filled positions, 3.0 vacancies)--This division administers juvenile justice programs for delinquent and incorrigible youth in coordination with the juvenile courts.



Information technology in the judiciary

As part of its responsibility to provide support for the Chief Justice of the Supreme Court in administering the courts, AOC implements state-wide technology initiatives under the guidance of the Commission on Technology (Commission). The 1989 Report of the Commission on the Courts recommended that a Commission on Automation be created that represented the diverse perspectives and needs of the courts, and in 1990 the Commission was established. The Commission oversees technology for the judiciary, providing policy recommendations and direction on matters related to information technology. For example, the Commission determines how to allocate available funding for information technology (IT) projects to courts who submit grant requests and among proposed technology projects. The vice chief justice heads the Commission, and the membership includes judges, clerks of court, court administrators, and public members, as well as a representative from the State Bar, the Executive Branch, the County Supervisors' Association, and the League of Cities and Towns. As of May 4, 2006, the Commission had 19 members. The Commission is also supposed to have a representative from the Legislature, but as of June 2006, this position was vacant. AOC reports that under the direction of the Commission, it is developing or planning to develop 11 information technology systems for state-wide use during fiscal years 2006 through 2008 with an estimated total cost of $10.5 million. As illustrated in Table 1 (see page 4), these include new case management systems for both limited and general jurisdiction courts, as well as a new adult probation tracking system and an updated juvenile probation tracking system. In addition to system development, AOC maintains the infrastructure on which information technology systems operate, including networks and databases, as well as provides support to users throughout the State. AOC also upgrades and modifies existing systems to provide enhancements and comply with statutory changes. In fiscal year 2006, the Supreme Court expected to spend $9.8 million on infrastructure and support, and $2.8 million on information technology system development.



The Commission on Technology provides policy recommendations related to technology.



Office of the Auditor General

page



3



Table 1:



Information Technology Projects in Development Fiscal Years 2006 through 2008

Projected Cost1 $3,089,000 3,034,000 Project Description Develop and implement case management system for superior courts Develop and implement Tempe Municipal Court case management system, which may be implemented state-wide2 Develop and implement an updated juvenile probation tracking system Develop and implement collections system Develop and implement a state-wide adult probation tracking system Implement imaging system at AOC, Supreme Court, and Court of Appeals Develop central index and access for all court documents in the State Develop and adopt standards and processes for electronic signatures on court documents Study, develop, and implement a system to automate and integrate the flow of information between the courts, law enforcement, and prosecutors Statistical reporting tool Electronic transfer of case information for appeal to the Court of Appeals and Supreme Court



Project Name General Jurisdiction Case Management System Limited Jurisdiction Case Management System Juvenile Online Tracking System AZ Fines, Fees, and Restitution Enforcement Adult Probation Tracking System Electronic Document Management System Electronic Case File Documents Repository Electronic Signatures Justice Integration Disposition and Warrant Information Flow



1,867,000 695,000 643,000 358,000 300,000 225,000 110,000



Crystal Enterprise Project Electronic Filing for Appellate Courts

1



93,000 90,000



These costs reflect only the time period of fiscal years 2006 through 2008. Some projects may have costs associated with them from prior fiscal years and may have costs in future fiscal years. According to AOC's chief information officer, the case management system for limited jurisdiction courts is being monitored and evaluated as a potential replacement for the current state-wide system. The Commission will make the final decision on whether to adopt the new system for state-wide use after it is implemented in Tempe.



2



Source:



Auditor General staff analysis of The Arizona Supreme Court Administrative Office of the Court's 2005 Report on Court Automation Projects, September 2005.



The Supreme Court also provides many of the courts in the State with computer hardware, software, and network services through the Arizona Court Automation Project (ACAP). The ACAP program also provides participating courts access to the



State of Arizona

page



4



judiciary's state-wide network, the Arizona Judicial Information Network (AJIN). This network links most of Arizona's courts and allows these courts to transmit data and information to each other and to state agencies, including AOC. Individual courts that participate in ACAP also have access to various court systems, such as the statewide case management and probation systems. In fiscal year 2005, the Supreme Court spent over $4 million from the Judicial Collection Enhancement Fund and the Defensive Driving School Fund to provide ACAP services. It recovered over $1.4 million of its costs by charging fees to the courts for ACAP services. Finally, AOC developed enterprise architecture standards to ensure consistency in information technology throughout the judiciary. These standards are a set of principles, standards, and products that guide how systems throughout the judiciary should be developed and implemented. Some courts have developed their own information technology projects, such as the Maricopa County Superior Court, which has developed both a financial management system and a case management system. These courts are expected to follow AOC's enterprise architecture standards.

AOC has developed enterprise architecture standards.



Operating budget

Table 2 (see page 6) illustrates the Supreme Court's actual revenues and expenditures for fiscal years 2004 through 2006. This includes operating expenditures for AOC. As shown in Table 2, the Supreme Court received revenues totaling nearly $39.7 million in fiscal year 2005 and approximately $44.8 million in fiscal year 2006 for an increase of nearly 13 percent. AOC attributes this growth in revenues to increased collections from the Fines, Fees, and Restitution Enforcement (FARE) program collections. However, the Supreme Court expenditures increased by nearly $6.6 million, or 18 percent, from approximately $36.6 million in fiscal year 2005 to nearly $43.2 million in fiscal year 2006. AOC attributes some of this increase in expenditures to an increase in the Court's building lease costs of approximately $1.7 million, costs for leasing instead of purchasing equipment, and an additional $2.3 million paid to a contractor providing collection services for the FARE program. In addition to receiving State General Fund appropriations, the Supreme Court receives monies from six different funds, including the Judicial Collection Enhancement Fund (JCEF) and the Defensive Driving School Fund (DDSF). The Legislature appropriates monies from these Funds for various purposes, such as for the Court Appointed Special Advocate program, Foster Care Review Board, and information technology projects. For fiscal year 2006, the Supreme Court budgeted nearly $13.4 million in revenues from JCEF and DDSF, which provide a majority of the Supreme Court's funding for technology projects (see Table 5, page 32). The Supreme Court uses monies from these two funds to provide technology services to the courts, support and maintain existing IT systems, and develop new IT systems.



Office of the Auditor General

page



5



Table 2:



Supreme Court Schedule of Revenues and Expenditures And Changes in Fund Balance1 Fiscal Years 2004 through 2006 (Unaudited)

2004 2005 $12,113,109 12,419,900 9,958,450 4,573,886 306,411 52,500 256,161 39,680,417 18,600,936 2,946,293 228,967 8,404,158 7,515,401 317,178 (1,446,863) 36,566,070 52,544 36,618,614 $ 3,061,803 2006 $12,618,832 16,235,108 10,546,532 4,681,450 611,251 11,750 117,230 44,822,153 19,881,534 5,404,521 4 256,291 9,439,288 9,510,683 5 252,335 (1,593,107) 43,151,545 42,994 43,194,539 $ 1,627,614



Revenues: State General Fund appropriations Charges for services2 Fines, forfeits, and penalties Intergovernmental Interest earnings Private grants Other Total revenues Expenditures and remittances to the State General Fund:3 Personal services and employee-related Professional and outside services Travel Aid to counties Other operating Equipment ACAP fees6 Total expenditures Remittances to the State General Fund Total expenditures and remittances to the State General Fund Excess of revenues over expenditures and remittances to the State General Fund7

1



$11,823,643 10,561,489 9,298,057 3,614,033 176,069 24,565 59,513 35,557,369 17,412,353 2,551,440 273,067 8,172,477 7,505,027 578,228 (993,059) 35,499,533 49,571 35,549,104 $ 8,265



2 3 4 5



6



7



Excludes county probation fees collected by the Supreme Court (Court) and the distribution of those fees to the counties. While these fees are deposited into the Court's Judicial Collection Enhancement Fund, they are distributed to the counties and are not available to pay for operating costs of the Court, and therefore, are not reported in the schedule. AOC reports that the fiscal year 2006 charges for services revenue increased significantly because of increased Fines, Fees, and Restitution Enforcement (FARE) program collections. Administrative adjustments are included in the fiscal year paid. AOC reports that the 2006 professional and outside services expenditures increased significantly primarily because an additional $2.3 million was paid to a contractor providing collection services for the FARE program. The AOC reports that the increase in fiscal year 2006 other operating expenditures is primarily attributable to an increase in the Court's building lease costs of approximately $1.7 million and costs for leasing equipment instead of purchasing equipment. Arizona Court Automation Project (ACAP) fees are collected from courts that receive computer equipment and services, and access to court systems and the judiciary network. See Other Pertinent Information, pages 31 through 36, for further information. The fees collected are considered reimbursements of the Court's costs and, therefore, reduce expenditures. AOC reports that the excess of revenues over expenditures in fiscal years 2005 and 2006 primarily relates to the collections in the Lengthy Trial Fund, the FARE program, and State Aid to Courts Fund exceeding the expenditures in those funds. In addition, the Court accumulated monies for the development and acquisition of new automated case and cash management systems. The excess of revenues over expenditures in the FARE program is expected to be spent in subsequent fiscal years to cover the cost of the tax intercept project, FARE program enhancements, and for participating court collection reimbursements. The excess of remaining revenues over expenditures is expected to be spent during fiscal year 2007 for the Lengthy Trial Fund due to new legislation and for the State Aid to the Courts Fund when the monies are transmitted back to the courts pursuant to A.R.S. 12-102.2. An appropriation increase is being requested for the State Aid to the Courts Fund in fiscal year 2008 to allow the courts to spend at the level that revenues can support. Auditor General staff analysis of the Arizona Financial Information System (AFIS) Accounting Event Transaction File and financial information provided by AOC for fiscal years 2004 through 2006.



Source:



State of Arizona

page



6



Scope and methodology

This audit focused on the methods that AOC uses to oversee the development of information technology systems and its management and oversight of the FARE program and vendor collections contract. The report presents findings and recommendations in the following areas: AOC has established a fairly comprehensive project management framework for information technology systems and has instituted several mechanisms to oversee information technology systems that are developed by individual courts, it can take some additional steps to improve its project management framework and oversight practices.

While While AOC has taken several steps to help Arizona courts improve the collection



of court fines and fees, it can improve its management of the judicial collections program by increasing oversight of its collections contract and by implementing performance measures. The report also presents other pertinent information regarding the use of monies in the Judicial Collection Enhancement Fund and the Defensive Driving School Fund, and information on the steps that the Supreme Court has taken to secure its IT resources. Auditors used various methods to study the issues addressed in this report, including interviewing AOC staff and other court staff, reviewing AOC's policies and procedures related to FARE, reviewing statutes, and analyzing financial information. Auditors also used the following specific methods: assess the adequacy of AOC's information technology project management and oversight practices and the application of these practices to IT systems under development, auditors reviewed AOC's project management framework and associated documentation, including checklists for the concept, initiation, and planning phases of the project development process; planning documentation, including detailed project plans, and scope, for the Juvenile Online Tracking System and the Adult Probation Enterprise Tracking System; the Arizona Supreme Court Administrative Office of the Court's 2005 Report on Court Automation, prepared in September 2005; project status reports submitted to the Court Automation Coordinating Committee of the Commission on Technology (Commission); meeting minutes from January 2000 through March 2006 meetings of the Commission; the Judicial Project Investment Justification for the Tempe Municipal Court case management system; an intergovernmental agreement between the City of Tempe and AOC; and funding plans for both the Pima County and Tempe case management systems.

T o



Office of the Auditor General

page



7



Auditors also compared AOC's project management framework to best practices from the IT Governance Institute's Control Objectives for Information and Related Technology (COBIT), an internationally recognized set of information technology guidelines, and COBIT Control Practices, which provide additional detail for the control objectives, as well as North Carolina's Implementation Framework for Statewide Information Technology Projects: Best Practices and Standards, which provides additional information technology guidelines and best practices.1--3

T o



evaluate AOC's oversight and management of its FARE collections program, including its oversight of the vendor contracted to provide services as part of the program, auditors reviewed the request for proposals; the selected vendor's proposal; AOC's vendor contract; the performance standards log; documents used to reconcile both financial and program data among the courts, the AOC, and the vendor; and observed the program data reconciliation process. Auditors also reviewed collections studies conducted by AOC staff, including the 2003 and 2004 data reports for both limited and general jurisdiction courts, federal government contracting guidelines recommended by the Federal Acquisition Regulation System, and information on assessing program performance from the National Center for State Courts.4--5 o T gather information regarding the revenues, expenditures, and projected fund balances in the Judicial Collection Enhancement Fund and the Defensive Driving School Fund, auditors reviewed documentation to support the projected revenues and expenditures for the funds for fiscal years 2005 through 2009 in AOC's spreadsheet, Judicial Collection Enhancement Fund (JCEF)/Traffic Case Processing Fund (TCPF) Summary Financial Position as of May 4, 2005, reviewed the fiscal year 2005 Arizona Court Automation Project expenditures and fees, analyzed financial information from AOC's accounting system, and reviewed the Joint Legislative Budget Committee appropriations reports for fiscal years 2003 through 2006, the Arizona Judicial Branch Information Technology Strategic Plan, 2006-2008, and the Judiciary Branch's legislative proposals for the 2006 legislative session.



1



IT Governance Institute. COBIT 4.0: Control Objectives, Management Guidelines, Maturity Models. Rolling Meadows, IL: IT Governance Institute, 2005. Auditors compared AOC's project management framework to 11 of COBIT's 14 project management control objectives. These 11 objectives address needed components of managing the development of individual IT projects, while the remaining 3 objectives address establishing a program and project management framework for the management of all IT projects, including the prioritization and coordination of all projects. IT Governance Institute. Control Practices. Rolling Meadows, IL: IT Governance Institute, 2004. North Carolina State Government CIO. Implementation Framework for Statewide Information Technology Projects: Best Practices and Standards. NC: Office of Information Technology Services, 2005. 48 C.F.R., 16.402.2. National Center for State Courts, CourTools: Trial Court Performance Measures. Williamsburg, VA: National Center for State Courts, 2005.



2 3



4 5



State of Arizona

page



8



determine the steps AOC takes to secure and control its IT resources, auditors reviewed AOC's IT security manual, toured its computer room, reviewed personnel files from AOC's human resources department for security acknowledgement forms, and observed the steps taken to ensure that users have the appropriate access to IT resources. Auditors also observed securityrelated devices and software, such as firewalls, anti-virus protection, and patch management to automatically update computers with the latest patches to prevent security breaches. gather information for the Introduction and Background, auditors reviewed the Arizona Constitution and unaudited information from the Arizona Supreme Court's Guide to Arizona Courts, the Supreme Court's Web site, the 1989 Report of the Commission on the Courts, information from AOC on the number of fulltime equivalent positions, and the Arizona Financial Information System (AFIS) Accounting Event Transaction File and AFIS Status of Appropriations and Expenditures and Trial Balance by Fund reports for fiscal years 2004 and 2005. This audit was conducted in accordance with government auditing standards. The Auditor General and staff express appreciation to the Chief Justice and the director and staff of the Administrative Office of the Courts for their cooperation and assistance throughout the audit.

T o



T o



Office of the Auditor General

page



9



State of Arizona

page



10



FINDING 1

Fundamentally sound program for managing technology projects could be improved

While AOC's IT project management framework covers many key areas, it could be improved. The Commission on Technology (Commission) plans to invest approximately $10.5 million in IT projects between fiscal years 2006 and 2008 and has established a fairly comprehensive framework to assist in developing and managing these projects. However, AOC can further improve its framework by addressing gaps in the areas of risk management, stakeholder communication, and independent reviews, as well as taking steps to ensure that project managers appropriately follow the framework. In addition, AOC has put several processes in place to monitor individual courts' IT system development, but it can take some additional steps, such as formalizing agreements with these courts and conducting project risk assessments, to help ensure these systems are developed as planned.



Project management framework covers most key areas

AOC has developed an information technology project management framework for projects being developed by AOC staff. This framework provides guidance to project managers on the tasks they need to complete when developing an IT system, and auditors found that it covers key areas recommended by IT industry standards. Good project management practices are important in ensuring that information technology projects meet established requirements and are done on time and within budget. The use of a project management framework helps to reduce the risk of unexpected costs and project cancellations, improve communications with and involvement of business-end users, and ensure the value and quality of project deliverables.1 As the administrative arm of the Supreme Court, AOC, under the Commission's direction, often provides state-wide technology applications to the courts, and in the past has either developed these applications in-house or contracted for their development. The Commission plans to invest an estimated $10.5 million in developing 11 new IT systems for state-wide use between fiscal years 2006 and 2008, and a

1 IT Governance Institute, COBIT4.0: Control Objectives, Management Guidelines, Maturity Models. Rolling Meadows, IL: IT Governance Institute. 2005.



The Commission plans to spend $10.5 million to develop systems between fiscal years 2006 and 2008.



Office of the Auditor General

page



11



comprehensive project management framework can help to ensure that these projects are completed on time and within budgets. AOC's project management framework compares favorably with 11 key project management control objectives recommended by COBIT, an internationally recognized set of information technology guidelines, and the North Carolina State Government's Implementation Framework for Statewide Information Technology Projects, which provides additional information technology guidelines and best practices.1--3 These practices include properly addressing the scope of a planned IT system project, quality management, change control, staffing, and postimplementation reviews. As shown in Table 3 (see page 13), auditors found that AOC's project management framework either substantially or partially addressed 11 of the project management control objectives recommended by COBIT. AOC's framework has five phases, and each phase includes several steps that provide general guidance to project managers on the processes they should follow for completing these phases. Additionally, the framework includes detailed checklists for some phases that project managers can use to help them ensure they complete all required tasks within a phase before moving on. Finally, AOC's framework requires approval by project management team members, including the project manager, AOC's IT director, and the business director at eight critical points within the five phases. This helps ensure that projects receive appropriate review at these points before proceeding. Various stakeholders can be added to the needed approval as well. AOC's application of its framework has facilitated the development of its Adult Probation Enterprise Tracking System (APETS). APETS is designed to enhance each county's ability to track adult offenders' contacts, addresses, conditions of probation, treatment, employment, and transfers between counties. APETS is scheduled to be implemented state-wide by December 2006, and as of May 2006, had been fully implemented in ten counties. Auditors' review of project management documentation found that APETS development has closely adhered to AOC's project management framework. This includes: members' roles and responsibilities have been clearly specified and documented;

Project



1



IT Governance Institute. COBIT 4.0: Control Objectives, Management Guidelines, Maturity Models. Rolling Meadows, IL: IT Governance Institute, 2005. Auditors compared AOC's project management framework to 11 of COBIT's 14 project management control objectives. These 11 objectives address needed components of managing the development of individual IT projects, while the remaining 3 objectives address establishing a program and project management framework for the management of all IT projects, including the prioritization and coordination of all projects. IT Governance Institute. Control Practices. Rolling Meadows, IL: IT Governance Institute, 2004. Control practices provides an additional level of detail for the control objectives defined in COBIT 4.0. North Carolina Government CIO, Implementation Framework for Statewide Information Technology Projects: Best Practices and Standards. NC: Office of Information Technology Services, 2005. The North Carolina Framework identifies 15 best practices and an additional 35 standards.



2



3



State of Arizona

page



12



Table 3:



Project Management Framework Compared to Selected COBIT Project Management Detailed Control Objectives1,2

AOC Requirements and Practices  Projects must receive approval from business sponsors before proceeding  Some projects provide status reports to Commission on Technology subcommittees  Project scope must be completed and approved  Project's functional requirements reviewed and approved  A communication plan is not required  Five phases for projects that include many steps  Eight checkpoints at which approval is required before proceeding  Project plan that includes:  Testing  Training  Risk  Implementation  Integration  Marketing  Production Turnover  Determine resources needed to complete project during initiation of the project  Judicial Project Investment Justification specifies roles and responsibilities of personnel.  Risk planning is supposed to be part of overall planning, but it is not yet standardized and still under development  Project planning includes developing a plan to test the new or updated technology  Scope and impact of project changes must be documented, reviewed, and approved  Planning and testing steps to ensure systems meet requirements  Weekly status reports and monthly reports at managers' meetings  Some larger projects report to Commission on Technology subcommittees and steering committees  No independent review and monitoring of project required  Post-implementation reviews required Objective Met Substantially



COBIT Control Objective Obtain stakeholder commitment and participation



Define, document, and approve scope Initiation and completion of project phases should be reviewed and approved Establish formal integrated project plan



Partially



Substantially



Substantially



Competent project resources assigned and responsibilities defined Identified project risks are eliminated or minimized A quality management plan is prepared and approved Establish a change control system Identify system assurance tasks Measure, monitor, and report project performance



Substantially



Partially Substantially Substantially Substantially Partially



Conduct post-implementation reviews

1



Substantially



IT Governance Institute. COBIT 4.0: Control Objectives, Management Guidelines, Maturity Models. Rolling Meadows, IL: IT Governance Institute, 2005 IT Governance Institute. Control Practices. Rolling Meadows, IL: IT Governance Institute, 2004. Auditor General staff analysis and comparison of AOC's project management framework to selected COBIT project management detailed control objectives.



2



Source:



Office of the Auditor General

page



13



development and implementation plans exist for each phase of the project, and these plans are regularly updated to reflect progress;

Detailed



approvals are obtained at the appropriate points in the project, which ensures that project team members and stakeholders agree when the project is ready for the next phase;

Required



reviews have been held with users after implementing the system in many counties, which have identified areas for change for future implementations; and

Post-implementation



team regularly meets with a project steering committee and a users' group to discuss progress, potential changes, and the effect potential changes might have on the project schedule.

Project



Although by February 2006 the APETS project had fallen 6 months behind the original implementation schedule set in 2003, according to the project manager, these delays were mostly attributable to staffing shortages, and scheduling adjustments were made. In addition, auditors found that the project manager updated the project plans and informed management as the delays occurred.



AOC can improve framework guidelines and application

While AOC's project management guidelines cover many key areas, AOC can take some steps to further improve these guidelines. Specifically, the guidelines should more fully address identifying and managing potential project risks, stakeholder communications, and project monitoring. In addition, AOC should take steps to ensure that IT projects developed by AOC staff follow the guidelines as designated by Information Technology Division management.



AOC framework deficient in three areas--AOC's



framework has deficiencies in three main areas. These deficiencies' effect can be seen in the Juvenile On-line Tracking System (JOLTS), an IT project AOC began developing in April 2004 to update the system for tracking juveniles on probation. Specifically: R Risk management--The framework only generally addresses risk management without providing any guidance on the steps that project management teams should take to identify potential risks and the actions that could be taken to manage and mitigate these risks. The framework only states that the project team should develop a risk plan as part of the planning phase. However, COBIT stresses the importance of minimizing the risks to a project by planning, identifying, analyzing, responding to, monitoring, and controlling all areas of risk.



State of Arizona

page



14



Auditors' review of project documentation regarding JOLTS' development found that AOC inadequately managed risk. Early in the JOLTS project's development, the lack of project team experience with the system technology was identified as a risk. While AOC indicated that it addressed this identified risk by obtaining training and mentoring for programmers, it did not continually monitor and analyze this risk to ensure that the training sufficiently mitigated it. According to an AOC official, the lack of experience with the system technology continued to be a problem, and it was not until the project was 6 months behind schedule that management took further steps to address the risk, including changing the scope of the project. Although project managers are supposed to consider risk when planning a project, AOC's Project Management Office Manager acknowledges that the degree to which risk assessment is done varies between projects. Therefore, AOC needs to formalize the process that it uses to identify and mitigate risks. According to this manager, it hopes to include a formalized risk management process as part of the framework in August 2006. S Stakeholder communication--The framework only partially addresses stakeholder communication by requiring approval from relevant parties at various stages in project development. For example, the framework requires approval of the project scope before detailed planning can begin. However, it does not require project managers to establish a communication plan that specifies who to communicate with and how to most effectively communicate. COBIT recommends that projects obtain commitment and participation from stakeholders and create a communication plan that specifies both who to communicate with and how to communicate, and is updated during the project. The North Carolina Framework states that stakeholder participation is important in order to avoid disappointing results. However, stakeholder communication has been a problem in the JOLTS project. Specifically, according to an AOC official, JOLTS project documentation did not include a plan for communicating with stakeholders, resulting in inconsistent communication with the primary, intended users of the system and other project stakeholders. For example, according to this same official, the JOLTS project management team was supposed to work closely with the Pima County Juvenile Court, which would provide business analysis and act as one of the pilot sites for the new system. The team was also supposed to meet with its project steering committee and the Probation Automation Coordinating Committee, an advisory committee of the Commission. However, absent a communication plan, communication with these various parties occurred infrequently. For example, according to the AOC official and the JOLTS project manager, the project team did not have any contact with Pima County from June through October 2005. Therefore, AOC should develop and implement policies and procedures within its framework regarding communication. These policies should require



A communication plan helps to ensure communication with stakeholders.



Office of the Auditor General

page



15



each project management team to develop a communication plan that specifies the project's various stakeholders and how to most effectively communicate with these stakeholders. I Independent review--The framework does not include any policies or guidance requiring the independent monitoring and review of projects by staff outside the project management team. The North Carolina Framework emphasizes the need for regular, independent reviews of management processes and project progress to verify project status, the reasonableness of business requirements, and to identify unrecognized problems. Without these reviews, reported project status and progress cannot be confirmed. The JOLTS project was hampered by a lack of independent reviews that were not sufficiently specified by AOC. According to an AOC official, the project management team began to encounter problems with the system's development and provided inaccurate information to AOC's IT division management regarding these problems and the project's status. This included project scope changes that were not properly approved. Since an independent review was not conducted to examine actual progress versus reported progress, management was unaware of these issues. According to AOC management, it was not until the project was about 6 months behind schedule and dates for the delivery of pilot pieces of the system were missed that division management became aware of the problems and began to make needed changes. Regularly performing independent reviews of project progress and status may have informed division management of these problems earlier. Although AOC added two positions in late 2005 in its project management office to assist in monitoring the status of projects, it has yet to establish guidelines for monitoring and assessing project performance and status. According to AOC's Chief Information Officer (CIO), the staff in these positions will review project plans to ensure that they accurately reflect project progress and independently report this progress to division management. AOC should independently review the status and progress of its IT projects and establish and implement policies and procedures regarding these reviews. These policies and procedures should specify who should monitor projects to verify that the technical approach is sound and that progress is being accurately reported, the frequency of such reviews, and how the monitoring should occur.



Use of framework not always monitored--In addition to the need to address

gaps in its framework, AOC has not always monitored whether project management teams adhere to the framework. Although the framework has some gaps that contributed in part to delays in the JOLTS project, according to information provided by AOC's CIO, project managers also did not adhere to the framework. For example, although JOLTS had a project scope, the scope was not formally updated as the project changed, and changes to the project were not



JOLTS project managers did not always apply the framework.



State of Arizona

page



16



approved by agency management or other stakeholders. In addition, the project plan for JOLTS was not updated with the project's progress. As a result, neither AOC management nor stakeholders were fully aware of how the project was changing from the original scope and plan and the consequences of those changes. AOC needs to develop and implement policies and procedures regarding framework use. According to division management, the project manager and a management team made up of the managers from the various units within the Division, such as operations, strategic planning, and the project management office, will be responsible for determining how the project management framework will be used for each project. This management group began meeting in April 2006. Until this time, according to AOC's project management office manager, project managers determined how to apply the framework to their project. However, to guide this team in making its determinations and to help ensure the consistent application and use of the framework for all projects, AOC should develop policies and procedures requiring that all information technology projects under development use the framework or specify the circumstances under which projects will not be required to implement the full framework.



AOC and the Commission monitor development of courts' IT systems, but can do more

In addition to its project management framework, AOC and the Commission have established several processes to monitor IT system projects developed by individual courts for state-wide use, but can do more. Specifically, AOC has begun partnering with courts to assist in Arizona court case management developing certain systems. As a result, AOC and the systems being developed Commission have implemented several mechanisms to monitor these projects. However, AOC and the Case management systems track case information, Commission could improve project monitoring by including parties to the case, court documents, developing formal agreements with courts on the scope fines, fees, restitution, and court dates. Two case and deliverables of a proposed project and by adequately management systems are currently being assessing and managing the identified risks of these developed for potential use in the courts state-wide. projects.



Individual courts developing state-wide projects--AOC



has begun partnering with individual courts to develop information technology applications for state-wide implementation. One type of system where AOC has been working more closely with individual courts is in the development of new case management systems. Based in part on the potential costs of having a vendor supply a case management system to the courts, the Commission decided in June 2004 to support the development of case management systems in Pima County Superior



Office of the Auditor General

page



17



Court and Tempe Municipal Court and then evaluate them for use in other courts throughout the State.



Several mechanisms in place to monitor projects--AOC



and the Commission have instituted or used several oversight mechanisms to monitor the progress of IT systems developed outside of AOC, including: J Judicial Project Investment Justification--In September 2004, the Commission adopted a Judicial Project Investment Justification (JPIJ) form, similar to the Project Investment Justification form used by the State's Government Information Technology Agency. While the Commission must approve all IT projects requesting JCEF monies, IT systems costing more than $250,000 must complete and submit a JPIJ form to the Commission for its review and approval prior to initiating system development. The JPIJ form requires courts to include information on the type of system or application being requested, the need for the system, benefits of the system, its projected cost, and how it will be funded. AOC's strategic planning manager reviews the JPIJ, works with the court to clarify areas where he believes the Commission may have questions, and makes a recommendation to the Commission. The Commission then reviews and approves or denies the project. As of March 2006, the Commission had received five JPIJ applications, four of which had been approved, and one that was pending review. F Funding milestone plan--According to AOC's CIO, all projects that receive state funding must complete a funding milestone plan that AOC must then approve. This plan specifies when various project parts or phases will be completed and the funding that will be needed upon the phase's completion in order to start work on the next phase. When a phase is complete, AOC and the Commission review the work done during the phase before releasing additional funding. For example, the Pima County Case Management System has a plan that requires Pima County to complete, in sequence, a criminal module phase, a civil and arbitration module phase, and a probate module phase. Each phase needs to be completed before receiving funding for the next phase. M Monthly status reports--Both case management projects provide project status reports to the Court Automation Coordinating Committee, an advisory committee to the Commission, nearly every month. These reports include information on scheduled and unscheduled accomplishments, items that were not completed as scheduled, items scheduled for completion during the next reporting period, and any project issues or risks. These reports allow various stakeholders to monitor the progress and status of the projects. P Project demonstrations--Project demonstrations are regularly held for other court personnel to keep stakeholders apprised of these projects' progress and their functionality. For example, Pima County has held numerous demonstrations of its case management system to demonstrate various



State of Arizona

page



18



functions to other county clerks of the court and court administrators throughout the State. In addition to these oversight functions, AOC personnel are assigned to both the Pima County and Tempe projects. According to AOC's chief system architect, he reviews the code written for these projects to help ensure that the projects are using appropriate coding tools and adhering to judicial enterprise architecture standards. AOC also has analysts and programmers assigned to assist with both the Pima County and Tempe projects, and a person assigned to both projects to help with testing and quality control.



AOC reviews the courts' projects to ensure they follow enterprise architecture standards.



Further improvements to oversight could help--While both AOC and the

Commission receive information from a variety of sources relating to the projects being developed by individual courts, additional steps could help to provide more information about projects and help mitigate some of the risks involved in developing information technology systems. Specifically: A Agreements should be formalized--AOC should enter into formal agreements when partnering with individual courts for IT systems development. While AOC has a formal agreement with the City of Tempe regarding the development of its case management system, a similar agreement does not exist with Pima County. AOC officials indicated that they prefer not to enter into formal agreements because the courts all exist within the same court system and only entered into the Tempe agreement at the insistence of the City. However, the Tempe agreement specifies project deliverables for which the Tempe Municipal Court and AOC are each responsible, project staffing and responsibilities, and AOC's project responsibilities. This type of an agreement can help to ensure that all parties involved in the development of an IT system are fully aware of project requirements, roles and responsibilities, and expectations. Additionally, COBIT and North Carolina's framework recommend developing a project scope or charter. North Carolina's framework recommends developing a project charter to authorize the project that clearly details for all parties the business or program need, the project's scope, intended accomplishments, summary work plan and deadlines, and the commitment of project resources. AOC has recently begun relying more on partnering with the courts to develop state-wide systems. While auditors did not identify any problems resulting from the lack of an agreement for the Pima County Case Management System, formal agreements would reduce potential uncertainty regarding project deliverables and the roles and responsibilities of the different parties involved. Therefore, when AOC partners with a court to develop an IT system for state-wide use, it should enter into agreements, such as a project charter, that define the project scope, intended accomplishments, project processes and deadlines, and the commitment of project resources.



Project agreements detail roles and responsibilities.



Office of the Auditor General

page



19



R Risk assessments should be completed--AOC should also improve its oversight of individual court-developed IT systems by assessing and managing the risks associated with developing these systems. Both COBIT and the North Carolina Framework suggest that risk assessment and management is needed to eliminate or minimize the risks associated with a project. However, AOC does not have a process for assessing the risks for IT systems developed by individual courts. While the JPIJ requires that the court developing the application identify potential risks, AOC's risks in the project differ from the court's risks as AOC will likely be responsible for the state-wide implementation of a system after its development. For example, to facilitate the implementation of the Pima County Case Management System, each county superior court will need to standardize its business processes. According to AOC, each superior court may have different business processes, including processes for accepting filings and recording Arizona Judicial Council the payment of fines and fees. However, AOC has not formally assessed the risk to the project if the courts do not standardize their processes and identified how it might manage this risk if it occurs. The Arizona Judicial Council is a body of AOC's IT management believes that the risk rests with the courts and judges, court administrators, attorneys, and that the risk to the State is minimal, in part because both the public members that assists the Supreme Commission and the Arizona Judicial Council passed resolutions Court and Chief Justice in the development of calling for business process standardization. However, a formal risk policies and procedures for the administration assessment would also identify additional steps that might be taken of the courts. to manage the risk in the event that courts do not standardize their business processes.



State of Arizona

page



20



Recommendations:

1. AOC should improve its project management framework for internally developed information technology systems by: a. Continuing with its plans to develop and implement a formal risk management process that requires the identification of potential project risks and actions to mitigate these risks; Establishing and implementing policies and procedures within its framework that require each project management team to develop a communication plan that specifies the various stakeholders to the project and most effective form of communicating with these stakeholders; and Establishing and implementing policies and procedures for conducting independent reviews of technology projects in order to monitor and assess project performance and status.



b.



c.



2.



AOC should develop and implement policies and procedures regarding the use of the project management framework, including requiring all information technology projects to use the framework and the circumstances under which projects will not be required to implement the full framework. AOC and the Commission should improve oversight of state-wide information technology systems that individual courts are developing by: a. Entering into formal agreements with the courts that define the project scope, intended accomplishments, project processes and deadlines, and the commitment of project resources; and Implementing a formal risk assessment and management process that requires the identification of potential project risks and actions to manage these risks should they occur.



3.



b.



Office of the Auditor General

page



21



State of Arizona

page



22



FINDING 2

Additional collections contract oversight and program monitoring needed

AOC has taken several steps to help Arizona courts improve the collection of court fines and fees, but additional contract oversight and program monitoring could further improve collection efforts. One of these steps was the creation of the Fines, Fees, and Restitution Enforcement (FARE) program, a voluntary collections program that consists of several automated collections services provided by a third-party vendor. While AOC has focused on developing and implementing the FARE program, AOC should shift its focus to improving basic oversight of the collections contract and vendor. Additionally, AOC should institute performance measures to help assess FARE's performance.



AOC provides collection assistance to the courts

To assist the courts in collecting fines, fees, and penalties, AOC implemented FARE. As of July 2006, the outstanding debt owed to courts participating in FARE for both delinquent and nondelinquent fines, fees, and penalties was approximately $382 million. FARE is voluntary and includes services such as notice-serving, skip-tracing, payments by Internet or interactive voice response (IVR) in both English and Spanish, and referrals to intercept state tax refunds and lottery winnings, as well as providing motor vehicle registration holds (see textbox on page 24). Special collections services are also offered for fines and fees that are more than 55 days past due. Special collections include each of the previously mentioned services, as well as outbound collection calling, wage garnishment, and credit bureau reporting. In June 2003, AOC entered into a contract with a private vendor to provide these collections services to the courts. According to AOC, between August 2003 and July 2006, AOC collected nearly $60.6 million through FARE. FARE services begin from the date the vendor receives the case information. For example, according to the contract, when a case is received, the vendor provides a courtesy notice informing the defendant of the fine or fee amount, whether they are



The FARE program provides numerous collections services.



Office of the Auditor General

page



23



FARE Program Collections Services

Notice-serving--The vendor creates, sends, and monitors several collections notices to the defendant. Skip-tracing--The use of personal information obtained from private businesses or governmental sources to identify a current address when a notice is returned with an incorrect address. State tax and lottery interceptions--The courts can, through the appropriate agencies, intercept tax refunds and lottery winnings to cover up to the total amount of debt owed the courts. Vehicle registration holds--Holds placed against a defendant's motor vehicle registration record(s) that prevent the defendant from renewing the registration until all court debt is paid in full. Internet or IVR payments--The vendor provides these payment options to facilitate the payment of court debts. Wage garnishment--If a case is at least 55 days past due and a judge allows this option, the vendor can contact a defendant's employer to intercept a portion of his/her wages until the court debt is paid. Outbound collections calls--Provides the vendor with another method to contact the defendant in an effort to collect debt. Calls are automatically dialed and turned over to the vendor's trained collection staff. Credit bureau reporting--If the defendant does not pay his/her fines, the vendor can report this outstanding court debt to the credit bureaus. eligible for defensive driving school if it is a civil traffic violation, and payment options. If the defendant fails to pay the fines or appear in court, the vendor sends two additional notices encouraging compliance. As shown in Table 4 (see page 25), the vendor charges AOC a fee for each item submitted based on the service and the volume of transactions. To cover these fees, administrative order No. 2003-126 requires AOC to add an additional $7 general service fee to the fine. When fines are not collected within 55 days after the due date, the court can decide whether to send the case to FARE for special collections services. AOC does not require courts to send cases to special collections. Rather, each court decides on its own. If a court sends a case to special collections, a 16 percent special collections fee is added to the amount of the fine, which is retained by the vendor upon collection. Additionally, a 3 percent special collections fee is added to the amount of the fine, which AOC retains upon collection. These special collection fees are used to offset operational costs incurred by FARE.



Administrative Order--An order issued by the chief justice of the Supreme Court that institutes policies and procedures for the entire judiciary branch, including all courts, to use in conducting administrative functions.



State of Arizona

page



24



According to AOC management, AOC's goal is to have all Arizona courts enrolled in FARE. As of March 2006, there are 187 courts in the State, and as of July 2006, 58 of these courts either partially or fully participated in FARE. Specifically:



Table 4:



Vendor Charges for FARE Services

By Number of Items per Contract Year

Cost per Service Internet or Telephone Payment Processing2 Motor Vehicle Registration Holds3



Number of Items Notice C Courts that partially participate in FARE--As of Submitted Each Year Serving1 July 2006, 56 of the 142 courts that use the < 250,000 $1.33 $1.29 $0.52 state-wide case management system partially 250,001--500,000 1.12 1.19 0.41 participated in FARE through an interim option. > 500,000 0.77 1.08 0.26 According to AOC management, the state-wide case management system is built on older 1 Fees for notice-serving are assessed on a per-case basis. technology that would require extensive 2 modifications to incorporate processes that are Fees for Internet or telephone payment processing are assessed on a pertransaction basis. required for full FARE.1 While AOC is in the 3 process of replacing this system, as an interim Fees for motor vehicle registration holds are assessed when a hold is placed on a registration. option, AOC compiles case information from courts using the state-wide system into a single Source: Auditor General staff analysis of information in the AOC/vendor FARE collection- services contract. database and transfers to the vendor information on those cases that courts designate for collection. Courts that participate in the interim option can receive special collections services for delinquent cases that are at least 55 days past due, but because of limitations of their case management system, cannot obtain services for nondelinquent cases, such as the initial courtesy notice the vendor sends to a defendant.



C Courts that fully participate in FARE--As of July 2006, two courts, the Phoenix and Chandler Municipal Courts, fully participated in FARE. While 45 courts have their own case management systems, only the Phoenix and Chandler Municipal Courts can access services provided through the FARE program, which provides services for nondelinquent as well as delinquent cases. According to AOC management, in implementing the full FARE program in Chandler Municipal Court, it is implementing technology that should allow other courts that do not use the state-wide case management system to more easily participate fully in FARE. Whether or not a court participates in either FARE option, it can obtain other services from the vendor. These services may include data entry, handheld data collection and citation issuance hardware for local law enforcement, lockbox processing of mail-in payments, call center services, digital imaging, electronic report distribution, and ad hoc reporting. As of November 2005, 26 courts used the vendor for at least one of these other services without participating in the FARE program.



1



According to the director of AOC's Information Technology Division, the new municipal court case management system will be compatible with FARE when AOC receives it from Tempe Municipal Court, but AOC staff will need to modify the new superior court case management system before it will be compatible with FARE. However, AOC expects both systems to ultimately be compatible with FARE technology.



Office of the Auditor General

page



25



AOC can take additional steps to improve vendor oversight

While AOC has begun to conduct some vendor oversight activities, it should improve its oversight of the FARE program vendor. AOC has implemented a vendor contract that includes essential contract elements, a performance log that tracks the contract requirements, and data reconciliation processes. However, AOC can take additional steps to improve vendor oversight, including addressing gaps that exist in the contract and establishing additional oversight procedures.



AOC has implemented some oversight mechanisms--In implementing

the FARE program, AOC has taken several steps important for contract oversight. Specifically: C Contract includes key elements--AOC's contract with the FARE program vendor includes several important elements. These elements include the services to be provided, the responsibilities of each party involved, specific contract requirements for the quality and quantity of these services, and penalties if the vendor fails to meet the contractual requirements. For example, the contract stipulates that the vendor shall provide evidence of their systems' security, including physical location security and local area network security. The contract requirements indicate that the vendor is to periodically audit the access logs and notify AOC of any unusual occurrences, and passwords must be changed at least every 60 days or 2 calendar months. Penalties for productivity loss due to security breaches or damages to data may result in forfeited fees. Further, continued lapses in security controls may result in cancellation of the contract. P Performance log assists in tracking compliance with contract requirements-- In the fall of 2005, AOC staff created a performance log to assist in monitoring the vendor's compliance with the contract requirements. This log describes the 84 requirements that are in the contract, and allows AOC staff to note the vendor's status in meeting these requirements, including whether the vendor has only partially or has not addressed the requirement. As of March 2006, the log noted that 40 requirements had been met, while 27 contract requirements had been partially addressed, and 5 had not yet been met. In addition, the log notes that 19 contract requirements require contract changes for various reasons. For example, the contract requires the vendor to provide AOC a quarterly summary report of disputes. However, the courts, not the vendor, handle disputes, and the log notes that a contract change is needed. According to AOC management, this performance log represents AOC's only tool for contract and vendor oversight. D Data review processes well established--AOC has established several processes for ensuring that FARE program data is complete and reliable. A



The FARE vendor contract includes important security requirements.



State of Arizona

page



26



high volume of data is exchanged between the courts, AOC, and the FARE vendor. Each case entered into FARE for collections contains multiple pieces of information, such as the defendant's name, address, and phone number, as well as the case number, citation charge(s), penalty fee(s), case balance, and case status. Auditors observed the data review processes that AOC performs to help ensure that this data is accurate. Data is reviewed as it is compiled for the interim option of FARE or on a daily basis for data received from the one court fully enrolled in FARE. Specifically, AOC's system checks the data electronically for accuracy and generates reports of any discrepancies. Staff from AOC, the courts, or the vendor then manually review and reconcile any discrepancies, and resubmit the corrected data to the vendor. Additionally, AOC and vendor staff reconcile FARE case data in their respective databases on a weekly basis to ensure that both databases include the same data. Lastly, staff from the courts, AOC, and the vendor meet weekly to discuss any system errors that may be occurring.



A high volume of data is exchanged among AOC, the courts, and the FARE vendor.



Gaps exist in the contract--While the contract includes key elements, gaps

within the contract could potentially affect AOC's ability to enforce some of its provisions. Specifically: S Some contract requirements lack penalties--The contract does not include penalties for noncompliance for 48 of the 84 contract requirements. For example, although the contract includes 8 requirements related to the backlog for special collections services, only 1 requirement has an associated penalty. The Federal Acquisition Regulation System (FARS), which governs federal government procurement, recommends that to the maximum extent practicable, federal agencies consider using both positive and negative incentives in connection with service contracts when the quality of performance is critical and incentives are likely to motivate the contractor.1 According to an AOC official, AOC determined that certain activities were more important than others and included penalty provisions for the areas where AOC had to have strict compliance and where the vendor agreed to accept them. Although AOC's approach appears to be consistent with FARS' recommendation, AOC could not provide any documentation supporting how it determined which contract requirements required noncompliance penalties. As a result, some contract requirements that could potentially benefit from an attached penalty may still lack one. Therefore, while AOC may not need penalties for each contract requirement, it should review the contract and determine if any other contract requirements need penalties, document these determinations, and add penalties as necessary. G General assurance review not required, but the vendor has obtained one-- Auditors found that the contract does not require a general assurance review or similar type of audit be performed; however, the vendor has obtained one. These general assurance reviews should be performed by an independent auditor to reasonably ensure that a vendor's business operations incorporate

1 48 C.F.R., 16.402-2.



Office of the Auditor General

page



27



sufficient internal controls and security to properly safeguard a client's data, monies, and other assets. According to AOC officials, they were unaware of the importance of including this type of audit as a contract requirement for the vendor. However, at auditors' request, AOC determined that the vendor had obtained such an audit and the vendor provided AOC with a copy. To help ensure that a general assurance review or audit is conducted at least annually for the duration of the contract and that the audit addresses areas of specific interest to the vendor's work performed for AOC, AOC should revise the contract to include a requirement for a general assurance audit, including specific areas it would like addressed. Additionally, AOC should request a copy of the audit annually and review it to ensure that the vendor's internal controls adequately safeguard AOC's information and other assets.



Additional contract monitoring needed--In



AOC should determine how frequently to monitor the 84 contract requirements.



addition to addressing the identified gaps in the contract, AOC should improve its oversight of the vendor's performance relative to the contract. The contract is complex to administer, with 84 contract requirements that need monitoring. While the performance log can be a useful tool for tracking the status of the vendor's compliance with the contract, AOC needs to establish processes to guide its monitoring of the vendor and contract. Specifically, AOC has not determined how frequently it will monitor the vendor's compliance with the contract's requirements or how it will verify vendor reports of compliance with these requirements. For example, the contract requires that the vendor skip-trace notices that are returned as undeliverable. The log indicates that this requirement has been met, but it does not include any information regarding how AOC staff made this determination or if staff verified information or documentation provided by the vendor demonstrating compliance. Therefore, AOC should establish and implement processes for monitoring the vendor's compliance with the contract, including specifying how frequently contract requirements will be monitored for compliance and the procedures AOC staff should perform to verify vendor reports of compliance with the contract requirements.



Performance measures needed to assess program effectiveness

In 2002, AOC identified a need to improve the collection of court-ordered fines, fees, and restitution. At that time, they set four goals to increase court collections. These included requiring all courts to participate in the State Tax Intercept Program, obtaining the ability to suspend motor vehicle registrations, implementing a centralized collections payment processing center, and pursuing a federal tax intercept program. AOC expected that it could increase collections by at least $51 million by implementing these goals.



State of Arizona

page



28



Although the FARE program provides for tax interception, vehicle registrations suspension, and a centralized collections payment processing center, AOC has not taken steps to develop performance measures specifically for FARE despite the availability of the necessary program data. While AOC officials have expressed an understanding of the need for performance measures, AOC management indicated that program resources have focused primarily upon implementing the program and encouraging the courts to participate rather than on developing performance measures.



NCSC-recommended Information to develop performance measures:

Case Date Due



number of the order of sentence



date for final payment of the total monetary penalty

Total



monetary penalty in the case



Amount of total monetary penalty received The National Center for State Courts (NCSC) recommends to date several measures for court collections programs and has Total amount of restitution ordered identified eight pieces of information that courts would need to record to support the suggested performance measures (see Amount received that is applied to textbox). Auditors reviewed information from AOC's data restitution warehouse and found that AOC collects from the courts Amount of restitution disbursed to victims participating in the FARE program seven of the eight pieces of information needed to develop and support collections Source: National Center for State Courts. CourTools: Trial Court performance measures. AOC does not collect data regarding Performance Measures. Williamsburg, VA: National Center for State Courts, 2005. restitution disbursed to victims, but collects information regarding the restitution amount collected. As a result, AOC can establish several of the NCSC-recommended performance measures and begin to assess and measure the effectiveness of the FARE collections program. Measures suggested by NCSC include:



O Overall monetary penalties collected--The overall monetary penalties collected is the total penalty dollar amount collected by the courts in the program, and includes both the actual dollars collected and the dollar value of community service or jail served in place of payment. P Preliminary compliance rate--The preliminary compliance rate is the actual dollar amount collected divided by the total amount of debt assigned during a specified time frame. This percentage rate allows management to review the amount of fines being assessed and determine how well the courts are collecting on those amounts. O Overall compliance rate--The overall compliance rate is the overall monetary penalties collected, including restitution, divided by the total amount ordered. This measure allows management to verify how much restitution (both money and restitution paid by community service or jail) is being collected and how well courts are doing overall in complying with financial court orders. R Restitution collection rate--The restitution collection rate is determined by dividing the amount of restitution collected by the amount of restitution ordered.



Office of the Auditor General

page



29



This rate allows management to ascertain how well the courts are doing in the collection of outstanding restitution amounts. While the Supreme Court's 2005-2010 strategic agenda recommends that individual courts implement NCSC measurements as they apply to the individual court's needs, AOC can use these measures as a guide in developing performance measures for the FARE collections program. According to NCSC, once established, if the results of these measures are validated for accuracy and reviewed regularly, management could begin to establish baselines, set performance goals, and observe trends as they develop. Additionally, AOC could use this information to report to courts on FARE's effectiveness. Therefore, AOC should develop and implement performance measures for FARE, and establish and implement policies and procedures to collect and validate the data needed to support the performance measures it develops. This would be particularly important if it establishes performance measures for which it currently does not collect data.



Recommendations:

1. AOC should address gaps that exist in the FARE program vendor contract by: a. Reviewing the contract and determining if any additional contract requirements need penalties, documenting these determinations, and adding penalties as necessary; and Revising the contract to include a requirement for an annual general assurance audit, including identifying specific areas it would like addressed. Additionally, AOC should request a copy of the audit annually and review it to ensure that the vendor's internal controls adequately safeguard AOC's information and other assets.



b.



2.



AOC should improve its oversight of the FARE program vendor and contract by establishing and implementing processes for monitoring the vendor's compliance with the contract, including specifying: a. How frequently contract requirements will be monitored for compliance; and Procedures that AOC staff should perform to verify vendor reports of compliance with the contract requirements.



b.



3.



AOC should develop and implement performance measures for FARE, and establish and implement policies and procedures to collect and validate the data needed to support the performance measures that it develops.



State of Arizona

page



30



OTHER PERTINENT INFORMATION

During this audit and in response to a legislative inquiry regarding the Supreme Court's technology and prioritization of information technology projects, auditors collected other pertinent information related to the use of monies in the Judicial Collection Enhancement Fund (JCEF) and the Defensive Driving School Fund (DDSF) and the security of the Supreme Court's IT resources.



Available money for technology expenses decreasing

While the Supreme Court has been able to fund its information technology needs and some operational costs from monies in the JCEF and DDSF Funds, it projects that its needs will begin to exceed the available monies in these two funds in fiscal year 2008. Monies in the JCEF and DDSF Funds pay for most of the Supreme Court's IT needs, including maintaining current systems and developing new systems. However, JCEF and DDSF expenditures increased by approximately 19.3 percent from fiscal year 2000 to fiscal year 2005, and AOC projects that they will continue to increase, potentially exhausting all available monies in these two Funds by the end of fiscal year 2008. The Supreme Court has taken steps, such as authorizing funding for only the most important IT projects and requesting that the Legislature either increase its General Fund appropriation to its pre-2004 level or allow the Supreme Court to increase civil filing fees and deposit the increased fees into JCEF to help ensure that monies are available for technology needs.



Fund definitions

Judicial Collection Enhancement Fund-- Consists of monies paid by individuals for surcharges and fines. Uses include training court personnel; enhancing collections; management of monies assessed or received by the courts, including restitution and child support; improving court automation, case processing, or the administration of justice; and for probation services. Courts wishing to receive monies from JCEF must submit a plan to the Supreme Court. Defensive Driving School Fund--Consists of a fee paid by a person attending a defensive driving school. Use is restricted to supervising the use of defensive driving schools and to all traffic and driving under the influence cases.



Two funds provide most technology funding--As



Source: A.R.S. 12-113 and 28-3398. authorized by A.R.S. 12-113 and 28-3398, monies in the JCEF and DDSF Funds can be used to develop and maintain technology for the Supreme Court and other Arizona courts. As illustrated in Table 5 (see page 32), both of these Funds' revenues exceeded expenditures for fiscal year 2005. Combined revenues for fiscal year 2005 totaled over $13.1 million, while expenditures totaled nearly $12.4 million.1 This resulted in a combined balance of more than $8.6 million for both Funds at the end of fiscal year 2005. 1 These totals do not reflect the collection, distribution, or fund balance of county probation fees. While these fees are deposited into JCEF, they are distributed to the superior court adult and juvenile probation departments. Office of the Auditor General

page



31



Table 5:



Judicial Collection and Enhancement Fund and Defensive Driving School Fund1 Schedule of Revenues, Expenditures, and Changes in Fund Balance Fiscal Years 2004 through 2006 (Unaudited)

2004 2005 $ 8,217,366 4,547,212 217,354 150,637 13,132,569 6,137,014 531,851 71,410 1,607,011 5,439,065 56,408 (1,446,863) 12,395,896 736,673 7,877,194 $ 8,613,867 2006 $ 8,042,386 4,647,459 422,129 68,981 13,180,955 6,916,377 397,207 81,002 1,727,937 5,866,872 47,554 (1,593,107) 13,443,842 (262,887) 8,613,867 $ 8,350,980



Revenues: Charges for services Fines, forfeits, and penalties Interest earnings Other Total revenues Expenditures:2 Personal services and employee-related Professional and outside services Travel Aid to counties Other operating Equipment ACAP fees3 Total expenditures Excess of revenues over (under) expenditures Fund balance, beginning of year Fund balance, end of year4



$ 8,357,782 4,380,471 139,048 16,541 12,893,842 5,921,990 709,209 55,030 1,329,556 5,318,462 328,260 (989,559) 12,672,948 220,894 7,656,300 $ 7,877,194



1



2 3



4



Excludes county probation fees collected by the Supreme Court (Court) and the distribution of those fees to the counties. While these fees are deposited into the Judicial Collection and Enhancement Fund, they are distributed to the counties and are not available to pay for the Court's operating costs, and therefore are not reported in the schedule. Administrative adjustments are included in the fiscal year paid. Arizona Court Automation Project (ACAP) fees are collected from courts that receive computer equipment and services, and access to court systems and the judiciary network. The fees collected are considered reimbursements of the Court's costs and therefore reduce expenditures. AOC reports that amounts accumulated in the fund balance will be used for the development and acquisition of new automated case and cash management systems for the courts. Auditor General staff analysis of the Arizona Financial Information System (AFIS) Accounting Event Transaction File and Trial Balance by Fund report, and financial information provided by AOC for fiscal years 2004 through 2006.



Source:



Funds used for technology and operating expenses--As appropriated

by the Legislature, monies from the JCEF and DDSF Funds are used to fund a variety of technology and operating expenses. These include the following:

The ACAP program cost approximately $2.6 million in fiscal year 2005.



A Arizona Court Automation Project (ACAP)--According to AOC, in fiscal year 2005, the Supreme Court spent approximately $2.6 million in JCEF and DDSF



State of Arizona

page



32



monies to provide ACAP services to courts throughout the State. ACAP is a fee-based program in which individual courts receive computer equipment, software, maintenance and support, and access to several court systems and to the Supreme Court's state-wide network, AJIN. According to an AOC official, the ACAP program was established in 1993 to facilitate access to various court systems, including a state-wide case management system. According to administrative order No. 2001-8, courts must participate in the program or obtain permission from the Commission on Technology not to participate. One hundred forty-two of the 187 courts state-wide participate in the program.1 Courts are charged a fee for each device that is linked to AJIN, and the Commission establishes these fees for the ACAP program. In 2002, the Commission approved a fee increase effective in fiscal year 2005 and another fee increase to be ACAP Fees as of October 5, 20051 effective in fiscal year 2009, based on Fiscal Year recommendations made by a funding Current 2009 workgroup that the Commission established. Initial fee: According to an AOC official, the fees charged Desktop computer/laptop: Fully configured $ 260 $ 260 for program services represent a negotiated Limited access 200 200 amount rather than an amount intended to Configured for imaging (desktop only) 580 580 recover the program's costs or a portion of these Annual fees per device costs. These fees are charged to participating Desktop computer 750 1,000 courts for each device connected to AJIN and Printer 750 1,000 pay for computer equipment and other services Laptop computer 1,250 1,500 included within the ACAP program. For example, based on the current fees, to get a desktop 1 Fees are charged per type of device and include equipment, software, computer, and printer, and receive ACAP maintenance, support, and access to court systems. services, a court would pay $1,500 annually. Source: ACAP Web site located on the judicial state-wide network. Beginning in fiscal year 2009, the total fees for the same equipment and services will increase to $2,000 annually. While the ACAP program cost the Supreme Court $2.6 million, program expenses actually totaled over $4 million. AOC has not specifically tracked the program's expenditures, but at the request of auditors, calculated its fiscal year 2005 ACAP program expenses and determined that they totaled over $4 million. However, AOC was able to offset a portion of these costs by collecting over $1.4 million in ACAP fees from participating courts. AOC does not account for the fees as revenue, but instead considers them as reimbursements for ACAP program expenses. This accounting approach, in effect, reduced the cost of the ACAP program to approximately $2.6 million for fiscal year 2005, which represents approximately 21 percent of the fiscal year 2005 JCEF and DDSF expenses.



1



Some of the courts that do not participate in this program include the Superior Court in Maricopa and Pima Counties, Phoenix Municipal Court, Paradise Valley Municipal Court, Prescott Justice Court, Gilbert Municipal, and Maricopa Justice Court. Auditors interviewed four court officials of courts that do not participate in ACAP and each of these court officials indicated that they chose not to participate because they had their own case management systems, and therefore did not need access to the state-wide case management system.



Office of the Auditor General

page



33



D Development and maintenance of various IT systems--In addition to ACAP program costs, in fiscal year 2005, the Supreme Court spent approximately $4.86 million of JCEF and DDSF monies to maintain and support IT systems, assist in the development and implementation of two new case management systems, and provide IT training and support to Supreme Court and court personnel. The Supreme Court uses monies from the JCEF and DDSF Funds to develop and maintain a variety of IT systems. These include a state-wide case management system, network, and adult probation system that are needed for the successful operation of the court system. Additionally, the Supreme Court is assisting with the development and implementation of two new case management systems, the Pima County Case Management System, which it plans to implement in each county's superior court, and the Tempe Case Management System, which it plans to implement in municipal and justice courts state-wide (see Finding 1, pages 11 through 21, for more information regarding the development and implementation of these systems). In fiscal year 2005, the Supreme Court spent approximately $397,000 for the development of these two systems. S Supreme Court operations--In fiscal year 2005, the Supreme Court used a total of $4.93 million in JCEF and DDSF monies to fund some of its operational costs. These operational costs include rent, training, and the costs for conducting operational reviews of the courts to ensure compliance with mandated operating requirements and standards. The Supreme Court has increased the amount of JCEF monies it uses to pay for some of its operational costs. Beginning in fiscal year 2004, the Legislature reduced the Supreme Court's General Fund appropriation by $2 million, which it typically used for operations, and increased its JCEF spending authority by $2 million annually. In fiscal year 2005, the Supreme Court elected to use the additional $2 million in spending authority from JCEF to help cover its operational costs and specifically used these monies to pay a portion of its rent payment for the Supreme Court buildings in Phoenix and Tucson. The Supreme Court also used additional JCEF and DDSF monies to fund other operational costs. For example, in fiscal year 2005, the Supreme Court spent approximately $459,000 of JCEF and DDSF monies for operational reviews.



Demand may deplete fund balances--Based on its current and continuing

IT needs, AOC projected that it would exhaust available monies in both the JCEF and DDSF Funds sometime in fiscal year 2008. Even though JCEF and DDSF revenues have exceeded expenditures in fiscal years 2004 and 2005, expenditures for these two funds increased by approximately 19.3 percent from fiscal year 2000 to 2005, while revenues only increased by approximately 8.2 percent over this period (see Figure 1, page 35). AOC projected that expenditures would continue to increase. Based on projections it prepared in May 2005, AOC estimated its fiscal year 2006 JCEF and DDSF expenditures at approximately $13.63 million, with this amount growing to a projected $15.7 million in fiscal year 2009. By comparison, AOC projected that JCEF and DDSF revenues would total approximately $12.86



State of Arizona

page



34



million in fiscal year 2006 and only grow to $14.4 million in fiscal year 2009.1 AOC attributes the projected growth in expenditures to the following: I Inflation--According to an AOC official, AOC used a rate of 5 percent to project the growth in JCEF and DDSF expenditures for fiscal years 2006 through 2009. According to this same official, AOC considered inflation, retirement increases, and potential healthcare and salary increases to arrive at the rate of 5 percent, while projections of less than 5 percent were based on budget changes. A A renewed vendor contract--In fiscal year 2006, AOC renewed a vendor contract for the licensing and maintenance of certain programs and the performance of certain associated services. This contract requires AOC to pay a total of approximately $1.92 million to the vendor from December 31, 2005 through July 1, 2008.



Figure 1:



Judicial Collection Enhancement and Defensive Driving School Funds' Revenues and Expenditures Fiscal Years 2000 through 2005 (Unaudited)



$14 $12

Revenues and Expenditures (In Millions of Dollars)



$10



$8 $6 $4 $2 2000 2001 2002 2003 2004 2005



Fis cal Year



Rev enue



Expenses



Source:



Auditor General staff analysis of the Arizona Financial Information Systems' Revenues and Expenditures by Fund, Program, Organization, and Object reports for fiscal years 2000 through 2005.



In addition to these projected expenditures, if AOC implements the Pima County and Tempe case management systems state-wide, it expected to spend only approximately $158,000 in JCEF and DDSF funding in fiscal year 2006 to begin the implementation. During fiscal years 2007 through 2009, AOC expects implementation costs to be between $1.56 million and $2.6 million annually. Without sufficient monies in JCEF and DDSF, the Supreme Court may have to delay implementing the new case management systems in order to continue covering its operating and other technology expenses. If the Supreme Court has to delay implementing the new case management systems, 142 superior, municipal, and justice courts that use the current state-wide case management system will have to continue relying on an obsolete system to process cases. As a result, the courts may be not able to process cases properly or effectively.



Supreme Court taking steps to address deficiencies--The



Supreme Court has taken steps to help address the projected deficiencies in the JCEF and DDSF Funds. First, as part of its strategic planning process for information technology, the Commission prioritizes projects by ranking them by level of importance and project duration. Projects are then approved based on this



1



According to AOC officials, AOC plans to annually revise its financial projections to reflect actual financial activity. For example, while AOC projected fiscal year 2006 JCEF and DDSF expenditures to be approximately $13.63 million, actual expenditures were approximately $13.44 million, representing a difference of approximately $190,000. Office of the Auditor General

page



35



prioritization and the availability of monies in these funds. In addition, the Legislature increased the Supreme Court's General Fund appropriation by approximately $2.8 million for fiscal year 2007, which will allow the Supreme Court to continue meeting its information technology needs by reducing the amount of operating expenditures paid from the JCEF and DDSF Funds.



AOC has taken steps to secure IT resources

As part of its duties, the Supreme Court provides technology services for the judicial branch. Auditors reviewed the Supreme Court's technology security and found that the Supreme Court has taken steps to secure its IT resources, including controlling physical access to its IT facilities, installing network security devices and software, controlling AJIN connections between the Supreme Court and courts state-wide, and controlling user access to and security of its systems and applications. Specifically, the Supreme Court uses electronic card keys to restrict access to its IT facilities and maintains a log of all visitors who enter its computer room. In addition, the Supreme Court has installed firewalls and other devices designed to prevent or detect improper access to IT resources, and software programs such as anti-virus protection, to help prevent the compromise of IT resources. The Supreme Court also takes reasonable steps in establishing and maintaining secure connections between the Supreme Court and other Arizona courts by using private communication lines, encrypting communications, and preventing connections that are not authorized by the Supreme Court. Further, the Supreme Court conducts regular reviews to help ensure that users can access only those IT resources necessary for their authorized responsibilities, requires completion of security awareness and authorization forms, has a process to update programs on users' computers to mitigate security risks, and makes available security-related information to users through the Support Center Web site and e-mail alerts. In addition, according to AOC management, the Supreme Court is taking additional steps that will further increase its IT security, including removing signs outside its central computer center and dedicating an employee to monitoring systems and network security.



State of Arizona

page



36



AGENCY RESPONSE



Office of the Auditor General



Performance Audit Division reports issued within the last 24 months

04-07 04-08 04-09 Department of Environmental Quality--Air Quality Division Department of Environmental Quality--Sunset Factors Arizona Department of Transportation, Motor Vehicle Division--State Revenue Collection Functions Arizona Department of Transportation, Motor Vehicle Division--Information Security and E-government Services Arizona Department of Transportation, Motor Vehicle Division--Sunset Factors Board of Examiners of Nursing Care Institution Administrators and Assisted Living Facility Managers Letter Report--Department of Health Services-- Ultrasound Reviews Department of Economic Security--Division of Employment and Rehabilitation Services-- Unemployment Insurance Program Department of Administration-- Financial Services Division Government Information Technology Agency (GITA) & Information Technology Authorization Committee (ITAC) Department of Economic Security--Information Security Department of Economic Security--Service Integration Initiative Department of Revenue--Audit Division 05-07 05-08 05-09 05-10 05-11 Department of Economic Security--Division of Developmental Disabilities Department of Economic Security--Sunset Factors Arizona State Retirement System Foster Care Review Board Department of Administration-- Information Services Division and Telecommunications Program Office Department of Administration-- Human Resources Division Department of Administration-- Sunset Factors Department of Revenue-- Collections Division Department of Revenue-- Business Reengineering/ Integrated Tax System Department of Revenue Sunset Factors Governor's Regulatory Review Council Arizona Health Care Cost Containment System-- Healthcare Group Program Pinal County Transportation Excise Tax Arizona Department of Eduation--Accountability Programs Arizona Department of Transportation--Aspects of Construction Management Arizona Department of Education--Administration and Allocation of Funds Arizona Department of Education--Information Management



04-10



04-11 04-12



05-12 05-13 05-14 05-15 05-16 06-01 06-02 06-03 06-04 06-05 06-06 06-07



05-L1 05-01



05-02 05-03



05-04 05-05 05-06



Future Performance Audit Division reports

Arizona Department of Health Services--Behavioral Health Services for Adults with Serious Mental Illness in Maricopa County