Empirical evaluation of the progressively increasing consequences act program

              Final Report Prepared for the
Arizona Supreme Court
Administrative Office of the Courts
Juvenile Justice Services Division
by
Don M. Gottfredson and Stephen D. Gottfredson
Justice Policy Research Corporation
4
Empirical Evaluation of the
Progressively Increasing Consequences Act Program
September, 1995
Stanley G. Feldman
Chtef Justice
STATE OF ARIZONA
ADMINISTRATIVE OFFICE OF THE COURTS
David K. Byers
Admmtstrative D~rector
of the Courts
In accordance with A. R.S. $ 8-230.02(E), an empirical evaluation of the Progressively Increasing
Consequences Act (PIC-Act) was completed in September 1995. The evaluation was conducted
by Don M. Gottfredson and Stephen D. Gottfredson of the Justice Policy Research Corporation.
The evaluation studied 24,677 youth referred to the juvenile courts between July 1993 and March
1994 and followed these youth for at least 13 months and, in some cases, up to 21 months.
The following is a general summary outlining the major points of the evaluation.
Findings
PIC-Act youth are more likely than other juveniles referred to be Anglo with an average age of
14 and enrolled in school (junior high school).
PIC-Act youth are typically referred by law enforcement agencies for theft or a peace offense
(e.g., disorderly conduct, trespassing) and have had no previous offenses.
PIC-Act youth are most likely to be assigned consequences consisting of community work service
or a delinquency education program.
PIC-Act consequences most frequently complied with are non-residential (mainly day and evening
support) treatment and drug and alcohol education programs.
PIC-Act consequences most frequently not complied with are restitution and general counseling.
There is a relationship between type of consequence assigned and subsequent reoffending.
Youth participating in drug and alcohol education and non-residential treatment are less
likely to be referred for a new offense.
1501 WEST WASHINGTON STREET PHOENIX. ARIZONA 85007-3327 6020542.9300 (TDD) 602-542-9545
Youth assigned restitution/monetary assessment only are more likely to be referred for a
new offense.
Compliance with consequences decreases the likelihood of new offenses.
Compliance with consequences decreases the seriousness classification of new offenses, if there
is a new offense.
For those completing PIC-Act, if a new offense is committed, it is more likely to be a status
offense.
The type of consequence assigned has no effect on the seriousness classification of new offenses,
if there is a new offense.
Some youth although not currently eligible for PIC-Act, such as first time felony or third
misdemeanor offenders who are not adjudicated, could possibly benefit from a diversion program.
Any questions regding this evaluation can be addressed by caUhg the
Juvenile Justice Sewices Division of the Adminbtrative Office of the Courts
. at (602) 542-9443.
Empirical Evaluation of the PIC-ACT
Empirical Evaluation of the
Progressively Increasing Consequences Act Program
Final Report Prepared for the
Arizona Supreme Court
Administrative Office of the Courts
Juvenile Justice Services Division
by
Don M. Gottfredson and Stephen D. Gottfredson
Justice Policy Research Corporation
Sacramento, California
September, 1995
The opinions expressed in this report are those of the authors and do not necessarily
reflect the views or endorsement of the Supreme Court of Arizona, the Administrative Office of
the Courts, the Juvenile Courts in the various Arizona Counties, or any other agency or person.
Acknowledgment
The data used in this study were extracted from the data file of the Juvenile Services
Division, Administrative Office of the Courts, by Steve Stilwell of Stilwell Software Products,
Tempe, Arizona. Appreciation is expressed to Donna Noriega, Program Manager, Bobbie
Chinsky, Program Manager, and Dr. Betsie McNulty, of the Juvenile Justice Services Division of
the Administrative Office of the Courts, for their valued advice.
Empirical Evaluation of the PIC-ACT iii
Empirical Evaluation of the Progressively Increasing Consequences Act
Contents
Page
Acknowledgment i i
Summary
Purpose
Methods
Samples Used for the Statistical Design
Samples Used for the Quasi-Experimental Design
Questions for this Study
Results
Recommendations 12
Introduction 19
Questions for this Study 23
Central Questions 24
Classification of Youths into PIC-ACT, Other Court Program,
and Not Eligible Samples 24
Meanings of the Group Classifications 28
Secondary Questions 28
Methods
Sample
Data Sources
Measurement
Dependent Variables
Empirical Evaluation of the PIC-ACT
Time at Risk
Independent Variables
Descriptive Variables and Other Data Availability
Analysis Plan
Results of the Statistical Design
Sample Differences in Youths' Characteristics and Counties
A Naive Answer to Question 1
The Need for Statistical Controls
A Priori Risk
Development of a Risk Measure for the Present Study
A Model for Selection for PIC-ACT, Other Court Program,
and Not Eligible Samples
Overview of Analyses with Statistical Controls
Effects of the Study Sample Classifications on
New Referrals
Effects of Consequences on New Referrals
Variation in Types of Consequences Used and
in Compliance
Effect of Compliance with Consequences on New Referrals
Effects of PIC-ACT, Other Court Program, and Not Eligible
Classifications on Seriousness of New Offenses
Effects of Consequences on Seriousness of New Offenses
Effects of Compliance with Consequences on Seriousness
of New Offenses
Results of the Quasi-Experimental Design
Empirical Evaluation of the PIC-ACT v
Selection of Groups for the Quasi-Experiment 72
Naive Comparisons of New Referrals and New
Offense Seriousness
Effects of PIC-ACT on New Referrals 75
Effects of PIC-ACT on Seriousness of New Alleged Offenses 77
Limitations 78
Conclusions 84
Recommendations 89
Appendices
A. Analysis of Covariance Summary Table, New Referral Outcome 94
B. Discriminant Function Summary, Classification for Consequences 95
C. Analysis of Covariance Summary Table, Type of Consequence,
for New Referral Outcome 96
D. Analysis of Covariance Summary Table, Compliance and
County, for New Referral Outcome 97
E. Analysis of Covariance Summary Table, Seriousness Criterion 98
F. Analysis of Covariance Summary Table, Type of Consequence,
for Seriousness Criterion (New Referrals Only) 99
G. Analysis of Covariance, Compliance and County, for Seriousness
Criterion (New Referrals Only) 1 00
H. Analysis of Covariance Summary Table, County and Quasi-
Experimental Groups, for New Referral Outcome 101
I. Analysis of Covariance Summary Table, County and Quasi-
Experimental Groups, for New Offense Seriousness Outcome 102
Empirical Evaluation of the PIC-ACT
Tables
1. Study Sample Analyzed by County
2. Regression of New Referral Criterion on Various Predictors
3. Standardized Discriminant Function Coefficients, Sample
Group Assignments
4. Standardized Discriminant Function Coefficients, Consequence
Assignments, Functions 1 and 2
5. Examples of Seriousness Offense Scoring
6. Numbers of Youths in Offense Seriousness Outcome
Categories (Most Serious Offense at Next Referral) for Total
Sample, by Three Study Samples
Figures
1. Classification of Youths into Samples for Study
2. Percent of Youths in PIC-ACT Sample, Other Court Program
Sample, and Not Eligible Sample
3. Percent of Youths Included for Study from Each Arizona County
4. Percent of County Sample Youths in the PIC-ACT Study Sample,
Other Court Program Sample, and Not Eligible Sample
5. Numbers of Youths in Three Samples, Analyzed by Source
of Referral
6. Average Age of Youths in Three Study Groups
7. Numbers of Youths in Three Samples, Analyzed According
to School Status
8. Average Number of Prior Drug Complaints in Three Groups
9. Numbers of Males and Females in Three Samples
10. Numbers of Youths in Three Groups, Analyzed by Most
Serious Complaint
Empirical Evaluation of the PIC-ACT
Numbers of Youths in Three Samples, Analyzed by Grade
in School
Numbers of Youths in Three Samples, Analyzed by Ethnic
Classification
Comparisons of Youths' Characteristics in Three Samples
Percents with New Referrals, by Study Sample
Numbers of Youths with and without New Referrals
Average A Priori Risk Scores, by PIC-ACT, Other Court
Program, and Not Eligible Samples
Percents with New Referrals, Adjusted for Risk, Selection,
and Time at Risk
Actual and Adjusted Percents with New Referrals for
Consequence Groups, with Adjustments for Time at Risk,
A Priori Risk, and Selection
Percents Assigned to Consequence Groups (Combined
Samples)
Percents Assigned Consequences of Various Types,
by County
Percent Com pliance for Types of Consequences
Adjusted Percents of PIC-ACT Youths With New Referrals
After Compliance or Non-compliance with Assigned
Consequences (Adjusted for Time at Risk, A Priori Risk,
Selection of Consequences, and Independent of County) 61
Percents with New Offenses Alleged at New Referral,
PIC-ACT, Other Court Program, and Not Eligible Samples 65
Percents with Felony and Misdemeanor Complaints at New
Referral, PIC-ACT, Other Court Program, and Not Eligible Samples 65
Percents of Youths in Three Study Groups with Cases
Adjusted and Not Adjusted 66
Percents of Youths in Three Study Groups Who Had
Petitions Filed as a Result of the New Referral
Empirical Evaluation of the PIC-ACT viii
Actual (Observed) and Adjusted Mean Seriousness Scores
for PIC-ACT, Other Court Program, and Not Eligible Samples,
with Means Adjusted for Time at Risk, Selection, and Risk
(Showing Adjusted Means Not Statistically Significant)
Actual and Adjusted Mean Scores for Seriousness
Criterion for Consequence Assignment Groups, with
Adjustments for Time at Risk, A Priori Risk, and Selection,
Showing No Significant Effect of Type of Consequence
Average Seriousness Scores for Youths with New Referrals,
by County, Adjusted for Compliance, Controlling for Time at
Risk, A Priori Risk, and Selection
Classification of Youths in Quasi-Experimental Groups
Numbers of Youths in Quasi-Experimental Groups
Percents with New Referrals in Four Quasi-Experimental
Groups (Unadjusted)
Average Offense Seriousness Scores for Youths with New
Referrals (Unadjusted)
Adjusted Percents with New Referrals in Quasi-Experimental
Groups
Adjusted Average Offense Seriousness Scores for Youths
with New Referrals in Quasi-Experimental Groups
Explanation of Variance, New Referral Outcome (Statistical
Controls Employed),
Explanation of Variance, Seriousness Outcome (Statistical
Controls Employed) (New Referrals Only)
Empirical Evaluation of the PIC-ACT 1
Empirical Evaluation of the Progressively lncreasing Consequences Act
Summary
Purpose
The Arizona legislature established the statewide Progressively
lncreasing Consequences Program in 1984. The legislation required "... a
periodic evaluation to determine if the provisions of this article reduce the
number of repetitive juvenile offenders." ' This is a report of such an evaluation.
The act requires actions by juvenile probation officers when youths are
referred to the juvenile courts. If a felony is alleged, the complaint must be
submitted to the county attorney with the request that a petition be filed. If a
misdemeanor or alcohol offense is alleged, the referral is not ordinarily required.
An exception is that misdemeanor complaints must be referred to the county
attorney when allegations of delinquent acts have been "adjusted" twice before.
"Adjustment" means that the complaint is disposed of without filing a petition. For
other cases, and if the county attorney does not file a petition, an interview may
be conducted that can ultimately result in adjusting the complaint. The complaint
must be adjusted if and only if the youth (a) accepts responsibility for the act,
and (b) has complied with specific conditions.
The youth is expected to comply with one or more of several program
requirements, called "consequences," before the complaint is adjusted. If the
conditions required for adjustment are not met, then the complaint may be
submitted to the county attorney with a request that a petition be filed. The
consequences are: community service; counseling; education for delinquency
A.R.S. 8-230.02,E.
Empirical Evaluation of the PIC-ACT 2
reduction or for alcohol or drug abuse; non-residential treatment; restitution; and
fines.
Methods
Data from the Administrative Office of the Courts were used to study
youths referred to the juvenile courts of the 15 Arizona counties between July
1993 and March 1994. Their records were followed for at least 13 months and
up to about 21 months to obtain measures of subsequent offending.
"Subsequent offending" was measured by new referrals to the juvenile courts
and by a classification of "seriousness" of alleged new offenses.
Samples Used for the Statistical Design
Youths transferred to the adult courts, those committed to the Department
of Youth Treatment and Rehabilitation, and status offenders were excluded. The
remaining 24,677 youths were classified into three groups for comparison, as
follows:
The PIC-ACT Sample includes youths eligible for PIC-ACT under A.R.S. 8-
230, as revised, and assigned consequences listed in the Act. Inclusion in
this sample means "Legally Eligible for PIC-ACT and Assigned-PIC ACT
Consequences." "PIC-ACT Consequences" means only those specific
consequences that were specified by the legislature. There were 10,499
youths in this sample.
The Not Eligible Sample includes youths referred to the county attorneys in
accordance with the PIC-ACT requirements of A.R.S. 8-230, as revised,
against whom the county attorneys filed petitions. "Not Eligible" means "Not
Eligible for PIC-ACT." Because of differences in county reporting, these
youths could not be identified for all counties. There were 1,733 youths in this
sample.
The Other Court Program Sample includes all other youths except transfers
to the adult courts, adjudication dispositions to the Department of Youth
Empirical Evaluation of the PIC-ACT 3
Treatment and Rehabilitation, status offenders, and youths in administrative
classifications. Inclusion in this Sample means "non-PIC-ACT processing
by the courts, including any court programs other than the assignment
of PIC-ACT consequences." These youths were not assigned PIC-ACT
consequences but of course were assigned to many other programs or
treatments --- that is, to "consequences" in the usual sense of that term. This
is not a "no treatment" group, but it is a differently treated group. There
were 12,445 youths in this sample.
Comparisons of different treatments are most rigorously done as
experiments arranged so that the groups being compared are equivalent at the
start. Since in this case an experiment was not feasible, "statistical" and "quasi-experimental"
designs were used. It was necessary to take into account, in each
analysis of program outcomes, measures of (a) the selection factors associated
with placement decisions, (b) the risk of delinquent behavior presented by youths
at the time of referral, and (c) the amount of time "at risk for each youth (since a
variable length of follow up period was used). Otherwise, the apparent results
may have been misleading. The comparisons made were based on a statistical
study of the whole sample and also on a quasi experimental design based on
sub-samples.
Youths placed in the PIC-ACT program differ from youth not so assigned
in two important ways. The first is due to the selection and assignment process,
which is partly prescribed by the law but mainly a discretionary decision.
Compared with those otherwise processed through the court system, the youths
assigned to PIC-ACT are, for example, a little younger. They more often have
been accused of theft. They have fewer prior referrals and probation violations.
These are only general tendencies, however, and there is substantial
overlap of characteristics of youths assigned to PIGACT consequences
and other court programs. The second is also a by-product of the assignment
process. Youths in PIC-ACT programs and those not affected by the PIC-ACT
differ in their risks of future involvement with the juvenile courts. They are better
risks, that is, less prone to future offending. This can be shown by their differing
Empirical Evaluation of the PIC-ACT 4
background characteristics. Again, there is substantial overlap between the
groups of youths assigned to PIC-ACT programs and those who go to
other courl programs. There are high and low risk youths in both categories;
but, in general, the PIC-ACT youths are better risks. In this study, this risk is
called "a priori risk."
Similarly, youths assigned to different consequence programs within PIC-ACT
differ. The assignment is at the discretion of the probation staff. Therefore,
fair comparisons of outcomes of assignment to different consequences require
that the factors associated with their selection, as well as a priori risk, be taken
into account.
There is another possible bias in comparisons that must be considered.
The youths in samples to be compared may not have equivalent exposures to
the risk of new referrals. They may be followed for different lengths of time, and
they may be differentially detained. Since they are different ages, some will
reach their 18th birthdays sooner after their referrals to the juvenile courts. If the
youths in groups compared are in the community free to commit offenses for
different lengths of time before age 18, these differences also must be
considered.
Each of these factors was taken into account in the analyses that form the
basis for the conclusions presented. This was done by methods for controlling
statistically those variables that cannot be manipulated physically. The analyses
are complex, but the principle is simple. The differences in outcomes are
analyzed in such a way that the variability due to the potentially biasing
factors is subtracted before identifying the differences that remain to be
attributed to the factor we wish to study. That is what is meant by "statistical
control."
Empirical Evaluation of the PIC-ACT
Samples Used for the Quasi-Experimental Design
Since a true experiment is not feasible for the present study, an
approximation was sought. That is what is meant by a "quasi-experimental
design." The objective was to compare similar youths (where "similar" means
alike in terms of characteristics typically affecting assignment to PIC-ACT) given
PIC-ACT consequences or other court programs. Youths typically assigned to
PIC-ACT with consequences were identified by a statistical analysis. Most were
indeed placed in PIC-ACT, but some were actually placed in other court
programs. Similarly, youths typically assigned to other court programs were
identified. The majority were actually placed in other court programs; but some
were placed in the PIC-ACT program instead. A comparison of the outcomes for
these groups (again taking account statistically for remaining selection factors, a
priori risk, and time at risk) provided an additional assessment of the effect of
PIC-ACT programs.
When youths were identified as most likely, on the basis of their
characteristics, to be placed either in PIC-ACT or Other Court Programs, and
then the actual placement was seen, four groups of youths were identified, as
follows:
1. PIC-ACT 'Experimental' Group 1
2. PIC-ACT 'Control' Group
3. Other Court Program 'Experimental' Group and
4. Other Court Program 'Control' Group.
These groups were defined on the basis of the most probable assignment
by the decisions of the probation and county attorney staff. First, the most likely
placement was determined, according to the characteristics of youths usually
1 The single quotes around the words "experimental" and "control" are reminders that
these are quasi experimental and control groups, and this is only an approximation to a
true experiment.
Empirical Evaluation of the PIC-ACT
placed in PIC-ACT. Second, it was determined, for each youth, whether the
actual placement was that found to be most likely. When the most likely
placement was PIC-ACT, and so was the actual placement, the youth was
assigned to the PIC-ACT 'Experimental' Group. When the expected placement
was PIC-ACT but the youth was actually placed in other court programs, the
youth was assigned to the PIC-ACT 'Control' Group. Similarly, the youths
expected to be placed in the Other Court Program were divided into two groups.
Those actually placed in the expected Other Court Program were assigned to
the Other Court Program 'Experimental' Group, and those placed instead in the
PIC-ACT program were assigned to the Other Court Program 'Control' Group.
Thus, four groups were defined for the study, as follows:
The PIC-ACT 'Experimental' Group is comprised of youths
typically assigned to PIC-ACT programs with consequences and
actually assigned to PIC-ACT consequences;
The PIC-ACT 'Control' Group is made up of youths typically
assigned to PIC-ACT programs with consequences but actually
assigned to Other Court Programs;
The Other Court Program 'Experimental' Group is comprised of
youths typically assigned to other court programs and actually
assigned to them; and
The Other Court Program 'Control' Group is made up of youths
typically assigned to other court programs but actually assigned to
PIC-ACT consequences.
Questions for this Study
The questions derived from the legislative mandate to evaluate the Act to
determine whether it reduces the number of repetitive juvenile offenders were
called "central questions" for the study. Other questions required to be
investigated in order to answer the main questions were called "secondary."
Empirical Evaluation of the PIC-ACT 7
Central questions to be answered were whether it makes any difference,
for later offending or the seriousness of new offenses, if youths are included in
PIC-ACT programs, or types of PIC-ACT programs, or comply with the
conditions required of them. These questions were examined for the State as a
whole and, so far as data and resources available permitted, for the various
counties.
Secondary questions concerned the characteristics of youths affecting
different decisions by probation staff and county attorneys. These questions had
to be addressed in order to conduct the analyses done to answer the central
questions.
Results
The answers to the "secondary" questions will be summarized first. Then
the answers to the "central" questions will be reported.
"Secondary" Questions
What youth characteristics (known at the time of referral) are related to
the likelihood of new referrals, and how are they weighted?
The a priori risk measure developed for this study included items typically
found to be predictive of later delinquency. Examples are indices of age, prior
record, and type of offense. The best predictors are the age at referral (older
youths are better risks), race, the number of prior counts, and the number of
prior referrals to the juvenile courts. The a priori risk levels vary among the
counties and among the three study groups considered. The "not eligible"
sample youths were, on average, the best risks. The worst risks were in the
"other court programs" sample, and in between were the PIC-ACT sample
youths.
Empirical Evaluation of the PIC-ACT 8
What are the factors considered in the selection process for the three
study groups, and how are they weighted?
The variables most helpful in understanding which youths are assigned to
PIC -ACT, other court programs, or required exclusion are measures of age and
"commitment to delinquency." They are similar to the items measuring risk, but
they are weighted differently. Examples are the number of prior referrals to the
juvenile courts, the total number of prior counts of offenses alleged, and the
youth's age. The classifications into the three study groups are explained further
by the number of times the youth previously has been referred to the court with
dispositions made without the formal court process of adjudication, and whether
the youth was detained immediately upon referral. Other variables that help to
differentiate the three groups are whether the first referral involved drug abuse,
the number of prior adjudications, the number of days ever detained, and race.
The mixtures of cases in terms of offenses, prior records, age, and prior history
in the juvenile court differ from county to county, as well as among the study
groups.
Although youth characteristics such as those listed help to
differentiate the samples of youths in PIC-ACT and other court programs,
the two groups are similar in many ways. No single characteristic
differentiates the two groups completely; it can be said only that they
differ in general, or on the average, on such characteristics as age, number
of prior referrals, or other prior record or offense variables.
What are the factors considered in assignment of PIC-ACT cases to the
different consequences, and how are they weighted?
Probation staff decided, among PIC-ACT cases, which consequences to
assign. The consequence selected for a youth depends on the offense alleged,
Empirical Evaluation of the PIC-ACT
the history of drug abuse, the prior record (numbers of prior counts and petitions)
and other case characteristics. The consequences assigned are explained
further by the following: the numbers of accomplices; prior drug allegations;
whether detained immediately upon referral; the number of days then detained;
age; race; and gender. The mixtures differ among the counties, and so do the
types and frequencies of consequences used.
What are the rates of compliance with the requirements of the different
consequences assigned?"
Some youths comply with the conditions set by the probation staff; others
do not. Compliance ranged from 59 percent for restitution to 91 percent for non-residential
treatment. The rates observed were as follows: community service,
82 Oh; counseling, 72%; education for delinquency prevention, 85%; education
for alcohol or drug abuse, 90%; non-residential treatment, 91 %; restitution, 59%;
fines, 80%; other consequences, 78%; and combinations, 83%.
Central Questions
The analyses so far summarized were done mainly to get ready to answer
the "central" questions by measuring the a priori risk, selection, placement, and
compliance. Here are the three main questions, with the answers provided by
the analyses:
Does it makes any difference, for later juvenile offending, if the youth is
selected as a PIC-ACT case, with consequences assigned?
The answer is "Yes."
This answer was given by the two different procedures for making the
comparison: the statistical study of all three groups in the total sample and the
"quasi-experimental" study.
Empirical Evaluation of the PIC-ACT 10
The actual percents of youths with new referrals, not corrected for
differences among the PIC-ACT, Other Court Program, and Not Eligible groups,
were not different. When, however, these were adjusted for a priori risk,
selection, and time at risk, the corrected rates did differ. The adjusted percents
with new referrals, for the "PIC-ACT sample" of youths assigned consequences,
the "other court program sample" of youths, and the "not eligible sample" of
youths referred to county attorneys when required by the PIC-ACT legislation
(with petitions filed) were different. The adjusted new referral rates were highest
for the "not eligible sample" with PIC-ACT-required referral with petitions filed,
lowest for the "other court program sample," and in between for the "PIC-ACT
sample" of youths with consequences assigned.
Whether youths are assigned to PIC-ACT with consequences, processed
otherwise through programs of the juvenile court system, or filed upon after the
referral to the county attorneys as required by the Act does make a difference in
new referral rates. The probability of new referral, when relevant risk, selection
factors, and time at risk are considered equivalent, is greatest for the "not
eligible" youths and lowest for those in programs other than PIC-ACT. These
differences cannot be explained by the variables known to be related to risk,
selection, or time in the community.
The effects of the PIC-ACT programs were studied also using the "quasi-experiment"
previously described. The outcomes for the youths who, on the
basis of their characteristics, would be expected to be considered PIC-ACT
cases and were actually placed in PIC-ACT programs were compared with
similar youths not assigned PIC-ACT consequences (the PIC-ACT "quasi-experimental"
and PIC-ACT "quasi-control" samples). Both groups were made
up of youths typically placed in PIC-ACT. The comparisons took account of
differences in selection, time at risk, risk of new referrals, and counties.
Consistently with the statistical study already described, the members of the
Empirical Evaluation of the PIC-ACT 11
"Quasi-Experimental" sample had, over all, a higher percentage of new referrals
than did the members of the "Quasi-Control" group.
No differences were found in the seriousness of new offenses alleged
when new referrals occurred. This was true for the overall statistical analysis and
also for the comparison of the PIC-ACT "quasi-experimental" groups. Though
the three study groups of the total sample differ in actual average seriousness
scores before any statistical control for risk, selection, and time at risk, these
differences, when adjusted for the statistically controlled factors, disappear.
There was no significant difference in the adjusted mean seriousness scores; the
differences observed would be expected by chance about six percent of the
time. The observed differences are accounted for by a priori risk, selection, and
time exposed to the risk of new referrals. Similarly, the two groups compared in
the "quasi-experimental" design did not differ in the average new offense
seriousness scores after the adjustments.
Does the particular PIC-ACT consequence selected make any difference
for later juvenile offending?
The answer is "Yes."
There were marked differences in actual new referral rates according to
the type of consequence assigned. After adjustment of these for time at risk, a
priori risk of new referrals, and selection for the particular consequence program,
these differences remained. Although smaller than before the adjustment, the
differences were still significant. The actual percents with new referrals ranged
from 37 percent for education for drug or alcohol abuse and non-residential
treatment to 62 percent for restitution. After the adjustment for the known
potentially biasing factors, these ranged from 41 percent for each of the first two
Empirical Evaluation of the PIC-ACT 12
programs to 54 percent for restitution. The type of consequence assigned does
make a difference in new referrals.
There was no effect of type of consequence on the seriousness of new
offenses --- that is, the particular consequence program selected by the
probation officers had no effect on the level of seriousness of new offenses
alleged.
Does compliance by the youth with the conditions of the consequences
make any difference for later juvenile offending?
The answer is "Yes."
Compliance, which is most frequent for education for alcohol or drug
abuse and non-residential treatment, affects the new referrals outcome. Within
the PIC-ACT study group and independently of county, percents with new
referrals were examined after adjustment for time at risk, a priori risk, and
selection for different consequences. The adjusted percents with new referrals
were 46 percent for the youths who complied but 54 percent for those who did
not. The probability of new referrals is decreased by compliance.
Compliance with the assigned consequences also affects the seriousness
classification of new offenses. Those youths who failed to comply had more
serious offenses alleged with new referrals than did their counterparts who
complied with PIC-ACT program requirements.
Recommendations
Five recommendations are suggested by the results of the study
described in this report. They may be summarized as follows:
Empirical Evaluation of the PIC-ACT 13
4 Consider diversion options for mandatory referrals of specific
cases to county attorneys;
4 Investigate and extend the most promising PIC-ACT programs
to additional counties;
a Improve monitoring procedures to increase compliance;
a Clarify recommended procedures for assignment to PIC-ACT,
and
8 Establish a research file including needed additional follow up
information.
The recommendations, explained with comments providing their justifications,
are as follows:
Change the PIC-ACT requirement that felony complaints and
misdemeanant complaints with two prior adjustments be referred to the
county attorney with a request that a petition be filed (A.R.S. 8-230.01,
as revised, paragraph A). Amend to allow diversion.
Comment:
Referral of these cases, with a request for petition, is now mandatory
rather than permissive. In the sample studied, 1,310 youths out of 14,939
otherwise eligible for PIC-ACT programs (nine percent) were identified as
referred forthwith to the county attorneys, as required for felony and third time
misdemeanor complaints. (The data available did not permit the identification of
these cases for many counties.) Another 1,673 youths, or 11 percent, were
referred to the county attorneys after cite-in for a PIC-ACT interview, not as
required but as discretionary acts of the court personnel. These youths did not
admit responsibility or they did not comply with PIC-ACT consequences. Of all
Empirical Evaluation of the PIC-ACT 14
youths included for study, seven percent resulted in filings after these referrals
and were no longer eligible for PIC-ACT consequence programs. Some of these
youths were referred as discretionary acts. It is only that portion of referrals and
requests required that is the subject of this recommendation.
The subsequent delinquent behavior of the Not Eligible group, measured
by new complaint referrals, was compared with that of youths who were
assigned PIC-ACT consequences and also with those in other court programs.
The group required to be referred to the prosecutors, and for whom petitions
were filed, are better risks, on the average, than the youths in either of the other
two groups. Nevertheless, this group (otherwise eligible for PIC-ACT programs)
has a higher percentage of new referrals than either of the other groups. This is
true after considering the risk levels of the youths, the time at risk, and the
selection factors associated with the law and the exercise of discretion. The
probability of new referrals is greatest for this group, lower for PIC-ACT cases
with consequences assigned, and lowest for youths in other court programs.
The effect of the law as it stands is to transfer a specific area of discretion
from the court system to the county attorneys, with an apparent increase in
repetitive delinquency, rather than the reduction to which the Act adverted.
The classification of youths referred to the courts with delinquency
complaints on the basis of the simple legal classification of the alleged act only,
or on an arbitrary classification based on the number of prior adjustments,
ignores much information about the youth and the circumstances of the alleged
delinquency. This information can be taken into account in arriving at the
decision whether to petition or divert. The evidence of this study suggests that
this area of discretionary decision making should be considered, where informed
judgments can be made on the basis of additional information. The mandated
referral process does not appear to work as intended.
Empirical Evaluation of the PIC-ACT 15
The more successful types of consequence programs identified in this
report --- education for drug or alcohol abuse and non-residential
treatment --- should be examined further to determine why they appear
to be successful and should be extended to additional counties.
Comment:
These programs reduce the likelihood of new referrals. They are not used
as commonly as community service or education for delinquency prevention, and
they are used extensively only in Maricopa and Pima Counties. A more thorough
analysis and evaluation of these programs is suggested to determine the
program features that appear to be successful and that can be "exported" to
other counties. Although placement in these two programs is related to (fewer)
new referrals, this may be due to either or both (a) additional but yet unknown
characteristics of youths selected for these programs, or (b) the effectiveness of
the programs. The most desirable further program would use a research design
to more rigorously test effectiveness and a systematic program for development
of these programs in other counties.
Procedures to improve compliance with consequences are needed,
particularly for some types of consequences programs. Restitution as a
consequence is notable for a relative non-compliance by the youth
assigned it. Counseling also has a low rate of compliance.
Comment:
Compliance with consequences assigned in the PIC-ACT program
decreases the probability of new referrals. When new referrals do occur,
compliance is predictive of less serious new offense allegations. Those youths in
Empirical Evaluation of the PIC-ACT 16
the PIC-ACT program who complied with the consequence assignments had
lower rates of new referrals and less serious new offense complaints.
Careful monitoring systems in each county are needed to increase
compliance with all consequence programs. Special efforts are needed to
improve compliance with assigned community service and counseling.
Although compliance is substantially related to (fewer) new referrals, this
may be due to either or both (a) additional but yet unknown characteristics of
youth who comply or (b) the act of compliance. This question warrants further
study, but the available evidence suggests the recommended efforts to increase
compliance.
Clarify recommended procedures for assignment to PIC-ACT
There do not appear to be any clear guides or policy statements
governing the selection of eligible youths (that is, those not now precluded by
law) for the PIC-ACT program. There is substantial consistency in this
discretionary selection process, as may be seen from the differences between
PIC-ACT and non-PIC-ACT youths reported in this document. Moreover, there is
evidence that youths typically selected for PIC-ACT, compared with those more
often not selected for PIC-ACT (when both groups actually are assigned PIC-ACT
consequences) have fewer new subsequent referrals. This suggests a need
for greater consistency in the assignment process, which appears to be, more
often than not, but not invariably, appropriate. At the same time, there is a
substantial overlap among the two groups when offense, prior record, age, and
other attributes of youth in the two groups are considered. Also, there is
substantial variation among the counties in the kinds of youths selected for PIC-ACT
programs. It is recommended that a greater degree of consensus be sought
Empirical Evaluation of the PIC-ACT 17
and articulated to describe the types of youths believed to be suitably assigned
to PIC-ACT. The specification of a policy describing the kinds of youth for whom
PIC-ACT programs are believed to be appropriate and desirable could help to
provide a greater consistency in selection and result in more effective PIC-ACT
programs.
Establishment of a research file, associated with the Administrative
Office of the Courts data file assembled from the various county
systems, is needed for a more efficient, reliable, and informative
research and management system within the Administrative Office of
the Courts.
Comment:
A file with recoded data elements suitable for analyses required by
program development, evaluation, and information dissemination programs
should be developed and maintained as a routine activity of the AOC. This would
markedly reduce costs of program evaluations, which now require repeated,
extensive reconstruction of the file for specific analyses. Associated with it
should be a program of "data audits," comprised of periodic sample tests of the
reliability of data elements included in the file. Although audits of financial
accounts are routinely expected, the auditing of the reliability of data to inform
major decisions, with potentially costly consequences, rarely is done.
In preparation for further evaluations of the PIC-ACT programs
specifically, data should be collected to permit the identification of youths eligible
for PIC-ACT, ineligible youths (with identification of the reason or reasons),
youths selected for PIC-ACT, all PIC-ACT consequences assigned, * and the
2 Some counties have indicated that only one consequence is reported in the data
collection system even though more than one actually was required of the youth. it is
recommended that all consequences assigned be reported.
Empirical Evaluation of the PIC-ACT 18
dates of PIC-ACT interviews. These data are needed for assessments of the
fidelity of the program with legislative requirements and for the evaluation of the
effectiveness of the programs.
Systems for follow up data collection for youths with adjudicated
dispositions to the Department of Youth Treatment and Rehabilitation, youths
transferred to the adult courts, and youths after age 18 are needed for more
complete evaluations of the PIC-ACT and other court programs.
Empirical Evaluation of the PIC-ACT
Empirical Evaluation of the
Progressively Increasing Consequences Act Program
Introduction
The Arizona legislature, in establishing the Progressively lncreasing
Consequences Program initiated a statewide program that began in 1984. The
general purpose of the program is implied in the act, since the legislation
required "... a periodic evaluation to determine if the provisions of this article
reduce the number of repetitive juvenile offenders." This is a report of such an
evaluation.
The act requires several actions by juvenile probation officers when
youths are referred with complaints or citations (and permits others). A
delinquency complaint is defined in the Act as " ... a report prepared by a law
enforcement agency and submitted to the court, alleging that a juvenile has
violated the criminal law." If the referral is for a delinquency complaint alleging
the commission of a felony offense, it must be submitted to the county attorney
with the request that a petition be filed. If the allegation is that of a misdemeanor
offense or an alcohol offense, then the complaint or citation may be submitted to
the county attorney, except that if the allegation is for a misdemeanor offense
and allegations of delinquent acts have been "adjusted" on two prior separate
occasions, then the complaint or citation must be referred to the county
attorney. If the county attorney does not file a petition, or if the allegation does
not fall within the category that must be referred to the prosecutor, then the
probation officer may interview the youth and at least one parent or guardian; if
so, the probation officer then must adjust the complaint, conditionally upon (a)
acknowledgment by the juvenile of responsibility for the act; and (b) compliance
by the juvenile with specific conditions specified in the act. "Adjustment" means
3 A.RS 8-230,a s revised.
4 A.R.S. 8-230.02,E .
Empirical Evaluation of the PIC-ACT 20
that the complaint or citation is disposed of in a manner that obviates the filing of
a petition.
If the youth does not acknowledge responsibility for the delinquent act or
alcohol offense alleged, or fails to comply with the conditions set by the juvenile
probation officer, then the complaint or citation may be submitted to the county
attorney with a request that a petition be filed.
Before adjusting a complaint or citation, the juvenile must comply with one
or more of several specified conditions, as follows:
1. Participation in unpaid community service work;
2. Participation in a counseling program ... designed to strengthen
family relationships and to prevent repetitive juvenile delinquency;
3. Participation in an educational program ... which has as its goal the
prevention of further delinquency;
4. Participation in an education program ... designed to deal with ...
alcohol or drug abuse;
5. Participation in a nonresidential program of rehabilitation of
supervision offered by the court, or offered by a community youth
serving agency ... ;
6. Payment of restitution to the victim of the delinquent act;
7. Payment of a monetary assessment.
5 These are the consequences prescribed in the Act. in practice some counties use
additional or atternative sanctions, as described in Juvenile Justice Services Division,
Administrative Office of the Courts, Arizona Supreme Court, PIC-ACT Reviews by
County, Phoenix, Arizona: Administrative Office of the Courts, February, 1995. Examples
are the use of (I) an Outdoor Education Program by Apache County; (2) informal
probation supervision, tutoring, and assigned essays in Cochise County; (3) written
reports, apology letters, and Teen Court in Coconino County; (4) informal probation,
tutoring, and Teen Court in Gila County; (5) Teen Court and tutoring in Graham County;
(6) curfew imposition, detention, informal probation, graffiti patrol, psychologist
interviews, apology letters, essays, summer program activities, and day support in
Greenlee County; (7) Teen Court, essays and apology letters in La Paz County; (8) a
Graffiii Abatement Program, the Renewing Arizona Family Traditions Program, a Victim
Empirical Evaluation of the PIC-ACT 2 1
Whether this act "reduces the number of repetitive juvenile offenders" is
the subject of the study proposed. Before explaining the procedures designed to
investigate this question, some problems in answering such questions, and
potential solutions to them, will be discussed briefly. Next, the central questions
to be answered by the study proposed, and also some secondary ones, will be
listed. The specific methods, including the sample to be studied, sources of data
to be used, methods of definition and measurement, and the analytic methods
proposed will be described. Then the results of the study can be reported and
discussed, and recommendations toward program improvement can be
considered.
Attempted reforms aimed at reduction of delinquency are rarely informed
by rigorous analyses of the effectiveness of the policies and practices thereby
changed. Expected consequences of legislative changes, guidelines policies, or
mechanisms limiting judicial discretion are announced and argued about; but
rarely are the underlying expectations based on evidence that can come only
from careful examination of the results of the new practices. Arguments for and
against the use of various alternative ways of dealing with delinquent youth
typically are made in the absence of information about the probable results of
choices --- whether made legislatively or judicially.
This lack is partly due to the fact that experiments designed to test central
sanctioning questions usually are not feasible. The term "experiments" implies
that groups treated differently can be considered equivalent in all respects
except for that differential treatment. Then observed differences in outcomes can
Offender Reconciliation Program, and Teen Court in Maricopa County; (9) Teen Court
and essays in Mohave County; (1 0) informal probation and Teen Court in Navajo
County; (1 1) a monetary donation to a chariiy, informal probation, random drug testing,
and a Stop Assaultive Children program in Pima County; (12) informal probation in Pinal
County; (13) admonishment, informal probation including curfews, urinalysis, tutoring,
and apology letters in Santa Cruz County; (1 4) a Court Obligated Program, a Volunteers
in Probation Program, detention tours, and tutoring in Yavapai County; and (15) in-home
detention, tutoring, and Teen Court in Yuma County.
Empirical Evaluation of the PIC-ACT
be said (with a known probability of error) to be due to the treatment. The
equivalence of the groups compared usually is sought by random assignments
to the treatment conditions. Since that is rarely possible when actions to be
taken to dispose of complaints or citations must be selected, groups given
different consequences cannot be considered equivalent at the outset, and the
delinquency reduction effects of sanctioning therefore cannot be compared fairly.
Since experimental designs for study of the impact of the assignments of
consequences on the subsequent delinquent behavior of juveniles referred are
not feasible, the next most rigorous designs should be used. Rarely, however,
are the data available to permit that, since data demonstrably relevant to
selection biases due to factors associated with the decisions ordinarily are
absent. As a result, little is known about the consequences for later delinquency
behavior of choices concerning consequences assigned or other means of case
disposition. Sanctioning policies are therefore usually developed without sound
information about how sanctioning modifies, controls, or enhances the likelihood
of future delinquency behavior by the juveniles sanctioned.
The research reported here was advantaged by a unique opportunity for
application of rigorous statistical and quasi-experimental designs for the
assessment of the effects of sanctioning choices made by probation personnel
(and county attorneys) consistent with the provisions of the PIC-ACT. This
opportunity was given by the foresight of the Administrative Office of the Courts
and the juvenile courts in the various counties that led to the availability of a
comprehensive data file that includes data on all referrals during the period July
1, 1993 through mid - May, 1995, including the results of major decisions in the
juvenile justice system process and subsequent outcomes in terms of
delinquency behavior. The opportunity was thus available to answer some
6 Possible effects on others, i.e., general deterrence effects, are ignored in this report as
beyond its scope.
7 Analyses of time series of events such as delinquent referrals are sometimes used for
evaluations such as this one, and sophisticated methods are available to assist in ruling
Empirical Evaluation of the PIC-ACT 23
central questions about the effects of sanctions through designs that corrected
for the non-equivalency of differently sanctioned groups.
The subsequent delinquency referrals of youth referred to the courts in
the 15 Arizona counties were examined, with analyses of the effects of sanctions
(consequences) using statistical methods and quasi-experimental designs based
on multivariate models of selection factors that would affect the validity of
comparisons. These are determined partly by the law but mainly by discretionary
decisions made by probation staff and by county attorneys. The designs were
based also on a model of "a prior/' risk, that is of the probability of a new referral
later, based on youth characteristics known at the time of the instant referral.
This enabled comparisons of the effects of PIC-ACT classification, assignments
to consequences, and compliance with consequences assigned on the nature of
subsequent offending. The statistical design, based on the entire sample of
youths studied, was supplemented by a quasi-experimental design. "Subsequent
delinquency" was measured by new referrals to the juvenile courts and, for new
referrals, a classification of the "seriousness" of the next (most serious) offense
alleged.
Questions for this Study
This research was intended to answer three central questions through the
use of statistical and quasi-experimental designs. Answering these questions
first requires examining secondary questions (discussed subsequently) about
selection and risk. Answering the latter questions (important in their own right)
can contribute to the strength of the research designed to answer the central
questions. They will be discussed after considering the central questions and the
designation of samples of youths available for comparisons.
out rival hypotheses to explanations that changes over time are due to the policy
intervention. In the case of the present problem, data that would permit such analyses
are not available; and in any case it is believed that the methods used permitted a more
rigorous analysis.
Empirical Evaluation of the PIC-ACT
Central Questions
The first general question is whether it makes any difference, for later
juvenile offending, if the youth is selected as a PIC-ACT case, with
consequences assigned.
The second question is whether the particular PIC-ACT consequence
selected makes any difference for later juvenile offending.
The third question is whether compliance by the youth with the
conditions of the consequences makes any difference for later juvenile
off ending.
For each of these questions, "subsequent juvenile offending"
means new referrals; and, when these occurred, the seriousness
classification of the (most serious) alleged new offense.
Classification of Youths into PIC-ACT, Other Court Program, and Not
Eligible Samples
The probation staff may, within the constraints specified in the act, adjust
cases or refer them. Referred cases not filed upon by the county attorney also
may be adjusted. Thus there may be three groups of youth whose later
delinquency may be compared: those selected for PIC-ACT programs and
assigned the consequences listed in the Act; those legally eligible for PIC-ACT
programs but assigned to other court programs, and those youth who were not
only referred to the county attorneys in accordance with PIC-ACT requirements
but for whom delinquency petitions were filed. These groups were called the
PIC-ACT Study Sample, the Other Court Program Sample, and the Not Eligible
Sample.
Empirical Evaluation of the PIC-ACT 25
A youth's classification in either of the two samples other than the PIC-ACT
Study Sample does not mean that no "consequences" in the usual sense of
that term were assigned. Subjects in the Other Court Program Sample were not
assigned PI C-ACT consequences according to the PI C-ACT procedures. If
consequences (in any sense) were assigned, these may have included many
other types of programs not specified in the Act. This could include various
probation programs and other diversion programs, so it is clear that classification
for this study into the Other Court Program Sample does not imply that the
youths in that group were subjected to "no treatment." Subjects in any of these
samples could be expected to proceed through stages of an Advisory Hearing,
an Adjudication Hearing, and a Disposition Hearing. At any stage,
"consequencesJiJn the usual meaning of that term, could ensue. Data are not
available to describe or assess the treatments and program assignments of the
Other Court Program and Not Eligible Samples.
The definition of the three groups of youths to be compared first may be
seen in Figure 1. As the chart shows, youths who were, as a result of the sample
referral event, transferred to the adult court or to the Department of Youth
Treatment and Rehabilitation, excluded from the study. The reason for exclusion
was that in both cases the required follow up data to determine subsequent
delinquency are not available. Status offenders, and those youths classified as in
an administrative category only, were excluded also, since the PIC-ACT
legislation addresses only delinquency complaints. It was then determined
whether the initial PIC-ACT criteria (specified in the Act) were met. If not, and the
case was submitted to the County Attorney forthwith (before the "cite in" date for
8 For a detailed discussion of the decision process in one county from referral through
adjudication, see Gottfredson, Don M., Gottfredson, Michael R., Gottfredson, Stephen
D., Etten, Tamryn J. and Petrone, Robert F., Needs for System Development in the
Maricopa County Juvenile Justice System. Sacramento, California: Justice Policy
Research Corporation, May, 1994.
9 Some counties, notably Maricopa, may include youths with status offenses (who are
not on probation) in PIC-ACT programs; see, Research and lnformation Specialists, Inc.,
An Evaluation of the PIC-ACT Program in Maricopa, Pima, and Coconino Counties.
Mesa, Arizona: Research and Information Specialists, Inc., February, 1988, p.22.
Empirical Evaluation of the PIC-ACT 26
the interview), then it was determined whether a petition was filed. If so, the
youth was classified into the Not Eligible sample. If not, the youth was
considered a candidate for PIC-ACT. If, however, any case initially eligible for
PIC-ACT was submitted thereafter to the County Attorney (which could be the
case if the youth does not admit to responsibility for the alleged act or acts or
does not agree to comply with assigned consequences) then it was also
determined whether the County Attorney decided to file a petition. If so, the
youth was classified into the Not Eligible category. If not, the youth was again
considered to be eligible for PIC-ACT. From the pool of youth thereby
determined to be eligible for PIC-ACT, those for whom the assignment of
consequences was recorded were placed into the PIC-ACT Study sample. The
remaining youth were assigned to the Other Court Program sample.
"Not Eligible" as used in this study means "not legally eligible according to
the procedures specified in the PIC-Act legislation." In order to understand the
meaning of the Not Eligible sample, however, it should be noted that legally
ineligible cases could not be determined for all counties. Eight counties had no
cases classified by our procedure into the "Not Eligible" group, and two others
had only one or two. At least part of the reason is that not all counties reported
the "cite in" date which was used to determine whether the youth was referred
"forthwith" to the county attorney. This means that some youth ineligible for PIC-ACT
are included in the Other Court Program group. It is clear that the "Not
Eligible" sample should not be taken as including all youths not legally eligible,
but only those not eligible by the PIC-Act rules specified by the legislature and
who could be identified by the available data.
Empirical Evaluation of the PIC-ACT 27
Classification of Youths into PIC-ACT
Study, Other Court Program, and Not
Eligible Samples
All Cases Referred
the Arlzona Juven~leC ourts
Transferred
to Adult Court . or Commltted to
3, - DYTR?
%
-%
. --- - \
' Status Offender
orAdm~n~strat~ve Exclude from Study!
-a
,+'
ln~t~PaICl -ACT Crlter~a --- . -- 7-
-- -- - . - ---
El~g~bfloer PIC-ACT
'S ubm~ttedto ,
Petition Flled?
Go to Adjudication
-+
3
PIC-ACT Sample Other Court Program Sample 2 Not El~gibleS ample 3
'? '5
9 I 3
* 3t!&~*%' t.>-W* > -S ,-<W ,,S, -W-3"S .4J:"+~.aw - &Y.YP . 2-2-r-7 E%P&S&+LV~%.:*-XL~>*.Lek td
z
Figure I: Classification of Youths into Samples for Study
Empirical Evaluation of the PIC-ACT
Meanings of the Group Classifications
The meanings of classification into the three study groups may be
summarized as follows:
The PIC-ACT Sample includes youths eligible for PIC-ACT under A.R.S. 8-
230, as revised, and assigned consequences listed in the Act. Inclusion in the
PIC-ACT Sample thus means "Legally Eligible for PIC-ACT and Assigned
Consequences."
The Not Eligible Sample includes Youths known to be referred to the
county attorneys in accordance with the PIC-ACT requirements of A.R.S. 8-
230, as revised, against whom the county attorneys filed petitions. "Not
Eligible" thus means "Not Legally Eligible for PIC-ACT."
The Other Court Program Sample includes all other youths except transfers
to the adult courts , adjudication dispositions to the Department of Youth
Treatment and Rehabilitation, status offenders, and youths in administrative
classifications. Inclusion in this sample thus means "non-PIC-ACT processing
by the courts, including any court programs other than the assignment of
PIC-ACT consequences." These youths were of course assigned to many
other programs or treatments. It is not a "no treatment" group, but it is a
differently treated group.
Secondary Questions
The secondary questions concern, first, the possibly different kinds of
"risks" presented by the youths assigned to the various classifications, and,
second, the characteristics of youths that affect the decisions taken by probation
Empirical Evaluation of the PIC-ACT
staff and the county attorneys. In both cases, the question is whether the
characteristics of youth, at the time of referral, differ in groups to be compared.
The secondary questions may be listed as follows:
What youth characteristics (known at the time of referral and before
placements) are related to the likelihood of new referrals, and how are
they weighted? This will be referred to as the a priori risk. If groups differ in
a priori risks, those differences must be taken into account in any
comparisons.
What are the factors considered in the selection process for the three
study groups, and how are they weighted? Probation staff (and in the
cases referred to them, the County Attorneys) made discretionary decisions
affecting the classification of youths into the three groups. If there are
different case characteristics for these three sets of cases, these are
important selection factors that must be taken into account in any fair
comparison of outcomes for the three groups.
What are the factors considered in assignment of PIC-ACT cases to the
different consequences, and how are they weighted? Probation staff
made discretionary decisions, among PIC-ACT cases, as to the particular
consequences to be assigned. Also, not all consequence programs are used
in all counties. As a result, differences in the youth assigned the different
programs must be considered in the comparison of results of the different
programs.
What are the rates of use, among counties, of the various PIC-ACT
consequences?" If counties differ in this respect, then the comparisons
made of the effects of consequences should be made independently of
any county effects.
Empirical Evaluation of the PIC-ACT
These "secondary" questions are essential to answering the "central
questions." Consider, for example, the selection by the probation officer from
among the alternative consequence assignments available. There are seven
basic choices of consequences for the child (although there may be also
combinations of these). The general problem of assessment of this decision is
one that typically is found at each step in the decision tree of juvenile justice
decisions. This is the problem of taking account appropriately of bias in the
comparison of outcomes from the different placements. By this we do not refer to
any supposed bias on the part of the probation staff or county attorneys; rather,
we are concerned about circumstances in the decision process that may, if not
properly recognized, result in misleading comparisons. The observed differences
in outcomes may be mistakenly attributed to the treatment, when actually they
were due to the selection for treatment or to the a priori risk.
That is, a direct comparison of outcomes of these seven programs may
be unfair, since the characteristics of youth differentially assigned to them may
differ (as reasonably to be expected). As a result, the groups assigned to the
different placements cannot be regarded as equivalent; and therefore the
outcomes cannot be compared fairly without taking these differences into
account in the comparison.
This merely points out that the data considered in making the decisions,
and also the differences, if any, in the a priori "risk" classifications of youth
assigned different treatments, must be taken into account so far as possible in
order that the comparison of outcomes of different treatments is fair. In the
analyses that follow, we sought to control statistically for observable differences
in the youths associated the decisions and those differences in a priori risks. lo
lo For a concise but more technical and detailed discussion of the problems as issue here,
see Berk, Richard A., "Causal Inference as a Prediction Problem," in Don M. Gottfredson
and Michael Tonry (Eds.), Prediction and Classijication: Criminal Justice Decision Making,
Chicago and London: The University of Chicago Press, 1987, 183-200. For early, less
technical discussions of the same topic, see Wilkins, Leslie T., "What is Prediction and is
Empirical Evaluation of the PIC-ACT 3 1
As will be seen, it is necessary to control also for "time at risk," that is, for the
length of time the youths were in the community before age 18 and thus subject
to the risk of being referred to the juvenile courts.
Methods
Sample
Included in the total sample were all youths (except specified transfers
and certain youth committed to the Department of Youth Treatment and
Rehabilitation) for whom referrals were received by the courts during the nine
month period July 1, 1993 through March, 1994. The sample was defined by first
referrals during this time period, regardless of any prior referrals. Note that
persons in the sample may have had prior referrals and that persons in the
sample may have subsequent referrals during the time period. Both males and
females of any age were included. Data concerning these youths were collected
and recorded through mid May, 1995 to determine outcomes of the referrals of
the youths sampled. l1
This means that all cases in the cohort of youth referred during the eight
month period were followed for at least 13 months (and up to about 21 months)
it Necessary in Evaluating Treatment?" in Research and Practical Application ofResearch in
Probation, Parole, and Delinquency Prevention, New York: Columbia University, New
School of Social Work, Citizen's Committee for Children of New York, 1961 ; and
Gottfredson, Don M., "The Practical Application of Research," Canadian Journal of
Corrections, 1963, 5:212-228.
11 All cases were followed at least until the end of March, 1995. As a result of the specific
methods required to abstract the data from the various counties, some data were
collected up to May 12, 1995. This minor variation in length of follow up for some
counties was ignored.
Empirical Evaluation of the PIC-ACT 32
after the date of referral (complaint), or until the 18th birthday l2to determine
whether there were new referrals.
The sampling of an eight month cohort, selected for reasons of availability
of data for both referral and follow up data, may introduce a seasonal bias into
the sample. It was believed that this potential bias could be ignored safely,
however, since the months with marked differences in typical numbers of youth
received are included. 13
Excluded from the sample were all youths transferred to the adult court
or committed to the Department of Youth Treatment and Rehabilitation as a
consequence of an offense or offenses for which the youth was first referred
during the time period. Note that this does not necessarily exclude all youth
transferred or committed during the time period.
After excluding also youths who were classified as status offenders and
those categorized as in an administrative class only, the remaining youths were
classified into the three comparison groups described previously.14
l2 Since the juvenile court jurisdiction in Arizona expires at age 18, no new referrals to the
juvenile court can occur thereafter.
l3 In order to examine the potential seasonal bias in the sample selection, the numbers of
referrals to the juvenile courts in Arizona during the calendar year 1993 (a full year
overlapping the selected sample) was analyzed by month. The average number for the
12 months was 5,814,w hile the average for the three months not included in the cohort
was 5,867. Months in which there may be obvious biases, such as truancy not expected
during summer months, are partly included, as well as the months with the highest
(October) and lowest (January) numbers of 1993 referrals. In order to follow a cohort of
youth referred to the courts for a full year or more after referral, the eight month period
for sample selection was used. It is assumed that youth referred during this period are
reasonably representative of youth referred during a full year period.
l4 A very small number of cases were excluded from the sample either because their
instant referral actually occurred before the intended date of the data initiation, or
because the date of their "next" referral occurred before the date of the instant referral.
Empirical Evaluation of the PIC-ACT
Data Sources
The Administrative Office of the Courts' data file was the source of data
for the analyses. A listing of the data elements in the file created for this study,
together with codes for the variables included, has been provided to the AOC. It
shows the codes for the data elements actually available and used for the
analyses described. The meanings of codes and abbreviations shown are
provided in order that replications of the analyses presented, or additional
analyses, can be completed using the data file, also provided to the
Administrative Office of the Courts, and the system file created to permit
statistical analyses by the SPSS (Statistical Package for the Social Sciences)
program. 15
Measurement
The Administrative Office of the Courts' file provided an unusually
complete record of background characteristics of the juveniles referred,
placements, compliance, county attorney filings, and outcomes. Measurement
issues will be discussed in terms of the dependent variables (outcomes)
included, the independent variables for the quasi-experimental designs, and
other data availability for descriptive purposes.
Dependent Variables
The outcome measures studied were derived from the apparent
legislative intent of reducing repetitive delinquent offenses. Only the criteria "new
referral(s) during the follow up period," the offense alleged if new referrals
occurred, and the seriousness rating of the latter were studied. That is, cases
were classified as to outcomes as to whether there was a new referral to the
juvenile courts during the time frame of the study, if so, the offense alleged, and
its seriousness rating.
l5 Most of the statistical analyses presented in this report were completed using this
program (Norusis, Marija A., SPSS for Windows, Release 6.0. Chicago: SPSS, Inc.,
1993.
Empirical Evaluation of the PIC-ACT 34
Among the most critical variables in research such as reported here are
those defining the outcomes used to measure new delinquent behavior. Reviews
of problems are available. l6 These include: the validity of available data as a
measure of outcome; the inability of dichotomous success/failure criteria to
capture the full range of post-release or post-sentencing adjustment (and
statistical difficulties inherent in the use of a dichotomous criterion); the possibly
confounding effect of "time at risk" when comparing experiences of offenders
who have been in the community for varying lengths of time; and differing error
rates depending upon the nature of the criterion chosen (e.g., new referrals,
petitions filed, or adjudications). Other concerns include frequent failures to
observe a long enough follow-up period, which typically is too short to measure
subsequent offending adequately; use of fixed follow up periods with a failure to
examine failure rates over time; and the use of narrow definitions without
recognizing the complexities of the concept of "recidivism."
In this project, outcomes were measured by new referrals and charges.
(Data on filings by the County Attorneys and on court adjudications of
delinquency or incorrigibility, when they occurred, also were collected.) The
referral data were assumed to be more reliable indicators of new delinquent acts
than were adjudications or new complaints or citations adjusted in the follow-up
records to be used. Referrals and charges record dates that are nearest in time
to offense behaviors. The direction of errors expected (accepting as failure a
referral for an offense not committed versus excluding offense events as a result
l6 See, e.g., Blumstein, A., and Larson, R.C., "Problems in Modeling and Measuring
Recidivism," Journal of Research in Crime and Delinquency, 8, 1971 , 124-1 32; Waldo,
G. and Griswold, D., "Issues in the Measurement of Recidivism, "in National Research
Council, The Rehabilitation of Criminal Offenders, Washington, DC.: National Academy
of Sciences, 1979, 225-250; Gottfredson, D.M. and Gottfredson, M.R., "Data for Criminal
Justice Evaluation: Some Resources and Pitfalls," in Handbook of Criminal Justice
Evaluation, edited by M.W. Klein and K.S. Teilman, Beverly Hills: Sage, 1980; Maltz, M.,
Recidivism Orlando: Academic Press, 1 984; Schmidt P and Wittee, A.D. Predicting
Recidivism Using Survival Models, New York: Springer Verlag, 1988; Blumstein, A.,
Cohen, J., Roth, J.A., and Visher, C.A., (Eds) Criminal Careers and "Career Criminals,"
Washington, D.C.: National Academy Press, 1968.
Empirical Evaluation of the PIC-ACT 35
of attrition in the juvenile justice processing from referral to adjudication) was
assumed to be the better choice.
The measures used included a classification of the seriousness of new
offense behaviors. A major development in the measurement of delinquency and
crime has been the effort to improve upon behavioral representations by
assessing the seriousness of offense acts. Measurement of the seriousness of
crimes dates from Thurstone; 17 replications suggest that these judgments
remain remarkably stable over time. l8 Others, using similar methods, have
developed more comprehensive measures. 19 For this study, we used the
"severity index" developed by the juvenile courts in Arizona and included in the
AOC file.
The dependent variables to be used in the study proposed (according to
the purposes of the various analyses) were as follows:
1. Any subsequent referral (O=No, 1 =Yes)
2. Seriousness score, Most Serious Charge, Next Referral Episode 20
l7 Thurstone, L.L., "The Method of Paired Comparisons for Social Values," Journal of
Abnormal and Social Psychology, 21 , 1 927, 384-400.
l8 Coombs, C.H., "Thurstone's Measurement of Social Values Revisited, Forty Years
Later," Journal of Personalityand Social Psychology, 6, 1967, 91-92; Krus, J. Sherman,
J.L., and Krus, P.H., "Changing Values Over the Last Half Century: The Story of
Thurstone's Crime Scales," Psychological Reports, 40, 1977, 207-21 1.
19 Rossi, P.H., Waite, E., Bose, C.E., and Berk, R., "The Seriousness of Crime: Normative
Structure and Individual Differences," American Sociological Review, 39, 1974, 224-237;
Sellin, T., and Wolfgang, M.E., The Measurement of Delinquency, New York: Wiley,
1 964. Gottfredson, S.D., Measuring Offense Seriousness: A Dimensional Approach,
Baltimore: Center for Metropolitan Planning and Research, The Johns Hopkins
University, 1981; Gottfredson, S.D., Young, K. and Laufer, W., "interaction and Additivity
in Offense Seriousness Scales," Journal of Resexch in Crime and Delinquency, 17,
1980,26-41; Gottfredson, S.D., and Taylor, R. B., "Community Context and Criminal
Offenders, in Communities and Crime Prevention, edited by T. Hope and M. Shaw,
London: Her Majesty's Stationery Off ice, 1988..
20 The data abstracted from the Administrative Office of the Courts file include data that
may be used to define other outcomes of interest, such as new filings by the County
Attorneys, new Adjudications, new Dispositions, and detailed data on subsequent
offenses alleged, including multiple offenses for the next referral.
Empirical Evaluation of the PIC-ACT
Time at Risk
It was necessary, for all analyses, to control for time at risk of offending.
The exposure to this risk varies among youth according to the dates of initial
referral in the sample studied and also with sanctions imposed and as actually
implemented; it varies as a result of further detention due to repeated offending
during the follow-up period, and with age due to the upper limit of age 18. This
variable was calculated by determining the maximum number of possible days of
exposure to the risk of new referrals in the community after the instant referral,
taking into account the number of days before the youth would reach his or her
18th birthday and the number of days in detention. 21
Independent variables
For each of the research designs intended to answer the "central
questions" posed above, the independent variable is a classification of youths
according to the outcome of a decision by either the probation staff or the county
attorney. These decision outcomes, however, result in groups of youths that may
be classified in several ways and depend also on the subsample of offenders
that are the subjects of study.22 A first analysis was based on the classification
"PIC-ACT vs. Other Court Program Sample vs. Not Eligible Sample." 23 A
21 It should be noted that new offense rates for youths nearing their 1 8th birthdays, when
the juvenile jurisdiction ends, may be underestimated, since only new referrals to the
juvenile courts are counted.
22 Youth in this sample of course may have prior referrals to the juvenile court. They may
have been referred more than once during the period of data collection, however, and
therefore were included in the sample on the basis of the first referral during the period
July 1, 1993 through September 30, 1994.
23 The Administrative Office of the Courts data file did not include these classifications, so it
was necessary to identify the cases to be included for the purposes of the study as "PIC-ACT
study sample cases." The procedure described above, depicted in Figure 1, was
followed for this purpose. It should be noted that not all cases initially treated as PIC-ACT
cases would necessarily be included as PIC-ACT study sample cases by this procedure.
That is, the study group is more accurately defined as "PIC-ACT cases Assigned
Consequences." Considering these cases as including all PIC-ACT cases would be
subject to several possible sources of error. If a youth, considered to be a potential PIC-ACT
case, was instructed to appear for an interview (the "cite-in" interview) but failed to
appear and no action was taken, then the case would not be included as a PIC-ACT
Empirical Evaluation of the PIC-ACT 37
second classified the youths included in the PIC-ACT program according to the
consequence programs listed previously. After examination of the numbers of
cases in each of these categories, however, this classification was modified to
include nine classes, as follows: (a) community service; (b) counseling, (c)
education for delinquency prevention; (d) education for alcohol or drug abuse;
(e) non-residential program of rehabilitation; (f) restitution; (g) fine; (e)other, and
(f) combination. The "other" classification was used because some data on
consequences was recorded without designation of any program otherwise
listed. The "combination" category was used when more than one of the PIC-ACT
designated consequences were used. Other classifications of
consequences, based on multiple assignments, were too small to permit
statistically powerful analyses.
For the various "secondary questions" listed previously, the independent
variables were selected according to the nature of the questions. For the
analysis of decisions by probation (and county attorney) staff as to inclusion in
the PIC-ACT, Other Court Program, and Non Eligible groups and, for the
selection of consequences, selected background variables (i.e., case
characteristics of the youth and of the alleged offense) were the independent
variables. These were combined according to methods described subsequently,
case. If the youth met the PIC-ACT criteria but nevertheless was referred to the County
Attorney with either a petition filed or no subsequent PIC-ACT consequences assigned,
then that youth would not be included as a PIC-ACT case. If the youth, considered to be
PIC-ACT eligible appeared for the interview but was not assigned consequences, then
the youth would not be considered to be a PIC-ACT case. The latter circumstance could
occur if the youth (a) admitted the offense but no consequence was assigned or (b)
denied the offense with the complaint referred to the County Attorney. Very young
children with first time minor misdemeanor complaints may be adjusted as a discretionary
act of the probation officer in some counties; these children would not be included. It
should be noted that cases not recorded in the AOC file used as PIC-ACT cases even
though they meet the legal definition for PIC-ACT eligibility would not be identified in the
AOC file as PIC-ACT cases. Our procedure was intended to identify all the PIC-ACT
cases by the legal definition. In the PIC-ACT sample used for the present study, there
are 1,389 or 13.2 percent of cases not designated in the AOC file as PIC-ACT cases.
The Other Court Program group used in this study includes 16 youths designated in the
file as PIC-ACT cases, but if our procedure is correct, these are cases not eligible for
PIC-ACT.
Empirical Evaluation of the PIC-ACT 38
in order to reduce various sets of variables to weighted linear composites. These
composite variables were used in the analyses of covariance to be described.
Descriptive Variables and Other Data Availability
The juvenile, consequence, and juvenile justice system process variables
available for this study, and for other studies based on the data set prepared, are
extensive. As noted previously, a list has been provided to the Administrative
Office of the Courts, along with the reconstructed data file and codes used. This
report does not contain descriptions of the consequence programs operated by
the various counties, but these are available from the Administrative Office of the
Courts. 24
Analysis Plan
A variety of analytical methods were used to address the variety of
questions posed. Some were addressed by statistical designs intended to
provide tests in which the expected bias due to lack of a true experiment were
removed so far as possible, with the goal that these may be ignored. 25 Others
required methods commonly used in prediction studies. In order to clarify the
nature of the analyses, these will be described in some detail for a few of the
questions; other analyses were similar. 26 The typical problem is one of
comparison of outcomes of different treatments while controlling for selection
24 Juvenile Justice Services Division, PIC-ACT Reviews by County. Phoenix: Arizona
Supreme Court, Administrative Office of the Courts, February, 1995.
25 For a detailed discussion of the issues involved, see Berk, R. A., "Causal Inference as a
Prediction Problem," in Gottfredson, D.M., and Tonry, M., (Eds.), Prediction and
Classification: Criminal Justice Decision Making, Volume 9 of Crime and Justice: A
Review of Research, Chicago: University of Chicago Press, 1987, 183-248.
Gottfredson, S.D., Gottfredson, D.M., and Gottfredson, M.R., "Turning Data Into
Information," Sacramento: Justice Policy Research Corporation, May, 1 994;
Gottfredson, S.D., Gottfredson, D.M., and Gottfredson, M.R., "Risk Measures for
Operational Use: Removing Invidious Predictors," Sacramento: Justice Policy Research
Corporation, May, 1994.
Empirical Evaluation of the PIC-ACT
biases due to the (a) factors associated with the decision, and (b) factors
associated with a priori risk. The latter is an estimate of probability of a given
outcome (e.g., new referrals during a specified period) that may be made on the
basis of information about the alleged offender before the placement is made.
The former can be estimated directly from the data about youth in the various
classifications. These are the two main sources of bias that must be considered
in the analysis if the non-equivalence of the groups in different treatments is to
be ignored safely. When the analysis of the effects of different consequences is
to be done, then it is necessary to consider also the selection of consequences
for different types of youths.
Results of the Statistical Design
Sample Differences in Youths' Characteristics and Counties
Before describing the analyses and results, some description of the
youths in the sample, and in the three groups to be compared for the total
sample, may be helpful. After the exclusions indicated previously (cases
transferred to adult court or the Department of Youth Treatment and
Rehabilitation, status offenders, and youth in administrative categories) from the
total sample, it consisted of 14,939 youths who met the initial legal criteria for
PIC-ACT. There were 1,310 youths whose complaints were known to be referred
forthwith to the County Attorneys. There were 1,673 youths whose complaints
were submitted to the County Attorneys after the cite-in (a notice to appear for
an interview) for PIC-ACT. Of these, 645 were submitted to the County Attorneys
after they failed to admit responsibility for the alleged offense or offenses; and
1,028 cases were referred to the County Attorneys for failure to comply with PIC-ACT
consequences. After completing the classification procedure depicted by
Figure 1, there were 24,677 youths in the total sample to be studied.
Empirical Evaluation of the PIC-ACT
When the total sample was sorted into the three groups to be studied,
there were 10,499 in the PIC-ACT Study Sample, 12,445 in the Other Court
Program Sample, and 1,733 in the Not Eligible Sample (Figure 2). These were
distributed by counties as shown in Table 1 and Figure 3.
Not E..l"i,g ible
Program
50%
-ACT
43%
I I UPIC-ACT =Other Court Program BlNot Eligible I I
Figure 2: Percent of Youths in PIC-ACT Study Sample, Other Court Program
Sample, and Not Eligible Sample
Table 1 : Study Sample Analyzed by County
Empirical Evaluation of the PIC-ACT 4 1
Figure 3: Percent of Youths Included for Study from Each Arizona County
There was a rather marked variation in the proportions included in the
total sample according to the county examined. Moreover, there was
considerable variation in the percents assigned, in the various counties, to the
three study sample groups (Figure 4). As noted previously, the data available did
not permit identification of the Not Eligible group from all counties.
UPICACT UOther Court Program ONot Eligible I
Figure 4: Percent of County Sample Youths in the PIC-ACT Study Sample,
Other Court Program Sample, and Not Eligible Sample
Empirical Evaluation of the PIC-ACT 42
The youths in the three groups vary in terms of characteristics that might
be expected to be related to selection for the groups or to the a priori risk of new
referrals. z7 Most youths in any group are referred by law enforcement agencies
(Figure 5).The average ages are different for the three samples (Figure 6). The
generally younger PIC-ACT youths are more apt to be enrolled in school (Figure
7). Those in the Other Court Program sample have, on the average, more prior
drug complaints (Figure 8). Females are more likely to be placed in PIC-ACT
(Figure 9).
The samples differ according to complaints (Figure 1 O), grade in school,
and ethnic classification (Figures 11 and 12). They differ on various measures of
prior record; generally, the more prior record the less likely the placement in PIC-ACT
(Figure 13).
Figure 5: Numbers of Youths in Three Samples, Analyzed by Source of
Referral
27 Differences reported in this report all are statistically significant by relevant tests at the
one percent level of confidence.
Empirical Evaluation of the PIC-ACT
15.4 =Other Court
15.2
15
14.8
14.6
14.4
14.2
14
10000
4000 "" 1 4 : , 2991 : a ,219 , ,
2000
0
PIGACT Other Court Not Eligible
Program
Female Male
Figure 6: Average Age of Youths
in Three Groups Figure 9: Numbers of Males and
Females in Three Samples
9000
8000
7000
6000
5000
4000
3000
2000
I000
0
PIC-ACT Other Not Eligible
Court
Program
Although the three groups
differ according to youths'
characteristics as illustrated by
these figures, it should be borne
in mind that there is a substantial
overlap among the groups on any
characteristic selected. For
examples: youths with felony
offenses often are found in both
Figure 7: Numbers of Youths in the PIC-ACT and Other Court
Three Samples, Analyzed Program samples, which include
According to School Status both young men and young
women; youths with thefts are
more often placed in PIC-ACT, but
many are found in the Other Court
Program group. Similarly, group
distributions of age, numbers of
prior drug allegations, prior
adjudications, prior petitions,
prior PIC-ACT placements, prior
offenses, and prior probation
violations overlap. There is no
single youth characteristic that
Figure 8: Average Number of distinguishes between the PIC-Prior
Drug Complaints in ACT and Other Court Program
Three Groups samples.
Not Eligible
Other Court 0.1
Program
PIC-ACT
0 0.02 0.04 0.06 0.08 0.1
Empirical Evaluation of the PIC-ACT
- -
4000
3600
3000
2600
2000
1600
1000
600
0
PIC-ACT Other Court Program Not Eligible
OPerson Felony ClO rand Theft OObstruction ClPerson
M lsdemeanor
ODrugs OPublic Peace UTheft =Other
Figure 10: Numbers of Youths in Three Groups,
Analyzed by Most Serious Complaint
4500
4000
3500
3000
2500
2000
1500
1000
500
0
PIC-ACT Other Court Program Not Eligible
Figure 11 : Numbers of Youths in Three Samples, Analyzed by Grade in
School
7000
6000
5000 Hispanic
4000 Gi African American
3000 White
2000 tl Native American
1000 tl Asian I Pacific Islands
0 Other
PIC-ACT Other Court Not Eligible
Program
Figure 12: Numbers of Youths in Three Samples, Analyzed by Ethnic
Classification
0.8
0.7 B o t h e r Court
0.6
0.5
0.4
0.3
0.2
0 .I
0
45
Average Number of Prior
Adjudications in Three Groups
=Other Court
1
=Other Court
0.8
0.6
=Not Eligible
0.4
0.2
0
0.5
0.4 =Other Court
0.3
0.2
0 .I
0
I I
Average Number of Prior Petitions
in Three Groups
Average Number of Prior Drug Average Number of Prior Person
Allegations in Three Groups Offenses in Three Groups
0.4
0.35 BOther Court
0.3
0.25
0.2
0.15
0 .I
0.05
0
I I
Average Number of Prior PIC-ACT
Consequences in Three Groups
1.2
1 =Other Court
0.8
0.6
0.4
0.2
0
Average Number of Prior Property
Offenses in Three Groups
2.5
BOther Court
2
I .5 l N o t Eligible
I
0.5
0
C
Average Number of Prior Counts
in Three Groups
0.35
0.3 mother Court
0.25
0.2
0.15
0.1
0.05
0
I I
Average Number of Prior Probation
Violations in Three Groups
Figure 13: Comparisons of
Youths' Characteristics in Three
Samples
Empirical Evaluation of the PIC-ACT
Does classification as a PIC-ACT
case, with consequences
assigned, affect outcomes? This first
"central question" was the question
for the first statistical design. The
dependent variable was the
dichotomous attribute "new referral
or not." The independent variable is
a classification variable: the PIC-ACT
Study Sample, the Other Court
Program Sample, and the Not
Eligible Sample. Thus, we wish to
compare the youths in the three
samples in terms of whether or not
they had new referrals to the juvenile
courts during the period of follow up
study. Two different procedures
were used to control for selection in
order that fair comparisons of the
new referral (and subsequently, the
offense seriousness) outcomes can
be made.
Before considering those
analyses, we may examine the
observed --- that is, the actual ---
differences in new referrals, with no
adjustment for bias due to a priori
risk or to selection.
A Naive Answer to Question I
A naive answer to Question 1,
whether it makes any difference, for
later offending, if the juvenile is
selected as a PIC-ACT case, with
consequences assigned, is provided
by Figure 14. This Figure (along with
Figure 15) summarizes the
occurrence of the new referral
criterion by the group classification
(PIC-ACT, Other Court Program,
and Not Eligible Samples).
The overall proportion of New
Referrals (the percent "failures"),
regardless of the group classifica-tion,
was .46, which also was the
percent observed for the PIC-ACT
sample. There were 47 percent
"failures" (new referrals) for the
Other Court Program sample and 45
percent for the Not Eligible sample.
The differences were not statistically
significant, and it must be concluded
on the basis of these data alone that
the groups do not differ in "failure,"
defined by new referrals.
4 7
4 7
46.5
4 6
45.5
4 5
44.5
44
OPIC-ACT =Other Court EYNot Eligible
Program
Figure 14: Percents with New
Referrals, by Study Sample
7000 6650
6000
5000
4000
3000
2000
1000
0
No New New
Referral Referral
OPIC-ACT =Other Court =Not Eligible
Program
Figure 15: Numbers of Youths
with and without New Referrals
Empirical Evaluation of the PIC-ACT 47
The Need for Statistical Controls
It has been seen, from figures presented previously, that the three sample
groups differ quite markedly in various youth characteristics plausibly related to
the a priori risk of new referrals --- that is, in the risk of new referrals that is
presented by the youth at the time of referral. The differences shown can not
be regarded as providing evidence of PIC-ACT effectiveness.
The naive interpretations of observed differences in new referral rates by
the study groups may well be biased by (a) factors associated with the decisions
that result in placement of youths in the three different groups and (b) factors
associated with a prioricharacteristics of the youths that also are associated with
the outcome measures. In the present study, the latter are most likely to be
represented by a priori differences in at-risk characteristics that are not randomly
distributed across the three groups. The former can be estimated directly from
the placement decisions.
A Priori Risk
Risk, as measured by the juvenile courts in Arizona, often is measured by
an instrument for the assessment of the probability of new referrals within a six
or twelve month period. The risk assessment device used requires an
assessment of such variables as the total number of prior referrals; the total
number of petitions; parental concern; parental cooperation; parental
supervision; adults in the home who have a drinking or drug problem; whether
there are children in the home who have a drinking or drug problem; and the
child's age. Unfortunately, neither the risk score nor most of the items needed for
its calculation are available for the vast majority of cases represented in the
Administrative Office of the Courts' data file. Thus, the risk scores used by the
various counties could not be used for the present purposes. And, the calculation
of a new risk measure on the basis of the present sample, including youths from
Empirical Evaluation of the PIC-ACT 48
all counties, could be expected to provide a better measure for the statistical
control needed in this study.
Development of a Risk Measure for the Present Study
Since the risk measures used by the various counties were not available
for most cases, it was necessary to develop one for the present study. The
ordinary least squares regression method was used, even though the outcome
criterion (new referrals) is dichotomous. This was done for three reasons. First,
the outcome distribution is not extreme, and ordinary least squares models are
remarkably robust under this condition. Second, the theoretically more
appropriate logistic models are interpreted only with considerable difficulty by
most decision makers, and the ready communication of results of the study is of
much concern. Third, our analyses using logistic models with closely comparable
problems has indicated no substantive difference from those developed using
ordinary least squares regression. 28
The a priori risk model developed for the new referral outcome criterion is
summarized in Table 2. It contains predictor variables typically observed in such
models, such as indices of age, prior record, and type of offense. The observed
value of the multiple correlation coefficient is well within the range of power
typically observed for risk instruments; its square shows that about 11 -5 percent
of the variation in new referrals is accounted for by the variables listed, in
com bination.
In the context of the variables listed in the table, the best predictor of new
referrals are the number of prior counts, the age at the instant referral, and the
number of prior referrals. Age is inversely related to new referrals; that is, as
typically is found, older youth tend to have fewer new referrals. More prior
record, measured by prior counts and petitions, is associated with new referrals.
28 Note 26, supra.
Empirical Evaluation of the PIC-ACT 49
Table 2: Regression of New Referral Criterion on Various Predictors
Note: R = .338; F ,,5,,M,, = 210.95, p < .0001
As illustration of the variability in risk presented when different groups are
compared, consider the average risk scores, calculated according to the
equation described by Table 2, for the three study samples. Figure 16 shows
that the youths in the study samples differed on the average, in terms of their
risk scores. The "worstJ' risks are found in the Other Court Program sample, the
"best" risks are those youths in the Not Eligible sample, and the cases in the
t
5.00
-7.85
2.00
4.70
20.18
-2.69
-3.86
2.94
7.63
2.35
-3.96
-13.76
5.78
5.97
-1 1.00
29.86
Beta
,031
-.053
.014
.I 16
.I45
-.017
-.033
.023
.I68
.034
-.041
-.I64
.070
.036
-.067
Predictor
Number of Counts, Instant Referral
Seriousness, Most Ser. Inst. Offense
Class (Felony, Misdemeanor)
Number of Prior Referrals
Seriousness, Most Ser. Prior Offense
Number of Prior Drug Allegations
Number of Prior Probation Violations
Number of Prior Person Offenses
Number of Prior Counts
Number of Prior Petitions
Number of Prior Detentions
Age at Instant Referral (Years)
Age at First Referral (Years)
Race (White vs. Non-White)
Sex (Male vs. Female)
Constant
P
.OOO 1
.0001
.0460
.OOO 1
.0001
.0076
.0001
-0033
.0001
.0189
.OOO 1
.0001
.0001
.0001
.OOO 1
.0001
B
.015
-.013
,009
.026
.029
-.024
-.018
.013
.032
.012
-.019
-.041
.016
.036
-.077
.761
Empirical Evaluation of the PIC-ACT 50
PIC-ACT sample are in between. In examining Figure 16, it may be remembered
that the overall percent of new referrals was 46.
49
50
48
46
44
42
40
38
36
CI PIC-ACT lother Court Not Eligible
Program
Figure 16: Average A Priori Risk Scores, by PIC-ACT, Other Court Program,
and Not Eligible Samples
A Model for Selection for PIC-ACT, Other Court Program, and Not-Eligible
Samples
Since probation officers and County Attorneys do not make decisions at
random, and since it may be presumed that they attempt to take likely outcomes
into account as they make placement decisions, it is important that these
selection factors also be controlled if the comparisons we seek are to be made
without systematic bias. To model these decisions, we used Fisher's multiple
discriminant function analysis. Given a nominal classification as a dependent
variable, the discriminant function seeks that linear combination of independent
variables that maximizes the between to within groups variance ratio. It is
possible to extract one less function than the number of groups to be predicted
(or independent variables, whichever is fewer), subject only to the constraint that
each be orthogonal to (uncorrelated with) the rest. This means that we wish to
identify the linear equations using these variables, and their weights, that best
separate the distributions of scores for these equations for the sample groups.
Empirical Evaluation of the PIC-ACT 5 1
The definition of "best" is that the distributions are separated as much as
possible, in relation to their variabilities.
For the present problem the task is to define the expected two functions
(equations) that best indicate the variables that distinguish the groups PIC-ACT,
Other Court Program, and Not Eligible, and their weights. The analysis
proceeded under an assumption of equal a priorigroup sizes (a very
conservative assumption), and two functions were extracted. Each is significantly
associated with the group classifications: The canonical correlations
summarizing these associations are .32 for the first function and .04 for the
second. The variables important for the classification into the three study groups
are indicated in Table 3, which summarizes the standardized discriminant
function coefficients. These coefficients are the weights to be applied to the
variables (in standardized form, that is, with equal means and variances) in order
to obtain the most efficient classification.
Table 3: Standardized Discriminant Function Coefficients, Sample Group
Assignments
Function 1 accounts for 98 percent of the accounted for variance, but both
functions nevertheless were used in the analyses to be described. The youths'
Independent Variable
Seriousness, Most Serious Instant Offense
Number of Prior Referrals
Seriousness of most serious prior offense
Drug Abuse at First Referral?
Number of Prior Adjudications
Number of Prior Counts
Prior Non-Court Dispositions
Detained on Date of Instant Referral?
Total Days Prior Detentions
Days Detained, Instant Referral
Age at Instant Referral (Years)
Race (White vs. Non-White
Function I 1 Function 2
-. 131
.588
-.229
-.082
-.I54
.495
-.321
.237
.I12
.088
.339
130
-. 489
-.795
.598
.279
-21 4
.452
.328
-. 460
-.312
163
182
.I 19
Empirical Evaluation of the PIC-ACT 52
characteristics most helpful in understanding the classification into the PIC-ACT
and other groups are the numbers of prior referrals and counts, age, prior non-court
dispositions, whether detained on the day of the instant referral, and the
seriousness of the most serious prior offense. The two functions correctly
classify youth into the three study groups in 42 percent of the cases (with no
assumptions about the prior probabilities of the classifications, that is, about
group sizes, which approach provides a quite conservative measure of the
correct classifications).
In order to examine whether counties differ in terms of the average scores
on these functions, the mean scores on the first and second selection functions
were calculated for each county. The counties do differ in terms of the kinds of
youth received, in terms of the variables included in each equation. That is, the
mixtures of cases in terms of offenses, prior records, age, and prior history in the
juvenile court, variables that help explain the assignments to the PIC-ACT and
other groups, differ from county to county.
Overview of Analyses with Statistical Controls
In the previous sections of this report we have identified first, the best
predictors of the outcome "New Referrals," and second, the best predictors of
the classification decisions placing the youths in the total sample into one of the
three groups to be compared. We have observed also that there is no difference
in the percents with new referrals for the three study groups. We now are in a
position to examine the new referral criterion for the three study samples while
controlling simultaneously for the factors identified in these first steps.
The statistical method used is the analysis of covariance. Analyses were
conducted separately for the two dependent variables of interest --- new referrals
and the seriousness rating of offenses alleged when new referrals occurred. For
each analysis, the independent variables of interest are the classification (PIC-ACT,
Other Court Program, Not Eligible) and the county of origin. In some cases,
Empirical Evaluation of the PIC-ACT
the interaction of these (that is, case classification within county) also is of
interest. For convenience, we will first report the results with the new referral
criterion, then results concerning the seriousness of new referrals. Each youth in
the total sample was assigned predicted risk scores and also scores on the two
discriminant functions developed to model the classification decisions. These,
plus the time at risk, now may be used as covariates in the analyses.
A conservative hierarchical approach was taken, with covariates entered
first. Essentially, this leaves only residual variation in the outcome criteria to be
explained by the type of classification. That is, for example, it enables us to
estimate the effect of the classification into PIC-ACT on the other classifications
of the behavioral criteria of interest --- new referrals or seriousness of new
offense allegations --- independent of the effects of a priori risk, of classification
selection, and of time at risk (or, indeed of county) factors.
Effects of the Study Sample Classifications on New Referrals
A summary of the analysis of covariance in the New Referral criterion is
given in Appendix A. The purpose of this analysis is to determine whether there
is an effect on new referrals of the group classifications (PIC-ACT, Other Court
Program, or Not Eligible samples), of counties, and of the interaction of the
group classification by county. The analysis controls for (takes into account) the
variables time at risk, selection functions 1 and 2, and a priori risk and tests for
the effects of study group, county, and the interaction of study group by county.
In making these tests, the effects of the variables controlled is first subtracted
out.
Empirical Evaluation of the PIC-ACT
The nature of the analysis may be summarized as follows:
I Dependent Variable: New Referrals
I Classification Variables (Independent Variables):
0 Classification of Youths as PIC-ACT, Other Court Program and
Not-Eligible Samples
0 County
C] Interaction of classification and county
I Variables Controlled (Covariates):
0 Time at Risk
0 Selection for Youth Classification (two linear combinations of
independent variables explaining selection)
0 A Priori Risk (a linear combination of independent variables
explaining the probability of new referrals on the basis of
information known at the time of referral)
The results show that there are statistically significant effects on new
referrals of: (1) classification as a PIC-ACT, Other Court Program, or Not Eligible
youth; (2) County; and the interaction of study group by county. The latter effect
means that in addition to the effects of study group and county there is an
independent effect of the combination of county and classification group.
Recall that there were no differences in the new referral percentages
when these were examined for the three groups compared (Figure 14) and that
the observed (actual) percent of new referrals for the total sample was 46
percent. We now can examine adjusted outcomes for the three groups, with that
adjustment, based on the analysis just described, taking account of the effects of
risk, selection, and time at risk. The adjusted percents are depicted in Figure 17.
Empirical Evaluation of the PIC-ACT 55
56
60
50
40
30
20
10
0
PIC-ACT Other Court Not Eligible
Program
Figure 17: Percents with New Referrals, Adjusted for Risk, Selection, and
Time at Risk
If the new referrals for the three groups are to be compared, Figure
17 should be used rather than the unadjusted data of Figure 14. It may be
concluded that the three groups differ as shown on the new referral criterion,
when known effects of time at risk, risk of new referrals known at the outset, and
selection are taken into account for the comparison. The adjusted new referral
rates are highest for the Not Eligible sample, lowest for the Other Court Program
sample, and in between for the PIC-ACT sample.
Effects of Consequences on New Referrals
For the analysis of the effects of the various consequences, it is
necessary to control not only for time at risk and a priori risk, but also for the
selection for the specific consequence assigned. Accordingly, a discriminant
function analysis was completed to define measures to be used for the required
statistical controls for selection. The results are summarized in Table 4 and ,
more completely, in Appendix 9.
Empirical Evaluation of the PIC-ACT
Table 4: Standardized Discriminant Function Coefficients, Consequence
Assignments, Functions 1 and 2
The first two discriminant functions do most of the work of classifying the
youths into the consequences categories --- that is, of modeling the decisions of
the probation officers. The first four (shown in Appendix B) nevertheless were
used in the analysis of covariance. As can be seen from the variables included in
these functions, the selection for type of consequence seems to be related to the
seriousness of the offense, the history of drug abuse (whether the youth was first
referred to the court with an allegation of drug abuse), the number of prior counts
and petitions, and other case characteristics.
Independent Variable
Number of accomplices
Number of counts
Offense seriousness
Felony or misdemeanor
Drug abuse at first referral
Number of prior drugs
Number of prior adjudications
Number of prior counts
Number of prior petitions
Detained at first referral?
Days detained, instant referral
Age at instant referral
White vs. non-white
Male vs. female
The analysis of covariance is summarized in Appendix C. Within the PIC-ACT
study sample, consequences were coded according to nine categories:
community service; counseling; education for delinquency prevention; education
for alcohol and drug abuse; non-residential programs for rehabilitation,
restitution, fines, "others" and combinations. The observed and adjusted
percents with new referrals (from the adjusted means) by category of
consequence, are shown in Figure 18.
Function I 1 Function 2
-.I36
-.I34
.602
.093
.371
-.073
182
-.274
-.225
-.I87
-. 092
1 44
.011
154
-. 004
-. 065
-1 18
,324
-.772
.215
.003
-.046
-.092
.054
-. 045
-. 103
.313
130
Empirical Evaluation of the PIC-ACT 57
70
60
50
40
30
20
10
0
Observed Adjusted
ClCommunity Service lcounseling OEducation for OEducation for drug or ONon-residential
delinquency prevention alcohol abuse Treatment
CiRestitution IIFine Mother DCom bination
Figure 18: Actual and Adjusted Percents with New Referrals for Consequence
Groups, with Adjustments for Time at Risk, A Priori Risk, and Selection
When the adjustments were made for the covariates, the variability in
rates of new referrals was less marked than that for the actual (observed) rates.
Nevertheless, the data show that the type of consequence does make a
difference, with education for drug or alcohol abuse, non-residential treatment,
fines, and the "combination" category faring best. There is no support in the
figure for restitution, which has the highest rate of new referrals.
The Administrative Office of the Courts provided these summary program
descriptions of the education for drug or alcohol use and the non-residential
treatment programs:
Delinquency Prevention (PIC-ACT Programs)
This service is for juveniles who have a specific
educational need pertaining directly to the reason for
their referral. This service is usually provided as an
educational program in either singular or multiple
episodes which build upon one another. It may
include other outpatient counseling services. The
service intent is to educate the client by providing
necessary skills, tools, and knowledge which can be
utilized to make responsible choices and to
Empirical Evaluation of the PIC-ACT
discontinue behaviors that instigated court
involvement.
Evening Support Service
This service provides a minimum of 3 hours
(excluding meals and transportation) of supplemental
services to youth who may attend daytime school.
Services often include supplemental education,
tutoring, GED study, pre-vocational andior vocational
instruction, individual living skills developments,
general counseling activities, substance abuse
counseling, social and/or recreational activities.
Structure and supervision may be moderate to
intensive with flexibility to accommodate changes in
individual needs. Programming may take placer at a
provider location, andlor in various community
locales.
Variation in Types of Consequences Used and in Compliance
There is substantial variation in the frequency of use of the various
consequence types reported for each youth in the Administrative Office of the
Courts data file. For the State as a whole, Community Service is most often
assigned (Figure 19). Education for Delinquency Prevention programs often are
reported also, as is non-residential treatment. Restitution is not often used, and
reported combinations (multiple consequence assignments) are rare. Not all
consequence types are used by all counties (Figure 20).
These data, which rely on the reporting on each case to the
Administrative Office of the Courts, may not adequately portray the use of the
various consequences. For more detail, the reader should consult the AOC
report for counties cited earlier. 29 For example, Santa Cruz County, with
consequences reported only as "other," includes community service as
mandatory within an informal probation program. Similarly, Pinal County includes
community service in a work service program.
29 Note 5, supra.
Empirical Evaluation of the PIC-ACT 59
18
20
15
10
5
0
Community Counseling Education for Education for Non-residential
Service Delinquency Drug or Alcohol Treatment
Prevention Abuse
0 Restitution Fine Other Combination
Figure 19: Percents Assigned to Consequence Groups (Combined Samples)
Yuma
Yavapai
Santa Cruz
Pinal
Pima
Navaho
Mohave
Maricopa
La Paz
Greenlee
Graham
Gila
Caconino
Cochise
Apache
0 10 20 30 40 50 60 70
I7 Community Counseling I7 Education for Cl Education for I7 Non-residential
Service Delinquency Alcohol or Drug Treatment
Prevention Abuse
I3 Restitution Fine Other Cl Combination
Figure 20: Percents Assigned Consequences of Various Types, by County
Empirical Evaluation of the PIC-ACT
There is substantial variation also in compliance with consequences
according to the type of consequence. This is illustrated by Figure 21, which
shows the percent compliance with each of the types of consequence.
Compliance was most frequent for education for alcohol or drug abuse
and non-residential treatment; it was lower for restitution and counseling.
0 10 2D J3 43 Eu 63 m m 90 100
OCanrurty ICanseliq IIEduc;timfa ClEd~~4imfa ClNaFresiMid
Ser\nce Deiincluency AlcohdaCXLlg Tr-
M i m Abuse
ORestiidim lFine lotha- ICart)i~m
Figure 21: Percent Compliance for Types of Consequences
Effect of Compliance with Consequences on New Referrals
Within the PIC-ACT study group, we may ask whether compliance with
assigned consequences makes any difference with respect to the new referrals
outcome. At the same time, we may ask whether this effect, if any, varies
according to the county reporting. The analysis summarized in Appendix D
answers those questions after controlling for time at risk, a priori risk, and
selection. It shows that there is a county effect, but, independently of county,
compliance affects the percents with new referrals, adjusted for differences in
the variables controlled. Appendix D shows also that the interaction of
Empirical Evaluation of the PIC-ACT 6 1
compliance by county was not statistically significant (at the one percent level of
confidence), indicating that the effect of compliance does not depend upon the
county considered. The effect of compliance is illustrated in Figure 22.
Figure 22: Adjusted Percents of PIC-ACT Youths with New Referrals After
Compliance or Non-compliance with Assigned Consequences (Adjusted for Time
at Risk, A Priori Risk, Selection of Consequences, and Independent of County)
It may be concluded that within the PIC-ACT sample compliance affects
the new referral outcomes: the adjusted percents with new referrals are notably
higher for youths who did not comply with PIC-ACT consequence requirements.
Effects of PIC-ACT, Other Court Program, and Not Eligible Classifications
on Seriousness of New Offenses
Similar analyses were completed for the outcome measure of the
seriousness of new offenses alleged, given a new referral. The types of offenses
scored variously in this classification procedure are illustrated in Table 5, which
shows the score categories, the abbreviated label used by the Administrative
Office of the Courts, and examples of offenses included. (The complete listing is
Empirical Evaluation of the PIC-ACT 62
available from the Administrative Office of the Courts.) For this analysis, of
course, only the youths with new referrals were included for the study.
Table 5: Examples of Seriousness Offense Scoring
Value
1
2
3
4
5
6
7
Label
Violent (Felony
Against Person)
Grand Theft (Felony
Against Property)
Obstruction
(Hindering Justice)
Fight (Misdemeanor
Against Person)
Drugs (Possession
or sale)
Peace (Disturbing
the Peace)
Theft (Misdemeanor
Against Property)
Examples of Offenses
Aggravated assault, arson (occupied
structure) Murder, manslaughter, kidnapping,
robbery, sexual assault
Forgery, burglary, fraud, car theft, purse
snatching (no force), arson
Escape, attempts and conspiracies to commit
crimes, obstructing justice, solicitation,
tampering, resisting arrest
Assault, endangerment, threat, domestic
violence, unlawful imprisonment
Possession, use, sale, manufacture of
Narcotic drugs, controlled substances
Attempted carrying concealed weapon,
disorderly conduct, reckless driving,
trespassing, contributing to delinquency,
cruelty to animals, driving under the influence
of drugs or liquor, speeding, failure to
appear, gambling, loitering, pandering,
pimping, illegal weapon use
Attempted theft or fraud or criminal damage,
petty criminal damage, fraudulent use of
credit card, shoplifting, malicious mischief
Empirical Evaluation of the PIC-ACT 63
This provides an ordinal scale, with higher numbers indicating generally
less serious offenses. The numbers of youths in each of the three groups of the
total sample are reported in Table 6. The percents with a given offense
allegation at the next referral are depicted in Figure 23.
Table 6: Numbers of Youths in Offense Seriousness Outcome Categories
(Most Serious Offense at Next Referral) for Total Sample, by Three Study
Samples
The distributions of the three samples are significantly different. 30
Category
No new referral
Violent
Grand Theft
Obstruction
7
Fight
Drugs
Peace
Theft
Status
Hold
Total
30 Each study sample distribution was compared with each other study sample distribution
by the (non-parametric) Kolmogorov-Smirnov two sample test, which provides a test of
the hypothesis that two cumulative step distributions are drawn from a common
population. In each case, the differences are statistically significant.
PIC-ACT
5,734
220
61 3
420
480
384
608
830
1,197
13
10,499
Other Court
Program
6,720
353
899
677
599
405
755
745
1,259
33
12,445
Not Eligible
964
37
104
27
90
63
115
150
182
1
17,33
Total for
Category
1,341 8
61 0
161 6
1,124
1,169
852
1478
1,725
2,638
47
24,677
Empirical Evaluation of the PIC-ACT 64
Before describing the results of the analyses of variance and covariance
of the seriousness criterion, some indicants of other outcomes often used by the
courts, according to the group classifications, may be described. It should be
noted that none of these comparisons include statistical controls such as used
for the analyses of new referrals and the seriousness criterion. Thus, these
comparisons are "naive" in the sense used in this report. Nevertheless, it may be
of some interest for administrative purposes to note the actual (unadjusted)
percents of cases, for the study groups, for these other outcomes.
The youths in the three study samples differ in their outcomes when the
latter are defined by the seriousness of offense categories or by felony or
misdemeanor class. Figure 23 shows the percents in each of the offense
seriousness categories, according to the study group classification. The most
common new offense classification according to the seriousness ranking, for any
study sample, is that of status offense. Theft is the most popular misdemeanor
or felony offense. Cases classed as violent (felonies against persons) are
fortunately the most uncommon. When new offenses at the next referral are
classified according to legal and administrative classifications rather than the
seriousness groups, misdemeanor offenses are the most common (Figure 24).
The percents of each of the three study groups whose cases were
adjusted are shown in Figure 25. Adjustments are much more common in the
PIC-ACT sample, with 71 percent of youth eventually having their cases
adjusted within the time frame of this study.
The percents with new referrals resulting in the filing of a petition is a
criterion sometimes considered a measure of "new serious offenses," since this
requires an examination of the offense alleged and its circumstances not only by
the court staff but also by the county attorneys. The PIC-ACT cases had the
lowest percent (15 percent) with new filings (Figure 26).
Empirical Evaluation of the PIC-ACT
12
10
8
6
4
2
0
Vidert QardM CBbwtion Fight WS Peace mEft Sfims
PlGPXrr B Other Cart Rogm 01 Not Higble
Figure 23: Percents with New Offenses Alleged at New Referral, PIC-ACT,
Other Court Program, and Not Eligible Samples
Figure 24: Percents with Felony and Misdemeanor Complaints at New
Referral, PIC-ACT, Other Court Program and Not Eligible Samples
Empirical Evaluation of the PIC-ACT
100
80
60
40
20
0
Not Adjusted Adjusted
PICACT I Other Court Program Not Eligible
Figure 25: Percents of Youths in Three Study Groups with Cases Adjusted
and Not Adjusted
I
Nopetitionfiled M o n Filed
CI ACKT S other Cart m a n Not Eligible
Figure 26: Percents of Youths in Three Study Samples Who Had Petitions
Filed as a Result of the New Referral
An analysis of covariance analogous to those reported previously 31 was
done. Figure 27 shows the actual and adjusted average seriousness scores for
the PIC-ACT, Other Court Program, and Not Eligible samples for those
31 See the discussion of limitations in a subsequent section; improved scaling of this
measure is desirable. Although the analysis of variance methods used are generally
appropriate only for interval scales --- i.e., those for which the distances between
numbers may be considered equal --- the method is relatively robust with scales such as
this. That the distributions cannot be considered to have been drawn from a common
population has been determined by a non-parametric test, as previously noted.
Empirical Evaluation of the PIC-ACT 67
individuals who had new referrals. In examining the figure, remember that a
higher score means "less serious." The observed (actual) seriousness scores
are statistically significantly different among the three groups, suggesting the
naive interpretation that the PIC-ACT, Other Court Program, and Not Eligible
classifications affects the seriousness of new offenses when they occur. The
effect of the sample group classifications, however, is not statistically significant
after the inclusion of the variables controlled in the analysis of covariance. 32 The
adjusted means, after controlling for time at risk, selection functions 1 and 2, and
a prioririsk, are shown at the right, but the differences shown for the actual and
adjusted means should be considered as expected by chance about six percent
of the time in repetitions of this study with new samples. The county effect was
significant, but the interaction of study group by county was not. The analysis
summary table is Appendix D.
56 5.5
54
52
5
48
46
44
Achral M-PICACT
81 Other Court Frogran 13 Not Eligible 1
Figure 27: Actual (Observed) and Adjusted Mean Seriousness Scores for PIC-ACT,
Other Court Program, and Not Eligible Samples, with Means Adjusted for
Time at Risk, Selection, and Risk (Showing Adjusted Means Not Statistically
Significant)
32 The one percent level of confidence was assumed for all analyses reported. In this case,
the value of F for study group, with 2 degrees of freedom, is 2.85 (P = .06).
Empirical Evaluation of the PIC-ACT 68
Effects of Consequences on Seriousness of New Offenses
The results of a similar analysis of the variation in the seriousness scores
associated with consequences are shown in Appendix E, and a comparison of
observed and adjusted means is given in Figure 28. The effect of type of
consequence assigned, after controlling for time at risk, a priori risk, and the
selection functions (only selection function 4 is significant) is not statistically
significant at the one percent level of confidence, so it is concluded that the type
of consequence assigned makes no difference to this outcome as measured.
The differences observed would be expected by chance about two percent of the
time in replications of the study.
UCamurtySeuoe BCansding Ofa~Aicnfa UEdKdicnfa ONcnresiMd
Wimpmy AlcGdcrD~g Tlx~hxft
R#Rlticn Atxse
RgtiNian l Fine 4Ctt-l~ Mcu~hnaticn
Figure 28: Actual and Adjusted Mean Scores for Seriousness Criterion for
Consequence Assignment Groups, with Adjustments for Time at Risk, A Priori
Risk, and Selection, Showing No Significant Effect of Type of Consequence
Empirical Evaluation of the PIC-ACT 69
Effects of Compliance with Consequences on Seriousness of New
Offenses
Whether the youth complies with the consequences assigned, however,
does affect the level of seriousness of new offenses alleged when new referrals
occur. The analysis of covariance summarized in Appendix G shows that, after
controlling for time at risk, a priori risk, the four selection functions describing
variables explaining the selection for specific consequence programs, there is a
significant effect for compliance. There also is a significant effect for county, but
not for the interaction of compliance by county. The effect of compliance, after
taking the covariates into account, is independent of the county effe