|
|
|
TEST REVIEW: Substance Abuse Subtle Screening Inventory – Adolescent 2 (SASSI-A2) |
|
|
|
|||||||||
|
|
Substance Abuse Subtle Screening Inventory –
A. Title: Substance Abuse Subtle Screening Inventory – Adolescent 2 (SASSI-A2) B. Author: Glenn A. Miller, Ph.D. C. Publisher: The SASSI Institute, 201 Camelot Lane, Springville, IN 47462 D. Forms, groups to which applicable: Single form available in paper-and-pencil format as well as three types of computer administration and online administration. For use with adolescents, ages 12 – 18. E. Practical Features: The SASSI-A2 is a revision of the SASSI-A published in 1990. It is quick and easy to administer and score. Reading level required is 4.4; audiotape is available. The double-sided single-page inventory consists of 72 true-false items, 32 of which are new to this version, and 28 questions to which the client reports the frequency of various experiences and consequences of substance misuse. Instructions are minimal for true-false items, and easy to follow for frequency items. F. General Type: Screening for adolescent substance abuse G. Date of Publication: 2001, second edition H. Costs, booklets, answer sheets, scoring: Manual $45.00. User’s Guide $30.00. Scoring Key $10.00. Audiotape of test $20.00. Starter Kit (manual, user’s guide, scoring key, and 25 paper tests and profile sheets) $115.00. The costs of paper test sets (inventory and profile sheets) vary by quantity ordered and range from $1.80 per test for a package of 25 to $1.10 per test for 25 packages of 100. Computer Starter Kit for Type 1 (client completes test at computer) or Type 2 (client completes paper version and clinician enters responses) is $215.00 for 25 administrations and brief interpretations. A deluxe computer starter kit is available which includes a 1 hour 50 minute training videotape presenting interpretations of test results and feedback sessions with 2 adult and 2 adolescent clients. Materials are also available in sets combined with adult materials. I. Time required to administer: Approximately 15 minutes. J. Purpose for which evaluated: For screening of adolescents in a variety of settings: schools, mental health, and juvenile justice settings. II. Purpose and Nature of the Instrument A. Stated purpose: The SASSI A-2 is designed to discriminate between adolescents who have a high probability of having a substance use disorder (abuse or dependence) and those with a low probability. In addition, the profile can be used to generate clinical hypotheses about the respondent. The SASSI’s unique contribution has been to detect substance abuse problems even when the respondent denies or attempts to conceal such problems (Miller & Lazowski, 2001). B. Description of test, items, and scoring: The true-false items contain statements reflecting symptoms of substance misuse, risk for substance misuse, attitude and beliefs related to substance use, and subtle items which are not obviously related to substance use. The examiner checks one of four time frames from which the client responds to the frequency items. Tests are scored by use of a template, and a profile sheet is prepared from the results. The scorer answers a series of questions about the obtained scores (Decision Rules), which lead to a classification of the respondent as High Probability or Low Probability of having a substance use disorder. In addition to the overall classification, subscale scores are: Face Valid Alcohol, Face Valid Other Drugs, Family-Friends Risk, Attitudes, Symptoms, Obvious Attributes, Subtle Attributes, Defensiveness, Correctional, and Supplemental Addiction Measure. There is a Validity Check to detect random responding, and a Secondary Classification scale used to distinguish between substance abuse and substance dependence. Subscale scores can be converted to precise T scores using an Appendix in the manual. C. Use in Counseling: The SASSI-A2 can be used by counselors in school, mental health facilities, and juvenile justice programs as a screening inventory to determine if an adolescent is in need of further, more in-depth assessment of substance use disorders (User’s Guide, SASSI Institute, 2001). III. Practical Evaluation A. Usefulness of the Manual: The SASSI-A2 has both a Manual and a User’s Guide. The manual provides an overview of the instrument, administration and scoring instructions, information regarding clinical considerations, and validity data. The readability of the manual is greatly improved over the first edition. The User’s Manual is a “how-to” guide for administration and scoring. It includes sample profiles, which are scored and interpreted. For a novice user, this Guide is essential and is easy to follow. B. Adequacy of
directions; training required to administer: Individuals with minimal
training in assessment can easily administer the SASSI-2A to adolescents,
individually or in a group setting. Publisher’s requirements state
that the inventory may be administered by human service practitioners
whose training includes assessment (nature of the training is not specified),
or by those who have completed an authorized SASSI training. IV. Technical Considerations A. Normative Sample: The test was developed on a sample of 1,470 respondents. This group was further divided into development and validation groups. The decision rules for determining level of risk for developing substance abuse problems were based on a subsample of 1,244 respondents. 63.6% were from juvenile corrections programs, 21.6% were psychiatric inpatients, 11.6% were from outpatient behavioral health facilities, and only 3.2% were from addictions treatment centers. The sample was 75% male and 25% female. Ethnic group membership was as follows: 60.7% Caucasian, 12.7% Hispanic American, 10.8% African American, 9.6% Native American, 2.1% Asian American, and 4% Other or unknown. Age of the sample ranged from 12 – 18, with 76.2% being 15 – 17. B. Reliability: Test-retest and internal consistency data are presented by the publisher. A test-retest coefficient of .89 is reported based on a subsample of 70 respondents re-tested after a two-week interval. The method of selecting this subsample is not specified, and no demographic information is presented. As the SASSI-A2 does not yield an overall score, test-retest reliability makes sense only for subscale scores. Coefficients for the subscales range from .81 to .92, quite acceptable given the caveats above. Further evidence for the stability of results is given by stating the high number of cases (94%) in which the decision rule classification did not change between administrations. This consistency is misleading, as there are nine decision rules, and a “yes” on any one of them results in a decision of “high probability.” The number of decision rules answered “yes” could vary considerably from one administration to the next without changing the outcome. Coefficient alpha for the overall inventory was reported to be .75, based on a sample of 1,245 participants. This level is considered fair (Del Boca & Brown, 1996). The manual does not specify how this level was computed. The alpha coefficients are below .70 for four of the nine subscales used in the decision rules and the supplemental Correctional scale, which is generally considered unacceptably low for this type of measure. When important clinical decisions are based on test results, a higher reliability (at least .80) is desirable (Carmines & Zeller, 1979). C. Validity: Because the SASSI-A2 has not been available long enough to generate independent psychometric studies, potential users must rely on the information provided by the publisher for this essential information (SASSI, 2001). The publisher attempts to demonstrate criterion validity, which in this case is a DSM-IV-based substance use diagnosis by a clinician. Using the agreement data provided by the publisher, whose website reports an “overall empirically tested accuracy of 94%,” and combining the two substance use disorders for the clinical diagnosis so that there are four cells (substance use disorder vs. no substance use disorder; clinician diagnosis vs. SASSI-A2 decision), the kappa statistic is .78. According to Landis & Koch (1977), the strength of this statistic is considered substantial. This provides much stronger empirical evidence of the correspondence between clinical diagnoses and the classification on the SASSI-A2 than simple agreement figures. The publisher reports that no significant differences were found in accuracy across treatment settings (schools were not included), age, gender, education, ethnicity, living situation, and employment status. D. Generalizability: The development and cross-validation samples for the SASSI-A2 were selected from the larger group of research participants according to three criteria: completion of a sufficient number of items to allow classification based on decision rules, availability of collateral data regarding substance use disorders, and a score on a validity scale of the inventory that indicates a valid profile. The 1,244 respondents who met these criteria comprise 53% of the participants in the validation research. The authors do not report such analyses regarding systematic difference between those participants who met selection criteria and those who did not. Participants in both samples included 64% and 63% respectively from juvenile corrections programs, and no participants from school programs. Because the instrument is being recommended for use in the schools, the fact that no school program participants were in the development or cross-validation samples is a serious concern. Further, 85% of participants have had trouble with the law. By way of comment on the high proportion of participants from correctional facilities in the SASSI-A2 Manual (Miller & Lazowski, 2001), the authors speculate, “it may be that the juvenile justice system is the primary setting in which adolescents are screened for substance use disorders” (p. 27). One must question the applicability of the inventory to youth not involved in the legal system given the nature of the sample on which it was developed and validated. The authors’ suggestion in the User’s Guide (SASSI Institute, n.d.) to employ the inventory in “school counselors’ offices, mental health centers, and juvenile courts” (p. 1) needs to be approached with caution. V. Evaluation: The lack of adolescent-specific substance abuse assessment tools has been lamented by practitioners and researchers alike (Winters, 1990). The SASSI-A2 provides a much-needed and easy-to-use tool for clinicians who must make important decisions about referral and treatment. The absence of peer-reviewed psychometric data on existing instruments (Leccese & Waldron, 1994) makes scrutiny of this inventory particularly critical, to prevent imprudent decisions being made on this basis of an instrument with inadequate psychometric properties. The data provided by the publisher of the SASSI-A2 do not yet establish acceptable levels of reliability and validity for the instrument. Clearly, more basic research on this instrument is warranted. The publisher attempted to establish the validity of the SASSI-2 by reporting the strength of its correlations with non-test criterion measures. The criterion was “clinician diagnosis.” The Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) emphasized that “the choice of criterion and the measurement procedures used to obtain criterion scores are of central importance” (p. 14) because the validity evidence of the test is dependent on the validity of the criterion. That is, the criterion should be one whose reliability and validity have been well established and accepted. The publisher of this inventory acknowledged that the criterion is the “gold standard” (Miller & Lazowski, 2001, p. 25) against which the new instrument is compared. The problem is the lack of information regarding the criterion (clinical diagnoses based on DSM-IV criteria) used in the validity study. The report (SASSI, 2001) stated only that the diagnoses were obtained from clinicians. Standard 1.6 (AERA, APA, & NCME, 1999) addresses this issue, stating “information about the suitability and technical quality of the criteria should be reported” (p. 21). The publisher provides no information about the procedures used or the qualifications of the diagnosticians, information that should be routinely reported. When individual clinician’s diagnoses are used, inter-rater reliability should be reported as well. Moreover, in this case it is particularly important that information regarding procedures and clinician qualifications be provided, because the participants in the SASSI-A2 development and reliability samples were obtained from an unspecified subset of 48 different treatment and correctional programs. The possibility of inconsistent diagnostic procedures and differing clinician qualifications across sites limits the value of the reported data. It is essential to ascertain if clinicians whose diagnoses were used as the criterion were blind to the results of the SASSI-A2. Again, this information is not given in the publisher’s report, and its absence undermines the usefulness of the data. The nature of the sites creates another issue regarding the criterion measure. Sixty-four percent of sites from which participants were drawn were juvenile corrections programs. Correctional programs employ personnel with criminal justice training, and do not often have trained substance abuse clinicians on staff. Without information regarding the qualifications of the clinicians whose diagnoses were used in the validation study, and inter-rater reliability when appropriate, the findings cannot be properly evaluated. A final concern regarding the procedures used to validate this inventory is that the 1,244 cases used in the process required a “clinically derived diagnosis” (Miller & Lazowski, 2001, p. 26) to be included in the samples. Of those 1,244 participants, 86% were diagnosed with a substance use disorder (Miller & Lazowski, 2001). Such a sample is skewed, and some of the cells in the analyses contain very few cases, detracting from the credibility of the findings. Because this is a widely used inventory with such potential clinical utility, it is hoped that researchers will conduct and publish independent studies that address these important psychometric concerns, and that they provide sufficient information for consumers to judge the value of their results. It is also strongly recommended that clinicians and facilities currently using this inventory be mindful of the deficiencies in its psychometric properties, and that clinical decisions not be based solely on the results of this inventory. The SASSI-A2 Manual cautions users that this is a screening inventory, and that it is only one component of a thorough assessment process. That caveat should be heeded. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: AERA. Carmines, E. G. & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: SAGE. Del Boca, F. K., & Brown, J. M. (1996). Issues in the development of reliable measures in addictions research: Introduction to Project MATCH assessment strategies. Psychology of Addictive Behaviors, 10, 67-74. Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342. Landis, J. R., & Koch, G. G. (1977). The measurement of observed agreement for categorical data. Biometrics, 33, 159-174. Leccese, M., & Waldron, H. (1994). Assessing adolescent substance use: A critique of current measurement instruments. Journal of Substance Abuse Treatment, 11, 553-563. Miller, F. G., & Lazowski, L. E. (2001). The adolescent SASSI-A2 Manual: Identifying substance user disorders. Springville, IN: The SASSI Institute. SASSI (n.d.) The adolescent SASSI-A2 User’s Guide [pre-publication copy]. Springville, IN: Author. SASSI Institute. (2001). The reliability and validity of the adolescent SASSI-A2. Springville, IN: Author. Winters, K. (1990). The need for improved assessment of adolescent substance involvement. The Journal of Drug Issues, 20, 487-502.
|
|
|
|
Last update:
December 10, 2002 |
