Appendix A: Methods

Minimal research exists on the effectiveness of CAPs generally, and the same is true for the majority of CAPs in use by Maryland Collaborative member campuses. To expand the knowledge base regarding CAPs, Maryland Collaborative staff identified existing taxonomies of college alcohol policies and developed a method for assessing CAPs effectiveness.

Data Collection

Each school’s CAPs was collected in April 2022 via a Google search including the school name and the term “alcohol policy.” We followed up with specific search words as needed “e.g., related to Greek life) until we located the most recent and complete policy documents. All data are current as of May 1, 2022.
Each policy was then assessed based on four distinct factors: 1) accessibility, 2) clarity, 3) effectiveness, and 4) consequences for policy violations.

Accessibility

The accessibility measure addresses the extent to which policy information is dispersed across several documents or web pages. Two independent reviewers assessed CAPs accessibility, searching for member schools’ CAPs on their respective website.

Clarity

We measured clarity using the Flesch readability score from Microsoft Word, which assesses sentence length and number of syllables per word. Flesch scores range from 0 to 100, with higher scores indicating greater text clarity: for example, a score of 60-70 designates “standard” or “plain” English; a score of 30-50 is considered difficult, confusing, and best understood by those with some college education; and a score below 30 indicates the text is very difficult and best understood by college graduates. The “21 Model Campus Alcohol Policies” fact sheet provides clear language for 21 common campus alcohol policies (Appendix B).

Effectiveness

The effectiveness of CAPs is a multi-component measure determined by: (1) the presence of 30 specific potential policy elements (based on policies found at member institutions and research literature and on federal policy requirements), and (2) the relative effectiveness of 30 of those elements at preventing excessive alcohol consumption as determined by a panel of alcohol policy experts and a panel of practitioners (the effectiveness of the five federally-mandated policy requirements were not evaluated by either panel).

The 30  policy elements were categorized as most effective or somewhat effective. Certain policy elements were placed in an additional not scored category if they were determined to be important for reasons other than to modify college student drinking behavior.

We then generated a composite CAPs effectiveness score using the following parameters: 2 points were given for each policy adopted in full and 1 point for each policy adopted in part. We summed the points “earned” by the policy elements included in each college’s CAPs to obtain an overall CAPs effectiveness score. Expert panel methodology, the 30 policy elements, the federally mandated policy element requirements, and further details regarding the CAPs effectiveness score are described in detail below.

Expert Panel Methodology: The 30 Policy Elements – Expert panel methodology is used to provide expert guidance in research areas where data are incomplete, imprecise, or controversial. We employed a panel of five alcohol policy experts who evaluated the 30 specific policy elements for their effectiveness at preventing excessive alcohol consumption among the general college student population. The panel of experts compiled the 30 policy elements from various sources including the Maryland Collaborative colleges’ existing policies.

Of note, in this analysis, we analyzed policy elements according to their effectiveness, not their efficacy. The question posed to panel of experts was, “Were this policy fully implemented and fully enforced, what would its maximum level of effectiveness be?” Recognizing that varying levels of implementation may accompany even ideal policy language, we asked panelists to disregard the impact that potential variation in policy implementation may have on policy effectiveness in the interest of conducting the panel expediently.

The panel of experts in this study first rated each policy element on a five-point Likert scale ranging from 0 (least effective) to 5 (most effective). Then, using the average rating of each policy element as a reference, the panelists participated in a round-table conference call in which they re-categorized each policy element according to group consensus into one of three categories: most effective, somewhat effective, or ineffective. Certain policy elements were placed in an additional “non-scored” category if they were determined to be important for reasons other than to modify college student drinking behavior.

Practitioner Panel Rating – We also invited a panel of practitioners made up of members of the Advisory Board of The Maryland Collaborative to rate campus alcohol policy measures using the same guidelines as the expert panel. The panel of practitioners have a unique perspective as they are key in implementing the policy measures on campus and have a first-hand look at what has been successful. The panel of practitioners identified seven strategies as most effective that were rated as somewhat effective by the panel of experts.

These seven strategies have been moved into the Most Effective category to reflect this perspective. A high level of consensus was reached on the ratings for all other policy elements.

• Most Effective policy elements are those determined by panelists to comprehensively affect the physical and/or normative drinking environment on campus. Banning alcohol consumption in public places, for example, was classified by expert consensus as Most Effective both because it restricts campus alcohol consumption and is likely to influence social norms around drinking. Likewise, banning alcohol at student organization recruitment events both sets the normative tone for the school year (as recruitment often happens soon after students arrive on campus) and actively restricts student consumption.
• Somewhat Effective policy elements generally reflect little or mixed research regarding their effects on student drinking behavior, and panelists perceived them as having more limited reach than Most Effective elements. These include elements such as bans on hard alcohol, which, while they would affect the whole student body, have been shown in the literature to result in beverage switching by students, i.e. shifting from hard liquor to beer or beer to hard liquor, accordingly. Policy elements with more limited reach include mandated registration of campus events with alcohol and hosting alcohol-free events, which would only affect select segments of the student body.

Federally Mandated Policy Elements – In addition to state- and community-wide policy elements, colleges and universities are federally required to have their own set of CAPs. In order to receive federal funds, an institution of higher education must at minimum fulfill five policy requirements as established by the Education Department General Administrative Regulations (EDGAR); Part 86 of the federal Drug-Free Schools and Campuses Act lists these requirements. EDGAR requires colleges to provide students with a written policy that: (1) bans unlawful possession and use of alcohol; (2) states applicable local, state, and federal laws; (3) describes the health risks associated with alcohol use; (4) cites any available campus alcohol counseling/ treatment programs; and (5) clearly enumerates the sanctions to be imposed in instances of policy violation.⁴

Consequences for Policy Violations

We used the same rating methodology to evaluate the effectiveness of consequences for violating CAPs. We gathered these consequences, often termed “sanctions” in the colleges’ alcohol policies, from each college’s CAPs. We classified consequences as Most Effective, Somewhat Effective, Ineffective, or Not Scored and a CAP’s consequences score was calculated through the same process as described above. We evaluated a total of 13 sanctions in 2016. At that time we determined that two of these sanctions – warning and alcohol education – were ineffective; we did not include them in our 2022 policy reports.

• Most Effective sanctions are those that panelists agreed would have a strong, population-wide deterrence effect as standalone measures. These include student organization probation and loss of student organization status, which panelists agreed were potent deterrents due to their permanent, structural effects on the physical and normative drinking environments on campus.
• Somewhat Effective sanctions include those that would have some effect as a standalone sanction, but whose effectiveness would be heightened as part of a “package” of graduated sanctions or stepped- care procedures working in tandem to deter policy violations, and appropriately treat those found in violation, respectively. Sanctions labeled as Somewhat Effective include individual suspension and probation, which, because of their severity and the extended deliberative process often required to enforce them, become less swift and certain, and consequently less effective, at a population level as standalone measures. Additionally, panelists determined that alcohol evaluation/screening would be most effective as a part of a stepped-care model with a clear referral in the policy language to BASICS or an analogous program for the students in need. While this consequence does not necessarily have a population-level impact, it could prove crucial for the small minority of students needing referral to treatment.

Certainty of Consequences

Deterrence is critical to effective enforcement of environmental policies and rests on the perception that violations will incur swift, certain, and sufficiently severe sanctions. It should be noted that policies are not effective if there is no certainty that they will be enforced. Certainty of consequences has been shown to be the most relevant and effective factor in deterring behaviors that warrant sanctions, especially among college students.^5,6

Appendix B: 21 Model Campus Policies

Appendix C: Limitations

This report has numerous limitations. Although we developed the instrument to evaluate CAPs to maximize the objectivity of policy reviewer assessment, it is impossible to completely remove the effects of rater subjectivity. Additionally, although the policy measures and sanctions included in this instrument are a fair representation of the most commonly used policies and best practices available, they are not exhaustive. Further, other policy measures not currently included in this instrument may be contextually more or less appropriate depending on the campus. Finally, the panel of experts made their determinations about the effectiveness of policy measures/sanctions – not about the real-world efficacy of those measures or sanctions. Efficacy relies on the extent to which policy measures are implemented and enforced on a campus, which may warrant further research. However, the addition of ratings by Advisory Board practitioners does provide the perspective of key implementers of policy measures on campus that have a first-hand look at what has been successful.