The principle of evaluating new policy and legislation is widely accepted, but is less frequently observed in practice. In Australia and most Western nations, little has been done, especially by government agencies, to monitor and evaluate, in a systematic manner, their national drug policies. This is the case in instances where legislation has developed incrementally (as in most of Australia's States and Territories) and where new approaches have been implemented following detailed policy reviews (e.g. the Australian Capital Territory's cannabis expiation notice initiative). Too often evaluation is considered relevant only after a policy has been in place for some time and changes are being considered. To be of most value, new initiatives should be designed and implemented with an explicit and adequately resourced monitoring and evaluation component built into the initiative from the outset.
In the context of evaluating new legislative approaches to cannabis, we suggest that policy evaluation and program evaluation be considered separately. The bulk of this paper has addressed the evaluation of policies, as reflected through legislative options. The details of evaluating the implementation of policies, focusing on the programs themselves, are matters which need to be addressed after the broad policy issues are settled and particular new approaches are agreed upon. We provide below an overview of the steps involved in both evaluative tasks.
The National Drug Strategic Plan 1993-97 (Ministerial Council on Drug Strategy 1993) provides the framework within which we base our recommendations relating to monitoring and evaluation. The Plan is explicitly goal-attainment focused, the approach that we argued for in Chapter 2, where we highlighted the importance of clarity in the specification of the goals of drug policy, legislation and program implementation. The Plan provides an overarching goal (or 'mission') for Australian drug policy: 'The overall mission of the National Drug Strategy is to minimise the harmful effects of drugs and drug use in Australian society' (MCDS 1993, p6). Policies and programs in specific areas, including cannabis, should be evaluated in a manner consistent with this national goal.
Applying goal attainment models of evaluation is a more complex task than it sometimes seems to be initially. Substantial technical difficulties exist in both designing and conducting evaluation research; part of this is the often complex task of identifying policy and program goals. Our discussion in Chapter 2 referred to the fact that, in practice, a particular cannabis policy can relate to goals which are inconsistent: the attainment of one may militate against the attainment of another. Needs change over time; often policy and legislation are far behind. For a given policy, it is likely that a hierarchy of goals will exist: the mission statement of the National Drug Strategic Plan is an overarching goal, one which must be articulated in terms which encompass other goals such as those concerned specifically with cannabis.
In Chapter 4 we outlined five broad policy and legislative options for cannabis in Australia. The approach taken there is a contribution to policy evaluation in this area, and, by extension, a contribution to decision-making. But how are decisions about drug policy to be made? One approach is the making of ad hoc decisions. This must be rejected in favour of more systematic policy choice models. The literature differentiates two groups of decision-making models for use by governments: 'rational' and 'incremental' approaches (see, e.g. Corbett 1992, pp59-76; Davis et al.1993, pp157-81). The rational approach, illustrated below, is something of an ideal. It is not always easy to implement in practice, but the alternative (making decisions incrementally without ever reconsidering the overall needs and patterns of policy responses) is much less attractive.
Applying a rational approach to decision-making in evaluating new legislative approaches to cannabis use (as different from evaluating the actual implementation of new legislation and resulting programs), we suggest that the following issues be taken into account:
(b) how achievable the goals are; and
(c) if the goals are judged to be achievable, how effective, efficient and appropriate are the implementation programs.
It is beyond the scope of this report to present details on the techniques of program evaluation. These are to be found in standard textbooks (Brinkerhoff et al. 1983; Hawe et al. 1991; Guba & Lincoln 1989; Morris 1987; Patton 1990; Rossi & Freeman 1989). One point to be made, however, is that the evaluation research model and techniques employed should be selected on the basis of answering a precisely defined evaluation question. If the evaluation question is, for example, 'What has been the impact on policing of a new legislative approach to cannabis', quite different evaluation techniques will be used from a situation where the evaluation question is 'How cost-effective has the new approach been', or 'How relevant is current legislation to contemporary needs?'. No single approach covers all circumstances.
Again, the National Drug Strategic Plan emphasises the importance of monitoring and evaluation. Under the heading 'Policy Approach', it states:
The National Drug Strategic Plan explicitly identifies policy objectives and indicators to enable the effectiveness of the program to be evaluated.
The National Drug Strategic Plan is committed to the application of needs-based planning and evaluation activities to ensure the effectiveness and efficiency of strategies employed to minimise drug-related harm (MCDS 1993, p5).
We commend those responsible for this emphasis, but note that this policy cannot be implemented without the allocation of a significant level of resources. The oft-stated policy that evaluation should be an integral part of all drug-related programs can only be realised if program budgets include sufficient funds to cover the design and conduct of monitoring and evaluation, and if reporting time frames are appropriate to the evaluation model being employed.
A related strength of the National Drug Strategic Plan which is highly relevant to the present review is its focus on indicators of program implementation, short-term outcomes and longer-term and broader impacts. The indicators selected for this purpose will depend on the particular initiative being evaluated; accordingly, they cannot be comprehensively described here. Within Australia, the work of Sarre, Sutton and Pulsford (1989) and of Christie (1991) in evaluating the South Australian cannabis expiation notice scheme, and of the Queensland Criminal Justice Commission's Advisory Committee on Illicit Drugs (1993) are examples of the effective use of indicators in the monitoring and evaluation of cannabis legislation. Single's (1989) review of the USA experience of marijuana 'decriminalisation' demonstrates the dearth of information available in that country to evaluate the legislative changes initiated over the last 20 years. Elsewhere in this paper we have referred to such evaluation data as are available relating to other nations.
Without seeking to be exhaustive, we list here some examples of indicators which are significant inputs to policy and program evaluation relating to new legislative initiatives concerned with cannabis:
In this section we have discussed the evaluation of new legislative approaches to cannabis use in Australia. We have differentiated between policy evaluation and program evaluation, and have pointed to the need to develop indicators, tied to specific initiatives, of activities, outcomes and impacts.
Undertaking high quality, systematic policy evaluation concerning cannabis is essential and this paper contributes to that process. Research aimed at developing evaluation techniques and program indicators will be needed as part of the development and implementation of new legislative approaches in this area.
Brinkerhoff, R. et al. 1983, Program Evaluation, Kluwer-Nijhoff, Boston.
Corbett, D. 1992, Australian Public Sector Management, Allen & Unwin, St Leonards.
Davis, G., Wanna, J., Warhurst, J. & Weller, P. 1993, Public Policy in Australia, 2nd ed., Allen & Unwin, St Leonards.
Guba, E. & Lincoln, Y. 1989, Fourth Generation Evaluation, Sage, Beverly Hills.
Hawe, D., Degeling, P. & Hall, J. 1990, Evaluating Health Promotion: A Health Worker's Guide, MacLennan & Petty, Sydney.
McDermott, F., Pyett, P. & Hamilton, M. 1991, Evaluate Yourself: A Handbook for Alcohol and Other Drug Treatment Agencies, [National Campaign Against Drug Abuse, Canberra].
Ministerial Council on Drug Strategy (MCDS) 1993, National Drug Strategic Plan 1993-97, AGPS, Canberra.
Morris, L. (ed.) 1987, Program Evaluation Kit, 2nd ed. 1987, Program Evaluation Kit, 2nd ed., nine volumes, Sage, Beverly Hills.
Patton, M. 1990, Qualitative Evaluation and Research Methods, Sage, Beverly Hills.
Rossi, P. & Freeman, H. 1985, Evaluation: A Systematic Approach, Sage, Beverly Hills.
Rotem, A. 1985, 'Evaluating programs on drug and alcohol related problems', Australian Alcohol and Drug Review, vol. 4, pp181-186.