globaldrugpolicy.org, 07 juillet 2011
A critical evaluation of the effects of safe injection facilities
Research has overwhelmingly supported the proposition that safe injection facilities (SIFs) are successful in meeting their stated objectives. However, the methodological and analytic approaches used in these studies have not been scrutinized to any significant degree. Previous studies are compromised by an array of deficiencies, including a lack of baseline data, insufficient conceptual and operational clarity, inadequate evaluation criteria, absent statistical controls, dearth of longitudinal designs, and inattention to intrasite variation. This review suggests that much of the commonly-cited evidence regarding the effects of SIFs cannot be substantiated. Disentangling complicated casual mechanisms first requires that the identified shortcomings be addressed.
The first injection room appeared in Bern, Switzerland, in 1986. It was established as a means of reducing the nuisance associated with public injecting as well as public health problems such as HIV transmission and drug overdose (1). It is an interesting historical footnote that the Bern room was initially designed as simply a café for drug users: its original plan did not include an injecting room. The injecting room arose only after injecting drug users (IDUs) began to inject openly in the café (2). In the decade that followed, injection and consumption rooms spread to other cities in Switzerland as well as cities in Germany and the Netherlands. Since 2000, injecting rooms have been introduced in Spain (Madrid and Barcelona), Australia (Sydney), and Canada (Vancouver). In various countries, these rooms are referred to as drug consumption rooms (DCRs), safe injection rooms, or supervised injection sites/facilities/centres. To reduce terminological confusion, this review uses the Supervised Injection Facility (SIF) designation to represent all of these variants.
In most cases, SIFs arose during times of converging epidemics when problems related to both public disorder and public health were perceived to be “out of control.” With regard to public disorder, open drug scenes and street drug markets were characterized by threatening congregations of addicts, rampant criminal activity, public injecting of drugs, and improperly discarded syringes and other detritus. SIFs held the promise of lessening some of these symptoms. In terms of public health, many countries were experiencing or anticipating frightening escalations in the rates of infectious diseases, including HIV, AIDS and Hepatitis C (HCV). Curbing risk behaviors associated with injection drug use, such as syringe sharing, reusing syringes, and using unsanitary equipment was seen as an important step toward reigning in this epidemic. Advocates argued that the low-threshold environments fostered by SIFs were ideally and uniquely suited to this task. These dual trajectories of public disorder and public health continue to inform the debate surrounding these facilities.
Only 10 years ago, there were virtually no English publications on SIFs (1). Six years later, it was still possible to contend that no well-designed, scientific studies or systematic evaluations empirically documenting the extent to which SIFs had achieved any of their goals had yet been published (3). At that time, the evidence that was available consisted largely of descriptive reports and process/implementation studies of SIFs operating in Europe. Since then, the situation has begun to change quite rapidly. Proposals to establish SIFs, first in Sydney and then in Vancouver, sparked widespread interest in evaluation research. As well, because the initiatives were so controversial, explicit evaluation protocols were required before the proposals could be adopted. Still, one researcher recently noted that “there is a dearth of quantitative evaluations of these facilities in the public health literature” (4).
What is most noteworthy about the paucity of empirical data is that SIFs are nonetheless widely touted as being almost unequivocally successful. The lack of reliable information upon which to base evaluations has in no way inhibited SIF evaluations. On the contrary, popular accounts and “conventional wisdom” are uniformly glowing. It is this apparent disjuncture that animates this review. Several countries are said to be contemplating establishing their first SIFs, including Norway (in Oslo), Italy (Rome), Denmark, Portugal, Luxembourg, and France, while Spain and Australia are considering expanding their programs into other cities (Bilbao and Canberra) (5). These considerations are, in no small part, premised on SIF evaluations that have never been critically scrutinized.
Unfortunately, studies on the effects of SIFs have never been subjected to any critical evaluation. As a result, any decisions on whether to establish, continue, or expand existing facilities are being made in a near vacuum, on the basis of potentially incomplete and one-sided data. The intent of the review is to present a more systematic and critical evaluation of previous SIF research, paying particular attention to methodological and analytical issues. At the same time, this review eschews the rhetorical debates that bog down and compromise too many discussions. Rather, it poses two much more direct queries:
- What does the research say about the impacts of SIFs?
- How confident are we about the conclusions drawn from the research?
Critical Evaluation of Research on SIF Impacts
SIFs are principally founded on the belief that they can effectively address the problems of public disorder and public health associated with IDUs, especially public injecting. To evaluate the effectiveness of SIFs is essentially to assess how well they meet their various objectives. Although different countries and cities emphasize different objectives, a fair degree of agreement persists concerning the goals of an SIF. This review examines the degree to which SIFs have been successful in achieving the following goals:
- reaching the target population
- encouraging service uptake
- improving health and reducing risk behaviors
- reducing the incidence of infectious disease
- reducing drug overdoses and overdose deaths
- reducing public nuisance and disorder
- avoiding an increase in neighborhood crime
Reaching the Target Population
The addictions literature has long recognized that IDUs constitute a particularly elusive population, a difficult group to access and connect to social, counseling, and therapeutic programs. Efforts that preceded SIFs, including outreach and needle exchange programs (NEPs), were in part premised on the supposition that they could provide increased opportunities to interact with these difficult-to-reach individuals. SIFs are similarly expected to act as means of contacting IDUs, in hopes that this contact will improve access to a variety of other services (6). It has been suggested that, in contrast to the brief and often hurried contact that typically characterizes NEP transactions, the relaxed environment promoted by SIFs is better able to facilitate meaningful interaction with IDUs.
With few exceptions, SIFs have been found to be effective in reaching their target populations. Support for this proposition takes several forms. In some cases, evidence of accessing the target population is offered prospectively. A survey of 178 NEP attenders conducted in advance of the opening of the Medically Supervised Injection Centre (MSIC) in Sydney found that 71% of respondents would have preferred to use an MSIC. That number rose to 83% for those respondents who had injected in public (7). Two studies carried out just prior to the opening of Vancouver’s SIF (called “INSITE”) are similarly positive in their assessments. In the first study, 37% of IDUs and 52% of public injectors expressed their willingness to attend an SIF (8). The second survey, performed with a different sample, reported a 92% rate of willingness (9).
All three of these studies essentially draw an implicit link between the potential to reach the target population and actual usage. However, the contrasts between these findings raise some hard questions. With regard to the first Vancouver (INSITE) study, it is not clear that a 37% rate of willingness to attend an SIF constitutes affirmative support. Even among the most relevant group, public injectors, support barely raises above 50%. Translated into practice, it is possible to posit that INSITE might reach only half of its intended clientele. This conclusion is even more likely given that the sample used in the first Vancouver study is the most scientifically valid of the three. The Sydney survey was based on a sample of 178 clients who attended the needle syringe service of the Kirketon Road Centre over a two-day period, while the second Vancouver survey utilized street-based recruitment and snowball techniques that produced a sample of 458 respondents. In short, both methods yielded convenience samples, the representativeness of which is unknown. On the other hand, the 587 respondents in the first Vancouver study were drawn from a large prospective cohort study that the authors have consistently maintained is representative (10). So while there is clearly some level of prospective support for SIFs on the part of potential clients, the breadth of that support remains unclear.
In addition, it appears that the depth of client interest in SIFs is also tenuous. It is most telling that the 92% willingness to use SIFs noted in the second Vancouver study was contingent on the complete absence of Health Canada restrictions (8). That is, the vast majority favored SIFs where there were no rules limiting behavior. This positive outlook was reduced dramatically as various conditions were introduced. The willingness to use an SIF was 64% if drug sharing were prohibited; 62% if assisted injection were prohibited; and 54% if registration were required. Under all three Health Canada guideline prohibitions, willingness dropped to 31%. Though not as dramatic, the negative effects of rules against “deal splitting” and injection assistance were also observed in Melbourne (11). In sum, prospective examinations of the ability of SIFs to reach target populations are not conclusive. At least some IDUs express a willingness to use SIFs, but the best research places that level lower than one might hope. Rates of potential uptake are lower still when using an SIF is made conditional on the acceptance of “house rules.”
A second method for assessing whether SIFs are reaching their goal of accessing IDUs involves the use of utilization statistics. By these measures, the record is mixed. Support for SIFs is presented in research that finds high frequency and consistency of visits. Studies in cities such as Frankfurt and Zurich have found that, on average, clients used facilities five times per week (12). In Rotterdam (13), clients reported visiting an SIF on six days in the week prior to the interview, and twice in the preceding 24 hours. A non-random survey of 18 SIFs in Germany found that 51% of respondents used the rooms at least once a day (12).
Conversely, evidence from Sydney and Madrid undermines the assertion that SIFs are frequented by highly active and consistent users. Tracked over trial periods of 18 and 26 months, clients at Sydney’s MSIC and Madrid’s SIF (DAVE) averaged fewer than two (1.20 and 1.65 respectively) visits per month (12, 14). During the first 12 months of the MSIC trial period, only 26% of clients had, on average, attended once a month or more (15). In Vancouver, only 45% of a sample of active IDUs reported ever using INSITE. Of those, a majority (57%) used the facility for fewer than a quarter of their injections (16).
While the difference in contexts is clear, an accompanying explanation is less obvious. Several possible reasons for the lower rates of regular use of the MISC and DAVE have been advanced, including specific aspects of the facilities (i.e. house rules), larger transient populations, and higher rates of user turnover in the local drug scene (12). Whatever the reason, utilization statistics from several cites do not support the claim that SIFs reach their target populations. To the extent that a particular drug scene is characterized by transience and high turnover and transient populations, the ability of an SIF to establish a consistent roster of injectors is that much more suspect. Data from such areas conveys a picture of almost random and sporadic usage among a highly fluid clientele.
And what of the assertion that the regular, frequent use of SIFs seems to be more common in Germany and the Netherlands? Reports of clients using the facilities five or six times a week do seem to confirm a pattern of consistency. With regard to frequency, however, the data that would allow for conclusions is most often unavailable: information on the proportions of usage events is lacking. For example, surveys of SIF users suggest multiple daily episodes of drug consumption. In Hamburg, clients averaged 6.3 “consumption units” over the 24 hours preceding their interviews (6). What the report does not reveal is how many of those 6.3 units were “consumed” at the SIF. It is entirely possible that clients consistently use the facilities but for only a small proportion of their overall drug usage. The results from Vancouver cited above indicates that this may be the case. The problem is that very few studies present their findings in terms of proportionality of usage, so it is impossible to say with any certainty how deeply the SIFs are reaching into their neighborhood populations. This problem is compounded by the fact that there is tremendous variation in the frequency of use. Over a 12 month period, the attendance at the MSIC ranged from one to 583 visits (15). In this light, even the impressive finding from Rotterdam, where clients reported having visited an SIF twice in the past 24 hours on average, may be called into question. What if those clients are using upwards of 10 times per day? Without a more detailed picture of overall drug use, basic utilization statistics cannot adequately answer questions pertaining to the goal of reaching the target population.
Of course, attracting clientele is merely a first step. In addition to needing a safe place to inject, SIF attendees tend to suffer from a host of related difficulties, including physical and mental illness, homelessness, and joblessness. As well, they generally lack social networks capable of connecting them to assistance and services. Advocates argue that the low-threshold atmosphere of SIFs allows them to provide a range of alternative services as well as act as bridges to wider services. One measure of success, then, is the degree to which clients are judged to take advantage of SIF services other than the injecting rooms and use SIFs to gain access to alternative services. Collectively, these processes are referred to as service uptake.
The evidence available thus far appears overwhelmingly to support the service uptake supposition. In terms of services provided at the facility, studies have consistently held that utilization rates are high. Studies of SIFs in Rotterdam (13) and Hamburg (6) found that a vast majority of respondents (88% and 89%) made use of at least one of the facilities services. But the nature of these services is dubious. In the Rotterdam study, the services clients most frequently mentioned were “coffee and a chat” (73%) and “eat a meal” (57%). More conventional “services” such as “medical care” (37%), “information on therapy” (19%), and “information on safe use of drugs” (15%) were used much less often. Similarly, respondents in Hamburg most commonly cited “meeting people” (53%). Requests for “counseling offers” (36%), “medical services” (28%), and “information on safer drug use” (5%) lagged far behind.
Two points deserve mention. First, the inclusion of primarily social activities as “services” is dubious. This overly broad conception serves only to inflate uptake statistics. More generally, conceptual imprecision in SIF evaluations makes it very hard to distinguish exactly what is meant by “services.” About a facility in Frankfurt, one researcher commented that “the SIF provides ‘counseling,’ but this primarily means orienting injectors on how to behave in the SIF, responding to clients’ questions, and referring clients who request help to on- or off-site services” (3, p. 335). Monitoring house rules and answering questions are distinct from client referrals and, once again, conflating them only improves assessments of uptake. Such definitional expansiveness leaves a distinct impression of evaluators rushing to demonstrate that few clients use SIFs solely for the purpose of injecting. The deconstruction of the conceptual expanse reveals a more sober picture of SIFs, one in which services are available and are used sporadically, but where injection nevertheless remains the most important reason for client visits. The words of a professional social worker in Hamburg are revealing: “We’d like to think of ourselves as a counseling and referral agency that also offers a consumption room. But frequently we feel like we offer a consumption room and offer counseling on the side” (3, p. 334).
The other point that stands out from the survey findings above is how few clients utilize SIFs for the purpose of accessing information on safe injection practices. This must be viewed with disappointment, given that SIFs are routinely lauded for their unique ability to provide an ideal atmosphere to dispense advice. The emphasis on safe-injection information is made clear by the frequency with which staff members take advantage of injection events to talk about safe-use rules and offer advice on the minimization of risk. Still, despite the prominent role afforded dissemination, most visitors do not seek out safer-use information. It has been suggested that “respondents may not report a great need to obtain this information because they are already well versed in safer-consumption facts and recommendations” (6, p. 672).
In light of the population targeted by SIFs, the low levels of uptake for information highlighted above are not at all surprising. While initiates and novice users may not be well educated about safe injecting practices, they are most often dissuaded or barred from using SIFs. Conversely, the type of long-term, problematic injectors that SIFs hope to reach would be expected already to have a firmer grasp of safe-use practices. Furthermore, it is possible that the very decision to attend an SIF distinguishes SIF and non-SIF users, in that the former may be more attuned to the risks of injecting. If these groups of users are distinct with regard to their appreciations of risk, then it is the latter that are in greater need of education.
Ultimately, the link between safe-use information and safe-use practices upon which the dissemination hypothesis is based rests on an apparently tenuous presumption; that is, that knowledge alone animates behavior. This may be incorrect, and if it is, the more pressing problem is not an absence of information, but a disconnect between knowledge and behavior. Situational contingencies such as the need for help or the unavailability of proper equipment may overwhelm any awareness of safe-use practice. If it is true that SIF clients tend to be cognizant of safe-injecting procedures, this fact would raise uncomfortable questions about why “information dissemination” is advanced as a potential SIF benefit in the first place. Given that SIF clients may not require safe-use education and that education may not necessarily translate into consistent behavior, the rationale for information dissemination is somewhat hollow. At the very least, the supposition regarding this particular service uptake is not supported by the bulk of evidence.
In addition to utilization, the other common parameter for service uptake is referrals. There are numerous studies documenting the large number of referrals initiated by SIFs. In Vancouver, over 2100 referrals were made over a one year period (17). In its first two years of operation, the MSIC made in excess of 1800 referrals (18). Over a six month period, 276 referrals were noted for 736 registered service users in Geneva (12). Referrals statistics look equally as impressive when expressed as a percentage of SIF clients. In Germany, it was estimated that 54% of participants had at least once been referred to further drug and social services (12); the same study found an average of 1.5 referrals per client.
All of these presentations seem to be operating under, and take for granted, the assumption that the number of clients is the appropriate denominator when considering referrals. It is not. For several reasons, the number of clients is a poor indicator of the role that referrals play in the day-to-day functioning of an SIF. First, there is considerable disparity in the rate of usage for clients: while some visit daily, others appear much less frequently. Simply put, clients vary with regard to their individual probabilities of receiving a referral, all else being equal. Second, it is highly unlikely that the distribution of referrals across clients is equal (the data to verify this hypothesis is not yet available in the literature). It is more likely that the distribution of referrals varies such that some clients receive a great many referrals, while others receive few or none. This distributional variation may reflect differences in probabilities (e.g. clients that use the facilities more often are, on average, more likely to be aware of referral services), or differences in individuals (e.g. some clients may be more apt to ask for a referral, may be sicker and in greater need of a referral, etc.). The important consideration is that the number of clients is a poor measure for contextualizing SIF referrals.
The more realistic way to frame referrals is to examine them in relation to the total number of injection events during the time period in question. This produces a far more accurate picture of how referrals fit into SIF operations. In the Vancouver study, the correct denominator is the total of visits to INSITE: 243,701. Thus, the rate of referrals from INSITE is less than 1% (.89). Expressed as a ratio, INSITE produces 1 referral for every 112 visits. For the MSIC in Sydney, the rate of referrals is 2%, a referral-to-visits ratio of 1:49. Other ratios vary considerably, from 1 per 152 visits in Hanover to 1 per 68 in the North Rhine-Westphalia area of Germany (12). By this measure, referral statistics do not support the service uptake hypothesis. Very few injection episodes result in referrals. Viewed against the totality of SIF activities, referrals are a distal consideration.
More generally, there is the question of whether referrals are effective measures of service uptake. Referrals represent only the starting point of a process. As such, it is erroneous to treat them as an endpoint, as most SIF research does. Referrals may provide information regarding potential service uptake, but reveal nothing about actual uptake, which can only be examined through follow-up studies that examine the conclusion of the referral process. In assessing uptake, the key consideration is not the provision of a referral, but whether the client actually followed through and used the referral. The vast majority of studies do not follow up on referrals. The handful that have examined referral uptake have found worryingly low levels of completion. In Sydney, only 20% of referral cards were returned (14). A study in Frankfurt found that roughly one-third of referred clients were actually admitted to detoxification services (33%) or treatment programs (37%) (19). In sum, given that relatively few referrals are generated by SIFs and that the rates of utilization for those few referrals are quite low, one is forced to conclude that the impact of SIF referrals on service uptake has been negligible.
Leaving aside these analytic limitations, the causal logic underlying the entire service uptake assertion is highly suspect. To substantiate the effectiveness of SIFs as a link between IDUs and external services, it would be necessary to demonstrate that the IDUs would not otherwise have found these services. This is an exceptionally daunting hurdle. Many addicts have been in treatment programs at least once, and based on what is known about cycles of relapse, often more than once. In those previous circumstances, the addicts found some way to access these programs. There is no reason to believe that they could not do so again. In addition, it would be a disservice to discount the efforts of other service providers and agencies in this regard. Street drug markets often attract a veritable army of support workers and others offering assistance. There is no empirical support for the idea that SIFs are superior to other providers in this regard. Ultimately, the case for SIFs as a means for linking IDUs to other services has yet to be made.
Improving Health and Reducing Risk Behaviors
Drug abuse, particularly abuse involving intravenous drugs, has been linked to a number of severely deleterious consequences for the user’s health. In general, IDUs suffer from poor health and hygiene. Some specific behaviors associated with injecting, such as reusing or sharing syringes, put users at greatly elevated risk for contracting blood-borne diseases, including HIV, AIDS, and HCV. The use of unsanitary equipment and other non-hygenic practices can lead to infectious diseases such as endocarditis and cellulitis (20). When done in public settings, injecting drugs presents further risks. Because of the clandestine nature of drug use, IDUs must simultaneously attempt to avoid detection by both the police and predators that might assault or rob them. As a result, public injections are often hurried and produce health complications. Finally, all drug abuse, and especially that occurring in public spaces, carries with it the attendant risk of overdose. Not surprisingly, then, one of the primary factors motivating the implementation of SIFs in the 90s was the need to staunch the rampant health problems associated with injection drug use.
To date, evaluations about the effects of SIFs on health and risk behaviors have been somewhat mixed. To be sure, numerous studies have found evidence of attenuations in risk behavior, including being more careful with hygiene and cleanliness (e.g. using sterile equipment and not sharing needles) and taking more time and rest (see for example 6, 12, 21). But other findings have been decidedly less sanguine. While syringe sharing may have become quite rare in Biel, other injecting-related high-risk behaviors, such as sharing spoons, filters, and water were still high (33%, 24%, and 15%, respectively). In Hanover, only 22% of interviewees reported positive changes in injecting hygiene. The SIF in Geneva has actually witnessed an increase in levels of risk-taking injecting behaviors (12). An evaluation in Sydney determined that rates of sharing syringes and other equipment were still high and remained unchanged from 2000 to 2002 (14). As well, there was no statistical difference between MSIC and non-MSIC users in terms of syringe sharing. Perhaps most tellingly, in the month prior to their interviews, fully one-half of MSIC users reported health problems related to injecting, including soft tissue injury, scarring or bruising of veins, swelling of hands or feet, and abscesses or skin infections.
Research on the impact of SIFs on risk behavior suffers from some familiar deficiencies. Blanket statements regarding hygiene and cleanliness, or taking “fewer risks” are difficult to quantify, and as such serve as poor bases for evaluation. Also conspicuous by their absence are baselines that would allow for an estimation of change. It would be very helpful to know what proportion of injections are being done with previously unused equipment, in comparison with what that proportion had been before using the SIF. As designed, these studies are incapable of assessing empirically longitudinal change in patterns of risk behavior, much less whether such changes could be attributable to SIFs.
Reductions in Incidence of Infectious Disease
As it relates to health outcomes, the impact of SIFs has primarily been assessed though their effects on levels of infectious diseases and drug overdoses. In theory, the implementation of SIFs has been linked to reductions in infectious diseases such as HIV, AIDS, and HCV by improving risk behaviors. This assumption is questionable and has thus far not been substantiated. However, even if this provisional link were demonstrable, there are two related factors that would render it very difficult to make the case for a connection between SIFs and reductions in infectious diseases.
First, many of the countries that have SIFs also have low population prevalence rates of HIV (12). These low prevalence rates make it difficult to detect changes in rates; for the most part, it is unlikely that changes in the number of cases would be sufficient to detect statistically significant trends (22). This helps to explain, for example, why where there is no evidence that the incidence of HIV or HCV notifications in the area surrounding the MSIC increased or decreased.
Second, in the total population of individuals afflicted with HIV or AIDS, only a small proportion can trace their illness back solely to intravenous drug use. In the Netherlands and Germany, only 8% and 12% of AIDS cases are attributable to injection drug use. There is also a problem of attribution. In Australia, for example, 4% of HIV diagnoses since 2000 have been attributed to injecting drug use. But another 4% have been attributed to the joint category of “Male homosexual contact and injecting drug use.” That IDUs represent only a small slice of the epidemiological pie is an issue given the manner in which the proposition that SIFs limit the spread of disease is normally evaluated. Specifically, this assertion is commonly tested through recourse to aggregate data, a practice which obscures the specific role played by IDUs.
At present, there are no longitudinal studies that track changes in rates of infectious disease for IDUs. The logistics of the research that would be required to assess changes subsequent to the implementation of a SIF would be exceedingly complicated. At present, there is no evidence that SIFs have either a positive or negative impact on rates of infectious disease. Barring research far more ambitious that anything seen thus far in this area, the effects of SIFs will remain unknown.
Reductions in Drug Overdoses and Overdose Deaths
The other major health outcome credited to SIFs is a reduction in drug overdoses, and especially overdose deaths. Perhaps the most universally quoted statistic is that no one has ever died of a drug overdose at an SIF. Another commonly cited statistic is that the risk for a severe overdose is 10 times higher for street overdoses than overdoses occurring in SIFs (23). Because these facilities usually have available medical personnel or trained staff, emergencies are dealt with quickly. Clearly, SIFs prevent overdose deathsat the facilities. But if the question is whether they are making significant inroads into the target population of IDUs, the more relevant test is whether SIFs are reducing overdose deaths more generally. The MSIC evaluation found no evidence that the facility affected the number of overdose deaths in the King’s Cross area (14).
Assessing the relationship between SIFs and drug overdoses is complicated by the fact that, at the time of implementation, overdose deaths were already showing a consistently declining trend in many countries. To demonstrate an effect in these circumstances, one has to show that the SIFs produced decreases above and beyond those that would have been expected, based on the trajectory of the data. This has yet to be done. Also lacking has been a consideration of how other important factors, such as the expansion of substitution treatment, establishment of low-threshold services, and intensification of outreach health education, are to be incorporated into analytic models. It will always be difficult to know how much of any decrease (or increase for that matter) in overdose deaths is attributable to the operation of SIFs, “given concurrent significant changes in drug policy, the increased availability of substitution treatments, and targeted policing operations” (24).
Unfortunately, evidence of effect is almost always inferred from trend lines, with no consideration of rival plausible hypotheses or controls for same. In this regard, Figure 1 illustrates this point. This chart is cited often as proof that SIFs reduce overdose deaths (see 19), but this interpretation is unwarranted. Even a simplistic inference would suggest that the closure of the open drug scene and expansion of harm reduction services caused by far the greatest drop in overdose deaths. There is no way to estimate the effect of the opening of SIFs. It is possible that the continuing decline was produced by the residual effect of the earlier initiatives. Certainly, there is no evidence that the establishment of the SIFs caused a decrease in deaths above and beyond what would already have been expected from the prevailing trend. Bereft of context (not to mention statistical controls), these trends are incapable of producing defensible inferences.
Because longitudinal data on non-fatal overdoses are unreliable, evidence supporting the efficacy of SIFs is not presented through trend lines but takes the form of statistics showing that the rates of non-fatal overdose per 10,000 visits are quite low. A survey of facilities in the Netherlands, Germany, and Switzerland found that the estimated rate of non-fatal overdoses ranged from a low of one to a high of 36 per 10,000 visits (25). For Vancouver (17) and Sydney (18), the comparable figures were 11 and 63. What is absent, though, is a comparative framework. Is a rate of, say, 11 overdoses per 10,000 “good?” Once again, the research is plagued by a glaring lack of baseline data. To put these figures in their proper context, it would be necessary to know the rate of overdose in the total population of IDUs. As this number is unknown, we can draw no conclusions about the relationship between SIFs and the prevention of overdoses.
Public Nuisance and Disorder
In addition to client-centered issues, there are wider community-level considerations, predominantly involving concerns over public nuisance and disorder. In the 1990s, areas plagued by extensive drug market activity and open drug scenes came to be viewed as intolerable nuisances. The concentrated drug scenes produce large amounts of litter, which is both unsightly and costly to collect. Improperly discarded syringes also pose a severe health risk. Calls for the restoration of order in public spaces were a driving force and help to explain the expansion of SIFs in the 1990s, which coincided with efforts by authorities to curb the deleterious consequences of open drug scenes (26).
Because public disorder entails a broad range of possible behaviors, evaluations of reductions in public disorder have adopted several perspectives. One common method is to assess changes in public drug use. Here, the “success” of SIFs is questionable. A number of studies have reported decreases in public drug use. Unfortunately, these reports often raise as many questions as they purport to answer. In Rotterdam, 83% of access card-holders reported that they used drugs less often in public after becoming registered users (13). But 69% of respondents admitted to having used drugs outdoors in the month prior to the survey. These results would seem to highlight a significant problem of continued public drug use. Contradictory results of this sort are featured in numerous other studies. Among Hamburg clients, half stated that SIFs had been their most frequent location for drug use during the past 24 hours. But 37% reported having used drugs in public in the past 24 hours, and respondents overall had used in public an average of 4 times over the 24-hour period (12). A study of 18 facilities across Germany found that, while 64% of respondents claimed to inject most frequently at SIFs, 38% of interviewees admitted having used drugs in public in the previous 24 hours (12).
All of these studies suffer from a lack of precision that clouds the interpretation of results. For example, how often is less often? If a client used drugs 100% of the time prior to the opening of an SIF, and 80% following the opening, the client could truthfully claim to have used drugs less often. At the same time, the level of public use would still be very high. Even the implications of claims to “inject most frequently as SIFs” are unclear. For high volume drug users, this could still result in many public injections. Finally, as mentioned at several points, the lack of baseline data makes it impossible to estimate the effect of SIFs on the overall rate of public injecting in a neighborhood. For these reasons, there is no reason to conclude that SIFs have made a significant dent in overall rates of public injecting.
A second measure of public disorder is the number of syringes and the amount of injection-related litter (wrappers and other debris) found in an area. The most comprehensive review of drug consumption rooms to date found no evidence that these facilities increased or decreased the number of improperly discarded syringes. The number in the area surrounding the MSIC in Sydney decreased during the 18-month trial period, but the MSIC Evaluation Committee specifically acknowledged that there was no way to determine if the drop was the result of the establishment of the Centre or was related to the reduced availability of heroin (14). Conversely, Beil, Switzerland, realized a small increase in the number of syringes (27). The researchers argued that the SIF was not the cause of the increase, attributing it instead to increases in cocaine use. What the researchers did not do was explain why they didn’t try to control for this assumed effect. By choosing a less stringent analytic design, the researchers really have no way to determine what was responsible for the increase in syringes. Under those conditions, selecting one “cause” over another is merely post hoc rationalization.
Some of the strongest evidence supporting the “reduction of syringes and litter” hypothesis comes from Vancouver, where a drop in both measures was recorded in the three months following the opening of INSITE (28). For purposes of illustration, two graphs from this study have been reproduced and relabeled as Figures 2 and 3. Upon closer inspection, both reveal patterns that complicate and potentially compromise the conclusions drawn by the authors. In both figures, the counts of publicly discarded syringes and items of litter seem to be lower following the opening of the facility. But, owing to the truncated time frame presented, there is no way to properly contextualize the results. Notice that the average counts for Week 1 and Week 6, both occurring before the SIF, are actually quite comparable to many of the weeks following. Were Weeks 2 through 5 anomalous in some way? Without a much longer time frame for the lead-up period, there is no way to answer this question.
The truncated time frame also affects the interpretation of the results for the follow-up period. The evaluation literature is rife with examples of what may be referred to as the “policy bump,” when the implementation of significant policies or programs produces immediately impressive changes. Over time, however, behavior tends to return to “normal.” It is possible that, in light of the opening of this new SIF, clients made special efforts to keep discarded syringes and trash to a minimum. There are numerous credible accounts of staff picking up garbage near INSITE in attempts to minimize any adverse appearances. These behaviors complicate the longitudinal assessment of SIF effects, which can only be validated if they can be sustained over the long run. The three-month follow-up provided by the Vancouver research is not long enough to demonstrate such effects.
No Elevations in Levels of Crime
In response to concerns that SIFs might increase crime in their areas, a few studies have concluded this worry is unfounded. In the publicly available English language literature, studies from Sydney (29) and Vancouver (30) both found that there was no increase in crime following the opening of the MSIC and INSITE. The Sydney evaluation found that there was no significant upward or downward change in the level of acquisitive crimes such as robbery and theft. There was a significant upward trend in both crime types in the months leading up to the opening of the MSIC, but a shortage of heroin had caused the trends to begin moving down before the MSIC. Similarly, the Vancouver study found no pre/post change in drug trafficking or assaults and robberies. There was decline in vehicle break-ins and vehicle thefts, but this was not attributed to the SIF.
The analysis of the Sydney data was conducted using a sophisticated, interrupted time-series design, while the Vancouver assessment was based on a pooled pre/post-test. What neither of the studies did adequately enough was to control for contextual factors, in particular the effects of variations in enforcement. To reiterate a point made earlier, trend lines are, in and of themselves, not useful for making causal inferences. There are numerous other factors that must be taken into account before any outcome can be properly assessed. For example, for one year from the inception of the INSITE program the Vancouver Police Department stationed four officers immediately outside of INSITE to provide staff with any assistance that they might require. A further 60 officers were assigned to the neighborhood immediately surrounding INSITE as part of VPD’s Citywide Enforcement Team (31). It is entirely possible, and indeed probable, that this concentrated allocation of resources had an appreciable effect on crime in this area. At a minimum, the effects of enforcement cannot be ignored; rather, they, and any other relevant factors, have to be built into the analysis.
The Methodological and Analytical Shortcomings of SIF Research
Evaluations of SIFs are, from the outset, hampered by the fact that very little is known about IDUs and other problem drug users. Few jurisdictions even have reliable estimates regarding the total number of users, much less accurate counts. As detailed information about drug consumption events is non-existent, comparisons and assessments of change are necessarily of questionable validity. We cannot say with any degree of certainty whether SIFs are even reaching their target populations because we know very little about that population. We cannot draw conclusions about the effects of SIFs on reducing overdoses without knowing the rate of overdose in the general population of users. Until these lacunae are addressed, reasonable answers to a host of questions will continue to be elusive.
Conceptual and Operational Fuzziness
In certain instances, SIF studies have suffered from a degree of vagueness, such that their results obfuscate more than they enlighten. Multiple operationalizations of “public injector” have been advanced, all of them unsatisfactory. The fact that a respondent affirms that he/she has taken drugs in public in the last month, or even in the last day, is insufficient to justify the characterization of “public injector.” In this regard, a simple sense of proportionality is sorely missing. With regard to service uptake, the definition of “service” is also overly broad and in need of conceptual clarification. It is misleading to make strong statements about service uptake when the “services” in question are primarily social, such as “meeting people” (6) or “coffee and a chat” (13). Presumably, the purpose of attempting to connect with otherwise difficult-to-reach populations is to help them access resources they would otherwise have gone without. Surely SIFs are not required to provide a venue for people to meet and chat.
In too many cases, the criteria upon which an evaluation is to be judged a “success” are not specified a priori. How much of a reduction in public injecting is substantively important? How much of a decrease in risk behaviors such as multiple syringe uses or syringe sharing is needed before the claim can be substantiated? Where change over time is at issue, neat statistical comparisons are often not helpful. Clearly, there is an element of subjectivity involved in setting targets, but a logically defensible criterion for success is preferable to post hoc pronouncements with unknowable validity.
There is also a problem with measures of “success” that set the bar far too low. For example, number of referrals is not an effective indicator of service uptake. It leaves unexamined too many contingencies, not the least of which are “Did the client actually use the referral?” and “What was the outcome of the referral?” Without the requisite follow-up, referrals don’t have any real weight as a substantive measure.
Self-report methods have traditionally played the role of the Boogeyman in drug research. There is no benefit to be gained from rehashing this debate here. In fact, to dichotomize the issue of validity is almost surely to ask the wrong question. More pertinent to the current endeavor are efforts to identify those respondent characteristics that may negatively influence the validity of self-reporting (32). While researchers are increasingly finding that self-reports are fairly accurate in circumstances where there are no contingencies or other mediating factors, the presence of contingencies and other complications does seem to impact the validity of such measures (33). The notion of contingencies is especially germane with regard to the populations of IDUs expressly targeted by SIFs, given that their contingencies are extreme. Not-in-treatment, nonincarcerated IV drug users have specifically been associated with high levels of false negative self-report classifications (34).
Although their discussion is virtually absent in the SIF literature, any number of reasons could account for self-report inaccuracies. For example, these are not recreational drug users, but rather, are long-term addicts. In light of the well-known side effects of, for example, cocaine use, which include anxiety and paranoia (35), some research has speculated that the rate of misrepresentation among IDUs may be attributable to the psychopharmacological effects of the drugs. It is at least plausible that the memory problems and cognitive deficits attendant with sustained abuse would have implications for the validity of self-reports. The idea of socially desirable reporting (SDR) has also been advanced as an explanation for misreporting (36). SDR suggests that people may be less than fully truthful about using illegal drugs due to their perceived “social unacceptability”. The social desirability thesis is underdeveloped in the precise context of intravenous drug use, but there is evidence indicating that validity should be assessed in relation to specific drugs (37, 38). There also appears to be a hierarchy of unacceptability with regard to drug use: cocaine use, for example, is misreported with much greater frequency than is marijuana use (39). On the assumption that IV drug use would rank very high on such a hierarchy, elevated rates of inaccuracy for this population are at least a possibility. None of this expressly refutes the findings of SIF research per se. It is troubling, however, that so little attention is paid to a seemingly critical issue in the SIF literature. The accuracy and utility of these studies will remain suspect until the validity of self-report techniques is, if not established, at least addressed.
Absence of Statistical Controls
The majority of SIF analyses demonstrates a lack of statistical control. In many of these cases, purely descriptive methods are inappropriately used to support more causally-oriented inferences. The “outcomes” used in SIF research are contingent upon numerous factors, and not just the existence of the SIF itself. For example, the effects of enforcement and policing are routinely overlooked. This is especially problematic in those cities, such as Vancouver, where police practices were altered in attempts to assist INSITE with its mandate. The effects of situation exigencies, while sometimes noted, also are usually not controlled for in the actual analysis. This is most evident in Sydney, where the heroin shortage is consistently identified as a confounding factor but is never incorporated into the analysis in a meaningful way. Less obvious, but not less important, is the example of discarded syringes in Vancouver, where the practice of volunteers picking up litter around INSITE greatly complicates the assessment of whether SIFs are associated with increases in improperly discarded syringes.
Absent proper controls, it is unclear how the effects of the SIF can be separated from other contextual factors. This further extends to the effects of other programs. As mentioned earlier, the establishment of low-threshold services in Germany coincided with other significant changes, including the expansion of substitution treatment and the intensification of outreach health education. Without appropriate (and sophisticated) controls, it is impossible to adequately disentangle the effects of these various initiatives. The analytic frameworks currently being used to examine the effects of SIFs are simply not up to the task, and it is hubristic even to hint at, much less assert, causality.
Most SIF studies conducted thus far have been cross-sectional “snapshots.” The dearth of longitudinal data is problematic for several reasons. First, the basic premise of the research, the assessment of the effects, begs for longitudinal context. Even simple pre/post designs have largely been ignored. Second, cross-section designs are incapable of capturing the dynamic component of change. What is the trajectory of SIF effects? Are they immediately noticeable after implementation? If not, is there perhaps a lag period? Quite apart from onset is the question of sustainability. At present, there is no way of knowing the extent to which any SIF effects can be maintained, or over what duration they may persist. Some of the largest initiatives, such as those in Sydney and Vancouver, are relatively new. Only data collected over time can yield helpful answers to questions such as these. Unfortunately, political and professional considerations increasingly marginalize the need for longitudinal designs (2). Moreover, the expense of a true longitudinal study of behavioral change is frequently beyond the budget of projects. It is also worth noting that the few longitudinal studies attempted thus far are marked by serious limitations. First, they have insufficient follow-up periods. Second, the interpretation of trend results, such as they are, has been questionable to the point of being self-serving.
The SIF as Black Box
By all accounts, SIFs offer distinct environments. They may be distinguished on the basis of size, available amenities, drug specialization, and models of service delivery. Despite these differences, there are currently no comparative studies on these facilities. Where research has involved multiple sites, it has merely aggregated across those sites. No attempts have been made to account for variation between these facilities. Rather, SIFs have been treated as Black Boxes. Outputs are produced, but the causal mechanisms responsible for said outputs remain hidden, unspecified. This oversight has clear implications both for policy-makers and practitioners. The demonstration of SIF effects must be accompanied by more careful consideration of which aspect of the SIF (or combination thereof) actually produced the effect.
Conclusion: Taking Causality Seriously
On the subject of the effects of SIFs, the available research is overwhelmingly positive. Evidence can be found in support of SIFs achieving each of the goals listed at the beginning of the evaluation. In terms of our level of confidence in these studies, the assessment offered here is far less sanguine. In truth, none of the impacts attributed to SIFs can be unambiguously verified. As a result of the methodological and analytical problems identified above, all claims remain open to question.
Even to reasonably approximate causality with regard to SIFs would in practice constitute a Herculean undertaking. Many of the existing shortcomings, such as sample size, longitudinal data, and comparative analyses could be addressed with more rigorous research designs. Conceptual and operational definitions could be refined, and evaluation criteria could be more precisely specified. But other limitations would prove very difficult to surmount. The question of baseline data is likely to continue to be held hostage to the complexity of logistics and the scarcity of resources. Disentangling causal mechanisms will similarly remain tricky. The distribution of SIFs is not random. Instead, they are normally parachuted into pre-existing cauldrons of social ills, the dynamics of which are exceedingly complex. Although SIFs are but one part of a much larger systemic response to the problem of substance abuse and intravenous drug use, they are too often credited with generating positive effects that are not borne out by solid empirical evidence. As a policy issue, the potential impacts of SIFs are simply too important, and too divisive, to be left to conjecture and inferences that cannot be supported.
Garth Davies holds a Ph.D. (Criminal justice) from Rutgers University. HIs research affiliations have included the National Council on Crime and Delinquency, the Institute on Crime, Justice and Corrections at the George Washington University, and Columbia University. He is currently an assistant professor in the School of Criminology at Simon Fraser University. His areas of interest include communities and crime, research methods, and statistical analysis.
I declare that I have no proprietary, financial, professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled "A Critical Evaluation of the Effects of Safe Injection Facilities" except for the following:
The basis for this article was a report prepared under a consultancy with the Addictive Drug Information Council.
1. Dolan, K. and Wodak, A. (1996). Final report on injecting rooms in Switzerland. Unpublished manuscript.
2. Haemmig, R. and van Beek, I. (2005). Supervised injecting rooms. In R. Pates, A. McBride, and K. Arnold (Eds.) Injecting illicit drugs (pp. 160-169). Oxford: Blackwell Publishing.
3. Broadhead, R. S., Kerr, T. H., Grund, JP C., and Altice, F. L. (2002). Safer injection facilities in North America: Their place in public policy and health initiatives. Journal of Drug Issues, 32(1), 329-356.
4. Wood, E., Tyndall, M. W., Zhenguo, Q., Zhang, R., Montaner, J. S. G., and Kerr, T. (2006). Service uptake and characteristics of injection drug users utilizing North America’s first medically supervised safer injecting facility. American Journal of Public Health, 96, 770-773.
5. Kimber, J., Dolan, K., van Beek, I., Hedrich, D., and Zurhold, H. (2003). Drug consumption facilities: An update since 2000.Drug and Alcohol Review, 22, 227-233.
6. Zurhold, H., Degkwitz, P., Verthein, U., and Haasen, C. (2003). Drug consumption rooms in Hamburg, Germany: Evaluation of the effects on harm reduction and the reduction of public nuisance. Journal of Drug Issues, 33, 663-688.
7. van Beek, I. and Gilmour, S. (2000). Preference to have used a medically supervised injecting centre among injecting drug users in Kings Cross, Sydney. Australian and New Zealand Journal of Public Health, 24(5), 540-542.
8. Wood, E., Kerr, T., Spittal, P. M., Li, K., Small, W., Tyndall, M. W., Hogg, R. S., O’Shaughnessy, M. V., and Schechter, M. T. (2003) The potential public health and community impacts of safer injecting facilities: Evidence from a cohort of injection drug users. Journal of Acquired Immune Deficiency Syndrome, 32, 2-8.
9. Kerr, T., Wood, E., Small, D., Palepu, A., and Tyndall, M. W. (2003) Potential use of safer injecting facilities among injection drug users in Vancouver’s Downtown Eastside. Canadian Medical Association Journal, 169(8), 759-763.
10. Wood, E., Tyndall, M. W., and Spitall, P. M. (2001) Unsafe injection practices in a cohort of injection drug users: Could safer injecting rooms help? Canadian Medical Association Journal, 165, 405-410.
11. Fry, C. L. (2002). Injecting drug user attitudes towards rules for supervised injecting rooms: Implications for uptake. International Journal of Drug Policy, 13(5), 471-476.
12. Hedrich, D. (2004). European report on drug consumption rooms. European Monitoring Centre for Drugs and Drug Addiction.
13. van der Poel, A., Barendregt, C., and van de Mheen, D. (2003). Drug consumption rooms in Rotterdam: An explorative description. European Addiction Research, 9, 94-100.
14. Medically Supervised Injecting Centre [MISC] Evaluation Committee. (2003). Final report of the evaluation of the Sydney Medically Supervised Injecting Centre. The results of this evaluation were confounded by a heroin shortage that coincided with the opening of the MSIC.
15. Kimber, J., MacDonald, M., van Beek, I., Kaldor, J., Weatherburn, D., Lapsley, H., and Mattick, R. P. (2003). Sydney Medically Supervised Injecting Centre: Client characteristics and predictors of frequent attendance in the first 12-months of operation. Journal of Drug Issues, 33(3), 639-649.
16. Wood, E., Tyndall, M. W., Li, K., Lloyd-Smith, E., Small, W., Montaner, J. S. G., and Kerr, T. (2005). Do supervised injecting facilities attract higher-risk injection drug users? American Journal of Preventive Medicine, 29(2), 126-130.
17. Tyndall, M. W., Kerr, T., Zhang, R., King, E., Montaner, J. G., and Wood, E. (2005) Attendance, drug use patterns, and referrals made to North America’s first supervised injection facility. Drug and Alcohol Dependence, 83(3), 193-198.
18. van Beek, I. (2003). The Sydney Medically Supervised Injecting Centre: A clinical model. Journal of Drug Issues, 33(3), 625-638.
19. MacPherson, D. (1999). Comprehensive systems of care for drug users in Switzerland and Frankfurt, Germany: A report from the 10th international conference on the reduction of drug related harm and a tour of harm reduction services in Frankfurt, Germany. Vancouver: City of Vancouver, Social Planning Department.
20. Wood, E., Kerr, T., Lloyd-Smith, E., Buchner, C., Marsh, D. C., Montaner, J. S. G., and Tyndall, M. W. (2004). Methodology for evaluating INSITE: Canada’s first medically supervised safer injection facility for injection drug users. Harm Reduction Journal, 1, 9-13.
21. Kerr, T., Tyndall, M. W., Li, K., Montaner, J., and Wood, E. (2005). Safer injection facility use and syringe sharing in injection drug users. Lancet, 366, 316-318.
22. Medically Supervised Injecting Centre [MISC] Evaluation Committee. (2001). Evaluation protocols for the trial of a Medically Supervised in Centre in Kings Cross. Sydney: University of New South Wales.
23. de Jong, W. and Weber, U. The professional acceptance of drug use: A closer look at drug consumption rooms in the Netherlands, Germany and Switzerland. International Journal of Drug Policy, 10, 99-108.
24. Dolan, K., Kimber, J., Fry, C., Fitzgerald, J., McDonald, D., and Trautman, F. (2000). Drug consumption facilities in Europe and the establishment of supervised injecting centres in Australia. Drug and Alcohol Review, 19, 337-346.
25. Kimber, J., Dolan, K., and Wodak, A. (2005). Survey of drug consumption rooms: Service delivery and perceived public health and amenity report. Drug and Alcohol Review, 24, 21-24.
26. Wolf, J., Linssen, L., and de Graaf, I. (2003). Drug consumption facilities in the Netherlands. Journal of Drug Issues, 33, 649-662.
27. Burrows, D. (2000). Starting and managing needle and syringe programs: A guide for Central and Eastern Europe and the newly independent states of the former Soviet Union. New York: International Harm Reduction Development.
28. Wood, E., Kerr, T., Small, W., Li, K., Marsh, D. C., Montaner, J. S. G., and Tyndall, M. W. (2004). Changes in public order after the opening of a medically supervised safer injection facility for illicit injection drug users. Canadian Medical Association Journal, 171(7), 731-734.
29. Freeman, K., Jones, C. G., Weatherburn, D. J., Rutter, S., Spooner, C. J., and Donelly, N. (2005). The impact of the Sydney Medically Supervised Injecting Centre (MISC) on crime. Drug and Alcohol Review, 24, 173-184.
30. Wood, E., Tyndall, M. W., Lai, C., Montaner, J. S. G., and Kerr, T. (2006). Impact of a medically supervised safer injecting facility on drug dealing and other drug-related crime. Substance Abuse Treatment, Prevention, and Policy, 1, 13-16.
31. Personal communication, Inspector John McKay, Vancouver Police Department.
32. Maisto, S. A., McKay, J. R., and Connors, G. J. (1990). Self-report issues in substance abuse: State of the art and future directions. Behavioral Assessment, 12, 117-134.
33. Schumacher, J. E., Milby, J. B., Raczynski, J. M., Caldwell, E., Engle, M., Carr, J., and Michael, M. (1995). Validity of self-reported crack cocaine use among homeless persons in treatment. Journal of Substance Abuse Treatment, 12(5), 335-339.
34. Falck, R., Siegal, H. A., Forney, M. A., Wang, J., and Carlson, R. G. (1992). The validity of injection drugs users self-reported use of opiates and cocaine. Journal of Drug Issues, 22(4),