U.S. Can Learn from Australian Debate over Success Rates of Detection Dogs

A debate now going on in Australia deserves the attention of those in the U.S. involved in using, and particularly those responsible for funding, detection dogs. While American courts sometimes consider the relevance of a dog�s accuracy rate in prosecutions, there is far less discussion on this side of the Pacific as to whether the overall success rate of drug and other detection dogs justifies their costs. There is also far too little recognition that poorly designed and implemented protocols, procedures, and practices, including inadequate training and continuing education, allow or even encourage false alerts, cueing, violations of individual rights, lawsuits, misuse and mistreatment of dogs, and other consequences that add to the bottom line of canine programs and make them harder to justify to governments and taxpayers.

Success Rates of Drug and Explosives Dogs in New South Wales

The issue was touched off when David Shoebridge, a member of the New South Wales Upper House of Parliament, asked the Minister for Police and Emergency Services the following questions (in italics) and received the answers (in Roman type).

How many complaints about police sniffer dogs were received by the NSW Police Force in the following years:
(a) 2007? 10
(b) 2008? 8
(c) 2009? 9
(d) 2010? 11
(e) 2011 to date? 5

(2) How much money has been paid out by or on behalf of the NSW Police Force or the state of New South Wales in response to legal proceedings relating to police sniffer dogs in the following years:
(a) 2007? $35,000 (Australian dollars)
(b) 2008? $85,000 (plus costs as yet unresolved)
(c) 2009? None
(d) 2010? None
(e) 2011 to date? None to date

(3) What expenses have been incurred by or on behalf of the NSW Police Force or the state of New South Wales defending legal proceedings relating to police sniffer dogs in the following years:
(a) 2007? $19,827.79
(b) 2008? $164,086.78
(c) 2009? None
(d) 2010? None
(e) 2011 to date? $6,671.52

(4) What records are kept of each search by a police sniffer dog? Records are kept in relation to all requests that are received. Each individual search is then recorded on the COPS system in the case of drug detection dogs, and via spreadsheet in the case of firearm and explosive dog searches.

(5) How many searches were conducted using police sniffer dogs in the following years?
(a) 2007? Drug dogs: 7,603; firearm/explosive dogs: 1,160
(b) 2008? Drug dogs: 10,562; firearm/explosive dogs: 1,128
(c) 2009? Drug dogs: 17,321; firearm/explosive dogs: 791
(d) 2010? Drug dogs: 15,779; firearm/explosive dogs: 551
(e) 2011 to date (April 30)? Drug dogs: 6,965; firearm/explosive dogs: 108

(6) How many searches involving a police sniffer dog found any illicit substances in the following years? Only drugs were considered in the answers.
(a) 2007? 2,435
(b) 2008? 3,748
(c) 2009? 5,109
(d) 2010? 5,087
(e) 2011 to date (April 30)? 2,103

(7) How many searches involving a police sniffer dog found indictable quantities of illicit substances in the following years: 2007-2011 to date? The police replied that this information could not be determined from police records.

(8) How many searches involving a police sniffer dog found weapons in the following years:
(a) 2007? 11
(b) 2008? 7
(c) 2009? 8
(d) 2010? 5
(e) 2011 to date? 1

Thus, there were 58,230 searches using detection dogs in the nearly four and a half-year period, resulting in 13,892 drug finds, a success rate of 23.9%. In the same period, there were 3,738 searches for using firearm/explosive dogs, resulting in 32 weapons being found (though no bombs are mentioned), a success rate of less than 0.01%. Combining both types of detection dogs for which figures were sought produces a success rate of 22.5%.

The Sydney Morning Herald, in an article posted online on December 12, updated the figures for 2011, indicating that through September 14 of 2011, 102 searches produced drugs 2,854 times, a success rate of only 20%.

According to the Sydney Morning Herald, Shoebridge said that �no test which has an 80% error rate could be considered a reasonable basis on which to conduct an intrusive public search of a citizen going about their daily business.� He said that with such a high error rate, targeting young people and Aborigines, the program should be halted. �If this was happening in the car parks of merchant banks, there would be outrage.�

The police response was that often found in the U.S.: if drugs were not found, it means that the person was using them recently, or had been in their presence, meaning that the alert was valid, just unproductive. The New South Wales Police Minister, Mike Gallacher, said the government fully supported the use of dogs because the police had found them effective.

New South Wales Ombudsman�s Report

There has been a tradition in Australia of reviewing the cost effectiveness of detection dogs. Most dogs are used in New South Wales and Queensland, the most populous part of the country (see the CIA map), and the Australian Customs Service has a Detector Dog Program. New South Wales has long had general police dogs, some of which were trained in drug detection, but did not begin to use specialized drug detection dogs until the 2000 Olympic Games in Sydney. Most detection dogs in Australia are Labradors. Dogs are generally trained with food rewards and taught to give a passive alert.

In a report published by the New South Wales Ombudsman in 2006, Review of the Police Powers (Drug Detection Dogs) Act 2001, the Ombudsman concluded that �despite the best efforts of police officers, the use of drug detection dogs has proven to be an ineffective tool for detecting drug dealers. Overwhelmingly, the use of drug detection dogs has led to public searches of individuals in which no drugs were found, or to the detection of (mostly young) adults in possession of very small amounts of cannabis for personal use.� (See also, New South Wales Council for Civil Liberties, analysis of the Ombudsman�s Paper.)

The high cost of the program, and its marginal results in targeting drug suppliers, led the Ombudsman to question �whether the drug Dogs Act should be retained at all.� The Act was in fact repealed, though this did not mean the end of detection dogs, only the end of certain expanded police powers regarding their usage. The Drug Dogs Act had conferred on police the power to use drug detection dogs without a warrant in certain public places such as pubs, on public transport routes, and sporting and entertainment venues. The Act also required the Ombudsman to review the success of the use of dogs under the Act.

Use of Detection Dogs in New South Wales

The Ombudsman�s Report describes police procedures:

�Police distinguish the screening of people by the drug detection dog from the searching of people conducted by police. The distinction is important because the Drug Dogs Act does not require police to have a reasonable suspicion about a person in order to use the drug detection dog for screening. To search a person, police must have a reasonable suspicion that a person is in possession or control of a prohibited drug.�

The use of one dog was described in a drug dog case, DPP [Director of Public Prosecutions] v. Darby, NSWSC 1157 (2002) as follows:

�The evidence established that Rocky [the drug detection dog] had been trained to detect the scent of cannabis. When he did so his training caused him to put his nose in the air, flair his nostrils, and sniff rapidly. He would then follow the scent to its source. When he reached the source of the scent, he was trained to put his nose on such source and sit down beside it. If the scent emanated from a person�s pocket, Rocky was trained to put his nose on the pocket and then sit down beside the person.� (emphasis added)

In this particular case, Rocky placed his nose on the suspect�s pocket, resulting in a find of 2.89 grams of methamphetamine and 1.9 grams of marijuana. Darby was charged with two counts of possession of prohibited drugs.

Vapor Wake Analogy?

Following scent to source bears a significant resemblance to Auburn�s trademarked Vapor Wake Detection Program, where dogs are �trained to alert if they detect any explosives in the air and follow the explosive to its source.� Government Accountability Office, Technology Assessment: Explosives Detection Technologies to Protect Passenger Rail, GAO-10-898 (July 2010). Although the GAO accepts Vapor Wake Canines as �an emerging use of EDCs [explosives detection canines] that may be applicable to the passenger rail environment,� and refers to �generally positive results,� there is no indication that the agency has requested or received any statistical evidence of the sort available in Australia as to the success rates of the dogs. This would not be an unreasonable request, given that the Metropolitan Atlanta Rapid Transit Authority, according to the GAO, has been using such dogs since 2006 (the Ombudsman was gathering such data from the beginning of the sniffer dog program).

That searches using such scent-to-source trained dogs could be conducted on the street is indicated in a description of a 2003 incident from notes used in preparing the Ombudsman�s Report:

�Dog indicates a man seated with a group of other people at a table. The handler taps him on the shoulder and explains about the dog and cautions him. The man says that he had a smoke about three hours ago. He is given an explanation about the scent of the cannabis still being on him. Police officer says, �Just come out with me, mate. We�ll get it sorted.� The man says he is �freaked�.�

About 84% of incidents in the period reviewed by the Ombudsman (from 2002 to 2004) involved marijuana, though ecstasy, methamphetamine, cocaine, and other drugs were also found. All but one of the incidents involved less than 300 grams, for which no legal proceeding followed. Australian police can issue warnings and cautions in non-prosecutable situations.

The Ombudsman concluded that in only 26% of the time where a dog indicated were drugs subsequently found. This accuracy rate was not consistent with statements made by the dog handlers, such as, �my dog never lies,� though handlers were willing to accept that other handlers make mistakes, but often insisted on the accuracy of their own dogs:

�We�re not going to turn around and say, �oh, the dogs sniffed them, OK, well we�ll stop them�. The dog�s just sniffing, and as the trained handlers, we know that. Perhaps the other police we work with don�t know, but we�ll just go, �nup that�s wrong�. So it�s a load of crap. The dogs don�t indicate on food or cats or dogs. They might sniff them. They might pay them a little more attention, but that person�s not being stopped or searched or anything else just because they�ve been sniffed.�

Should There Be a Minimum Rate of Success to Justify a Program�s Existence?

As to the question of what level of accuracy is acceptable, the Ombudsman�s Report stated:

�It is not possible to point to a rate of finding drugs which is universally thought of as acceptable in terms of accuracy and effectiveness. While some might say that a rate of 1 in 4 is commendable, others will be concerned that three quarters of those searched are not found to be committing any offence. Others point to the number of admissions of drug use that accompany the searches where no drugs have been found to indicate that the dogs are highly accurate, detecting miniscule residual traces. Still others might comment that it is unduly narrow to focus on statistical rates when other benefits of the use of drug detection dogs and high visibility police patrols are no less important but are less quantifiable. For example, it has been argued that the drug detection dogs contribute to reducing fear of crime in the community, deterring and disrupting drug networks and reducing other forms of crime such as assaults and thefts.�

It is to be noted that finding �miniscule residual traces� might be acceptable in some environments, such as schools, but the Drug Dogs Act did not include schools in the definition of a �public place� where a warrantless search could be conducted. The Ombudsman continued:

�What is clear is that in practice, drug detection dogs provide the sole basis for the police reasonable suspicion to search a person. The dog can be viewed as a tool used to engage this suspicion. One might expect a higher standard of accuracy and reliability when using a tool, such as metal detectors or breathalysers, than of a police officer acting in the course of his or her duties without the assistance of any tools.�

The Ombudsman then reviewed several U.S. cases discussing accuracy rates, for which see Police and Military Dogs, Chapter 8, �Accuracy Rates.� One professor questioned whether a 27% accuracy rate was any better than chance, noting that about 35.5% of 20 to 29 year-olds and 27.7% of 14 to 19 year-olds in Australia were using drugs according to one survey. In other words, randomly searching individuals in these age groups would probably produce results similar to those from using a dog, at much less cost.

The Report acknowledged that one sniffer dog used by the police had a 56% success rate, though another had only a 7% success rate. The table reproduced here compares the rates of 17 dogs over a two year period from February 22, 2002, to February 21, 2004. All supervisors of canine units should keep such tables of dogs in their units, and all governmental authorities should demand such recordkeeping, along with cost figures.

Deterrent Effect Doubted

The Ombudsman�s Report rejected any deterrent effect in the use of police dogs:

�We were not able to find, nor were NSW Police able to provide, any evidence that the use of drug detection dogs disrupted low-level street dealing in a sustained manner. Similarly, we were not able to identify any evidence that the use of drug detection dogs has had a deterrent effect on drug users, or led to a reduction in drug-related crime. Nor were we able to measure any appreciable increase in perceptions of public safety as a result of high visibility policing operations utilising drug detection dogs. Further, there was no evidence that police obtained intelligence information during drug detection dog operations that led to further investigation of drug supply.�

At another place in the Report, however, the Ombudsman acknowledges that �it is not possible to determine the deterrent effect of drug detection dogs or whether they contributed to the general downward trend in drug offenses.�

The Australian experience is clearly different from that in the U.S., where it has often been possible to obtain significant information from street-level users regarding their suppliers.

Significance of the Australian Debate

Trolling sports and entertainment venues with drug dogs, given that most individuals caught are going to be given warnings even if caught with something, seems a waste of police time at the expense of the public. The failure to target environments where drug suppliers may be found is a good argument for altering police practice, but not necessarily for eliminating drug detection dogs.

In prior blogs we have been concerned that dogs should not become a failsafe method of justifying a search when nothing else pans out. Pressure to produce an alert is a primary source of cueing, cueing that will often result in a false positive. As we have argued, the results of cueing are insidious throughout the criminal justice system, even if American courts have been slow to recognize the issue and prosecution witnesses reluctant to acknowledge it. False alerts, whether from cueing, poor training, or otherwise, reduce the cost effectiveness of canine units. Handlers and police administrators should understand that ignoring cost effectiveness may ultimately result in costs being cut altogether.

Even if residual odor can continue as a mantra to excuse a low field accuracy rate in criminal prosecutions, the number of times drugs are not found by a department�s canine unit after alerts has to be accounted for an expense without result. Australia has been more focused on the cost benefit of canine units than have politicians and administrators in the U.S. This is likely to change as more governments at all levels become strapped for cash and are forced to consider how crime can be fought most efficiently, and cheaply.

We believe that the dogs will win in the end, that there are thousands of good law enforcement dog handlers in the United States, but we also believe that recordkeeping could substantially improve, and should be made more comprehensive and uniform across the U.S., on a model similar to that used by the New South Wales Ombudsman. Such results should regularly be reviewed by police administrators and governments, and published for the benefit of governments, courts, and the public.

This blog was written by John Ensminger and L.E. Papet. Picture and table reproduced here as noncommercial use, � State of New South Wales through the NSW Police Force. Thanks to Sherri Minhinnick for suggesting this topic to us.