Next: 5 THE FARRINGTON AND WELSH META-ANALYSIS Up: OUTDOOR LIGHTING AND CRIME, PART 1: LITTLE Previous: 3 OVERVIEW OF LIGHT AND CRIME STUDIES Contents
In general, the lighting and crime studies reviewed are notable for their shortcomings in design, conduct and analysis, and the disparities in results. Several of these studies were also reviewed by Lab (1997) [55], who was likewise critical:
``There are a variety of methodological problems throughout the lighting studies. One of the most problematic of the issues relates to the measurement of lighting. Various studies tend to differentiate between ``relit'' and ``unrelit'' areas of town without producing evidence of the increased level of illumination or the uniformity of the lighting (Tien et al., 1977 [107]). Simply altering the light fixtures does not guarantee an actual change in the amount of illumination. A related problem is the lack of information on the control areas and their lighting, besides the fact that these areas did not receive the new lights (Nair et al., 1993 [71]; Tien et al., 1977 [107]). Targeting high-crime areas and comparing them to lower crime areas may account for the failure of the project. Reduced crime in a high-crime area could be a regression artifact. This means that the unusually high offense levels return to a lower, more natural level over a period of time. A related problem is that of using short-term follow-up times, which could mask true results (Nair et al., 1993 [71])... Perhaps the most strident support for lighting is offered by Painter (1993) [78], based on a series of analyses conducted in England. Unfortunately, Painter fails to address a number of methodological concerns and inconsistencies (Nair et al., 1993 [71]), which leads to serious doubts about the efficacy of the results.''
Problematic studies are examined in detail in this chapter. This includes the Dudley and Stoke-on-Trent projects in particular, because:
Marchant (2001) [65] criticised the publication vehicle for Painter and Farrington (1997) [82], a book restricted by title to claimed successful applications of Situational Crime Prevention, and therefore unacceptably biased in scientific terms.13This is no fault of the contributing authors, of course. The book editor's preface to the paper claimed that conventional wisdom about lighting being ineffectual for crime prevention is changing largely because of Painter's work and in the face of much skepticism. This assertion by the book editor is rather uncritical.
Marchant (2001) [65] noted that a lighting company had funded the research described by Painter and Farrington (1997) [82]. He pointed out that some of the statistical tests used were one-tailed, justifiable only if there was prior evidence of no possibility for results to involve the other tail (an adverse effect of lighting). In some cases, the one-tailed test gave a statistically significant result when the appropriate two-tailed test does not. This leads to doubt about the conclusions drawn. Marchant also criticised the interview procedures, first quoting Painter and Farrington (1997):
`` `Unfortunately it was not possible to link up the before addresses with the after addresses, in order to carry out longitudinal analyses with each address acting as its own control.' No explanation is given of the reasons but it results in key information being lost. This inability to link the address, is strange since further up the page it says `Of those re-interviewed 90 % were the same respondent as in the before survey, 7 % were the same household but a different respondent and 3 % were a different household at the same address'. So the interviewers certainly seem to have gone back to the same addresses and knew who they spoke to, so why could not the responses from each address before and after the introduction of brighter lighting be linked? Such a flaw seriously undermines the research and raises the question `if this aspect was messed up, were there other errors?'
It would also be useful to be told, which organisation actually carried out the interviewing.''
Following on from the last sentence, the writer thinks it would also be helpful to know the qualifications or training of the interviewers, details of how the quality of the interviews was monitored, whether any interviews were of unacceptable quality and what action, remedial or otherwise, was taken in those cases.
Crelin and Granata (2002) [24] drew attention to funding of the Painter and Farrington (1997) work by a lighting company. They also found some other problems:
``This study concludes that a higher level of illumination introduced into residential areas shows an effect on crime, determined by short-term results and established solely through before-and-after interviews with the area's residents. It is true that great care has been taken to keep several aspects of this study from extraneous influence, however, much is still left open to interpretation... [The validity of the study appears] to rest upon what was alleged through resident interviews - not documented criminal acts.''
Police records of reported crime are known to underestimate total crime substantially by comparison with crime survey results. For this and other reasons, criminologists generally prefer to work with survey results. However, for Uniform Crime Reporting (UCR) total crime in the USA, the extent of underestimation did not change much since 1975 according to Bastian (1993) [6]. Marvell and Moody (1996) [67] used UCR data in their extensive study after acknowledging its shortcomings. Despite the problems, police records are independent of research studies done using these records. On the other hand, ad hoc surveys done in connection with particular studies are potentially open to bias.
Painter and Farrington (1997) pointed out that police records for Dudley were unsuitable for checking the survey results or as sources of experimental and control data ``because of changes in recording procedures and inadequacies of available data.'' More details would have been helpful.
In the case of pedestrian and resident interviews done specifically to determine whether `improved' lighting has affected crime, bias could arise from the wording or order of questions, despite the use of `double-blind' procedures. In particular, the effect of brighter lighting in reducing fear of crime (see Chapter 6 below) and the common belief that lighting reduces actual crime could possibly bias recollections of crime. Places that were less brightly lit at night might seem more prone to crime. Any such bias could possibly apply to the places in daytime as well as night. Reliance on interviews could therefore have contributed to the difference between the Painter and Farrington results and the generally smaller or inconclusive effects found by other researchers using recorded crime data. In the absence of resolution of this issue, it could be argued that inclusion of complete figures for recorded crime should have been a condition for publication rather than an option.
There are other problems with Painter and Farrington (1997), including bias in the literature review. From the time when the experiment was done (ca 1992) to the time of publication, it was not justifiable to claim that a consensus on a beneficial effect of lighting on crime had been established in the journal literature. A beneficial effect may have been the result reported in Painter's unpublished PhD thesis of 1995, but as is stated in the acknowledgement section of the paper, the thesis included the same (Dudley) experimental results. The literature review and the rest of the paper therefore should have been neutral in its approach to the topic. Instead, it is strongly biased towards the view that lighting does prevent crime, as is shown by the following quotes from the paper:
``The main aim of this project is to investigate the effects of improved street lighting as a crime prevention technique.''
It should have been a test for any effects of lighting changes on crime measures. As it stands, it pre-empts the results of the experiment.
``Modern interest in the relationship between street lighting and crime began in North America amidst the dramatic rise in crime which took place in the 1960s.''
This was a time when street lighting was also increasing dramatically, but that is not mentioned.
``In summary, the relationship between visibility, social surveillance and criminal opportunities is a consistently strong theme to emerge from the literature.''
This is based on Situational Crime Prevention theory, not experimental results. Taking it to its logical conclusion, there should be little crime in daytime, far from what actually happens.
``This design controls for the major threats to internal validity outlined above.''
No, an increase in lighting as a treatment should be counterbalanced in some way, such as by a decrease in lighting of similar magnitude as a simultaneous treatment in another experimental area, or, less effectively, a subsequent return of the lighting levels in the experimental area to their original values.
``The design and layout of the estates, and the type of dwellings, facilitated natural surveillance, which was particularly important for street lighting to be effective as a crime prevention strategy.''
These features doubtless also facilitated criminal choice of target and commission of crime, but this is not mentioned.
``The new lighting replaced the older type mercury lamps.''
The implication is that new is better than old, when there was no good reason to believe that the characteristics of the replacement lamps were any better or worse than those of the existing type of lamps in terms of any beneficial effect on crime, and they were certainly worse from the viewpoint of experimental design. (See below in this section for more details.)
``The British Standard (BS5489 Part 3) lists three categories of lighting levels corresponding to low, medium and high crime risk areas and levels of traffic and pedestrian usage.''
This statement pre-empts the experiment, as greater risk of actual crime is stated in the standard as requiring brighter lighting. This is a fault of the standard rather than the paper, but the statement is not queried in the paper. It also raises the issue of why the standard expressed this view when the balance of available evidence was inconclusive if not against it at the time it was written.14
``Also, it permits the displacement of crime from the experimental area to the control area.''
This pre-empts the experiment by failing to mention the opposite effect as an equal possibility.
``In addition to leading to a positive change in resident opinions and physically creating a brighter and safer environment, street lighting...''
The inclusion of `safer' pre-empts the experiment.
The following statement is made in the conclusions section of Painter and Farrington (1997):
``In short, improved street lighting has no negative effects and has demonstrated benefits for law abiding citizens.''
At the time, there was an extensive and readily available literature to the contrary about the adverse health, safety, ecological, greenhouse and other environmental effects of artificial light at night (eg IDA 2002 [49], LiteLynx 2002 [61]). Nobody should recommend increasing what is already an environmental problem without seeking expert advice and discussing or at least drawing attention to the broader ramifications.
Additional problems of the study relate to possible confounding effects of unreported changes in colour rendering, light distribution and glare. In the case of the mercury-vapour lamps that were replaced, it is well known that they have a long operational lifetime, during which the light output drops steadily and substantially. Local council staff decided that the existing lighting was in a bad state of repair and that the area would be relit. If this state arose because the lamps were nearing, or at, the end of their useful life, then a far better experimental treatment would have been to replace the used lamps by unused ones of the same type as is usually done. This would have avoided the confounding by colour, beam pattern and column spacing that actually happened with the substitution of high-pressure sodium. The mean increase in illumination would have been comparable in the two cases, or could have been made so.
In the `before' survey, the proportion of respondents who reported seeing police in the last month in the experimental area was 17.4 %, and for the control area, 27.5 %. In the `after' survey, the values were 38.2 % and 30.7 % respectively. This means that for the sampled months, police presence in the experimental area relative to the control area increased by (0.382/0.307)/(0.174/0.275), ie a 1.97 times increase after the relighting. The extent to which a relative doubling of police presence is expected to have affected the commission of crime is important.
According to Weatherburn (2002) [117], the best-conducted US study on effect of police numbers on crime is Marvell and Moody (1996) [67]. Using their result on p 632, a 10% increase in number of US city police will bring about a 2.9% decrease in total crime, corrected for under-reporting. Levitt (1997) [58] pointed out that (US) cities tend to hire more police as election time approaches, and that such increases reduce violent crime more than they reduce property crime. It is not known if elections or any other event affected the number of police in Dudley during the experiment, but the total number of sightings in the after survey was 53 % greater than in the before survey.
Not all examinations of the problem find a beneficial effect of police on crime. After a major terrorist attack in Buenos Aries, police were reallocated to guard ethnic properties around the clock. The police had to stay close to the assigned properties. Di Tella and Schargrodsky (2001) [26] made use of police records of car theft before and after the attack. Car thefts decreased in the blocks including the guarded properties, but increased concurrently in surrounding blocks. Overall, car thefts were not decreased, merely displaced. In this case, substantially increased police presence did not reduce car theft. While this has lessons relating to the spatial extent of deterrence around a single stationary police officer, it seems unlikely to apply to the change in police presence during the Dudley study.
Goodman (2002) [41] modelled total recorded crime as a function of socioeconomic and demographic variables for 92 midsize US cities. Up to thirteen variables such as population, percent of vacant houses, number of school dropouts and unemployment rate were used. A decomposition process allowed isolation of crime as a function only of police-related variables such as the growth of police numbers resulting from prior city budgeting decisions. For the mean number of police per city, 356.6, the crime rate per 100 000 population is 8926.5. From the model results, a 10% increase in police number over the mean would reduce crime by 98.8, ie a 1.1% decrease. This takes account of the increase in recorded crime as an artifact of increased police numbers (eg Walker 2002 [115]).
Goodman was further able to decompose the police-related variables, isolating the actual crime-reducing effect of increasing the number of police in a city with all other variables held constant. For a 10% increase in the mean number of police, the effect of this in isolation is a decrease of 10.3 % in crime. Obviously crime would not disappear altogether or even become negative if police numbers merely doubled, so linear extrapolation is not appropriate for increases of this order.
A doubling of police numbers represents a 10% increase repeated about 7
times. Using Marvell and Moody's estimate, a doubling of police should
reduce crime to (0.971)
, ie 0.814, or a reduction of 18.6 %. Using
Goodman's first estimate, the reduction should be to (0.989),
ie 0.925 or
a reduction of 7.5 %.
Goodman's second estimate is (0.897), ie 0.467 or a
reduction of 53 %.
Of Goodman's two estimates, the second is more relevant
to the Dudley situation.
The relative reduction in prevalence of all crime actually reported by Painter and Farrington was 21.1 %. This is certainly of a magnitude that could result from a relative doubling in police presence that took place after the lighting intervention, insofar as the `previous month' response given by respondents is representative of the situation for the `previous 12 months'.
Painter and Farrington performed a logistic regression analysis to check whether differences in police presence and proportions of people over 60 on the estates influenced the results. According to this test the differences did not influence the results, so they dismissed the importance of the variation in police presence and concluded that the relighting caused the observed crime reduction. This appears to fly in the face of the facts.
It seems more likely that the reduction in crime in the relit area was a consequence of a relatively greater police presence than a lighting change. If the lighting change did anything at all, it would seem possible that it affected police presence instead. Presumably, police are affected by the dark like anyone else, including criminals, and it would hardly be a surprise if they spent relatively more time in the brighter area after relighting, to the extent that they had any discretion to exercise. This could even have been a conscious choice on the basis that the relit area must be a higher crime area requiring their presence because the council had relit it. Given the known effect of police presence on crime and the unknown effect of lighting, the former is the more parsimonious explanation of the crime results.
Pedestrian use of two streets in the Dudley experimental and control areas was monitored for 3.0 hours on each of two nights in March 1992 and March 1993. The paper stated that the weather in each of these two periods was similar, cold and dry. No reason is given for the absence of quantitative measures of weather characteristics. It would have seemed important to report dry-bulb temperature readings at least. Effective temperature measures incorporating humidity and wind chill factors would have been better again. However, any effect of weather on pedestrian numbers would have applied equally to control and experimental areas, so the absence of physical measures of weather does not justify discounting of the pedestrian results in this case.
There is no doubt that substantial changes were observed in behaviour of the residents. Standard statistical tests indicated small probabilities of chance occurrence of many of these changes, provided that unknown confounding effects were absent. But as demonstrated by the New Jersey quasi-experiments of Section 3.4, real world quasi-experiments are notoriously subject to non-trivial unknown influences. This adds to the reasons why the Dudley results are unconvincing.
The Dudley experiment involved many different before-after measures.
Farrington and Welsh (2002a,b) [34,35]
derived an odds ratio (see Chapter 5
below), a single measure representing a relative change in total crime for
the control area divided by the change for the experimental area.15The
value they derived is 1.44 (representing a 44% relative increase in crime
in the control area), with p
0.05, meaning that a result of this size
could be expected to arise by chance less than once in every twenty of a
large number of trials. A larger change than this took place with the
first pair of counties in Table 1, with a probability of getting this
result by chance less than once in every ten thousand comparisons of
county pairs.
The point of this comparison is that such variations can arise more readily than expected in the real world, because of unknown influences rather than lighting or other deliberate interventions. The argument is strengthened by the larger numbers of crimes in the county pairs compared with those in the Dudley experiment: for example, the New Jersey county crime counts were far less vulnerable than the crime counts for the Dudley areas to artifacts from one or a few habitual criminals changing their preferred locality of operations. The comparison casts considerable doubt on the attribution of the observed changes in crime in Dudley to the lighting intervention. The many other faults described in the Dudley experiment and its reporting add to this doubt, suggesting that the claimed beneficial effect of lighting on crime is not reliable.
This conclusion appears to be generally applicable to other existing real-world experiments on lighting and crime. Future experiments on this topic will need to involve more safeguards against confounding by uncontrolled variables.
Painter and Farrington (2001a) [85] is a report of crime surveys of young people in the Dudley study. The research was funded by a lighting company. The paper begins with a discussion of the Scientific Methods Scale used by Sherman et al. (1997) [102]. Eck's (1997) [31] brief description of this ordinal scale is:
``As in earlier chapters, evaluations were graded using the scientific methods score (1 = correlations between tactics and crime and studies without pre-intervention measures; 2 = pre-post designs without control places; 3 = pre-post designs with controls or time-series designs with at least five time periods prior to the intervention; 4 = studies of interventions in a large sample of places compared to similar places without interventions; and 5 = randomized controlled experiments.''
Painter and Farrington used the score of various experimental designs to indicate `methodological quality', which appears to make too much of it. Methodological quality might be regarded as a somewhat broader term that could include, for example, the rigour with which scientific method is followed and the full range of precautions taken to minimise threats to validity.
Regardless of this semantic issue, Painter and Farrington claimed that their Dudley experiment reached Level 4 on the Scientific Methods Scale. Although they quoted a different definition that allows for the control of additional relevant factors, it seems problematical whether this is sufficient to overcome the limitations of just one experimental area and one control area in the Dudley experiment. If their claim is accepted, then the New Jersey experiment of Section 4.2 might also have refinements contrived for it to reach Level 4. Possibly Level 4 and certainly Level 3 and all lower levels of real-world quasi-experiments on lighting interventions and crime, as performed to date, appear to be unacceptably likely to generate unreliable results.
As Painter and Farrington explained, Level 5 is impracticable to implement for studies of crime effects of lighting interventions. The Level 5 requirement for substantial numbers of randomly selected `units' could usually be met for other purposes with households or individuals as the units, but hardly with areas such as blocks of houses as the units required in street lighting studies. Therefore, lower-level designs have to be used, but they need to be bolstered by some further safeguards. Eck mentioned time-series designs in Level 3, but Painter and Farrington's list does not include this detail. It seems that time-series measures at least would need to be incorporated into Level 3 designs and perhaps even into Level 4 designs in order to get reliable answers in real-world studies of lighting interventions and crime.
Painter and Farrington (2001a) did not mention whether there were any shops, pubs or other commercial areas present in or adjacent to the Dudley experimental and control areas as potential or actual crime hotspots. They cited Painter and Farrington (1999b) [84] about the claimed cost savings of the Dudley and Stoke-on-Trent crime reductions attributed to relighting but this has no bearing on the credibility of either study.
The adult `before' survey results indicated ``If anything, the experimental area was slightly worse on crime''. In the self-reported delinquency survey also, the experimental area was again worse on crime before the intervention. The effect of this would be a small artifactual increase in the prospects of an apparent reduction in crime in the experimental area. This does not negate the results, but makes them less reliable.
In the young persons self-reported delinquency before survey, the questions asked about previous offences were for `ever' rather than for the previous 12 months, as had to be the case in the `after' survey. This could be expected to have inflated the before results for the experimental and control areas. The experimental area before score for self-reported delinquency was 1.55 and for the control area, 1.50. The outcome would again be a small artifactual increase in the probability that a relative reduction in offending would be found in the experimental area after the treatment. The authors did recognise that the `ever' condition clouded the issue, but the explanation for why this was done (``advantage of obtaining more complete information on offending before'') seems quite inappropriate.
A survey of perceived effects of improved street lighting was also done. Although most residents in both estates were aware of the lighting change, no explanation is given for the absence of the control after part of this survey.
In discussing the results for outdoor victimisations, Painter and Farrington (2001a, p 275) stated ``Disappointingly, there was no significant tendency for victimization to decline more in the experimental area than in the control area.'' The first word indicates that the authors were not disinterested in the outcome. Furthermore, in the discussion of the overall results, the distinction between supposed lighting effects on fear of crime and actual crime seems blurred at times.
The results were `mixed' to some extent, but the authors did not see this as a warning that their basic premises were flawed. Instead, they tried to explain some discrepancies with a supposition that stretches credibility (p 279):
``The most surprising result is that victimization of young people did not decrease more in the experimental area than in the control area. The qualitative data suggested that, whereas crimes by young people decreased, pestering of young people by older people did not decrease. Possibly, the improved street lighting inhibited offending by younger offenders against older victims but not offending by older offenders against younger victims.''
Painter and Farrington (2001a, p 278) gave more information than in their 1997 paper about the conduct of the experiment: there was regular contact between the principal experimenter and the fieldwork supervisor, local estate housing officers and the police. The principal experimenter also attended Tenants' Association meetings. Numerous opportunities could therefore have arisen for unwitting bias in comments to influence the surveys, which is not to say that this ever happened. Regardless, contact with tenants who may have been survey respondents before the final interviews does not seem to have been good practice. The double-blind interview procedure would not have provided an effective barrier against any bias thereby introduced.
Painter and Farrington (2001a) also provided information that was not in the 1997 paper about the absence of police records from the before-after comparisons. In the Conclusions, it is stated that police-recorded crime had been planned for inclusion in the study. This raises the issue of why it did not occur, given the potential value of the study for crime prevention. It seems reasonable to expect that the police should have ensured the provision of appropriately compatible crime records for the duration of the experiment, along with records of patrol durations in the experimental and control areas.
Given the potential importance of the work in influencing the expenditure of large sums on lighting in the UK and elsewhere, more pro-active cooperation might also have been expected from the council. In the planning stages, this would have allowed discussion of prospects of counterbalanced lighting treatment (decrease as well as increase), temporarily extended use of mercury-vapour lamps to reduce confounding and photometric surveys of the experimental and control areas. Neither of the Dudley papers mention these matters, but to be fair, the issues are doubtless more obvious in hindsight.
If the confounding of the Dudley experiment by the change in relative police visibility or any factor other than increased light is accepted as an explanation of the results, then the Painter and Farrington (2001a) [85] paper has rather limited value.
The Stoke-on-Trent study (Painter and Farrington (1999a) [83]) broadly follows the arrangements used in the Dudley study. The literature survey runs to eight pages. It is clearly based on the assumption that increased lighting reduces crime and virtually no mention is made of the possibility of no effect or an increase in crime. At the time the experiment was performed, 1992 and 1993, there was no justification for such a one-sided view.
On the basis of perceived need,16local council staff decided that the street lighting in a certain area of Stoke-on-Trent would be converted to high-pressure sodium luminaires. This became the experimental area. Nearby areas separated from the experimental area by physical features were chosen to be the control. Untreated areas contiguous with the experimental area were called `adjacent', providing a second control. The paper does not describe the existing street lighting in the adjacent and control areas; presumably it was ``older, domestic-type incandescent lamps'' like that originally present in the experimental area. The column spacing was reduced from 50 m to 38 m. The maintenance and energy costs doubled in the relit area and the ``amount of useful light increased fivefold''. As in the Dudley study, the photometric details are inadequately described, again ignoring the Tien et al. (1977) [107] warning of this as a contributory factor in poor experimental results.
Crime surveys were conducted in the experimental, adjacent and control areas before and after the lighting intervention. In the before survey, the prevalence of all crime in the experimental area was 69.2 % greater than in the control area, and the adjacent area crime was 63.0 % greater than in the control area. In the after survey, these values had changed to 11.7 % greater and 14.4 % greater respectively. Prevalence of crime in the control area increased from 34.1 % before to 38.3 % after. These figures suggest that crime in the experimental and adjacent areas was relatively elevated by comparison with the control area to begin with and fell in the course of the experiment. Regression to the mean would be a possible explanation for much of the observed changes.
In the before survey, the proportion of respondents who reported seeing police in the last month in the experimental area was 21.1 %. For the adjacent area, the figure was 25.9 %, and for the control area, 58.0 %. In the after survey, the values were 9.7, 12.4 and 11.1 % respectively. The surveyed police presence fell in all three areas but the largest change was in the control area, from 58.0 % before to 11.1 % after. This means that for the sampled months, police presence in the experimental area relative to the control area increased by (0.097/0.211)/(0.111/0.580), ie a 2.40 times increase after the relighting. For the adjacent area the relative increase was 2.50 times. This suggests strongly that the observed reductions in prevalence of crime in experimental and adjacent areas relative to the control area were a result of the relative more-than-doubling in police presence that took place after the lighting intervention, insofar as the `previous month' response given by respondents does actually represent the situation for the `previous 12 months' of the crime surveys. Police force areas were restructured in March 1992, ie during the period covered by the before crime survey. No other information is given in the paper about police deployment. Crimes recorded by the police for the police area covering the experimental, adjacent and control areas showed little overall change over the period of the study.
Painter and Farrington were aware of this confounding by the changes in police presence but as in the Dudley study, dismissed it on the basis of results of a logistic regression analysis. This again flies in the face of reason, given the known substantial effect of police presence in deterring crime. The similarity of the relative reductions in crime for the experimental and adjacent areas strongly suggests that the cause was the relative increase in police presence, a factor common to both areas, rather than the relighting, which was confined to the experimental area.
Given that the relative police presence in the experimental area apparently doubled or more after the relighting in both the Dudley and Stoke-on-Trent studies, it seems odd that Painter and Farrington did not discuss this in appropriate detail.
In interviews of seven police officers who had patrolled the Stoke-on-Trent areas, all expressed a preference for the relit area as being easier for them to work in. No information was given on whether the police had the discretion, or had been directed, to spend relatively more time in the experimental area after it had been relit.
Pedestrian street use of relit, adjacent and control areas was monitored for 2.5 hours on each of two nights in December 1992 and December 1993. The paper stated that the weather in each of these two periods was similar, cool and dry. As in the Dudley study, no reason was given for the absence of quantitative measures of weather characteristics.
The pedestrian counts showed an increase of 71 % in all pedestrian traffic after relighting in the experimental area. If it is accepted that this is reliably greater than the increase of 34 % in the adjacent area and 32 % in the control area, this raises a substantial issue that is not dealt with by the authors. These counts indicate that the adjacent area was like the control area. In practical terms, the brighter lighting of the experimental area would have affected the illumination in each part of the adjacent area only to a distance of about one pole-spacing, 50 m or less. This suggests that the adjacent area should have been treated as a control, in that it was largely unaffected by the relighting. But Painter and Farrington ignored this in concluding:
``Interestingly, decreases in crime in the adjacent area were almost as great as in the experimental area. This suggests that there was no displacement of crime, but rather a diffusion of the benefits of improved street lighting. Conceivably, the improved lighting in the experimental area deterred potential offenders not only in this area but in the adjacent area as well, since the areas were not clearly delimited. The qualitative data showing how information about the areas was communicated, and how relighting led to increased community pride in the adjacent area, supported this hypothesis.''
If the adjacent areas were indeed controls unaffected by the treatment and closely matched to the experimental area by proximity and similarity of housing, the most parsimonious explanation of the results is that relighting had no reliable effect on crime, and that crime in the original control area increased relatively because of the relative reduction of police presence or because of other asymmetric factors.
For the several reasons given, the writer is unable to accept that the claimed beneficial effect on crime in the Stoke-on-Trent study was caused by increased illumination when it could also have been caused by any one or more of choice of control areas, change in police presence, change in glare, change in beam pattern, change in inter-pole distance, change in lighting colour, or change in flicker modulation depth. On top of all this is the potential for unwitting bias from conflict of interest. Funding for the work was provided by a representative of a lighting company and by the Midlands Electricity Board. Both funding sources stood to benefit from the finding that a beneficial effect on crime resulted from increased lighting.
Painter and Farrington (1999b, 2001b) [84,86] are successive recalculations of the monetary value of the crime reduction claimed for the Dudley and Stoke-on-Trent relighting projects. Painter and Farrington (1999b) [84] stated:
``Thus, in the case of lighting improvements, if crime is being shuffled from relit to darker areas, or from night to day, the total amount of crime would not be reduced.''
It is not at all obvious how or why brighter lighting at night as a claimed crime prevention measure might displace crime into the much brighter conditions of daylight while there was also the expectation that crime would be displaced into darker conditions.
High-pressure sodium lights are yet again described as white, but the ``older domestic type tungsten lamps'' they replaced in Stoke-on-Trent are actually whiter.
Information not previously given about the Stoke-on-Trent project is that other crime prevention strategies continued to run during the course of the experiment. These strategies were monitored but no details are given about how this was done, whether or not they were a threat to the validity of the experimental results, and what actions were taken if they were a threat, and when.
Following the Painter and Farrington (2001b) [86] paper are printed comments by a representative of the lighting company that funded the work. He said he was ``glad that [the work] had a positive outcome'' and that it was a powerful tool he hoped would be used to seek funds for public lighting in the face of competing interventions.17Another commentator wrote of the ``good news that it [the paper] brings to those who believe in the value of lighting.'' Revealing though they may be, comments like these appear out of place in a scientific journal and lower its standing. The authors' reply then puts the need for dose-response studies along with interviews of offenders and victims. They might usefully consider or reconsider the offender interviews described by Ramsay and Newton (1991) [96] (see Section 3.2), in which lighting was generally discounted as a factor.
An additional serious problem affecting the Dudley and Stoke-on-Trent studies is described in Section 5.2 below. Meanwhile, any one or more of the several shortcomings already described may be sufficiently serious to invalidate the conclusions, regardless of Scientific Methods Scores, the number of variables studied and the sophistication of the statistical analysis. Most of the findings of the two studies appear to be unreliable, including the supposed cost savings brought about by relighting expenditure. The effects of brighter lighting in increasing pedestrian numbers and reducing fear of crime might be thought reasonable and innocuous, but there is more on this in Section 5.2, and much more to come in Part 2 of this work.
The two works [89,90] discussed here are reviews rather than experiments. They are included because they give further insights into lighting and crime experiments, especially the Dudley and Stoke-on-Trent studies, and the influence of Situational Crime Prevention theory.
Pease (1999) [90] reviewed the lighting and crime literature and claimed that Painter's recent research provided firm evidence that where street lighting improvements were successful, they reduced crime by day as well as at night, probably because of changes in street use, enhanced community pride and sense of area ownership. Pease's re-analysis of the data also suggested that beneficial lighting effects are greater in chronically victimised areas.
Pease (1999) drew attention to the ``rash'' of existing reviews on lighting and crime while adding another. Apart from typographical errors and other signs of hasty preparation, the many more serious faults in his review virtually guarantee that more reviews will follow. An example is inconsistency between the review's own summary and similar material in another document issued earlier as a summary:
``4. In the most recent and sophisticated studies, street lighting improvements are associated with crime reductions in daytime as well as during the hours of darkness. This invites speculation that the effects of lighting work through community pride and sense of ownership as well as more directly through surveillance of offenders. Re-analysis of data from these studies suggests that lighting effects are greater in chronically victimised areas;'' (Pease 1999, the full review).
``4. Street lighting improvements, where successful, are associated with crime reductions in daytime as well as during the hours of darkness. This result is of fundamental importance. It means that the effects of lighting work through something more general than improvement in the surveillability of potential offenders at night. The most plausible reasons for this pattern concerns [sic] changes in street use, enhanced community pride and sense of area ownership. Re-analysis of data from these studies suggests that lighting effects are greater in chronically victimised areas, which is of particular importance for integration of street lighting in other schemes devised under the provisions of the Crime and Disorder Act 1998;'' (Pease 1998, the separate summary).
At least some of the factors that Pease saw as important are unlikely to have been recognized as potentially important when the original studies were designed. It is therefore unlikely that these factors would have been properly controlled for, if at all, nor might sufficient data have been collected, if any, to allow reliable conclusions to be drawn about them. Even if Pease's speculation about the reasons for the claimed effects of lighting is accepted as worth testing scientifically, new studies rather than reanalysis of existing data would be preferable to test the hypotheses.
In the summary document, Pease (1998) [89] stated:
``Crime prevention practitioners have always included lighting as part of their toolbox, and have advocated its use accordingly.''
The statement may be true but is no proof at all of the effectiveness of lighting, merely a belief about it that has been maintained in recent years despite a shortage of reliable supporting evidence. Failure to qualify the statement accordingly could be interpreted as bias by omission.
Pease mentioned a view attributed to the UK Home Office that street lighting does not have effects in reducing crime, and he expressed concern that this view could exclude or limit the role of street lighting in local crime and disorder prevention strategies required under the UK Crime and Disorder Act 1998. At the time, neither the view nor the attribution was at odds with the consensus of research findings on the subject. The Home Office had funded some of the studies involved. A computer search through the text of the Act found no mention of lighting. Guidance documents on the Home Office website indicate that lighting can be included in programs set up under the Act to reduce fear of crime. Any lack of effectiveness in preventing crime does not appear to block the use of lighting. There is nothing to prevent trials of, say, reduced intensity or reduced glare street lighting as possible crime and fear of crime reduction measures. But this is clearly not what Pease had in mind when he stated it was ``timely to consider the effect of street lighting on crime afresh''.
Pease showed that the Atkins, Husain and Storey (1991) [3] data could be re-analysed to indicate that relighting in the Wandsworth part of London, over the years 1984 to 1989, reduced crime in the area, contrary to the original findings. In the absence of sufficient data for a proper time-series analysis, Pease used an abridged method that was invalid for several reasons. Although admitting that the procedure was only suggestive, he still quoted a numerical probability to support his views. But Pease's work does draw attention to serious shortcomings in the Atkins, Husain and Storey study: for example, the weak experimental design, the presence of insufficiently identified data in an appendix, and an unjustified belief that lighting changes would not bring about social changes that could have some effect on crime in daylight.
Pease stated that his concern provided the motivation for his review of research evidence. The (UK) Lighting Industry Federation funded this review (Pease 1998). Publication of the review by the Institution of Lighting Engineers was prearranged to take place, regardless of the findings, as an indication of the reviewer's independence. This information about motivation, funding and publication is given only in the summary document. Under the constraints of scientific method, the prearrangement might have been a necessary condition to indicate an effectively unbiased approach, but not a sufficient condition. The funding source and publication guarantee should not have been omitted from the full text of the review, which carries a later date than the summary document.
The Campbell Collaboration is a body committed to publishing high quality reviews of the effects of sociological and educational interventions. Its policy on conflict of interest includes the following statements (Campbell Collaboration 2002 [17]):
``Reviewers should report any conflict of interest capable of influencing their judgments, including personal, political, academic, and other possible conflicts, as well as financial conflicts. It is impossible to abolish conflict of interest, since the only person who does not have some vested interest in a subject is somebody who knows nothing about it... Disclosing a conflict of interest does not necessarily reduce the worth of a review and it does not imply dishonesty. However, conflicts of interest can influence judgments in subtle ways.''
``It is a matter of Campbell Collaboration policy that direct funding from a single source with a vested interest in the results of the review is not acceptable.''
Pease's review does not comply with the long established requirement for a scientific review to have face validity in terms of freedom from conflict of interest.
In Pease's review, the strength of the scientific evidence against lighting effects on crime is given unduly scant coverage. The massive and rigorous study of Sherman et al. (1997) [102] is referenced but its conclusions are ignored. The sole mention it gets from Pease is dismissal of part of just one sentence as opinion.
Pease's review includes a `selected annotated bibliography' of 13 papers and book chapters limited to those claiming some sort of beneficial effect of lighting in reducing crime or fear of crime, a further demonstration of unacceptable bias. Other specific examples are Pease's descriptions of those sharing his views as ``children of light'', and of those ``...yet to be convinced of lighting effects on crime'' as ``disciples of darkness'' having ``dogmatic'' and ``reactive'' views, not to mention his assessment of Painter's Dudley and Stoke work as a ``technical tour de force'' and ``the last word''.
Pease (1999) [90] made much of Painter's conclusion that increased lighting is an effective crime prevention measure when targeted to small `crime hotspots'. But this approach actually favours the return of false beneficial results, because:
Painter and Farrington mentioned the regression to the mean effect in their papers as a potential threat to internal validity but Pease did not discuss it.
Overall, Pease's review looks to be one-sided and unconvincing. Although it appears justifiable to continue testing for effects of lighting changes on day and night crime rates and displacement in new ad hoc studies, researchers do need to be reasonably disinterested in the outcome and a necessary but not sufficient condition for this is financial independence from lighting and related industries. In no way does this statement imply that the works referred to were subject to any sort of conscious bias because of the support arrangements, far from it. The issue is purely about the known need for a high degree of compliance with this aspect of scientific method.
The Quinet and Nunn (1998) [95] paper reads as though based on a belief that more lighting does prevent crime and all that is required to demonstrate this is a sufficiently well designed and executed study. Nowhere do they mention the possibility that lighting may have no effect or even aid the commission of crime, as discussed by Eck (1997) [31], a work not reviewed although it had been available for well over a year before their publication date of December 1998. Their review does include uncritical brief summaries of Painter (1990, 1994b) [77,80] and Painter and Farrington (1997) [82], but their own field study was based on data ``... for particular crimes, and only for crimes occurring at night (the only meaningful time period to use when assessing the impact of lighting)''. This flatly contradicts one of the key claims of Painter and Farrington (1997). The work was supported by funding from the Indiana Electric Association.
Quinet and Nunn analysed the number of calls for police service before and after additional streetlights were placed in Indianapolis neighbourhoods to `enhance' the lighting. Residents groups had the extra lights installed in areas where they thought they were needed, ie areas with apparently high crime incidence. Non-contiguous areas without lighting changes were included in the analysis as controls. Apart from the numbers of lights, no details are given of the existing and additional lighting: whether the lamps were identical in type, output, beam spread, glare, mounting height etc., whether the numbers of poles and their spacings were changed and so on.
The analysis did not include displacement effects: ``Either you have a control area free of influences from the experimental area, or you do not.'' Some of the results suggest that criminals either did not know of this constraint, or ignored it. As Quinet and Nunn put it, some of the results of their ``vigorous [sic] scientific assessment'' were ``mixed'', ``very mixed'', ``extremely mixed'', and ``counter to expectations'', and ``Although none of these differences were statistically significant changes, they are nonetheless suggestive of the expected deterrent influence of enhanced street lighting.'' Despite their attempts to explain away the unexpected results, the overall finding is at best inconclusive. Of course, `mixed' results are precisely what a statistically based study is likely to produce when the variables are unrelated, and also when a genuine relationship between the variables is either too weak for reliable detection using the available data and tests or is substantially affected by uncontrolled factors.
Quinet and Nunn concluded ``The analysis of the target areas suggests that enhanced street lighting in particular neighborhoods is sometimes associated with concurrent reductions in reported crime.'' The results really justified them saying `increases' instead. Farrington and Welsh (2002a,b) [34,35] reassessed their results and found an odds ratio of 0.75, meaning that crime was 25 % more likely in the experimental areas than in the control areas after the lighting treatment.