Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-02-04T21:09:09.757Z Has data issue: false hasContentIssue false
  • English
  • Français

Judging Surveys: Experts, Empirical Evidence and Law Reform

Published online by Cambridge University Press:  24 January 2025

Gary Edmond
Gary Edmond*
Affiliation:
Faculty of Law The University of New South Wales, Sydney 2052, Australia
*
E-mail: <g.edmond@unsw.edu.au>.
Get access Rights & Permissions [Opens in a new window]

Extract

This article examines the conduct of empirical legal research and its relationship to law reform. Through a detailed analysis of the largest survey of State and federal judges conducted in Australia it explores some of the limits to empirical investigation, particularly the tendency to rely primarily on judicial perspectives as the basis for law reform. Focusing upon empirical legal research on the subject of expert evidence the article initially examines research methodologies, then extends the analysis to consider the correspondence between the collection, interpretation and presentation of empirical data and recommendations for legal change. This involves an assessment of a broad range of methodological and theoretical issues with implications extending well beyond the particular survey. Last, the empirical research on expert evidence will be evaluated using the principal reform proposal suggested by the investigators. This exercise will provide an indication of methodological problems which beset the survey and demonstrate practical limitations with the particular approach to expertise.

Type
Research Article
Information
Federal Law Review , Volume 33 , Issue 1 , March 2005 , pp. 95 - 139
Copyright
Copyright © 2005 The Australian National University

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Earlier versions of this paper were presented in a special session on ‘Science and Law’ at the Australasian Association for the History, Philosophy and Social Studies of Science Annual Conference 2003, University of Melbourne, 30 June – 3 July 2003, at the Staff Seminar Series, Faculty of Law, University of Newcastle, August 2003 and at the 21st Annual Law and Society Conference, University of Newcastle, 8–10 December 2003. The author would like to thank those who commentated on drafts, along with the referees and editors.

References

1 Similar issues arise in relation to the recent reforms to Federal Court and NSW Supreme Court procedures: Practice Direction: Guidelines for Expert Witnesses in Proceedings in the Federal Court of Australia 2003 (Cth) and the ‘Expert Witness Code of Conduct’ in the Supreme Court Rules 1970 (NSW) sch K.

2 Ian, Freckelton, Prasuna, Reddy and Hugh, Selby, Australian Judicial Perspectives on Expert Evidence: An Empirical Study (1999)Google Scholar and Ian, Freckelton, Prasuna, Reddy and Hugh, Selby, Australian Magistrates’ Perspectives on Expert Evidence: A Comparative Study (2001)Google Scholar. This article is primarily focused on the survey of judges, though most of the comments are applicable to the subsequent survey of magistrates. References to Australian Judicial Perspectives are abbreviated as ‘AJP'.

3 AJP, above n 2, 16.

4 Ibid.

5 The reader is told that the opinions of other participants will be examined at some future stage.

6 AJP, above n 2, xi–ii.

7 Ibid 1.

8 This article does not attempt to systematically analyse the judicial comments collected with the questionnaire. Primarily because: the multiple-choice questions and answers appear to be the principal basis for the empirical assertions in the text, and the basis for all numerical claims; the comments are not available to readers whereas the assessment of the prefigured responses and data provide materials which are readily accessible; and finally because there is no obvious methodology associated with the presentation of judicial comments. In general, the comments are used to confirm or reinforce, and occasionally qualify, results. We are not always told how representative particular comments are, nor how many judges actually produced written comments (to particular questions).

9 AJP, above n 2, 7.

10 Ibid 38.

11 Steve, Woolgar and Dorothy, Pawluch, ‘Ontological Gerrymandering: The Anatomy of Social Problems Explanations’ (1985) 32 Social Problems 214Google Scholar. One caveat. It might be thought that there is a certain irony in criticising the work of others, especially where it involves the identification of social problems, by attributing methodological inadequacies (or problems) to their own practices. These kinds of debates, concerned with manipulating the existence, identity and extent of social problems, have been described by Woolgar and Pawluch as forms of ontological gerrymandering. The concerns espoused by Woolgar and Pawluch might have been (more) pertinent if this article was purely theoretical or simply rejected the existence of problems — including problems with bias — associated with the use of expertise. But it does neither of these things. The article does not attempt to suggest that the use of expertise is without difficulties. And, quite intentionally, it reinforces the importance of methodologically competent social scientific research and the value of incorporating perspectives from research traditions beyond conventional areas of legal scholarship.

12 AJP, above n 2, 142. The questions and responses are set out in appendices to the Report.

13 Ibid 143–4.

14 Interestingly, those people who actually interview and study scientists (and experts) have not consistently identified forms of bias (or partisanship) as serious problems in the practice of science. Rather, studies of scientists suggest that the simplistic normative models of science (or institutional imperatives), conventionally and most famously associated with Robert Merton — such as commun(al)ism, disinterestedness, organised skepticism and universalism — do not adequately capture or explain scientific practice. Indeed, studies by Mitroff indicate that scientists may actually value ‘counter-norms’ — such as dogged commitment to ideas, even in the face of apparently disconfirming evidence. Further, more recent work on norms and rule following (inspired by Wittgenstein) would seem to indicate that even where strong norms or prescriptive rules exist, the rules rarely explain how they should be applied and this enables a range of divergent — even competing — interpretations. Consequently, norms like disinterestedness or skepticism are practically incapable of guiding practice in any specific context, especially in areas of uncertainty and controversy. See Robert, Merton, The Sociology of Science: Theoretical and Empirical Investigations (1973)Google Scholar; Ian, Mitroff, The Subjective Side of Science: A Philosophical Inquiry into the Psychology of the Apollo Moon Scientists (1974)Google Scholar; Harry, Collins and Trevor, Pinch, Frames of Meaning: The Social Construction of Extraordinary Science (1982)Google Scholar; Michael, Mulkay, ‘Interpretation and the Use of Rules: The Case of the Norms of Science’ in Thomas, Gieryn (ed), Science and Social Structure: A Festschrift for Robert Merton (1980) 111Google Scholar.

15 AJP, above n 2, 38. See also at 12, 29.

16 Ibid 113.

17 There is repetition in the treatment of bias and partisanship (Questions 2.2, 2.3, 2.11, 6.8 and 6.9). This may help to explain why judges ‘raised the issue more than once'. Furthermore, there is a considerable literature on the ordering of questions, the use of repetition and how early questions may stimulate or structure the responses to subsequent questions. See, eg, Jon, Krosnick and Duane, Alwin, ‘An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement’ (1987) 51 Public Opinion Quarterly 201Google Scholar; William, Locander and John, Burton, ‘The Effect of Question Form on Gathering Income Data by Telephone’ (1976) 13 Journal of Marketing Research 189Google Scholar.

18 ‘The judges’ identification of clarity of explanation as the single most important factor in the oral delivery of evidence highlighted the importance of expert witnesses developing communication skills of a high order if they are to supply to decision-makers information with which they are able to deal adequately': AJP, above n 2, 116 (emphasis added).

19 Ibid 143.

20 Ibid 146.

21 Ibid 13, 40.

22 Research suggests that respondents are more likely to endorse answers incorporated with the survey than supply their own. See William, Belson and Judith, Duncan, ‘A Comparison of the Check-list and the Open Response Questioning Systems’ (1962) 11 Applied Statistics 120Google Scholar; Howard, Schuman and Stanley, Presser, Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording and Context (1981)Google Scholar; Peter, Hiller, ‘The Subjective Dimensions of Social Stratification: The Case of the Self-Identification Question’ (1973) 9(2) Australian and New Zealand Journal of Sociology 14Google Scholar; Aravind, Joshi, ‘Varieties of Cooperative Responses in Question–Answer Systems’ in Ferenc, Kiefer (ed), Questions and Answers (1983) 229Google Scholar.

23 AJP, above n 2, 154.

24 A sizeable portion did, however, identify ‘bias’ as some kind of problem. This raises several issues, including whether or not: this is an artifact of the survey; can be explained in other ways; and (depending on how we define bias) represents an incorrigible feature of (the representation of) expertise.

25 AJP, above n 2, 26. Presented as non-controversial or self-evident, the categories ‘bias', ‘partisanship’ and ‘lack of objectivity’ appear repeatedly throughout Perspectives. The failure to clearly define bias (contrast AJP, above n 2, 24 n 3) and the extent to which biased expertise can be identified or remedied are serious oversights. On definitional problems consider Renate Mayntz, Kurt Holm and Roger Huebner, Introduction to Empirical Sociology (A Hammond, H Davis and D Shapiro trans, 1976 ed) 7–21 [trans of: Einführung in die Methoden der empirischen Soziologie]; Patrick, McNeill, Research Methods (2nd ed, 1990) 23–5Google Scholar.

26 AJP, above n 2, 144.

27 Paul, Lazarsfeld, ‘Evidence and Inference in Social Research’ (1958) 87(4) Daedalus 99Google Scholar. For a more general discussion of some of the implications of classification, consider Geoffrey, Bowker and Susan, Star, Sorting Things Out: Classification and Its Consequences (1999)Google Scholar.

28 AJP, above n 2, 36–8, 60. However, we should not assume that these categories are independent.

29 Ibid 3, 25–6. Intriguingly, more respondents reported encountering bias (Question 2.2, 92%: at 143–4) than partisanship (Question 6.8, 85%: at 154).

30 There are several comments about costs but no questions in the survey were directed to issues of cost. There are questions about taking experts out of play, but this is a very small part of the larger question about expert availability. See Carol, Jones, Expert Witnesses: Science, Medicine and the Practice of Law (1994) 128–64Google Scholar.

31 AJP, above n 2, 3. Numerous writers have commented upon how survey respondents do not necessarily agree on the meaning of commonplace terms. See William, Belson, The Design and Understanding of Survey Questions (1981)Google Scholar; Robert, Nuckols, ‘A Note on Pre-testing Public Opinion Questions’ (1953) 37 Journal of Applied Psychology 119Google Scholar; above n 25.

32 One possible explanation is that ‘bias’ is perceived as a problem for jurors. However we are provided with no reliable information about the ability of juries (or judges) to identify or assess the significance of ‘bias'.

33 Some social scientists might contend that questions pertaining to ‘bias’ should follow questions about judicial use of the rules of evidence. Adopting such an approach, questions about actual practice and the use of rules of evidence ought to have preceded and ‘filtered’ reflection on apparent problems. It is widely accepted that survey responses are shaped by question order effects. Consider the discussion of filtering in William, Foddy, Constructing Questions for Interviews and Questionnaires: Theory and Practice in Social Research (1993) 101–11Google Scholar.

34 AJP, above n 2, 155–6.

35 Evidence Act 1995 (Cth); Evidence Act 1995 (NSW); Evidence Act 2001 (Tas).

36 AJP, above n 2, 116.

37 Ibid 88–9. This rule was substantially abolished in the Uniform Evidence Acts s 80.

38 AJP, above n 2, 159.

39 Joe, Cecil and Thomas, Willging, Court-Appointed Experts: Defining the Role of Experts Appointed under Federal Rule of Evidence 706 (1993)Google Scholar; Laural, Hooper, Joe, Cecil and Thomas, Willging, ‘Assessing Causation in Breast Implant Litigation: The Role of Science Panels’ (2001) 64 Law and Contemporary Problems 139Google Scholar; Michael, Saks, ‘The Phantom of the Courthouse’ (1995) 35 Jurimetrics Journal 233Google Scholar.

40 Frank, Turner, Contesting Cultural Authority: Essays in Victorian Intellectual Life (1993) 201Google Scholar; Andrew, Abbott, The System of Professions: An Essay on the Division of Expert Labor (1988)Google Scholar.

41 Stanley, Cohen, Visions of Social Control: Crime, Punishment and Classification (1985)Google Scholar; Joseph, Gusfield, The Culture of Public Problems: Drinking-Driving and the Symbolic Order (1981)Google Scholar.

42 Gary, Edmond, ‘The Law-Set: The Legal–Scientific Production of Medical Propriety’ (2001) 26 Science, Technology and Human Values 191Google Scholar.

43 This shift of agency operates as a kind of supererogation, where experts are burdened with (some of) the ‘sins’ of the legal system. Consider Gary, Edmond, ‘Constructing Miscarriages of Justice: Misunderstanding Scientific Evidence in High Profile Criminal Appeals’ (2002) 22 Oxford Journal of Legal Studies 53Google Scholar.

44 Alfred, Schutz, ‘Phenomenology and the Social Sciences’ in Collected Papers (1962) vol 1, 118–39Google Scholar.

45 ‘Gaps’ between public accounting (what people say they do or believe) and actual behaviours (what they actually do or believe) are notorious in social scientific research. McNeill makes the point that:

The interview schedule or questionnaire means that the researcher is setting limits to what the respondents can say. … Fundamentally, the survey [or questionnaire] method finds out what people will say when they are being interviewed, or filling in a questionnaire. This may not be the same thing as what they actually think or do.

McNeill, above n 25, 47; Irwin, Deutscher, What We Say/What We Do: Sentiments and Acts (1973)Google Scholar.

46 AJP, above n 2, 16.

47 Compare Jonathan, Potter, Representing Reality: Discourse, Rhetoric and Social Construction (1996)Google Scholar.

48 Contrast the situatedness of all knowledge developed by Donna, Haraway, Simians, Cyborgs and Women: The Reinvention of Nature (1991)Google Scholar.

49 AJP, above n 2, 1 (emphasis added).

50 The Report indicates that this survey represents the initial stages of a more comprehensive enterprise. Without wanting to imply that a more expansive and diversified inquiry would repair the methodological limitations, the initial inquiry discloses an exclusive interest in the attitudes of judges and magistrates.

51 Roland Barthes, Mythologies (Annette Lavers trans, 1972 ed) [trans of: Mythologues].

52 Questions about whether judges should dominate law reform, or supply the ‘empirical’ evidence for law reform, are fundamentally political.

53 AJP, above n 2, 1.

54 Ibid 1–2.

55 Ibid 2.

56 Ibid 13.

57 Schuman and Presser, above n 22; Tom, Smith, ‘Non-Attitudes: A Review and Evaluation’ in Charles, Turner and Elizabeth, Martin (eds), Surveying Subjective Phenomena (1984) vol 2, 215Google Scholar.

58 Ian, Freckelton, ‘Judicial Attitudes Toward Scientific Evidence: The Antipodean Experience’ (1997) 30 University of California Davis Law Review 1137, 1212Google Scholar.

59 AJP, above n 2, 65 (emphasis added).

60 Ibid 43.

61 Ibid 66. This makes a discussion, which criticises mock jury research, especially ironic. The authors note that investigations using mock jurors ‘are open to criticism in terms of their methodology, their selection procedures, their cultural matrix, and the fact that most used mock, rather than real, jurors': at 45. These other inquiries, however flawed, did at least target jurors and mock jurors. Furthermore, the authors make few attempts to explain (in)consistencies between judicial perspectives and the claims promoted in other parts of the Report. For example, while answers provided by judges do not seem to support the abolition of the lay jury, elsewhere in the Report (at 38, 43–5) more critical approaches to jury (and implicitly judicial) competence are espoused and apparently endorsed.

62 Ibid 66. On the power of quantification, see Theodore, Porter, Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (1995)Google Scholar; Walter, Williams, Honest Numbers and Democracy: Social Policy Analysis in the White House, Congress and the Federal Agencies (1998)Google Scholar.

63 See Harry, Kalven and Hans, Zeisel, The American Jury (1966)Google Scholar.

64 AJP, above n 2, 150.

65 Other models of expertise, understanding and competence might be used to nuance, and even defend, the jury (and judges). Consider, for example, recent research into the public understanding of science (PUS) by Alan, Irwin and Brian, Wynne (eds), Misunderstanding Science: The Public Reconstruction of Science and Technology (1996)Google Scholar; Alan, Irwin, Citizen Science: A Study of People, Expertise and Sustainable Development (1995)Google Scholar; Harry, Collins and Robert, Evans, ‘The Third Wave of Science Studies: Studies of Expertise and Experience’ (2002) 32 Social Studies of Science 235Google Scholar; Alan, Irwin, ‘Expertise and Experience in the Governance of Science: What is Public Participation for?’ in Gary, Edmond (ed), Expertise in Regulation and Law (2004) 32Google Scholar and more general approaches to lay understanding such as Nicholas, Abercrombie and Brian, Longhurst, Audiences: A Sociological Theory of Performance and Imagination (1998)Google Scholar; James, Scott, Domination and the Arts of Resistance: Hidden Transcripts (1990)Google Scholar.

66 The sociologist Collins has argued that sometimes when things which are intended as demonstrations — deliberately staged to restrict the extent of interpretative flexibility — ‘go wrong’ they revert to the status of experiment — where the level of ambiguity is much greater. If we problematise some of the assumptions underlying the survey — such as the sufficiency of judicial perspectives or appropriateness of answering some of the questions — then this demonstration of the real, via judicial attitudes, can be understood as an experiment into judicial aptitudes and/or professional interests. See Harry, Collins, ‘Public Experiments and Displays of Virtuosity: The Core-Set Revisited’ (1988) 18 Social Studies of Science 725Google Scholar.

67 Jean, Converse, ‘Predicting No Opinion in the Polls’ (1976) 40 Public Opinion Quarterly 515Google Scholar; Howard, Schuman and Stanley, Presser, ‘Public Opinion and Public Ignorance: The Fine Line Between Attitudes and Non-Attitudes’ (1980) 85 American Journal of Sociology 1214Google Scholar; George, Bishop et al, ‘Pseudo-Opinions on Public Affairs’ (1980) 44 Public Opinion Quarterly 198Google Scholar; Clyde, Coombs and Lolagene, Coombs, ‘“Don't Know“: Item Ambiguity or Respondent Uncertainty’ (1976) 40 Public Opinion Quarterly 497Google Scholar; Jon, Krosnick et al, ‘The Impact of “No Opinion” Response Options on Data Quality: Non-Attitude Reduction or an Invitation to Satisfice?’ (2002) 66 Public Opinion Quarterly 371Google Scholar.

68 AJP, above n 2, 65.

69 Ibid 2 (emphasis added).

70 Ibid 111.

71 Ibid 144. See also at 40.

72 Ibid 145.

73 Ibid 150.

74 Ibid 151.

75 Ibid 40 (emphasis added).

76 Ibid 6–7. See Harrington-Smith on behalf of the Wongatha People v Western Australia (No 2) ('Wongatha’ Judgment No 7) (2003) 130 FCR 424, 427–8 [19]; Jango v Northern Territory (No 2) [2004] FCA 1004 [9].

77 Ibid 113.

78 Ibid 145.

79 Ibid 145–6.

80 Ibid 55 (emphasis added).

81 Ibid 116. Several sociologists characterise representations of complexity and uncertainty as symbolic forms of action: Brian, Campbell, ‘Uncertainty as Symbolic Action in Disputes Among Experts’ (1985) 15 Social Studies of Science 429Google Scholar; Susan, Star, ‘Scientific Work and Uncertainty’ (1985) 15 Social Studies of Science 391Google Scholar.

82 Compare Jerome, Ravetz, Scientific Knowledge and Its Social Problems (1971)Google Scholar.

83 Clifford, Geertz, ‘Thick Description: Toward an Interpretative Theory of Culture’ in The Interpretation of Cultures: Selected Essays (1973) 3Google Scholar.

84 In the influential Access to Justice report, Lord Woolf appeared to recognise the continuing need for ‘red-blooded adversarialism', albeit in fewer and fewer cases: Lord, Woolf, Access to Justice (1996) 138–9Google Scholar.

85 Gary, Edmond, ‘Judicial Representations of Scientific Evidence’ (2000) 63 Modern Law Review 216Google Scholar.

86 AJP, above n 2, 2. See also at 7, 12.

87 Ibid 10 (emphasis added).

88 Ibid 13 (emphasis added). See also at 70.

89 Barry, Barnes, Understanding Agency: Social Theory and Responsible Action (2000)Google Scholar.

90 AJP, above n 2, 57.

91 Ibid 10–12, 113–15.

92 Ibid 10, 113.

93 Ibid 113.

94 Ibid 10, 113.

95 Lord Woolf, above n 84.

96 AJP, above n 2, 57.

97 Ibid 149.

98 Ibid. Although, even combined (at 37%), these were lower than the number of ‘Missing responses’ (38%).

99 Ibid 5–6.

100 Ibid 149.

101 Ibid 61.

102 Ibid 12.

103 Ibid 149–50 (Question 4.5).

104 Ibid 63. See also at 12.

105 Ibid 116–17.

106 This strategy was extended to expert evidence (not explicitly intended for litigation) by Von Doussa J in the Hindmarsh Island Bridge litigation: Chapman v Luminis Pty Ltd (2001) 123 FCR 62. See Gary, Edmond, ‘Thick Decisions: Expertise, Advocacy and Reasonableness in the Federal Court of Australia’ (2004) 74 Oceania 190Google Scholar.

107 Steven, Connor, Dumbstruck: A Cultural History of Ventriloquism (2000)Google Scholar.

108 Historical work on the role of experts in court demonstrates their tremendous value as well as the negotiations around formal legal recognition and definitions of reliability and competence. Historical accounts frequently suggest close and continuing relations between courts and fields of expertise, particularly psychology, medicine and the forensic sciences, see Jones, above n 30; Tal, Golan and Snait, Gissis (eds), ‘Science and Law’ (1999) 12 Science in Context 3Google Scholar; Jennifer, Mnookin, ‘The Image of Truth: Photographic Evidence and the Power of Analogy (1998) 10 Yale Journal of Law and the Humanities 1Google Scholar; Simon, Cole, Suspect Identities: A History of Fingerprinting and Criminal Identification (2001)Google Scholar; Roger, Smith and Brian, Wynne (eds), Expert Evidence: Interpreting Science in the Law (1989)Google Scholar; Tal, Golan, Laws of Men and Laws of Nature: The History of Scientific Expert Testimony in England and America (2004)Google Scholar.

109 AJP, above n 2, 117 (emphasis added).

110 Ibid 118 (emphasis added).

111 Ibid.

112 Ibid 149.

113 Ibid 160.

114 John, Sink, ‘The Unused Power of a Federal Judge to Call His Own Expert Witness’ (1956) 29 Southern California Law Review 195Google Scholar; Tahirih, Lee, ‘Court-Appointed Experts and Judicial Reluctance: A Proposal to Amend Rule 706 of the Federal Rules of Evidence’ (1988) 6 Yale Law and Policy Review 480Google Scholar.

115 AJP, above n 2, 17.

116 Ibid 104.

117 Ibid 156–7.

118 Ibid 113.

119 Ibid 118.

120 Consider also Freckelton, above n 58, 1212:

The project is timely because of the assertions advanced in some quarters that Australia's commitment to the adversary process needs to be revisited, in part, on the basis that the community cannot underwrite its cost and on the basis that juries can no longer cope with the complexities and conflicting nature of modern-day expert evidence.

121 Some of these issues have been canvassed more extensively in Gary, Edmond, ‘After Objectivity: Expert Evidence and Procedural Reform’ (2003) 25 Sydney Law Review 131Google Scholar and Gary, Edmond and David, Mercer, ‘Experts and Expertise in Legal and Regulatory Settings’ in Gary, Edmond (ed), Expertise in Regulation and Law (2004) 1Google Scholar.

122 Interestingly, where judges comment on the partisanship of experts they may compromise their own independence. See Vakauta v Kelly (1989) 167 CLR 568.

123 These are not new observations. In Lord Abinger v Ashton (1873) 17 LR Eq 358, 374 Sir George Jessel MR wrote

'in matters of opinion I very much distrust expert evidence, for several reasons. In the first place, although the evidence is given upon oath, in point of fact the person knows he cannot be indicted for perjury, because it is only evidence as to a matter of opinion. So that you have not the authority of legal sanction.'

124 Carl, Cranor and David, Eastmond, ‘Scientific Ignorance and Reliable Patterns of Evidence in Toxic Tort Causation: Is There a Need for Liability Reform?’ (2001) 64 Law and Contemporary Problems 5Google Scholar.

125 AJP, above n 2, 23–9; Learned, Hand, ‘Historical and Practical Considerations Regarding Expert Testimony’ (1901) 15 Harvard Law Review 40Google Scholar; Jack, Weinstein, ‘Improving Expert Testimony’ (1986) 20 University of Richmond Law Review 473, 482Google Scholar; Margaret, Hagen, Whores of the Court: The Fraud of Psychiatric Testimony and the Rape of American Justice (1997)Google Scholar.

126 While not defined, the authors repeatedly insinuate an image of ‘reliability', similar to that promoted in a seminal US Supreme Court decision (Daubert v Merrell Dow Pharmaceuticals, Inc, 509 US 579 (1993) ('Daubert’)), notwithstanding apparent wholesale rejection by the surveyed judges (AJP, above n 2, 8, 75-79, 91, 93, 101, 115). More than 50 per cent of judges did not consider ‘reliability’ to be a prerequisite for the admission of expert evidence (Question 6.2) and fewer than 20 per cent considered falsification necessary for determining reliability (Question 6.3). See also Ian, Freckelton and Hugh, Selby, The Law of Expert Evidence (1999) 84, 267Google Scholar, 338–9; Ian, Freckelton, ‘The Challenge of Junk Psychiatry, Psychology and Science: The Evolving Role of the Forensic Expert’ in Hugh, Selby (ed), Tomorrow's Law (1995) 52Google Scholar; Ian, Freckelton, ‘Contemporary Comment: When Plight Makes Right—The Forensic Abuse Syndrome’ (1994) 18 Criminal Law Journal 29Google Scholar. Compare Gary, Edmond and David, Mercer, ‘Keeping “Junk” History, Philosophy and Sociology of Science Out of the Courtroom: Problems with the Reception of Daubert v Merrell Dow Pharmaceuticals Inc.’ (1997) 20 University of New South Wales Law Journal 48Google Scholar.

127 AJP, above n 2, 115. Two influential and politically conservative US commentators also advocate the use of ‘mainstream science': Kenneth, Foster and Peter, Huber, Judging Science: Scientific Knowledge and the Federal Courts (1997)Google Scholar. Huber is a senior fellow of the Manhattan Institute, a politically conservative pro-business think tank. Compare Gary, Edmond and David, Mercer, ‘Juggling Science: From Polemic to Pastiche’ (1999) 13 Social Epistemology 215Google Scholar.

128 Gary, Edmond and David, Mercer, ‘Daubert and the Exclusionary Ethos: The Convergence of Corporate and Judicial Attitudes Towards the Admissibility of Expert Evidence in Tort Litigation’ (2004) 26 Law and Policy 231Google Scholar.

129 AJP, above n 2, 115.

130 While interests are a useful means of interpreting social action they are analytical constructs and do not necessarily provide a direct correlation with actual motivations or the reasons for action. Some of the limits with interest explanations are canvassed by Barry, Hindess, ‘“Interests” in Political Analysis’ in John, Law (ed), Power, Action and Belief: A New Sociology of Knowledge? (1986) 112Google Scholar; Steve, Woolgar, ‘Interests and Explanation in the Social Study of Science’ (1981) 11 Social Studies of Science 365Google Scholar; Steven, Yearly, ‘The Relationship Between Epistemological and Sociological Cognitive Interests: Some Ambiguities Underlying the Use of Interest Theory in the Study of Scientific Knowledge’ (1982) 13 Studies in the History and Philosophy of Science 353Google Scholar.

131 Compare Gary Edmond and David Mercer, ‘Trashing “Junk” Science’ (1998) Stanford Technology Law Review 3 <http://stlr.stanford.edu/STLR/Articles/98_STLR_3/index.htm> at 24 February 2005.

132 AJP, above n 2, 59.

133 Kumho Tire Co v Carmichael, 526 US 137 (1999); Daubert v Merrell Dow Pharmaceuticals, Inc, 43 F 3d 1311 (1995). The second case was the decision of the Fifth Circuit Court of Appeal on remand. Citing Peter Huber, Galileo's Revenge: Junk Science in the Courtroom (1991), the Court of Appeal adopted a reading of Daubert 509 US 579 (1993) which made it significantly more onerous for plaintiffs to adduce expert evidence.

134 Consider the activities of experts in David, Mercer, ‘Hyper-Experts and the Vertical Integration of Expertise in EMF/RF Litigation’ in Gary, Edmond (ed), Expertise in Regulation and Law (2004) 85Google Scholar.

135 One question from the survey provides some indication of the practical utility of protocols (like the Declaration) and duty statements in situ. The answers to Question 8.4, directed to certification of expert reports under s 177 of the Uniform Evidence Acts, would appear to suggest that such reforms are perceived as irrelevant or largely ineffective. These implications remained undeveloped in the Report.

136 See Michael, Lynch, ‘The Discursive Production of Uncertainty: The OJ Simpson “Dream Team” and the Sociology of Knowledge Machine’ (1998) 28 Social Studies of Science 829Google Scholar; Edmond, above n 42; Stefan, Timmermans and Marc, Berg, The Gold Standard: The Challenge of Evidence-Based Medicine and Standardization in Health Care (2003) 94–104Google Scholar.

137 AJP, above n 2, 1 (emphasis added).

138 This approach can be characterised as naïve empiricism: a general perspective dismissed as implausible in virtually all texts of the social sciences, including introductory textbooks on methods. Consider the widely cited introductory work by Alan, Chalmers on the limits of inductivism, What is This Thing Called Science? (2nd ed, 1982) 1–21Google Scholar. As McNeill, above n 25, 128, explains:

“Data” means, literally, “things that are given”, i.e. there, waiting to be found. It assumes a positivist view of the world. But if knowledge is created and constructed, then data is not “given”, but produced … Every research method is a means of producing knowledge, not collecting it. None simply records “the facts” or “the truth” as an external object.

139 Mayntz et al, above n 25, 23. According to these authors, who were writing in the 1960s,

[i]t is now virtually undisputed that research without some theoretical basis is not only unfruitful but downright impossible.’ They continue: ‘If radical empiricists have questioned the dependence of research on theory it has been because they had too narrow a concept of theory and did not recognise that, in the social sciences, delineating the subject and attaching names to social realities are themselves theoretical decisions.

140 Another way of assessing the choice of methods is to compare Australian Judicial Perspectives with the discussion of surveys outlined in Freckelton and Selby's leading textbook on expert evidence The Law of Expert Evidence (1999) 110–48, especially 142ff. An examination of the principles and methods from The Law of Expert Evidence might be used to suggest that Perspectives does not conform to the principles required for legally admissible survey work. Perspectives would appear, for example, to encounter difficulties with the following: ‘(1) A representative cross-section … must be interviewed … (4) Leading questions must not be posed. … (12) Interviewers should be experienced … (13) Interviewers should not depart from prescribed procedures': at 143.

141 The value of including other subjects is that they provide a form of triangulation which may or may not support the authors’ one dimensional investigation (of judges and magistrates). If the authors had considered experts, for example, they may well have produced a range of results which were not consistent, or easily reconciled, with the answers provided by judges and magistrates. Such results might have destabilised the findings and required further investigation or more sophisticated interpretive techniques.

142 For a critical overview of several research methods see Emile, Durkheim, Suicide: A Study in Sociology (1897)Google Scholar; Aaron, Cicourel, Method and Measurement in Sociology (1964) 7–120Google Scholar; Ray, Pawson, A Measure for Measures: A Manifesto for Empirical Sociology (1989)Google Scholar; Charles, Briggs, Learning How to Ask: A Sociolinguistic Appraisal of the Interview in Social Science Research (1986)Google Scholar. More specifically, consider Colin, Robson, Real World Research: A Resource for Social Scientists and Practitioner-Researchers (2nd ed, 2002) 125–7, 243Google Scholar. This is not intended to suggest that other forms of inquiry are without difficulties but to reinforce limitations inherent in questionnaires, see also Martin, Hammersley, What's Wrong with Ethnography: Methodological Explorations (1992) 96–122Google Scholar, 159–73; Bent, Flyvbjerg, Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again (Steven, Sampson trans, 2001)Google Scholar.

143 John, Campbell, Richard, Daft and Charles, Hulin, What to Study: Generating and Developing Research Questions (1982) 97-103Google Scholar. Campbell et al contend that without appropriate theory research may be easier and faster but the results may have little value: at 102.

144 Several research traditions, including phenomenology and ethnomethodology — inspired by Garfinkel and Sacks — focusing on contextual inter-subjective meaning are exceptionally critical of survey research. See Harold, Garfinkel, Studies in Ethnomethodology (1967, reprinted 1996)Google Scholar; Harvey, Sacks, Lectures on Conversation (1998)Google Scholar; Cicourel, above n 142.

145 For an overview of recent sociological writings in the field of law and science, see David, Mercer, ‘The Intersection of Sociology of Scientific Knowledge (SSK) and Law: Some Themes and Policy Reflections’ (2002) 6 Law Text Culture 137Google Scholar; above n 65.

146 Harry, Collins, Changing Order: Replication and Induction in Scientific Practice (1985)Google Scholar; Michael, Mulkay and Nigel, Gilbert, ‘Putting Philosophy to Work: Karl Popper's Influence on Scientific Practice’ (1981) 11 Philosophy of the Social Sciences 389Google Scholar.

147 Accessible examples of scientific controversy studies include Harry, Collins and Trevor, Pinch, The Golem: What Everyone Should Know About Science (1993)Google Scholar and The Golem At Large: What You Should Know About Technology (1998).

148 Thomas, Kuhn, The Structure of Scientific Revolutions (2nd ed, 1970) 144–59Google Scholar; Harry, Collins, Gravity's Shadow: The Search for Gravitational Waves (2004)Google Scholar.

149 Randal, Albury, The Politics of Objectivity (1983)Google Scholar; Robert, Proctor, Value-Free Science? Purity and Power in Modern Knowledge (1991)Google Scholar; Bruno, Latour, Science in Action: How to Follow Scientists and Engineers Through Society (1987)Google Scholar; Helen, Longino, Science as Social Knowledge: Values and Objectivity in Scientific Inquiry (1990)Google Scholar.

150 Prominent citations include Australian Law Reform Commission, Review of the Federal Civil Justice System, Discussion Paper No 62 (1999) 1.8, 13.21, 13.22; Australian Law Reform Commision, Managing Justice: A Review of the Federal Civil Justice System, Report No 89 (1999) 6.76, 6.92, 6.95, 6.111, 6.113, 8.161; New South Wales Law Reform Commission, Expert Witnesses, Issues Paper No 25 (2004); Justice Alan Abadee, ‘Professional Negligence Litigation: A New Order in Civil Litigation — the Role of Experts in a New Legal World and in a New Millennium’ (Paper presented at the Australian College of Legal Medicine, Canberra, 16 October 1999); Justice Alan Abadee, ‘The Expert Witness in the New Millennium’ (Paper presented at the 2nd Annual Scientific Meeting of General Surgeons Australia, Sydney, 2 September 2000); Justice Alan Abadee, ‘Update on the Professional Negligence List and Expert Evidence: Changes for the Future’ (Paper presented at the Australian Plaintiff Lawyers Association Branch Conference, Sydney, 3 March 2000); JusticeHarold, Sperling, ‘Expert Evidence: The Problem of Bias and Other Things’ (2000) 4 Judicial Review 429Google Scholar; Justice James Wood, ‘Expert Witnesses — The New Era’ (Paper presented at the 8th Greek Australian International Legal and Medical Conference, Corfu, June 2001); Re W and W (Abuse Allegations; Expert Evidence) (2001) 164 FLR 18.

151 Smith and Wynne, above n 108; Sheila, Jasanoff, Science at the Bar: Law, Science and Technology in America (1995) 215–17Google Scholar.

152 Gary Edmond, ‘Capturing the Courts: Public Science and the Subversion of Legal Autonomy’ (unpublished).

153 Compare Thomas, Gieryn, Cultural Boundaries of Science: Credibility on the Line (1999)Google Scholar.

154 In recent decades, judges have adopted more interventionist, even managerial, roles and procedures. In some jurisdictions, such as the US, cases are frequently disposed of through pre-trial judicial decision making about the admissibility of expertise. Judicial definitions and interpretations of science, expertise, and models of causation can determine not only the outcome of particular cases but the shape of entire congregations (of cases), and extend to influence professional practice, research orientations and even the publication of results. See Gary, Edmond and David, Mercer, ‘Litigation Life: Law–Science Knowledge Construction in (Bendectin) Mass Toxic Tort Litigation’ (2000) 30 Social Studies of Science 265Google Scholar; Edmond and Mercer, above n 131.

155 Philip, Mirowski and Esther-Mirjam, Sent (eds), Science Bought and Sold: Essays in the Economics of Science (2002)Google Scholar.

156 Lorraine, Daston and Peter, Galison, ‘The Image of Objectivity’ (1992) 40 Representations 81Google Scholar; Steven, Shapin, ‘Cordelia's Love: Credibility and the Social Studies of Science’ (1995) 3 Perspectives on Science 255Google Scholar; Steven, Shapin, A Social History of Truth: Civility and Science in Seventeenth-Century England (1994)Google Scholar.

157 John, Schuster and Richard, Yeo (eds), The Politics and Rhetoric of Scientific Method: Historical Studies (1986)Google Scholar; Nigel, Gilbert and Michael, Mulkay, Opening Pandora's Box: A Sociological Analysis of Scientists’ Discourse (1984)Google Scholar; Timothy, Lenoir, Instituting Science: The Cultural Production of Scientific Disciplines (1997)Google Scholar; Turner, above n 40.

158 Huber, above n 133. For a critical discussion of Huber's work consider Edmond and Mercer, above n 131.

159 The adoption of more realistic models of science and expertise raises important questions for those studying and practicing politics, regulation and law. Consider Yaron, Ezrahi, The Descent of Icarus: Science and the Transformation of Contemporary Democracy (1990)Google Scholar; Richard, Sclove, Democracy and Technology (1995)Google Scholar; Stephen, Hilgartner, Science on Stage: Expert Advice as Public Drama (2000)Google Scholar.

160 Unfortunately, such criticisms often appear to support the status quo, see William, Haltom and Michael, McCann, Distorting the Law: Politics, Media and the Litigation Crisis (2004) 76Google Scholar, 106, 289–90; Michael, Saks, ‘Public Opinion About the Civil Jury: Can Reality Be Found in the Illusions?’ (1998) 48 DePaul Law Review 221, 234–5Google Scholar.

161 More attention ought to be devoted to the roles played by experts in the very large number of disputes that do not go to trial, see Richard, Miller and Austin, Sarat, ‘Grievances, Claims, and Disputes: Assessing the Adversary Culture’ (1980–81) 15 Law and Society Review 525Google Scholar.