John Pfaff
Associate Professor, Law
Fordham Law School, United States

John Pfaff is an Associate Professor of Law at Fordham Law School. Prior to joining Fordham, he was the John M. Olin Fellow at Northwestern University School of Law and a clerk for Judge Stephen F. Williams of the United States Court of Appeals for the District of Columbia Circuit. He received his JD in 2003 and his PhD in Economics in 2005, both from the University of Chicago. Pfaff’s research focuses on the proper use of social science and other empirical evidence in the legal system, as well as on more specific empirical questions related to criminal law and sentencing policy. His work on the use of empirical evidence explores how to develop rigorous evidence-based quality guidelines and systematic reviews of studies using non-experimental, observational data; and how to use such reviews in legal settings to assess scientific evidence as an alternative (or at least an ameliorative) to the law’s iconic reliance on dueling partisan experts. His sentencing research examines the forces driving the growth in the U.S. prison population, the relationship between prison growth and crime rates, and the effects of guidelines on sentencing outcomes and behavior.

Incorporating Systematic Sources of Knowledge into the Social Sciences and the Law 
This project consists of two goals. First, it seeks to fundamentally improve how empirical disciplines that rely on observational data determine what they know. It focuses on developing evidence-based systematic reviews for observational work. Such reviews provide substantially more transparent and objective assessments of knowledge than their alternatives, but they are unused in most empirical fields. This part of the project will produce two papers and a conference. The first paper, aimed at legal empiricists, will point out the shortcomings of current methods of assessment and examine the theoretical benefits and practical concerns of systematic reviews. The second will generate such a review, thus demonstrating its feasibility, using empirical work on incarceration’s effect on crime as an illustrative example. This part of the project will culminate with a roundtable conference bringing together researchers from several disciplines who work on evidence synthesis. The conference will establish general protocols for constructing evidence-based systematic reviews of observational studies, and it will lay the groundwork for a more formal institution to advance their use and development in empirical fields more generally.

The second goal is to develop ways to incorporate such reviews into an adversarial legal system. Well-designed systematic reviews emphasize collaboration among independent analysts; the legal system, competition between partisan experts. This part of the project will produce two papers over the period of the grant. The first, a companion paper to the theoretical piece mentioned above, will be aimed at the legal audience, pointing out the epistemic superiority of systematic reviews while discussing legal issues they raise. The second paper will address concerns raised by the fact that legal and scientific fact-finding will never be identical. In particular, it will examine how to reconcile independent systematic reviews with adversarialism. Though lauded for its ability to ferret out knowledge, adversarialism also advances normative goals other than fact-finding, and it is essential to determine ways to incorporate systematic reviews while honoring these values. This article will also focus on pragmatic issues of implementation, such as funding and the selection of reviewers. This project begins a larger effort to reform how empirical disciplines and the law determine knowledge. The current, inferior methods are firmly entrenched in both law and the sciences. Displacing and reforming them will require the substantial efforts of numerous actors in both areas, but the payoff will be a profoundly deeper understanding of what we in fact know.

The first paper under this grant is complete, and will soon be submitted to a peer-reviewed law journal by the end of summer 2009.  This paper lays out the basic problems with how the social sciences currently produce knowledge, explains the key benefits of a systematic approach, and points out the key challenges of adopting such methods in the social sciences.  The groundwork for two more papers is underway. The second paper begins to consider how to incorporate systematic reviews into the US legal system, and the third starts to develop prototype quality guidelines for producing systematic reviews of observational studies.  An early draft of the legal paper was presented at a faculty workshop at Fordham in May.

Watch Video

View Grantee Profile

Back to Project Descriptions