On the perils of embedded experiments

There is growing interest in 鈥榚mbedded experiments鈥, conducted by researchers and policymakers as a team. Aside from their potential scale, the main attraction of these experiments is that they seem to facilitate speedy translation of research into policy. Discussing a case study from Bihar, Jean Dr猫ze argues that this approach carries a danger of distorting both policy and research. 

Evidence-based policy is the rage, to the extent that even village folk in Jharkhand (where I live) sometimes hold forth about the importance of 鈥ebhidens鈥, as they call it. No one, of course, would deny the value of bringing evidence to bear on public policy, as long as evidence is understood in a broad sense and does not become the sole arbiter of decision-making. However, sometimes evidence-based policy gets reduced to an odd method that consists of using randomised controlled trials (RCTs) to find out 鈥榳hat works鈥, and then 鈥榮cale up鈥 whatever works. That makes short shrift of the long bridge that separates evidence from policy. Sound policy requires not only evidence 鈥 broadly understood 鈥 but also a good understanding of the issues, considered value judgements, and inclusive deliberation (, 2020a).

Enormous energy has been spent on the quest for rigorous evidence, much less on the integrity of the process that leads from evidence to policy. As illustrated in an earlier contribution to Ideas for India (), it is not uncommon for the scientific findings of an RCT to be embellished in the process. This follow-up post presents another case study that may help to convey the problem. It also illustrates a related danger 鈥 casual jumps from evidence to policy advice. The risk of a short-circuit is particularly serious in 鈥榚mbedded experiments鈥, where the research team works 鈥榝rom within鈥 a partner government in direct collaboration with policymakers.

The case study pertains to an experiment conducted in Bihar in 2012-2013 and reported in Banerjee, Duflo, Imbert, Mathew and Pande (2020)1. This is a large-scale, influential experiment by some of the leading lights of the RCT movement 鈥 indeed, a formidable quartet of first-rate economists reinforced by one of India鈥檚 brightest civil servants, Santhosh Mathew. The high technical standards of the study are not in doubt, and nor is the integrity of the authors. And yet, I would argue that something is amiss in their accounts of the findings and policy implications of this study.

Read More »

The limits of 鈥渄esign ethics鈥 under capitalism

Working as a product designer in media for the past five years, I鈥檝e witnessed the topic of 鈥溾 raised at industry conferences, presentations, and meetups. Yet I鈥檝e noticed that in our discussions, designers rarely mention the economic context within which we design. We hold up examples like and and we ask: how might we design better? Conventional discourse presents these unintended consequences of our work as technical problems: how might we design and code ethically, while maintaining profitability and growth? (Perhaps the most well-known example of this framing is The Center for Humane Technology鈥檚 鈥淭he Social Dilemma,鈥 which confuses correlation with causation by attributing negative mental health and political trends to technology, with no mention of technology鈥檚 place in capitalism.)

We will not solve problems of authoritarianism, racism and xenophobia, misinformation and addictive technology, mental health and public health, or climate change with design ethics. While designers should thoroughly consider the consequences of our work, the problems facing the design and technology industry are not ones of individual bad actors (though some exist). Rather, we must acknowledge that design decisions are economic decisions鈥撯揳nd in our current economic system, the economic interests of individuals often conflict with their social consequences. Technology firms are not cultural or ideological actors, but 鈥渆conomic actors within a capitalist mode of production…compelled to seek out profits in order to fend off competition鈥 (Srnicek 2017, 3). If we truly want to design ethically, we must first consider how technology is embedded in capitalism. Our ability to make technology work better for society as a whole depends upon our willingness to reorder our priorities and redefine value as more than profit maximization.

Read More »

The Power of the Market

business-1730089_960_720.jpg

鈥淭he 鈥榤arket鈥 is a bad master, but can be a good servant.鈥
– (1993: 420)

In the world today, more and more interpersonal interactions are replaced by market transactions. The market system is both an economic and a cultural phenomenon, yet we seem to be hardly aware of the values that are bound up in it. This phenomenon is manifest at many levels: from the family, through the neighbourhood and the enterprise, to the nation and the globe. If there is such a thing as global ethics, I suggest, then they are 鈥 like it or not 鈥 the ethics of the market. My purpose here is to elaborate this claim, and to assess its implications. I shall distinguish between the market as a theoretical construct in economics, and the market as a social institution.

My main hypothesis can be briefly stated as follows: the most convincing ethical argument currently being made in favour of the market is its neutrality. Whether the market is in fact neutral may be disputed. But if one accepts this claim, it implies that the market is amoral, rather than immoral, and there remain, I suggest, two objections to allowing the market ethic to prevail. The first is that this is an abrogation of moral responsibility. It implies delegating decisions of major social and material significance to powers which are beyond our control, and whose outcome is uncertain. Second, the neutrality of the market comes at a cost in social and human terms; social relations between persons are replaced by contractual relations between economic agents.Read More »

Poor Behavior, Good Behavioral Policies? Double Standards for the North and South

behavioral.jpg

Behavioral approaches to development economics and policy have gained momentum in recent years. A growing number of papers studying behavior of people in poor countries have been published in top journals, accompanied by the rise of randomized controlled trials (RCTs). In 2015, the was dedicated to behavioral and cognitive research and policy. Papers studying how to or have become classics in the field. Lots of hope has been placed into social experiments and behavioral policies .

Behavioral policies are of course not reserved for policy-making in poor countries. In fact, nudges became famous with a . Many behavioral instruments have been discussed and tested in and for rich countries. But there has been an important difference as compared to the debates in development economics: when debating behavioral policies in rich countries, scholars have also devoted lots of time to consider normative and ethical concerns. For example, following Thaler and Sunstein鈥檚 exposition of (see also ), a debate unfolded on whether nudges could be anti-libertarian (, , , or ). Implications of the use of nudges as a new form of government policy have been analyzed, for example, from a , or with a focus on . have been written about . The debate has reached a great level of differentiation, e.g. when authors argue that so-called (these are nudges that seek to stimulate voluntary cooperation in social dilemma situations) may be justified for different reasons than those targeting individual welfare. Overall, the debate has become really sophisticated, and the autonomy, welfare, and dignity of citizens in rich countries as well as consequences of the use of behavioral policies for these countries鈥 modes of government have received lots of careful scrutiny (recently again ).Read More »