Those of you who know me know that I am very proud of the four and a half years I worked at the U.S. EPA’s Office of Water, first as an Oak Ridge Institute for Science and Education Fellow and then as an environmental protection specialist and economist. Fewer of you will know that every major effort I worked on as a federal employee more or less failed. This was not due to any failings on the part of the people there, the staff and management remain one of the smartest and dedicated groups I have ever met. And even the undeniable budgetary and partisan political troubles can not take the full share of the blame. No, our failures stemmed from how poorly institutions and organizations set up in the 1970s are suited to deal with 21st century environmental problems.
So what I took out of my time at U.S. EPA were the lessons of failure. And nothing taught more lessons than the high profile failure of the Stormwater Rule.
Taking dramatic action to deal with the damage to U.S. waterbodies caused by stormwater running off buildings and roads wasn’t a bad idea. Dramatic changes were necessary in how stormwater was managed and regulated in the United States to prevent further degradation of said waterbodies. The U.S. EPA had more or less left the issue to the states for many years and while some had made significant and exciting progress, many were doing virtually nothing.
Our specific regulatory approach itself wasn’t bad or uninformed either, it had been developed by some very smart people with decades of experience with the issue area itself and the nuts and bolts of how to get things done in Washington, D.C. In many ways we were implementing a tried and tested U.S EPA approach to environmental regulation: let the states experiment, and then use the lessons from that experimentation to set a minimum technical standard that the states that were straggling behind would have to meet.
No, the problem was the institutional context in which we were operating, which as I noted above, was still tuned to fix the environmental problems of the 1970s where the problems could be easily traced to some big building with a pipe spewing gross stuff that killed the fish.
Before U.S. EPA management and the White House could come to a final decision about the proposed regulation, however, the agency was going to have to go through the full regulatory process, which means that even the most sensible sounding and pragmatic policy (such as we thought ours was) has a lot of hoops to jump through before it becomes law.
This is because decisions, particularly decisions by public authorities, often have unintended consequences and attempts to solve a problem in one area may lead to frustration, misunderstanding, and even adversely affect public health, the environment, and people’s livelihoods in other areas. The solution is to create an opportunity for affected and interested parties to make their voices heard, and in the U.S. a federal requirement that authorities include the public in their decision making process has existed at least since the National Environmental Protection Act of 1969. And “public participation” requirements in some form or another are pretty commonplace in Europe as well, as I would imagine they should be in any nominally democratic society.
And we can imagine the ideal public participation process. New information is brought to the public authorities, they reconsider and redesign their policy appropriately, and the harms the policy would have caused are avoided or at least minimized. Perhaps the process takes a bit longer, and the final policy costs a bit more, but overall society is richer, more equitable, and just plain better off. But how often is this ideal realized?
I can’t say exactly. On the margins, when minor tweaks to the policy are required, maybe exempting some small population from the regulation or giving them extra protection, say not requiring businesses below a certain size to fill out a lengthy report to the government, or putting some extra resources into inspecting potentially toxic substances in schools, public input and participation can be very effective.
But when the changes to the anticipated policy become more than little tweaks, simple participation may not lead to any meaningful improvement of the policy, and may just lead to confusion and distrust. Some problems just aren’t that easy to characterize or solve, and the public authority may very well find its attempt to solve the problem characterized as being a problem in itself. In these “Wicked Problems” even formulation what the problem is, is a problem, often problems are not understood until after a solution has been formulated.
Let’s say for example you have too much waste piling up for the city dump to handle. The problem may be that the dump is too small, that you need a recycling program, or simply that the citizens should not consume as much stuff (NRC 1996).
As stakeholder groups (waste haulers and environmental activists in the above example) are likely to have varying solutions, they likely won’t be able to agree on what the problem was in the first place.
So we moved forward with our formulation of the solution (which was tied to our formulation of the problem, i.e. the need for U.S. EPA to set a standard) for the damage caused by stormwater runoff from streets and buildings. Our solution being a regulatory requirement that would require newly developed and redeveloped land to try to mimic natural hydrology. We believed that this could be implemented in a relatively cost effective manner as we had seen a lot of very attractive projects which had used green infrastructure such as rain gardens and green roofs and new “low impact development” techniques to accomplish just this. Some had even saved money and seen higher sale prices because, well, people like plants.
We had listening sessions across the country where people commented on our proposed changes to administrative law. Some people liked our program, some didn’t. And many just brought a lot of new information to the table. For example, a major barrier to the use of green infrastructure was that it wasn’t allowed under local building codes. For something like this we could issue guidance and allow local authorities some time to revise these codes. Other issues weren’t so easily resolved. For example, we were only looking at regulating new and redevelopment within urbanized areas, what if this perversely pushed more sprawl as developers sought to avoid the costs of regulation? And could the new techniques, which infiltrated the rainwater and anything it was carrying into the ground lead to groundwater contamination? And how burdensome would this be for the construction industry, would it substantially increase home prices?
These questions and many others were debated inside and outside of the U.S. EPA for the entire rulemaking process, and many of the debates go on even though the rule itself died in March 2014. And the process and the extensive information gathering, modeling, and analysis that were connected to it were not able to build any kind of broad consensus for or against the regulation we were proposing.
Which sounds funny. Having collected and analyzed all this information we should have “known” more and so it should have been clearer whether the regulation was a good or bad idea. But we hadn’t dealt with the fact that we had a wicked problem on our hands, or rather we were trying to deal with one aspect of a whole host of interconnected issues relating to urban development, and we couldn’t convince stakeholders that we could pursue our goals without stepping on their toes and causing problems in the issue areas they cared about.
Now, as I learned in our efforts to make sense of what happened, the problem of building consensus and understanding around government actions which involve difficult to interpret science and political disagreement was not unique to this effort. Back in 1996 the National Research Council (NRC) took a stab at addressing similar issues which had been encountered by a number of U.S. entities, including the U.S. Department of Defense, Health and Human Services, Agriculture, EPA, the U.S. Nuclear Regulatory Commission, and American Industrial Health Council, and the Electric Power Research Institute in its report Understanding Risk: Informing Decisions in a Democratic Society.
The thinking in the report remains state of the art today and it contains many concrete examples of where the process to inform the public and develop the appropriate scientific analysis was done correctly, and where it was done poorly. If the recommendations of the report had been implemented broadly at U.S. EPA, we would have been far more likely to have avoided the expensive and very public failure that was the process to establish a Stormwater Rule.
And process is the key word here. The final rule could be challenged in court if we did not comply with certain federal requirements which included when you have to publish notices in the Federal Register, how long comment periods need to be open for, what additional statues have to be considered, and what analyses have to be conducted. Where we fell down with the Stormwater Rule is that we formulated the broad outlines of the problem, and the solution, relatively early, and then treated the (admittedly substantial) process requirements that had been given to us as boxes to be checked, or hurdles to be jumped, on our way to a final decision. Don’t get me wrong, we did listen and conducted the elements of the process with complete sincerity, but how open we could be about changing our approach was limited by the fact that we knew basically where we wanted to go, and we didn’t have much time to get there as we started work in earnest in 2009 and planned to propose by the end of 2012. That is a very very aggressive schedule for a government action of the size we were conceiving.
In the above mentioned report (Understanding Risk) the NRC is emphatic that the process must be gotten right (NRC 1996, p. 22), and the process element which the NRC warns most strongly against using without broad based deliberation was exactly the element which killed our rulemaking, that is the cost benefit analysis (NRC 1996, p. 104) and the many “judgements that are implicit” in the techniques that makes such an analysis possible.
Now the superficial reason that the cost benefit analysis killed the Stormwater Rule was that costs were higher than benefits, and no doubt having higher benefits than costs would have strengthened the political hand of U.S EPA management and made them more likely to issue the rule. But getting high “monetized benefits” around water issues is notoriously difficult, and many regulations have been issued in which costs were much higher then benefits (I recall a rule of thumb for water rules being that costs should be no more than three times benefits).
The reason that federal agencies are allowed to issue regulations is even when a cost benefit analysis is unfavorable for taking an action, it may still make sense to take the action. This is because the meaning of costs and benefits in the context of a formal analysis is deceptively narrow and technical and understanding precisely the final numbers of any given analysis requires not only a significant background in economics but some familiarity with how methodologies were implemented in that specific study. In the environmental context, these terms are even less intuitive because one must rely on “non-market valuation” which depends on a very specific and even more technical body of research.
To cover quickly how “cost” and “benefit” are used in everyday speech as opposed to in a formal analytic setting, let’s consider that while many of us consider increasing equality to be a “benefit,” there is no way that equality can be considered in the framework of a cost benefit analysis . Other qualitative factors of interest such as sustainability, resiliency, or innovativeness, might be translated into monetized terms in a cost benefit analysis, but it is likely that after the translation process such terms would only bare a tangential relation to our everyday usage of the terms, and thus the analysis would be likely misleading in broader debate.
The point is: cost benefit analysis is only useful if decision makers and stakeholders understand how to interpret the results. And when it comes to complicated environmental regulations this is rarely if ever the case. Air regulations often have much higher benefits than costs but this is because small particulate matter is likely to lead to human deaths. Now it does seem that avoided deaths should translate into HUGE benefits, obviously human deaths are first and foremost what we wish to prevent, but discussing human death in a cost benefit analysis requires putting a monetary value on human life, in the case of the U.S. government standard values are between $7 million to $9 million per human life (these come from very technical studies about how much money Italian miners demand for more dangerous work). Check your intuition on these numbers for a second. Then consider that simply multiplying one of these numbers together by the number of expected deaths likely forms the core of the several billion dollars in potential benefits reported in the news regarding a big new air regulation.
If you have any skepticism about placing a value of $7 million to $9 million dollars on each individual human life, you probably would have preferred to look simply at the number of people likely to die without the new regulation. Then you might be able to make a judgement call based on what you thought was a manageable cost for industry. The problem we are now faced with is that we as a society don’t have a nice analysis to make the implicit judgement for us. What you decide is “manageable” will say something about your judgement, and thus also about you, and your judgement is likely to be inextricably linked to whether your child suffers from asthma or you have to install the control technology on your coal plant, whether you’re a Greenpeace volunteer or a Koch Industries lobbyist. Very quickly what seemed to be an entirely technical matter has become a political one.
And here are the Scylla and Charybdis which any decision making process which is at the same time scientific and yet political must steer between.
Overly technically and analytically oriented decision making processes threaten to confuse and obscure understanding of relevant stakeholders, the public, and even the decision makers themselves; overly politically oriented processes may never be able to arrive at a consensus on the facts at hand. But in a well tuned process, consensus and understanding go hand in hand, just as technical analyses can support and inform political deliberation.
Now designing and carrying out such a process is more of an art than a science. But this is an art that can be informed by prior cases, as the “Understanding Risk” report shows. An example of a successful effort comes from the Man and Biosphere Program (MAB) which was organized by the U.S. Department of State. I assure you, given the scope of the issues involved, success was not assured:
In a MAB activity over several years, more than 100 natural and social scientists from various federal and state agencies and from universities have considered policy options for managing surface water so as to maintain a sustainable ecosystem in and around Florida’s Everglades (Harwell et. al., in press). Changes in the ecosystem and possible responses to them entail risks to endangered species, to drinking water quality in nearby metropolitan areas, and to the livelihoods of sugar growers. The scientists considered all these risks carefully, but from a perspective different from that typical to risk assessments.
They defined the problem not as one of estimating and reducing risks, but as one of developing a shared vision of desired conditions of the ecosystem. The then identified development strategies consistent with such a vision and proposed governance structures that could adaptively manage the social-ecological system and it changed and new knowledge developed. They considered several scenarios for change in human management of the ecosystem and analyzed them in terms of their compatibility with goals of sustainable economic and social development and with a widely shared vision of ecosystem use. The MAB effort is noteworthy for its problem driven approach, particularly its extensive and explicit efforts to understand the decisions to be made, rather than presuming that decision makers would gain the understanding they needed from estimates of the ecological, health, and economic costs and benefits of previously defined choices. In fact, the process generated policy options that had not previously been considered and might be more acceptable, both socially and ecologically, than any that might otherwise have been considered (NRC 1996, p. 18-19). (Emphasis added)
In addition to specific cases such as the one mentioned above, there is a good deal of general practical knowledge out there and agencies would do well to direct more of the attention to the “craft” of meaningful public participation. Such effort shouldn’t be seen as a distraction for scientific work to inform policy, but rather as a necessary step to ensuring that the data gathered and analyzed answer the relevant questions. And what the relevant questions are is never clear at the outset of a policy making process which involves a complicated mix of politics and science.
And those of you who know me very well or read my post at the beginning of the year, have heard of the huge potential I see in participatory modeling, particularly participatory modeling exercises aided by computer simulation. Such exercises have been shown to help stakeholders from diverse backgrounds to come to a common representation and understanding of the problem, and thus a common and trusted vision for a way forward. As I develop my work with an approach that uses these exercises, my experiences with the Stormwater Rule and stormwater issues more generally will certainly be foremost on my mind. And I invite any practitioners or researchers struggling with these issues to contact me through the comment form on this blog or at chris.moore [at] ecologic.eu
Stern and Fineberg. Understanding Risk: Informing Decisions in a Democratic Society. National Research Council. 1996. http://www.nap.edu/openbook.php?isbn=030905396X