67 Am. U. L. Rev. 733 (2018)

*Edward B. Shils Professor of Law and Professor of Political Science, and Director, Penn Program on Regulation, University of Pennsylvania Law School.  I am grateful for assistance provided by Laurent Abergel, Justin Berg, and Kelly Funderburk, as well as John Boulé, Laura Collins, Jody A. Stafford, and other members of the staff of the American University Law Review.  This Article builds on testimony before the U.S. Senate Committee on Homeland Security and Government Affairs Subcommittee on Regulatory Affairs and Federal Management.  See Reviewing Independent Agency Rulemaking:  Hearing Before the Subcomm. on Regulatory Affairs and Fed. Mgmt. of the S. Comm. on Homeland Sec. and Gov’t Affairs, 114th Cong. 79 (2016) (statement of Cary Coglianese, Edward B. Shils Professor of Law, Director, Penn Program on Regulation, University of Pennsylvania).

Independent regulatory agencies—such as the Federal Communications Commission, Nuclear Regulatory Commission, and Securities and Exchange Commission—regularly issue highly consequential regulations.  When they issue their regulations, however, they do not have to meet the same requirements for analysis that apply to other agencies.  Consequently, courts, policymakers, and scholars have voiced serious reservations about a general lack of high-quality prospective analysis of new regulations at independent agencies.  These agencies’ track records with retrospective analysis of their existing regulations raise similar concerns.  In this Article, I approach the quality of regulatory analysis at independent agencies as a policy problem, assessing the current quality and offering possible solutions Congress could adopt to improve these agencies’ regulatory analysis.  I present three options for improving prospective analysis by independent agencies:  (1) continuing to allow courts to encourage better analysis; (2) subjecting independent agencies to the same White House review that currently applies to executive agencies; and (3) amending the Unfunded Mandates Reform Act (UMRA) to impose a requirement for analysis but not White House review.  The UMRA option would best balance the desire to improve independent agencies’ prospective regulatory analysis with the prevailing norms of autonomy that surround these agencies.  In addition to improving prospective analysis, independent agencies should seek to produce more rigorous retrospective analysis of their existing regulations, both to improve the substantive performance of their existing regulations and to learn better what to expect when analyzing new regulations.  I thus offer options for improving retrospective analysis by independent agencies, each of which could be adopted without undermining autonomy norms.  Ultimately, to improve independent regulatory agencies’ performance in fulfilling their public missions, these agencies’ leaders must make smarter regulatory decisions—and the first step toward smarter decisions lies with improving regulatory analysis.

Introduction

Over the last fifteen years, a group of seventeen major independent agencies—including the Federal Communications Commission (FCC) and the Securities and Exchange Commission (SEC)—collectively issued nearly 5000 federal regulations.[1]  Yet not one of these rules has been subject to the usual legislative or presidential requirements for regulatory analysis that executive branch agencies must follow when developing new rules.[2]  The Unfunded Mandates Reform Act (UMRA) and Executive Order 12,866 exempt independent agencies from the normal requirements for regulatory impact analysis.[3]  Perhaps not surprisingly then, over forty percent of major regulations from independent agencies reportedly lack any information on the anticipated costs or benefits of these new rules.[4]

To be sure, all agencies could improve their analysis of regulations, but regulatory scholars and commentators have raised particular concern in recent years about weak or insufficient analysis at independent regulatory agencies.[5]  The emergence of such concern seemed to coincide with much more active and consequential regulatory agendas at a number of major independent agencies during the Obama Administration, as illustrated by the FCC’s 2015 adoption of an “Open Internet” regulation[6] and the SEC’s and the Commodity Futures Trading Commission’s (CFTC) promulgation of major new regulations under the 2010 Dodd-Frank financial reform legislation.[7]  Litigants, as well as some judges and commentators, have criticized independent regulators for failing to produce adequate analysis before adopting new regulations, with courts remanding some agencies’ rules for further analysis.[8]

Much concern to date has focused on prospective regulatory analysis, which takes place before agencies adopt new regulations, informing decision makers about whether to proceed with new rules or how to design them.  But another type of analysis matters too:  retrospective analysis, which takes place after an agency promulgates a rule and seeks to measure its impacts.  Retrospective and prospective analysis are interrelated.  Prospective analysis clarifies the goals of a new regulation and identifies expected outcomes; this in turn informs the subsequent process of retrospective analysis by identifying benchmarks against which the regulation’s actual effects can be assessed.[9]  Conversely, when retrospective analysis shows how well a regulation has (or has not) worked, it informs future prospective analysis about whether to retain or modify that regulation, as well as how to design other regulations.[10]  Both types of analysis—prospective and retrospective—are essential ingredients for smart decision making about how to deliver high-quality regulatory outcomes.[11]  Although litigation and scholarly work has so far focused most attention on the adequacy of prospective analysis at independent agencies,[12] no reason exists to think that these same agencies are doing any better than other agencies when it comes to evaluating their rules after the fact.

The purpose of this Article is to gauge what we know about how independent agencies are performing both types of analysis and to offer steps that Congress might take to encourage improvements in both kinds of analysis at such agencies.  In this Article, I have in mind primarily the agencies that Congress has stipulated to be independent in its definition of the term in the Paperwork Reduction Act,[13] but I recognize that what constitutes an independent agency can itself be open to discussion.[14]  Agency independence has long been understood in terms of structural features related to the appointment of agency heads—for-cause removal restrictions, fixed terms, and, with multi-member agencies, bipartisan distribution requirements.[15]  Agencies with these features have generally been considered independent, while those lacking them are instead considered executive agencies, operating under the closer oversight of the White House.  More recently, though, some scholars have properly recognized structural independence as more of a matter of degree, rather than as a binary characteristic.[16]  Other commentators have acknowledged that de jure structural independence is but one possible, even if not always sufficient, means by which agency officials can make de facto independent judgments about how best to advance a statutory mandate and deliver public value.[17]

Even the independent agencies listed in the Paperwork Reduction Act do not uniformly share the same structural features.  Most have agency heads protected by for-cause removal limitations, but some do not (e.g., the Office of the Comptroller of the Currency (OCC) and the Office of Financial Research).[18]  Moreover, the Paperwork Reduction Act’s list does not include some agencies headed by administrators who do enjoy for-cause removal protection (e.g., Social Security Administration).[19]  Despite these nuances, the list of agencies stipulated as independent in the Paperwork Reduction Act proves particularly relevant to this Article’s treatment of prospective and retrospective regulatory analysis because existing regulatory analysis requirements apply to executive agencies but do not apply to agencies that the Paperwork Reduction Act lists as independent.

In each of the two parts of this Article that follow, I turn first to what we can infer about the quality of independent agencies’ analyses, suggesting that it is harder than it might seem to say definitively how deficient are the analyses conducted at these agencies.  However, on the not-unreasonable assumption that independent agencies’ analyses are far from optimal in their current level of rigor and completeness, I next turn in each subsequent part of this Article to possible legislative actions that might help encourage agencies to improve their analysis.  I begin with prospective analysis in Part I, followed with a similar treatment of retrospective analysis in Part II.

I.  Prospective Analysis

Although administrative law scholars sometimes pine for a bygone era when so-called informal rulemaking was truly informal[20]—if such a day ever actually existed—the process of making new regulations today often involves numerous procedural steps and the building of what can sometimes be an extensive administrative record.[21]  New rules are also always susceptible to judicial review under the Administrative Procedure Act’s arbitrary and capricious standard, which effectively compels agencies to justify their rules based on evidence and reasoning.[22]

For the most significant new rules, administrative procedures demand that agency officials explicitly define the problem they seek to solve, offer justifications for their proposed regulations, consider alternatives, and estimate the anticipated benefits and costs of both their preferred actions and other alternatives.  UMRA and Executive Order 12,866 impose precisely these sorts of analytical requirements when agencies plan to issue rules having certain kinds of annual economic effects in excess of $100 million (or higher for UMRA, due to inflation adjustments).[23]  Under Executive Order 12,866, agencies must clear their benefit-cost analyses of new rules through the White House Office of Information and Regulatory Affairs (OIRA).[24]  The Executive Order further states that each agency shall “propose or adopt a regulation only upon a reasoned determination that the benefits of the intended regulation justify its costs.”[25]

Statutes also contain additional analytical requirements.  The Regulatory Flexibility Act requires analysis when an agency expects a new rule will impose substantial impacts on small businesses.[26]  The Paperwork Reduction Act calls for estimates of costs and time associated with any paperwork requirements found in new regulations.[27]  The National Environmental Policy Act (NEPA) demands that federal agencies analyze the environmental impacts of major actions that will affect the environment.[28]  The Congressional Review Act requires agencies to report to Congress and the Comptroller General on new rules that would have an annual economic effect above the $100 million threshold and to provide a copy of any benefit-cost analysis prepared for those rules.[29]

Procedural requirements such as these reflect the sensible expectation that agencies engage in analysis before adopting new rules.  Just as it is true in other consequential endeavors, it is better for regulators to look before they leap.[30]  Conducting prospective analysis can help reduce the possibility of mistakes, unintended consequences, and wasted resources.[31]

All regulatory agencies must follow most, but not all, of the analytical requirements applicable to new rulemaking in statutes such as the Regulatory Flexibility Act and NEPA.  However, the main analytical requirements calling for agencies to conduct prospective benefit-cost analyses of major rules do not apply to independent agencies.[32]  The definition of an “agency” under UMRA “does not include independent regulatory agencies,”[33] and the terms of Executive Order 12,866 expressly do not apply to agencies the Paperwork Reduction Act lists as independent regulatory agencies.[34]

As a result, it should hardly be surprising that independent regulatory agencies have come under considerable criticism for failing to conduct extensive or even adequate benefit-cost analyses of many of their rules.  As Curtis Copeland notes in a report prepared for the Administrative Conference of the United States, “studies indicate that independent regulatory agencies often do not quantify or monetize regulatory benefits, and often quantify and monetize only paperwork costs.”[35]  Richard Revesz observes that when it comes to producing benefit-cost analyses of their rules, executive agencies are “more proficient,” and “[t]he less successful agencies are independent and outside the purview of OIRA review.”[36]

The evidence supporting such claims typically derives from the reports of new major rules and their underlying analyses that independent agencies submit to the Comptroller General, pursuant to the Congressional Review Act.  For example, Copeland reports that, of twenty-two major rules that independent agencies issued in 2012, “[o]nly one rule contained any quantitative benefit information.”[37]  OIRA compiles this information in its annual reports to Congress and observes that “[i]ndependent agencies still have challenges in providing monetized estimates of benefits and costs of regulation.”[38]  Other commentators reach much the same conclusion based on similar evidence.[39]

Despite this consensus about independent agencies’ analytical deficiencies, it is actually more difficult than it may seem to assess exactly how well or poorly independent agencies are doing in analyzing their rules.  The mere fact that these agencies report to the Comptroller General that some—or even many—of their rulemakings do not include quantified or monetized estimates of benefits or costs only says so much.[40]  When researchers have looked in depth at the specific materials prepared by independent agencies in individual rulemakings, they have sometimes found that agencies have given more attention to the benefits or costs of their rules than the summaries they share with the Comptroller General might suggest.[41]  More importantly, analysts lack a clear benchmark against which to measure the quantity and quality of benefit-cost analyses produced by any regulator.  For how many rules exactly is it reasonable to expect independent agencies to have produced monetized estimates of benefits and costs?  The answer almost surely is not “all rules.”

Quantification and monetization of regulatory impacts are not always possible because of a lack of data, fundamental uncertainties, or insuperable conceptual challenges with respect to making particular estimates.[42]  Estimating the benefits of homeland security regulations, for example, has proven more difficult than for other regulations because of the ex ante low probability of the underlying problem and the likelihood of strategic, adaptive responses by terrorists to any regulatory interventions.[43]  Taking account of challenges like these, Executive Order 12,866 expressly “recogniz[es] that some costs and benefits are difficult to quantify.”[44]

Trying to assess the general quality of independent agencies’ prospective analyses by comparing their level of quantification or monetization with analyses that executive agencies produce will be at best suggestive.  What constitutes quality analysis, after all, will be specific to each individual problem that a new rule addresses, taking into consideration the availability of relevant data.  Some scholars have argued that it is much more difficult to quantify the effects of financial regulation, a domain dominated by independent regulators.[45]  In addition, research indicates that agencies produce less thorough analysis for rules that must be completed under tight statutory deadlines,[46] and we know that many of the rules that independent financial regulators have issued in recent years under the Dodd-Frank Act have faced such deadlines.[47]  Without controlling for factors such as these, comparisons of independent agencies’ analyses with those of other agencies will be incomplete and even potentially misleading.

Still, it seems reasonable to assume that independent agencies—presumably like most agencies—could do a better job of analyzing the benefits and costs of their new regulations.[48]  As Ryan Bubb has suggested, it may be that benefit-cost analysis “plays little role in financial regulation not because it is especially challenging but rather because institutional structures do not produce incentives for financial regulators to develop and employ” such analysis.[49]  If this is correct, then we should consider what steps Congress could take to change those institutional structures so that independent agencies would have more of an incentive to improve their prospective regulatory analysis.  Three main options can be considered.[50]

A.  Make No Legislative Changes

Due to the demands that courts have started to impose on independent regulatory agencies,[51] as well as the generally heightened salience of the issue of regulatory analysis at independent agencies (including the prospect of legislative change), independent regulators appear already to be taking some steps to improve their institutional capacity for producing quality analysis.[52]  The SEC, for example, has made notable strides in strengthening its economic staff in the wake of the Business Roundtable v. SEC[53] decision.[54]  The FCC has released plans to create an Office of Economics and Analytics intended to “expand and deepen the use of economic analysis” at the Commission.[55]

The first option for Congress, then, would be simply to wait before doing anything further, as the quality of prospective analysis at independent agencies seems likely to improve, at least to some degree, on its own over time.  Of course, not all independent agencies will make the same strides.  Another disadvantage of waiting is that, in the intervening time, independent agencies will continue to make regulatory decisions that could have important consequences for economic activity and could be in place for a long time.

B.  Codify the Requirements of Executive Order 12,866 for Independent Agencies

Scholars have long considered the question of whether Presidents may legally apply the requirements in Executive Order 12,866 to independent agencies, and Presidents themselves have been reluctant to do so for the past thirty years.[56]  By contrast, Congress would not face any such legal question if it were to codify the requirements of Executive Order 12,866 and apply them to independent agencies.

This option would have the advantage of creating symmetry in the analytical requirements for regulation by both executive agencies and independent agencies.  After all, regulations affect the public and the economy regardless of whether executive or independent agencies issue them.  Legislatively imposing those requirements on independent agencies would cure an anomaly in the law, providing independent agencies with the same institutional structures and incentives for producing quality prospective analysis as executive agencies.

Although subjecting independent agencies to the same requirements for producing regulatory analysis as executive agencies could be easily justified on the grounds of sound regulatory management, applying the entirety of Executive Order 12,866 to independent agencies would make a significant alteration in the policy autonomy that has long been afforded to independent agencies.  Executive Order 12,866 does not merely call for agencies to conduct prospective analysis; it also creates an institutional review process that gives the OIRA Administrator, and ultimately the President, oversight and gate-keeping influence over agencies’ regulatory decisions.[57]  As Executive Order 12,866 expressly states in numerous places, the regulatory review process is one that aims at ensuring regulation will be consistent with the “President’s priorities.”[58]  In addition, under section 6(a)(3)(A) of the Executive Order, the OIRA Administrator can ultimately determine which rules it will deem significant and thus subject to regulatory analysis and review.[59]  In addition, section 8 of the Executive Order precludes an agency from publishing a rule while it is still under review at OIRA, and section 7 establishes a process through which conflicts between OIRA and the agency head can be elevated to the President for resolution.[60]

Legislatively applying the entirety of Executive Order 12,866 to independent agencies would not only compel such agencies to adhere to the Order’s principles for sound regulatory analysis—a laudable objective—but it would also apply to such agencies the Executive Order’s institutional and process provisions, several of which would prove incompatible with the structure of most independent agencies.  The wholesale codification of Executive Order 12,866 would thus present three concerns.

First, the wholesale application of Executive Order 12,866 to independent agencies headed by multimember bodies would be problematic because the procedures in Executive Order 12,866 are drafted to apply to “the agency head.”[61]  Although it is conceivable that the phrase “the agency head” could apply to an entire multimember body that constitutes the head of an independent agency,[62] the Executive Order’s procedures are most naturally intended for agencies headed by a single administrator.  A multimember body simply could not engage in the kind of back-and-forth interactions contemplated by Executive Order 12,866.[63]  The practice of regulatory review under the Order routinely involves a working interchange and dialogue between an agency head (or designee) and the OIRA Administrator (or designee),[64] a process which as a practical matter would not work well with multimember bodies.  If the entire multimember body were taken to constitute the agency “head,” then the review process would become extremely cumbersome.  Merely determining an agency’s position on OIRA’s feedback on the agency’s regulatory impact analysis would presumably necessitate a meeting with all commission members in accordance with various Government in the Sunshine Act requirements[65]—including open meetings.[66]

Second, legislatively applying the entirety of Executive Order 12,866 to independent agencies would signal a major shift in the norms and practices of autonomous regulatory decision making that have long prevailed at independent agencies.  The Order makes clear that any conflicting viewpoints between OIRA and an agency about whether or how to proceed with a rule “shall be resolved by the President.”[67]  To the extent that operational autonomy for such agencies remains valued, Congress should not apply wholesale to independent agencies the institutional mechanisms in Executive Order 12,866.  One alternative approach could be to follow the model of the Paperwork Reduction Act, which does subject independent agencies to some OIRA oversight of their information collection efforts but which also expressly allows independent agencies to override OIRA’s decisions.[68]  Another alternative can be found in the proposed Independent Agency Regulatory Analysis Act, introduced by Senator Rob Portman (R-OH).[69]  This bill would authorize the President to extend the analysis requirements of Executive Order 12,866 to independent agencies and to require them to submit their analyses to OIRA for review; however, the agency would not be bound, either by the statute or any executive order pursuant to it, to respond to any feedback from OIRA.[70]  The agency’s analysis and OIRA’s feedback would, however, become part of the administrative record on judicial review, thus effectively requiring the agency to give a reason for any departure from what OIRA recommends.[71]

Finally, even with changes to Executive Order 12,866 along the lines of those in Senator Portman’s bill, the process of having the White House review actions by independent agencies would still result in some palpable shift in longstanding norms of agency independence.  Congress would do well to consider the practical implications that would follow from assigning or authorizing White House review of independent agencies’ regulatory analyses.  In particular, such a shift would present obvious institutional challenges for OIRA, which possesses a very tiny staff compared with the many executive agencies it oversees.[72]  Legislation that would thrust responsibility on OIRA for overseeing the actions of as many as twenty additional regulatory agencies would necessitate a substantial increase in OIRA personnel and funding.

C.  Eliminate UMRA’s Exemption for Independent Agencies

A third option for Congress to consider would be to remove the independent agency exemption contained in UMRA.[73]  This option would address the current statutory asymmetry in independent agencies’ analytical requirements vis-à-vis executive agencies, while avoiding any questions or concerns about the White House intruding on independent agencies’ policy autonomy.  It would also obviate any need to increase the funding and size of OIRA, as compliance with UMRA’s benefit-cost analysis requirement does not involve OIRA.[74]  UMRA simply imposes a legal obligation on agencies to produce a statement of benefits and costs of rules covered by the Act.  This obligation to produce such a statement is judicially enforceable, but the Act precludes courts from ruling on the adequacy of agencies’ analyses.[75]

One small potential downside of this approach might be that UMRA’s analytic requirements do not apply to as many rules as Executive Order 12,866.  UMRA’s threshold applies to rules that impose $100 million or more in annual “costs,”[76] rather than rules having similar levels of what the Executive Order describes as economic “effects,” a term which presumably encompasses both costs and benefits.[77]  Plus, the $100 million amount in UMRA adjusts over time for inflation, so today the threshold is much higher.[78]  Still, if UMRA’s somewhat more limited scope were a concern, Congress could simply adjust the threshold to make it comparable to the one in Executive Order 12,866.

The larger question about eliminating UMRA’s exemption for independent agencies would be whether it would provide enough of an institutional incentive for agencies to produce better quality analysis.  Although eliminating the UMRA exemption would not provide for any institutional peer review role of the kind that OIRA provides for executive agencies, UMRA does ensure that agencies are required, under the threat of a potential court order, to prepare benefit-cost analyses.  Moreover, the analyses they prepare to comply with UMRA can form part of the agency record and thus are reviewable by courts under the general arbitrary and capricious standard in the Administrative Procedure Act.[79]  For this reason, if the UMRA exemption were removed, independent agencies would clearly have an added institutional incentive to conduct benefit-cost analysis and to take its findings seriously.  A further advantage of removing the UMRA exemption would be that independent agencies could no longer claim, as they now do, that benefit-cost analysis is simply not required of them, which could help in shifting organizational norms within these agencies in a positive direction that promotes the value of producing quality prospective regulatory analysis.

This third option—removing UMRA’s exemption—would offer a middle ground between doing nothing and involving OIRA in overseeing independent agencies’ regulatory development.  Removing the UMRA exemption would not disrupt existing norms of independence nor would it demand the development of significant new review capacity at OIRA.  It would nevertheless advance the objective of achieving legal parity between independent agencies and executive agencies with respect to regulatory analysis.  Pursuing a middle ground, rather than going to the extreme of involving OIRA in the work of independent agencies, would seem especially prudent in light of the limitations in current assessments of the adequacy of regulatory analysis at independent agencies.[80]  Moreover, it is important to keep in mind that OIRA review has not cured all inadequacies in regulatory analysis at executive agencies.[81]

In contemplating whether to take action, members of Congress should focus on what steps will best promote improvements in prospective analysis and regulatory decision making, taking into account the values that Congress has long recognized in institutional autonomy for regulators in certain policy domains, such as financial regulation.  In addition to giving overall consideration to the values served by both analysis and autonomy, members of Congress should also keep in mind several other considerations when deliberating about how to improve regulatory impact analysis at independent agencies:

• Continue to recognize the practical limits associated with conducting benefit-cost analysis. Currently, Executive Order 12,866 and UMRA recognize that full quantification and monetization of benefits and costs will not always be feasible for all regulations.[82]  Any further legislative action should similarly recognize these feasibility concerns and continue to allow agencies the discretion to adopt appropriate regulations even if some impacts cannot be quantified or monetized.
• Take into account specific legislative mandates applicable to individual agencies. Some agencies’ organic statutes preclude them from considering costs when making certain regulatory decisi[83]  Congress should approach any new legislation imposing general analytic requirements mindful of the implications such action might have for these individual statutory requirements.
• Recognize that conducting quality analysis demands resources. As Shelley Metzenbaum and Gaurav Vasisht have written, “Funding adequacy has a direct and profound impact on whether a regulator can be effective.”[84]  If Congress takes steps to mandate that independent agencies undertake additional analysis, it should also ensure that these agencies have the resources needed to fulfill any such mandate effectively.
• Do not expect perfection. Even with mandates, regulatory analysis will not always be completed well nor will it always influence regulatory decisions to the extent that it should.  Despite decades of experience with OIRA’s oversight of executive agencies, there remains substantial variation in these agencies’ compliance with best practices of economic analysis of regulations.[85]  What I wrote over a dozen years ago remains true today:  “The available empirical research indicates that simply mandating analysis does not eliminate inefficiency, and it may not even significantly reduce it.”[86]

The Administrative Conference has reinforced these considerations in recommending to Congress, should it impose new requirements on independent agencies, that it “recognize that agencies need (a) the flexibility to scale the analyses to the significance of the rules and (b) the resources to satisfy such requirements.”[87]  Any effort to eliminate independent agencies’ exemptions from requirements to conduct regulatory analysis should be approached with thoughtful attention to the conditions and capacities that agencies will need to ensure that they can prepare sound analysis and take its results seriously.

II.  Retrospective Analysis

The Obama Administration took a number of steps to build what it characterized as a “culture of retrospective review and analysis throughout the executive branch.”[88]  In early 2011, President Barack Obama issued Executive Order 13,563, proclaiming that the nation’s regulatory system “must measure, and seek to improve, the actual results of regulatory requirements.”[89]  That order directed executive agencies to develop plans for “periodic[] review [of] existing significant regulations to determine whether any such regulations should be modified, streamlined, expanded, or repealed so as to make the agency’s regulatory program more effective or less burdensome.”[90]

In response, over about five years’ time, executive agencies reportedly undertook more than 800 retrospective reviews and eliminated over seventy “regulatory provisions.”[91]  According to then-OIRA Administrator Howard Shelanski, these efforts “achieved an estimated $37 billion in cost savings, reduced paperwork, and other benefits for Americans.”[92]  His examples of such cost-savings all stemmed from regulatory changes at executive agencies,[93] including the now-famous EPA “spilled milk” regulation which effectively exempted certain milk storage containers from particular EPA oil spill rules.[94]  But what have independent regulatory agencies accomplished in terms of retrospective analysis of their stock of regulations?

In July 2011, President Barack Obama issued Executive Order 13,579, stating that “each independent regulatory agency should develop and release to the public a plan” for retrospective review of its existing significant regulations.[95]  Eleven days later, the OIRA Administrator at the time, Cass Sunstein, sent the heads of independent agencies a memorandum in which he offered expressly non-binding “guidance” on the President’s order, noting that independent “[a]gencies may well find it useful to engage in a retrospective analysis of the costs and benefits . . . of regulations chosen for review.”[96]  He emphasized that “[s]uch analyses can inform judgments about whether to modify, expand, streamline, or repeal such regulations, and can also provide valuable insight on the strengths and weaknesses of pre-regulatory assessments [that is, prospective analysis] which can be used to enhance the agency’s analytic capability.”[97]

The White House Council of Economic Advisors (CEA) reported that, as of November 2011, a total of twenty-one independent agencies had developed retrospective review plans as called for by the Executive Order.[98]  This included all the major regulatory agencies designated as independent under the Paperwork Reduction Act.[99]  The CEA report indicated that the independent agencies’ plans reflected “substantial efforts to reduce burdens” and it highlighted review efforts taken or underway at seven independent agencies (CFTC, FCC, Federal Deposit Insurance Corporation, Federal Energy Regulatory Commission, Federal Reserve, Federal Trade Commission, and OCC).[100]  Although the CEA report described most of the efforts at these seven agencies as still at an early stage, the report indicated that at least the FCC had already “eliminated 190 rules, many of which are no longer needed as a result of technological advances.”[101]

The rigor and depth of agencies’ analytic efforts in these retrospective reviews, whether conducted by executive or independent agencies, proved generally quite limited.  According to a report that the Administrative Conference commissioned, the “vast majority” of executive agencies’ efforts lacked “formal retrospective analysis, such as ex post estimates of benefits, costs, or efficacy.”[102]  What we know about the independent agencies’ efforts makes them look still less substantial.  Most of the plans submitted by independent agencies basically described existing, routine practices of consultation with the public and intentions to keep abreast of developments in the regulated industry.[103]  The Nuclear Regulatory Commission (NRC), for example, took nearly two and a half years to take its initial retrospective review plan submitted to the White House and approve it as a final plan—but even then that final “plan” consisted only of a compilation of existing principles and practices that guide NRC rulemaking activities.[104]  The CPSC took nearly five years before it approved a brief document that did little more than describe an internal process to follow, and questions for agency staff to consider, in selecting existing rules to review.[105]  Much as with the executive agencies, few, if any, of the independent agency plans could be said to contain or call for truly robust “formal retrospective analysis.”[106]

It is far from clear whether independent agencies have made much more progress over time in improving their retrospective analyses.  For the duration of the Obama Administration, anyone interested in executive agencies’ progress could go to the White House website and find status reports submitted twice each year.[107]  But no such repository ever existed of the status or accomplishments at independent agencies.  Indeed, it is not even clear how many of these agencies ever followed through at all on the initial plans they submitted.  Executive Order 13,563—the one that Executive Order 13,579 directed at independent agencies—only called for agencies to produce an initial plan.[108]  Regular progress reports on retrospective review were called for in a subsequent memorandum from the OIRA Administrator[109] as well as a subsequent presidential order,[110] both of which were directed just to executive agencies.

The Obama Administration’s regulatory lookback initiative aimed, laudably, to build a culture of retrospective review through the “continuing process of scrutiny of existing rules” fostered by the presidential requirement of regular progress reports.[111]  It remains to be seen, of course, to what extent the Administration’s lookback initiative has contributed to any enduring cultural shift at any agency.  Although the Trump Administration has not made retrospective analysis a centerpiece of its regulatory agenda, some commentators have suggested that Executive Order 13,771—which calls for executive agencies to eliminate existing regulations to offset the costs of new regulations[112]—may provide additional incentives for agencies to evaluate existing regulations.[113]  Of course, whatever positive, lasting change either the Obama lookback initiative or the Trump “one-in-two-out” requirement may have in terms of promoting retrospective review at executive agencies, presumably any such impact has been still more attenuated at independent agencies.

Clearly, more can be done to foster a governmental culture that embraces serious retrospective analysis of regulations at both executive and independent agencies.  One desirable cultural shift would entail refocusing and broadening the rationale for retrospective review.  As economist Joseph Aldy aptly notes in his report to the Administrative Conference, burden reduction was a common theme of the Obama Administration’s lookback initiative, as well as of similar efforts in earlier administrations.[114]  The Trump Administration’s regulatory agenda appears exclusively focused on burden reduction.[115]  Yet instead of just focusing on reducing regulatory costs or burdens, retrospective review should also consider regulatory benefits in an effort to help agencies overall create better-designed and better-implemented regulations.[116]  Smarter regulation means not only creating more cost-effective outcomes but also delivering greater overall benefits.

Retrospective review can provide valuable information that can be used to inform future regulatory decisions.[117]  Multiple regulatory agencies, executive and independent, face similar challenges, such as in regulating to promote private investment in anti-terrorism security efforts to protect key infrastructure, or in regulating to foster a “safety culture” within high-hazard industrial operations.[118]  Retrospective evaluation research can fruitfully illuminate such common challenges among regulators working in related areas.[119]  Learning how different types of regulatory strategies—such as market-based instruments, management-based regulation, behavioral nudges, or performance standards—have performed in one regulatory domain can be useful in designing regulations in other similar domains.[120]  Furthermore, by comparing the results of rigorous retrospective evaluations of individual rules’ costs and benefits with the prospective estimates that agencies make of these costs and benefits, agencies and their analysts can learn how to improve the regulatory analysis that takes place when agencies develop new rules.[121]

What concrete steps might Congress take to help independent agencies better realize retrospective review’s full potential for deepening regulatory knowledge and improving regulatory decision making?  Three possibilities merit consideration with respect to both executive and independent agencies.

A.  Codify and Extend Requirements for Regular Reporting on Strategically Focused Retrospective Reviews

The practices that emerged within executive agencies over the final five years of the Obama Administration under Executive Orders 13,563 and 13,610 provide a foundation on which agencies could be encouraged to build.  To ensure continuation of these practices, Congress could productively codify similar planning and progress reporting requirements—and extend them to independent regulatory agencies—helping to ensure that regular, strategic efforts of regulatory evaluation remain implemented.

If Congress were to take such action, retrospective review practices could benefit from a broadening of their purpose beyond the worthwhile objectives of streamlining and burden reduction, which have almost exclusively characterized retrospective review efforts in the past.[122]  In the Regulatory Flexibility Act, for example, Congress has already required both executive and independent agencies to undertake mandatory periodic reviews (at least every ten years) of all rules imposing “significant economic impact[s]” on small businesses.[123]  The statute’s stated purpose for such reviews is narrow:  “to minimize any significant economic impact of the rules upon a substantial number of such small entities.”[124]  Instead of limiting the focus of retrospective review so narrowly, Congress could amend the statute to encourage retrospective analysis that promotes smarter, more strategic regulatory decisions—that is, analysis that measures and potentially increases benefits, in addition to finding cost reductions.  In other words, Congress could direct agencies to design evaluations under the Regulatory Flexibility Act that would support the goal, as stated in Executive Order 12,866, of developing a better system of regulation “that protects and improves . . . health, safety, environment, and well-being and improves the performance of the economy without imposing unacceptable or unreasonable costs on society.”[125]

Agency officials currently have considerable discretion over which rules to target for retrospective analysis and how to conduct that analysis.[126]  Presumably any legislation codifying Executive Order 13,563 would continue to allow agencies to have the discretion and responsibility to determine which rules to review, along with when and how to review them, as these decisions will depend on each agency’s overall priorities and available resources.[127]  Legislation, though, can help shape these priorities in a direction that promotes public value through genuine learning.  For example, agencies might appropriately be encouraged to analyze rules that they issued under conditions of high uncertainty about their costs or benefits, or rules that rely on common assumptions or present common problems of interest to regulators.[128]

Maintaining OIRA’s current role in overseeing agency reporting about retrospective review—acting as a government-wide clearinghouse of sorts—would make sense for several reasons, even for independent agencies.  First, it could help ensure that OIRA staff can benefit from the knowledge generated from agencies’ retrospective analyses.  Such learning would inform OIRA staff members in their efforts to oversee executive agencies’ prospective analyses.  OIRA staff would also be well-positioned to articulate any government-wide best practices or other methodological guidelines for retrospective analysis, much as OIRA has done for prospective analysis with its Circular A-4.[129]

Second, since OIRA coordinates the implementation of the Paperwork Reduction Act across the entire federal government,[130] keeping its staff members apprised of independent and executive agencies’ data needs may enable them to streamline any information requests necessary to evaluate existing rules.  Often the only way to conduct meaningful retrospective analysis will be to require reports or survey responses from individuals or organizations in the private sector, the very kind of agency “collection of information” that the Paperwork Reduction Act conditions on OIRA approval.[131]

Third, OIRA could incorporate overall progress and key findings from agencies’ retrospective reviews into its annual reports to Congress on the benefits and costs of regulation.  Currently, these reports only provide estimated or forecasted benefits and costs of regulation,[132] but Congress could also benefit from systematic reporting of ex post identification of regulatory benefits and costs.

Finally, OIRA could be encouraged or authorized to issue non-binding “evaluation prompts” to agencies, identifying specific rules that would benefit from careful retrospective study.[133]  OIRA is especially well-positioned to identify either particular rules or general regulatory or analytical issues where evaluation findings could help improve prospective regulatory impact analysis.  Its role in making suggestions to independent agencies about evaluations to undertake would not intrude on such agencies’ core policy autonomy.  Such prompts could be stipulated by law to be completely non-binding for independent agencies—and for executive agencies, for that matter.

Even if all four suggestions above were implemented, the overall role contemplated for OIRA with respect to retrospective analysis would be largely one of information aggregation.  OIRA would serve as the recipient of reports and the facilitator of learning.  None of these suggestions need disrupt prevailing norms of agency independence nor require any dramatic changes to OIRA’s resources or staffing levels.

B.  Require Agencies to Issue Evaluation Plans when Promulgating New Major Rules

In principle, a well-developed regulatory impact analysis—the kind prepared prospectively to comply with UMRA and Executive Order 12,866—will provide much helpful information that evaluators could use to organize an evaluation of a regulation later.  Still, as noted in Part I, regulatory impact analyses are not always of uniform quality.[134]  Moreover, the exercise of completing even a brief, standardized evaluation plan at the time of a rule’s establishment can discipline and sharpen a decision maker’s thinking.

Such required plans need not be onerous.[135]  At a minimum, they simply need to include: (a) a description of concrete criteria, indicators, or proxies of regulatory impacts (specific benefits as well as costs); (b) known existing data that could be used to measure the rule’s impacts, or a statement of the type of new data that would be needed to measure the rule’s impacts; (c) an estimated time period after which the rule’s impacts should begin to be observable and evaluation would be appropriate; and (d) sources of variation and possible research strategies or designs, whether experimental or quasi-experimental, that could take advantage, at the appropriate time, of that variation to try to draw inferences of the rule’s impacts.[136]  OIRA could establish guidelines for appropriate research designs and other plan features, which could be instructive for independent agencies, even if not binding.[137]

In 2015, Senator Heidi Heitkamp (D-ND) introduced the Smarter Regs Act, which would amend the Administrative Procedure Act to require that when agencies propose a major rule—that is, one with annual economic effects greater than $100 million—they also “include a potential framework for assessing the major rule, which shall include a general statement of how the agency intends to measure the effectiveness of the major rule.”[138]  The bill—which would apply to both executive and independent agencies—would then require agencies to follow through with their plans and conduct assessments of major rules in accordance with the time intervals and methods provided in the agencies’ published frameworks.[139]  Even without requiring these follow-on assessments, the mere process of developing evaluation plans at the outset of a rulemaking could help reinforce an evaluation culture within agencies as well as provide useful guidance for future evaluation of the rule by outside evaluators and the public.[140]

C.  Invest in Regulatory Evaluation and Related Research in Behavioral and Regulatory Science

Taking retrospective review seriously demands resources:  time, personnel, and funding.[141]  Limitations in resources present tradeoffs between the breadth and depth of retrospective analysis.  From 2011 to 2016, the Obama Administration’s lookback initiative took retrospective review seriously by generally favoring breadth (number of rules reviewed) over depth (the empirical rigor and sophistication of the underlying reviews).[142]  By some estimates, executive branch agencies conducted more than 800 retrospective regulatory reviews during this time period.[143]  That means that the average executive branch agency reportedly undertook about thirty reviews, or about six per year, although a few agencies reviewed over fifty rules, or more than eight per year.[144]  Most of these reviews appear to have relied mainly on expert judgments, impressions, and assumptions.[145]  Few, if any, reviews involved in-depth empirical evaluation of the kind needed to draw valid inferences about what impacts the regulation under review actually may have caused.[146]

The back-of-the-envelope nature of most of the Obama Administration’s retrospective reviews is hardly an intrinsic flaw.  Building a portfolio of reviews that favor breadth over depth is certainly better than not looking back at all.  Even quick glances back in the rearview mirror can be helpful.  Moreover, an agency presumably does not need a randomized controlled experiment, for example, to surmise that replacing paper filings with electronic filings will save processing time and money.  Yet, a retrospective review portfolio devoid of any in-depth evaluation research misses a critical opportunity to draw a causal connection between regulations and intended as well as unintended benefits and costs.[147]  Regulations, after all, aim at causation.  They seek to change behavior of regulated entities in ways that reduce or solve problems.  To look back without trying to make any causal inferences is to miss learning whether regulations are accomplishing what they are supposed to accomplish—as well as whether they might be causing new problems altogether.

For these reasons, agencies need to conduct some retrospective analysis that draws causal inferences about their regulation.  They need to engage in evaluations that compare the world with a regulation to a counterfactual world without that regulation.  Since counterfactuals cannot be directly observed, agency evaluators must estimate them by using careful research designs, such as randomized controlled experiments, or by deploying various statistical techniques that effectively approximate randomized experiments.[148]  Such research can take time and effort to design and conduct.

In building a portfolio of retrospective reviews, regulatory officials need to make choices based on available resources.  Not every rule will necessarily require rigorous, in-depth evaluation.  Banning the use of lead as an additive in gasoline, for example, might not demand a sophisticated evaluation to validate that such a rule caused observed declines in air concentrations of lead, especially if few or no other major sources of lead emissions exist.[149]  In many instances, though, it will be important to determine what the actual benefits and costs of a rule have been.  Those benefits and costs, if properly monetized, represent the value of the negative and positive impacts that the rule has caused.  Axiomatically, the only way to know what difference a regulation may be making—whether for good or for ill—is to conduct a careful, causally-oriented evaluation.[150]

Even when agencies are selective and strategic in choosing rules to assess retrospectively, conducting rigorous evaluations will require adequate resources.  For some government agencies, these costs may be quite palpable and Congress will need to ensure agencies have appropriate budgetary resources.  The needed resources, though, will almost always amount to only a tiny fraction of overall estimated costs and benefits of the rules themselves, especially when evaluations are targeted toward major regulations.  From the standpoint of overall social welfare, investing in evaluation is worthwhile if it provides decision makers with options to lessen the costs or increase the benefits of major regulations even modestly.

Congress might also consider ways that research capacities available at other institutions—for example, at the National Academies of Sciences, Engineering, and Medicine or the National Science Foundation—might be used to help support the evaluation of independent agencies’ regulations.  Other institutions could undertake or fund such research directly, or they could provide more fundamental research in behavioral sciences that indirectly helps to inform retrospective analysis.  These efforts could improve regulatory decisions by enhancing regulators’ understanding of how and why different regulations have the effects they do.

Conclusion

Improving the quality of regulatory analysis remains an ongoing challenge for all agencies.  When it comes to independent agencies, it is clear that requirements for high quality analysis—whether prospective or retrospective—do not apply to these agencies as they do to executive agencies.[151]  And yet the regulations that independent agencies adopt show no meaningful disparity in substantive significance that would justify continuing to exempt them entirely from analytic requirements.[152]  Independent agencies’ regulations are in fact highly consequential to the economy and to overall societal well-being.

Closing the gap in the treatment of independent and executive agencies’ analyses would be feasible using options outlined in this Article.  By taking actions such as those presented here, Congress could encourage the heads of independent agencies to improve their agencies’ regulatory analyses and ultimately produce smarter regulatory decisions—even without causing any dramatic diminution in independent agencies’ core autonomy.  Taking steps such as those discussed here would enhance the incentives for agency decision makers to look carefully before they leap, as well as to ensure that they look backwards from time to time to learn how well existing regulations are working.  Improving the quality of regulations issued by independent agencies ultimately depends on improving the quality of, and reliance on, both prospective and retrospective regulatory analysis.

        [1].    The Paperwork Reduction Act defines “independent regulatory agency” by listing nineteen federal agencies:

[T]he Board of Governors of the Federal Reserve System, the Commodity Futures Trading Commission, the Consumer Product Safety Commission, the Federal Communications Commission, the Federal Deposit Insurance Corporation, the Federal Energy Regulatory Commission, the Federal Housing Finance Agency, the Federal Maritime Commission, the Federal Trade Commission, the Interstate Commerce Commission, the Mine Enforcement Safety and Health Review Commission, the National Labor Relations Board, the Nuclear Regulatory Commission, the Occupational Safety and Health Review Commission, the Postal Regulatory Commission, the Securities and Exchange Commission, the Bureau of Consumer Financial Protection, the Office of Financial Research, [and] Office of the Comptroller of the Currency . . . .

44 U.S.C. § 3502(5) (2012).  A search in the Federal Register using Lexis yielded 4848 final rules issued by these listed agencies from 2002 to 2016.  The search yielded no final rules issued by two of these agencies:  Interstate Commerce Commission (which no longer exists) and the Office of Financial Research.

        [2].    2 U.S.C. § 1532(a) (2012) (mandating that executive branch agencies that seek to develop a rule which may result in the private-sector expenditure of $100 million or more annually prepare an analysis of the anticipated benefits and costs of the rule); Exec. Order No. 12,866, 3 C.F.R. 638 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06 (establishing principles and procedures for executive branch agencies to conduct benefit-cost analyses of new rules and subject them to review by the Office of Information and Regulatory Affairs).  Executive Order 12,866, adopted by President William J. Clinton, was reaffirmed by President Barack Obama in Exec. Order No. 13,563 § 1(b), 3 C.F.R. 215 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 816–17.  President Donald J. Trump has retained both executive orders.

        [3].    UMRA’s definition of an agency “does not include independent regulatory agencies.”  2 U.S.C. § 658(1) (2012), incorporated by 2 U.S.C. § 1502.  The agencies covered by Executive Order 12,866 are those covered by the Paperwork Reduction Act “other than those considered to be independent regulatory agencies.”  Exec. Order No. 12,866 § 3(b), 3 C.F.R. 641.

        [4].    The Office of Management and Budget analyzed major regulations that thirteen independent agencies issued between 2005 and 2014 and found that only eighty-four out of 143 major regulations (fifty-nine percent) issued by those independent agencies contained at least some information about costs or benefits—a necessary but quite minimal criterion for sound analysis.  Office of Mgmt. & Budget, Exec. Office of the President, 2015 Report to Congress on the Benefits and Costs of Federal Regulations and Agency Compliance with the Unfunded Mandates Reform Act 97–98, app. c (2015) https://obamawhitehouse.archives.gov/sites/ default/files/omb/inforeg/2015_cb/2015-cost-benefit-report.pdf.

        [5].    See, e.g., Richard L. Revesz, Cost-Benefit Analysis and the Structure of the Administrative State:  The Case of Financial Services Regulation, 34 Yale J. on Reg. 545, 549 (2017) (providing case studies that “illustrate the shortcomings of independent agencies with respect to cost-benefit analysis”); Joseph P. Mohorovic, Improving Regulatory Analysis at Independent Agencies, Reg. Rev. (Jan. 10, 2017), https://www.theregreview.org/2017/ 01/10/mohorovic-improving-regulatory-analysis-independent-agencies (observing that “too often I have seen [the Consumer Product Safety Commission (CPSC)] depart from . . . analytical best practices, which then can lead to misinformed and even unnecessary regulations” and arguing that “the kind of cost-benefit analysis required of executive branch agencies would lead to better rules at CPSC and other independent agencies”).

        [6].    Protecting and Promoting the Open Internet, 47 C.F.R. pt. 8 (2015) (codifying rules promulgated by the FCC in response to the net-neutrality debate).  The FCC subsequently repealed this order after President Donald J. Trump was elected and appointed a new chair of the FCC.  FCC Acts to Restore Internet Freedom (Dec. 14, 2017), https://transition.fcc.gov/ Daily_Releases/Daily_Business/2017/db1214/DOC-348261A1.pdf.

        [7].    See, e.g., Dodd-Frank Final Rules, Final Guidance, Final Exemptive Orders, and Other Final Actions, U.S. Commodity Futures Trading Comm’n, https://www.cftc.gov/LawRegulation/ DoddFrankAct/Dodd-FrankFinalRules/ index.htm (last visited Feb. 7, 2018) (listing the CFTC’s seventy-nine finalized rules, orders, and guidance actions under the Dodd-Frank Act); Implementing the Dodd-Frank Wall Street Reform and Consumer Protection Act, U.S. Sec. and Exch. Comm’n, https://www.sec.gov/spotlight/dodd-frank.shtml (last visited Feb. 7, 2018) (detailing the final rules adopted by the SEC under the sixty-seven mandatory rulemaking provisions of the Dodd-Frank Act).

        [8].    For recent litigation raising challenges to independent financial regulators’ analyses, see Bus. Roundtable v. SEC, 647 F.3d 1144, 1150 (D.C. Cir. 2011); Am. Equity Inv. Life Ins. Co. v. SEC, 613 F.3d 166, 172–73 (D.C. Cir. 2010); Chamber of Commerce v. SEC, 443 F.3d 890, 894 (D.C. Cir. 2006); Chamber of Commerce v. SEC, 412 F.3d 133, 145 (D.C. Cir. 2005); Sec. Indus. & Fin. Mkts. Ass’n v. CFTC, 67 F. Supp. 3d 373, 384, 390, 437–38, (D.D.C. 2014); Inv. Co. Inst. v. CFTC, 891 F. Supp. 2d 162, 190 (D.D.C. 2012), aff’d, 720 F.3d 370, 372 (D.C. Cir. 2013); Int’l Swaps & Derivatives Ass’n v. CFTC, 877 F. Supp. 2d 259 (D.D.C. 2012).  Not all of these challenges, of course, have been successful.  Litigants raised similar objections, unsuccessfully, in an action challenging the FCC’s decision making that led to its Open Internet Order in 2015, arguing that, as Gordon Crovitz has stated, “the FCC skipped the economic analysis.”  L. Gordon Crovitz, ‘Economics-Free’ Obamanet,’ Wall St. J. (Jan. 31, 2016, 6:20 PM), https://www.wsj.com/ articles/economics-free-obamanet-1454282427.

        [9].    Cary Coglianese, Measuring Regulatory Performance:  Evaluating the Impact of Regulation and Regulatory Policy, Org. for Econ. Cooperation and Dev. Expert Paper No. 1, at 18 (Aug. 2012), https://www.oecd.org/gov/regulatorypolicy/

1_coglianese%20web.pdf [https://perma.cc/6ZZZ-NERP] [hereinafter Coglianese, Measuring Regulatory Performance] (observing that “whatever the criteria that are used in prospective impact analysis can also be used to evaluate regulations after the fact”).

      [10].    Id. at 47–50 (elaborating an integrated framework for using “[e]valuation research [to] inform decision making about a broad range of policy relevant questions,” including prospective decision making about new rules).

      [11].    Cary Coglianese & Lori D. Snyder Bennear, Program Evaluation of Environmental Policies:  Toward Evidence-Based Decision Making, in Decision Making for the Env’t:  Soc. and Behavioral Sci. Research Priorities 246, 249–52, 263 (Gary D. Brewer & Paul C. Stern eds., 2005); Michael Greenstone, Toward a Culture of Persistent Regulatory Experimentation and Evaluation, in New Perspectives on Regulation 111, 116, 121–22 (David Moss & John Cisternino eds., 2009); Cary Coglianese, Empirical Analysis and Administrative Law, 2002 U. Ill. L. Rev. 1111, 1121 [hereinafter Coglianese, Empirical Analysis]; Cary Coglianese, Thinking Ahead, Looking Back:  Assessing the Value of Regulatory Impact Analysis and Procedures for its Use, 3 KLRI J.L. & Legis. 5, 14, 23 (2013).

      [12].    See supra notes 5, 8 and accompanying text; infra notes 35–39 and accompanying text.

      [13].    See supra note 1 (defining “independent regulatory agency” in the Paperwork Reduction Act by listing nineteen federal agencies).

      [14].    See generally Geoffrey P. Miller, The Debate over Independent Agencies in Light of Empirical Evidence, 1988 Duke L.J. 215.

      [15].    See Paul R. Verkuil, The Purposes and Limits of Independent Agencies, 1988 Duke L.J. 257, 259 (detailing the three main structural features characteristic of “independent” agencies).

      [16].    Kirti Datla & Richard L. Revesz, Deconstructing Independent Agencies (and Executive Agencies), 98 Cornell L. Rev. 769, 772–73 (2013) (arguing that agencies are best viewed on a sliding scale of independence from presidential influence); see also David E. Lewis & Jennifer L. Selin, Political Control and the Forms of Agency Independence, 83 Geo. Wash. L. Rev. 1487, 1507 (2015) (developing “numerical estimates of agency independence”); Christopher Carrigan & Lindsey Poole, Structuring Regulators:  The Effects of Organizational Design on Regulatory Behavior and Performance, Penn Program on Reg. Research Paper, at 7 (2015), https://www.law.upenn.edu/live/ files/4707-carriganpoole-ppr-researchpaper062015pdf (discussing “the notion that independence is best thought of as a continuum”).

      [17].    See, e.g., Cary Coglianese, Listening, Learning, Leading:  A Framework for Regulatory Excellence 42 (2015), https://www.law.upenn.edu/live/files/4946-pprfinalconvenersreportpdf (noting that formal legal structures do not necessarily guarantee agency independence); Org. for Econ. Co-operation & Dev., Creating a Culture of Independence:  Practical Guidance Against Undue Influence 9 (2017), https://www.oecd-ilibrary.org/creating-a-culture-of-independence_5jfxjd55fzd2.pdf? contentType=%2fns%2fOECDBook%2c%2fns%2fBook&itemId=%2fcontent%2fbook%2f9789264274198-en&mimeType=application%2fpdf&containerItemId=%2fcontent %2fserial%2f24151440&accessItemIds=&option6=imprint&value6=http%3a%2f%2foecd.metastore.ingenta.com%2fcontent%2fimprint%2foecd (finding that the “de jure” independence found in legal structures does not necessarily provide the “culture of independence” that is necessary for regulators to function independently and effectively); Peter Conti-Brown, The Institutions of Federal Reserve Independence, 32 Yale J. on Reg. 257, 261 (2015) (arguing that the complex legal structure of the Federal Reserve, which was intended to balance independence and accountability, does not always serve its intended purpose); Carrigan & Poole, supra note 16, at 4–7 (discussing how the structure and composure of regulatory bodies impacts their effectiveness).  See generally Miller, supra note 14, at 215.

      [18].    The OCC is located within the Department of the Treasury.  The enabling statute for the OCC section explicitly lacks for-cause removal protection, instead providing that the Comptroller of the Currency “shall hold his office for a term of five years unless sooner removed by the President, upon reasons to be communicated by him to the Senate.”  12 U.S.C. § 2 (2012).  The statutory provision establishing the Director of the Office of Financial Research lacks any provision on removal, indicating the absence of any for-cause protection. 12 U.S.C. § 5342(b).

      [19].    Admittedly, the Paperwork Reduction Act’s enumerated list is not intended to be exclusive; it can encompass “any other similar agency designated by statute as a Federal independent regulatory agency or commission.”  44 U.S.C. § 3502(5) (2012).  What is “similar” is hardly self-evident, given that the nineteen agencies are not identical in their structural features.

      [20].    See, e.g., Thomas O. McGarity, Some Thoughts on “Deossifying” the Rulemaking Process, 41 Duke L.J. 1385, 1385 (1992) (lamenting that “the bloom is off the rose” because “the rulemaking process has become increasingly rigid and burdensome”).

      [21].    Mark Seidenfeld, A Table of Requirements for Federal Administrative Rulemaking, 27 Fla. St. U. L. Rev. 533, 533–46 (2000) (charting the source of multiple steps that agencies are required to take in order to engage in rulemaking).

      [22].    For an agency to avoid a judicial remand under the arbitrary and capricious standard of the Administrative Procedure Act, it must consider evidence and pay attention to the likely impacts of regulation when promulgating rules.  5 U.S.C. § 706(2)(A) (2012); see also Michigan v. EPA, 135 S. Ct. 2699, 2706, 2711–12 (2015); Motor Vehicle Mfrs. Ass’n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 31, 57 (1983).  See generally John D. Graham & Paul R. Noe, A Paradigm Shift in the Cost-Benefit State, Reg. Rev. (April 26, 2016), https://www.theregreview.org/2016/04/26/graham-noe-shift-in-the-cost-benefit-state (discussing courts’ expectation that agencies consider the impacts of new rules and the increasing judicial receptivity to benefit-cost analysis in the absence of a statutory prohibition on the consideration of costs).

      [23].    2 U.S.C. § 1532(a)(2) (2012); Exec. Order No. 12,866 § 3(f)(1), 3 C.F.R. 638, 641 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06.  These requirements not only do not apply to less significant rules but they also do not apply to executive agency actions that are not formally rules but nevertheless may serve “quasi-regulatory” purposes.  See John D. Graham & Cory R. Liu, Regulatory and Quasi-Regulatory Activity without OMB and Cost-Benefit Review, 37 Harv. J.L. & Pub. Pol’y 425, 426, 445 (2014).

      [24].    Exec. Order No. 12,866 § 6(a)(3)(B)(ii), 3 C.F.R. 644–45.

      [25].    Exec. Order No. 12,866 § 1(b)(6), 3 C.F.R. 638–39.

      [26].    5 U.S.C. § 601.

      [27].    44 U.S.C. §§ 3501, 3506 (2012).

      [28].    42 U.S.C. § 4321 (2012).

      [29].    5 U.S.C. § 801.

      [30].    See Frank Partnoy, Wait:  The Art and Science of Delay 240 (2012) (arguing that “[o]ur most important policy decisions—about the economy, jobs, health care, defense, the environment, and foreign relations—require that smart people spend long periods of time thinking strategically”).

      [31].    See, e.g., Susan E. Dudley, Make “Independent” Regulatory Agencies More Accountable to the Public, Forbes (May 9, 2017, 7:00 AM), https://www.forbes.com/sites/ susandudley/2017/05/09/make-independent-regulatory-agencies-more-accountable-to-the-public (urging President Trump to issue an Executive Order that will require independent agencies to conduct impact analyses and submit proposed regulations to OIRA).  Even those observers who express concern over an undue emphasis on benefit-cost analysis in the regulatory process recognize the value in prospective analysis of some kind.  See, e.g., Susan Rose-Ackerman, Putting Cost-Benefit Analysis in its Place:  Rethinking Regulatory Review, 65 U. Miami L. Rev. 335, 337–38 (2011) (seeking to “challenge the hegemony” of benefit-cost analysis while still “favor[ing] technocratic analysis that measures both costs and benefits in the most accurate way possible”).

      [32].    On occasion, of course, an independent agency’s organic statute may require it to conduct a benefit-cost analysis.  Such is the case, for instance, with the Consumer Product Safety Commission (CPSC), which must prepare a “description of the potential benefits and potential costs of [any new product safety] rule.”  15 U.S.C. § 2058(f)(2)(A) (2012).

      [33].    2 U.S.C. § 658(1) (2012); see also id. § 1502 (incorporating the definition in § 658 into UMRA).  The enacted version of UMRA does not define the words “independent regulatory agencies,” but the conference committee report indicates that they were intended to mirror the definition in the Paperwork Reduction Act, the only other place in the U.S. Code where these words are defined.  S. Rep. No. 104-2, at 31 (1995).

      [34].    Section 3(b) of the Executive Order states:  “‘Agency,’ unless otherwise indicated, means any authority of the United States that is an ‘agency’ under 44 U.S.C. § 3502(1), other than those considered to be independent regulatory agencies, as defined in 44 U.S.C. § 3502(10).”  Exec. Order No. 12,866 § 3(b), 3 C.F.R. 638, 641 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06.  The provisions of the Paperwork Reduction Act have been re-numbered, so that the definition of independent regulatory agencies is now at 44 U.S.C. § 3502(5) (2012).

      [35].    Curtis W. Copeland, Economic Analysis and Independent Regulatory Agencies 4 (2013), https://www.acus.gov/sites/default/files/documents/Copeland %20Final%20BCA%20Report%204-30-13.pdf.

      [36].    Revesz, supra note 5, at 560.

      [37].    Copeland, supra note 35, at 4.

      [38].    Office of Mgmt. & Budget, supra note 4, at 32.

      [39].    See, e.g., Arthur Fraas & Randall Lutter, On the Economic Analysis of Regulations at Independent Regulatory Commissions, 63 Admin. L. Rev. 213, 236 (2011) (reporting that their “review suggests that the economic analyses prepared by independent regulatory commissions do not measure up to those of the executive branch agencies”); see also Jerry Ellig & Rosemarie Fike, Regulatory Process, Regulatory Reform, and the Quality of Regulatory Impact Analysis 43–45 (Mercatus Ctr. Working Paper No. 13-13, July 2013) (reporting study results that suggest independent agencies’ economic analysis is of lower quality than such analysis conducted by executive agencies).

      [40].    See Fraas & Lutter, supra note 39, at 218–19, 221 (noting that the Comptroller General does not “analyze or comment on the substance or quality of rulemaking”).

      [41].    See id. at 227 (reporting that rules issued by the Federal Reserve contained greater attention to regulatory burdens that indicated by summaries); Copeland, supra note 35, at 111 (noting that, despite what studies based on reported summaries suggest, “[i]n fact . . . the agencies often at least qualitatively discuss regulatory costs and benefits (and often discuss at least some costs in quantitative or even monetary terms)”).

      [42].    Kenneth J. Arrow, et al., Is There a Role for Benefit-Cost Analysis in Environmental, Health, and Safety Regulation?, 272 Sci. 221, 222 (1996) (acknowledging that “not all impacts can be quantified”).

      [43].    Tanya Xu, How Should We Measure Terrorism Risk?, Reg. Rev. (Aug. 25, 2016), https://www.theregreview.org/2016/08/25/xu-how-should-we-measure-terrorism-risk (discussing the challenges of applying typical analytic approaches to intentional human behavior like terrorism).

      [44].    Exec. Order No. 12,866 § 1(b)(6), 3 C.F.R. 638, 639 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06 (2012).

      [45].    John C. Coates IV, Cost-Benefit Analysis of Financial Regulation:  Case Studies and Implications, 124 Yale L.J. 882, 886–87, 997–99 (2014) (arguing against the reliability and feasibility of quantitative benefit-cost analysis of financial regulations); Jeffrey N. Gordon, The Empty Call for Benefit-Cost Regulation in Financial Regulation, 43 J. Legal Stud. S351–S353 (2014) (arguing that benefit-cost analysis cannot be effectively applied to financial regulation).  Other scholars have contested the view that benefit-cost analysis of financial regulation is infeasible.  See, e.g., Eric Posner & E. Glen Weyl, Benefit-Cost Analysis for Financial Regulation, 103 Am. Econ. Rev.:  Papers & Proc. 393–94, 397 (2013); Revesz, supra note 5, at 548–49.

      [46].    Patrick A. McLaughlin & Jerry Ellig, Does OIRA Review Improve the Quality of Regulatory Impact Analysis?  Evidence from the Final Year of the Bush II Administration, 63 Admin. L. Rev. 179, 181 (2011).

      [47].    Copeland, supra note 35, at 4, 101–02, 111–12.

      [48].    See Robert W. Hahn & Patrick M. Dudley, How Well Does the U.S. Government Do Benefit-Cost Analysis?, 1 Rev. Envtl. Econ. & Pol’y 192, 194–97 (2007) (examining seventy-four regulatory impact analyses across three administrations and finding substantial quality shortfalls).

      [49].    Ryan Bubb, Comment:  The OIRA Model for Institutionalizing CBA of Financial Regulation, 78 L. & Contemp. Probs. 47, 50 (2015).

      [50].    The options discussed below all contemplate general changes to administrative procedures.  It bears noting that, if Congress wished to take a more incremental approach, it could target just one or more individual agencies.  To some extent, the organic statutes of individual agencies already vary in that they direct some agencies to consider—and others not to consider—costs when making regulatory decisions.  See Copeland, supra note 35, at 4.

      [51].    See Bruce Kraus & Connor Raso, Rational Boundaries for SEC Cost-Benefit Analysis, 30 Yale J. on Reg. 289, 290 (2013) (describing how the D.C. Circuit “has set a very high bar for economic analysis in rulemaking” for the SEC and other financial regulators through a string of cases spanning twenty years); Eugene Scalia, Cost-Benefit Analysis and Reasoned Agency Decision-Making, Reg. Rev. (Sept. 26, 2017), https://www.theregreview.org/2017/09/26/ scalia-cost-benefit-analysis-reasoned-agency-decision-making (defending court actions over the SEC’s mutual fund governance rule and its proxy access rule).

      [52].    Kraus & Raso, supra note 51, at 327–29, 342.

      [53].    647 F.3d 1144 (D.C. Cir. 2011).

      [54].    Kraus & Raso, supra note 51, at 292, 325–26 (explaining that “[t]he SEC began implementing significant changes only in the wake of the Business Roundtable decision.  The artificial separation of the SEC [benefit-cost analysis] and [efficiency, competition, and capital formation] consideration sections was finally abandoned,” resulting in the publication of one analysis for each issue the SEC addressed, which laid out the fact with the benefits and drawback of each proposed rule); see also Jerry Ellig, Systematic Study Shows Improvement in SEC Economic Analysis, Reg. Rev. (Mar. 20, 2017), https://www.theregreview.org/2017/03/20/ellig-systematic-study-shows-improvement-sec-economic-analysis.

      [55].    Fact Sheet:  Establishment of the Office of Economics and Analytics Order, Fed. Commc’n Comm’n (Jan. 9, 2018), https://transition.fcc.gov/Daily_Releases/ Daily_Business/2018/db0109/DOC-348636A1.pdf; see also Ajit Pai, The Importance of Economic Analysis at the FCC (April 5, 2017), https://s3.amazonaws.com/ media.hudson.org/files/publications/20170405PaiDOC-344248A1.pdf (remarks of FCC Chairman Ajit Pai at the Hudson Institute).

      [56].    See, e.g., Peter M. Shane, Presidential Regulatory Oversight and the Separation of Powers:  The Constitutionality of Executive Order No. 12,291, 23 Ariz. L. Rev. 1235, 1258–59 (1981) (“More difficult questions would surely be posed if the President were to extend all the provisions of [the regulatory review] Executive Order . . . to [independent] agencies, the heads of which are not removable at his discretion.”); Revesz, supra note 5, at 586 (observing that no President has extended regulatory review to independent agencies due to concerns about congressional reactions).

      [57].    Exec. Order No. 12,866 § 2(b), 3 C.F.R. 638, 640 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06 (2012).

      [58].    See Exec. Order No. 12,866 §§ 2(a), 2(b), 3(f)(4), 3 C.F.R. 640, 642.

      [59].    Exec. Order No. 12,866 § 6(a)(3)(A), 3 C.F.R. 645.

      [60].    Exec. Order No. 12,866 §§ 7–8, 3 C.F.R. 648–49.

      [61].    See, e.g., Exec. Order No. 12,866 § 6(b)(2)(C), 6(b)(3), 3 C.F.R. 647; see also Exec. Order No. 12,866 § 7, 3 C.F.R. 648 (describing the conflict resolution process as taking place between the President or Vice President and “the relevant agency head” or “the head of the issuing agency”).

      [62].    See Free Enter. Fund v. Pub. Co. Accounting Oversight Bd., 561 U.S. 477, 512–13 (2010) (“As a constitutional matter, we see no reason why a multimember body may not be the ‘Hea[d]’ of a ‘Departmen[t]’ that it governs.”).

      [63].    See, e.g., Exec. Order No. 12,866 §§ 4(c)(4), 6(b)(3), 3 C.F.R. 643, 647.

      [64].    Donald R. Arbuckle, OIRA and Presidential Regulatory Review:  A View from Inside the Administrative State 68–70 (2008), https://works.bepress.com/ donald_arbuckle/1/download (describing back-and-forth interactions between OIRA and agency officials); Sally Katzen, OIRA at Thirty:  Reflections and Recommendations, 63 Admin. L. Rev. 103, 105–07 (2011) (describing the normally collegial working relationships between OIRA and agencies).

      [65].    5 U.S.C. § 552(b) (2012) (outlining the basic requirements for meetings of multi-member bodies).

      [66].    It is highly doubtful that White House officials would want to conduct their meetings with multimember commissions in the open.

      [67].    Exec. Order No. 12,866 § 7, 3 C.F.R. 648.  Of course, the same provision qualifies the priority for the President by indicating that such a priority applies only “[t]o the extent permitted by law.”  Id.  Furthermore, section 9 of the Order makes clear that “[n]othing in this order shall be construed as displacing agencies’ authority or responsibilities, as authorized by law.” Exec. Order No. 12,866 § 9, 3 C.F.R. 649.  These qualifying provisions could naturally be read to preclude the President from having the final say in any disputes between OIRA and an independent agency because doing so would offend the agency’s legal autonomy.  If so, then this offers another reason not to codify unthinkingly Executive Order 12,866 verbatim, for doing so would likely mean that the OIRA review process would not have the same impact as it has for executive agencies.

      [68].    44 U.S.C. § 3507(f) (2012).

      [69].    S. 1607, 114th Cong. (2015).

      [70].    Id. §§ 3–4.

      [71].    See id. § 4.

      [72].    Robin Bravender & Emily Yehle, Wonks in Embattled Regulatory Office are Mysterious—But “Not Nefarious,” E&E News (Feb. 18, 2014), https://www.eenews.net/ stories/1059994711 (“OIRA has a staff of about 44 people.”).

      [73].    2 U.S.C. § 1502(1) (2012) (incorporating the definition of “agency” in 2 U.S.C. § 658, which “does not include independent regulatory agencies”).  At least three bills have been introduced in Congress that would eliminate this exemption from UMRA: (1) Unfunded Mandates Accountability Act of 2017, S. 686, 115th Cong. § 5 (2017); (2) Unfunded Mandates Information and Transparency Act of 2017, S. 1523, 115th Cong. § 5 (2017); and (3) Unfunded Mandates Information and Transparency Act of 2017, H.R. 50, 115th Cong. § 5 (2017).

      [74].    2 U.S.C. § 1532(a)(2).

      [75].    §§ 1532(a)(2), 1571(a)(2)–(3).  Under section 1571(a)(2) of the UMRA, if an agency fails to complete a required “statement” of benefits and costs for a qualifying rule, “a court may compel the agency to prepare such written statement.”  Id. § 1571(a)(2)(B).  Section 1571(a)(3) states that “the inadequacy or failure to prepare such statement . . . shall not be used as a basis for staying, enjoining, invalidating or otherwise affecting such agency rule.”  Id. § 1571(a)(3).  In other words, UMRA authorizes the courts to compel the preparation of a benefit-cost analysis, but courts cannot, on the basis of UMRA, remand a rule because the agency failed to prepare such an analysis or because a court finds the agency’s analysis to be inadequate.  Id. § 1571(a)(2), (3).

      [76].    § 1532(a) (requiring economic impact statement for rules that would demand an “expenditure . . . by the private sector, of $100,000,000 or more”).

      [77].    The Executive Order defines a “significant regulatory action” as one that is expected to “[h]ave an annual effect on the economy of $100 million or more.”  Exec. Order No. 12,866 § 3(f)(1), 3 C.F.R. 638, 645 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06 (2012) (emphasis added).

      [78].    See 2 U.S.C. § 1532(a).

      [79].    Section 1571(a)(4) of UMRA provides that “[a]ny information generated [in developing a benefit-cost analysis statement] that is part of the rulemaking record for judicial review under the provisions of any other Federal law may be considered as part of the record for judicial review conducted under such other provisions of Federal law.”  Id. § 1571(a)(4).  Thus, although courts cannot pass on the adequacy of an agency’s “statement” under UMRA, they can review the underlying “information” upon which the statement is based when reviewing agency rules under 5 U.S.C. § 706(2)(A) (2012), the arbitrary and capricious standard in the Administrative Procedure Act.

      [80].    See supra notes 40–48 and accompanying text.

      [81].    Coglianese, Empirical Analysis, supra note 11, at 1119–25 (reviewing the available empirical research on the impact of the economic analysis required under executive order and concluding that such a requirement “does not eliminate inefficiency, and it may not even significantly reduce it”); see also infra note 85.

      [82].    See 2 U.S.C. § 1532(3) (requiring agencies to estimate compliance costs only “to the extent that the agency determines that accurate estimates are reasonably feasible”); Exec. Order No. 12,866 §§ 1(b)(6), 6(a)(3)(C)(i)–(ii), 3 C.F.R. 638, 639, 645 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06 (specifically “recognizing that some costs and benefits are difficult to quantify” and requiring quantification of benefits and costs only to the “extent feasible”).

      [83].    See, e.g., Whitman v. American Trucking Ass’ns, 531 U.S. 457 (2001) (holding that the Clean Air Act does not permit the EPA to consider costs when setting ambient air quality standards).  See generally Richard J. Pierce, Jr., What Factors Can an Agency Consider in Making a Decision?, 2009 Mich. St. L. Rev. 67, 71 (“[A]n agency cannot consider a factor that Congress explicitly or implicitly prohibited it from considering.”).

      [84].    Shelley H. Metzenbaum & Gaurav Vasisht, What Makes a Regulator Excellent?  Mission, Funding, Information, and Judgment, in Achieving Regulatory Excellence 148, 153 (Cary Coglianese ed., 2017).

      [85].    Hahn & Dudley, supra note 48, at 206–07 (finding that the quality of agencies’ regulatory analyses varied greatly across agencies and from one administration to the next).

      [86].    Coglianese, Empirical Analysis, supra note 11, at 1125.

      [87].    Admin. Conference of the United States, Administrative Conference Recommendation 2013-3:  Benefit-Cost Analysis at Independent Regulatory Agencies 8 (2013), https://www.acus.gov/sites/default/files/documents/Recommendation%202013-2%20%28Benefit-Cost%20Analysis%29_0.pdf.

      [88].    Memorandum from Cass R. Sunstein, Office of Mgmt. & Budget, Exec. Office of the President for the Heads of Exec. Dep’ts and Agencies (April 25, 2011), https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2011/m11-19.pdf.

      [89].    Exec. Order No. 13,563 § 1(a), 3 C.F.R. 215 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 816–17 (2012).

      [90].    Exec. Order No. 13,563 § 6(b), 3 C.F.R. 217.

      [91].    Howard Shelanski, Retrospective Review, by the Numbers, White House Blog (Aug. 31, 2016, 9:00 AM), https://obamawhitehouse.archives.gov/blog/2016/08/ 31/retrospective-review-numbers-0.

      [92].    Id.  A review of the Obama Administration’s lookback initiative that the Administrative Conference commissioned suggests that many of these cost-savings came in the form of administrative changes, such as switching to electronic filings, rather than making substantive regulatory changes.  Joseph E. Aldy, Learning from Experience:  An Assessment of the Retrospective Reviews of Agency Rules and the Evidence for Improving the Design and Implementation of Regulatory Policy 52 (2014), https://www.acus.gov/sites/default/files/documents/Aldy%2520Retro%2520Review%2520Draft%252011-17-2014.pdf.  A random sample of fifty retrospective reviews, completed in July 2015 for a separate project at the Penn Program on Regulation, showed that, in slightly more than three-fourths of the reviews that resulted in changes, the changes were of an administrative or paperwork variety.  Reducing unnecessary paperwork burdens is no doubt to be applauded, but streamlining administrative processes is not squarely centered on improving “the actual results of regulatory requirements.”  Executive Order 13,563 § 1(a), 3 C.F.R. 215.

      [93].    See, e.g., Howard Shelanski, Reducing Costs and Burdens:  Further Progress in Regulatory Lookback Effort, White House Blog (May 7, 2014, 7:05 PM), https://obamawhitehouse. archives.gov/blog/2014/05/07/reducing-costs-and-burdens-further-progress-regulatory-lookback-effort (highlighting the Centers for Medicare & Medicaid Services change to a regulation, which was projected to save $3 billion over five years); Howard Shelanski, Regulatory Lookback Eliminates Major Paperwork Burden, White House Blog (Aug. 8, 2013, 12:42 PM), https://obamawhitehouse.archives.gov/blog/2013/08/08/ regulatory-lookback-eliminates-major-paperwork-burden (discussing retrospective review at the Department of Labor (DOL), Department of Transportation (DOT), and Department of Health and Human Services (HHS)); Shelanski, supra note 91 (discussing retrospective review of regulatory systems at DOL, DOT, and Environmental Protection Agency (EPA)); Howard Shelanski, Retrospective Review:  July 2015 Lookback Reports, White House Blog (Aug. 14, 2015, 11:30 AM), https://obamawhitehouse.archives.gov/blog/2015/08/14/retrospective-review-july-2015-lookback-reports-0 (reviewing the efforts to reduce regulatory compliance at the Attorney General’s office, Department of Homeland Security, Department of Commerce, Department of Interior, Department of Justice, DOL, DOT, EPA, and the Small Business Administration); see also Cass Sunstein, Smarter Regulation:  Reducing Cumulative Burdens, White House Blog (Mar. 20, 2012, 10:30 AM), https://obamawhitehouse.archives.gov/blog/2012/03/20/smarter-regulation-reducing -cumulative-burdens (explaining that a DOL regulation adjustment will save employers $2.5 billion).  For additional discussion, see Cass R. Sunstein, Simpler:  The Future of Government (2013).  Sunstein worked on regulatory reform initiatives as the first OIRA Administrator in the Obama Administration.  Id.

      [94].    Cary Coglianese, Taking Regulation Seriously, Reg. Rev. (Jan. 28, 2012), https://www.theregreview.org/2012/01/28/taking-regulation-seriously.

      [95].    Exec. Order 13,579, 3 C.F.R. 256 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 817–18 (2012).

      [96].    Memorandum from Cass R. Sunstein, Office of Mgmt. & Budget, Exec. Office of the President for the Heads of Exec. Dep’ts and Agencies (July 22, 2011), https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2011/m11-28.pdf [hereinafter Sunstein, Memorandum on Executive Order 13,579].

      [97].    Id.

      [98].    Exec. Office of the President, Council of Economic Advisers, Smarter Regulations Through Retrospective Review 10 (2012), https://obamawhitehouse.archives.gov/ sites/default/files/lookback_report_rev_final.pdf [hereinafter Smarter Regulations Through Retrospective Review].

      [99].    Id. at 10–11.

    [100].    Id.

    [101].    Id. at 4, 10.  Undoubtedly this sounds like a major achievement in regulatory reduction, but it is hard to imagine that the FCC’s actions could be attributable to any serious retrospective review conducted in just the four months following the signing of Executive Order 13,579.  The timing of the FCC’s actions suggests that either these revocations were already in progress before that order was issued or that the rules that were eliminated were so obviously outmoded that removing them was an inconsequential housekeeping matter.

    [102].    Aldy, supra note 92, at 52.

    [103].    Id. at 49–50 (explaining that, of thirty-nine independent agency rules issued in 2013 and 2014, “only eight monetized the costs of the regulatory action,” “none . . . monetized the benefits,” and the regulations did not result from retrospective review nor contain provisions for future review).

    [104].    See U.S. Nuclear Regulatory Comm’n, Final Plan for Retrospective Analysis of Existing Rules 5 (Jan. 24, 2014), https://www.nrc.gov/docs/ML1400/ML14002A441.pdf.

    [105].    U.S. Consumer Prod. Safety Comm’n, RCA:  Plan for Retrospective Review of Existing Rules 6–7 (Apr. 1, 2016), https://www.cpsc.gov/s3fs-public/PlanforRetrospective ReviewExistingRules.pdf.

    [106].    Overall, a Senate Committee found in 2015 that agency retrospective review “efforts had resulted in few completed reviews since the 2011 executive orders and that better data and more planning would allow agencies to conduct better reviews.”  S. Rep. No. 114-282, at 4 (2016).

    [107].    See, e.g., Retrospective Review of Regulations, White House, https://obamawhitehouse. archives.gov/omb/oira/regulation-reform (last visited Feb. 7, 2018) (listing twenty-six government entities and their status reports for 2015 and 2016).

    [108].    Exec. Order No. 13,563, 3 C.F.R. 215 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 816–17 (2012).

    [109].    Memorandum from Cass R. Sunstein, Office of Mgmt. & Budget, Exec. Office of the President for the Heads of Exec. Dep’ts and Agencies (Oct. 26, 2011), https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/inforeg/implementation-of-retrospective-review-plans.pdf [hereinafter Sunstein, Memorandum on Implementation of Retrospective Review Plans].

    [110].    Exec. Order No. 13,610, 3 C.F.R. 258 (2013), reprinted as amended in 5 U.S.C. § 601 app. at 820–21 (2012).

    [111].    Sunstein, Memorandum on Implementation of Retrospective Review Plans, supra note 109.

    [112].    Executive Order 13,771 § 2(c), 82 Fed. Reg. 9339, 9339 (2017).

    [113].    See, e.g., Sofie E. Miller & Susan E. Dudley, The Devil is in the Details of President Trump’s Regulatory Executive Order, Regulatory Stud. Ctr. (Feb. 1, 2017), https://regulatorystudies.columbian.gwu.edu/devil-details-president-trump%E2% 80%99s-regulatory-executive-order (“Although many [P]residents have encouraged agencies to retrospectively review their regulations, President Trump’s [executive order] may provide agencies with some of the strongest incentives for assessing the costs—and the benefits—of their existing rules.”).

    [114].    Aldy, supra note 92, at 34.

    [115].    Unlike other executive orders on regulation, President Donald J. Trump’s Executive Order 13,771 makes no mention of even the possibility of any benefits from regulation, emphasizing instead just a need “to manage the costs associated with the governmental imposition of private expenditures required to comply with Federal regulations.”  Exec. Order 13,771 § 1, 82 Fed. Reg. 9339.

    [116].    Coglianese & Bennear, supra note 11, at 250–51.

    [117].    Cary Coglianese, Measuring Regulatory Excellence, in Achieving Regulatory Excellence 299–300, 305–06 (Cary Coglianese ed., 2017) (explaining why measurement is essential for regulators to learn what works and to make smarter decisions); see also Admin. Conf. Recommendation 2017-6, 82 Fed. Reg. 61,738–42 (Dec. 29, 2017) [hereinafter Admin. Conf. of the U.S., Learning from Regulatory Experience] (emphasizing the importance of learning from experience in order to make better regulatory decisions).

    [118].    Reviewing Independent Agency Rulemaking:  Hearing Before the Subcomm. on Regulatory Affairs and Fed. Mgmt. of the S. Comm. on Homeland Sec. and Gov’t Affairs, 114th Cong. 79 (2016) (statement of Cary Coglianese, Edward B. Shils Professor of Law, Director, Penn Program on Regulation, University of Pennsylvania); see Transp. Research Bd., Nat’l Acads. of Scis., Eng’g, and Med., Designing Safety Regulations for High-Hazard Industries (2017), https://nap.edu/24907 (examining several industries that pose high safety risks and offering recommendations for regulating those sectors); Cary Coglianese & Thomas R. Menzies, Designing Safety Regulations for High-Hazard Industries, Reg. Rev. (Oct. 4, 2017), https://www.theregreview.org/ 2017/10/04/coglianese-menzies-safety-regulations-hazard-industries (summarizing and commenting on the National Academies of Sciences, Engineering, and Medicine’s report on regulatory design).

    [119].    Cary Coglianese, Moving Forward with Regulatory Lookback, 30 Yale J. on Reg. Online 57, 65 (2013), https://yalejreg.com/moving-forward-with-regulatory-lookback.

    [120].    For a helpful discussion of differences in regulatory strategies, see Coglianese & Menzies, supra note 118.

    [121].    Coglianese & Bennear, supra note 11, at 251–52.

    [122].    See Aldy, supra note 92, at 34 (recognizing that past reviews have sought to streamline rules and reduce burdens).

    [123].    5 U.S.C. § 610(a) (2012).

    [124].    Id.  Executive Order 13,563 contains a similar purpose:  “to promote retrospective analysis of rules that may be outmoded, ineffective, insufficient, or excessively burdensome.”  Exec. Order No. 13,563 § 6, 3 C.F.R. 215, 217 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 816–17.

    [125].    Exec. Order 12,866, 3 C.F.R. 638 (1994), reprinted as amended in 5 U.S.C. § 601 app. at 802–06.

    [126].    See Sunstein, Memorandum on Executive Order 13,579, supra note 96 (urging agencies to “exercise [their] discretion to develop a plan tailored to [their] specific mission, resources, organizational structure, and rulemaking history and volume”).

    [127].    If Congress seeks to direct an agency to evaluate a specific regulation or set of regulations, it always can do so through other legislation, as it already does from time to time.  In such cases, Congress may also need to consider appropriating additional funding to support the desired evaluation research.

    [128].    Cf. Coglianese, supra note 119, at 64–65 (recommending that agencies be encouraged to evaluate rules promulgated in the light of uncertainty over benefits or costs or in the face of common issues presented in multiple rulemakings).  As a report issued by the CEA has noted, “Retrospective analysis is an important complement to prospective analysis.  In some cases, prospective analysis of costs and benefits will be highly uncertain; retrospective analysis can provide valuable additional information and ultimately lead to better regulations.”  Smarter Regulations Through Retrospective Review, supra note 98, at 1.  In addition, recommendations issued by the Administrative Conference contain a further list of helpful considerations that agencies may consider when prioritizing retrospective analysis.  Admin. Conference of the United States, Administrative Conference Recommendation 2014-5:  Retrospective Review of Agency Rules 9–10 (2014), https://www.acus.gov/sites/default/files/ documents/Recommendation%25202014-5%2520%2528Retrospective%2520Review %2529_1.pdf.  Such recommendations could be usefully codified as criteria for agencies to draw upon when planning and conducting retrospective analysis.

    [129].    Office of Mgmt. & Budget, Exec. Office of the President, Circular A-4:  Regulatory Analysis 2–3 (Sept. 17, 2003), https://obamawhitehouse.archives.gov/ sites/default/files/omb/assets/omb/circulars/a004/a-4.pdf (identifying “three basic elements” that every prospective analysis should include).  Of course, independent agencies would need to have the good sense to refer to and rely on these best practices or guidelines, as absent any changes to existing analytic requirements these agencies would not fall within the OIRA orbit.  For a robust argument for making such changes, precisely so that independent agencies can benefit from quality guidance from OIRA, see Revesz, supra note 5, at 584–86, 588.

    [130].    See 44 U.S.C. § 3503 (2012) (establishing OIRA within the Office of Management and Budget).

    [131].    § 3507 (codifying the requirements for approval of agency collection of information from the public).  As previously noted, independent agencies can override an OIRA disapproval of an information collection request.  See supra note 68 and accompanying text.

    [132].    See, e.g., Office of Info. & Regulatory Affairs, 2016 Draft Report to Congress on the Benefits and Costs of Federal Regulations and Agency Compliance with the Unfunded Mandates Reform Act 5 (2016), https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/legislative_reports/draft_2016_cost_benefit_report_12_14_2016_2.pdf (“As has been the practice for many years, all estimates presented in this chapter are agency estimates of benefits and costs, or minor modifications of agency information performed by OMB.”).

    [133].    See Coglianese, supra note 119, at 64–66 (proposing a practice of “evaluation prompts,” according to which OIRA would identify rules that would present valuable learning opportunities).

    [134].    See Hahn & Dudley, supra note 48, at 194 (reporting that the quality of regulatory impact analyses across different administrations shows “a great deal of variation”).

    [135].    Coglianese, supra note 119, at 62 (“An evaluation plan would constitute only a small part of an overall [regulatory impact analysis], and it would be non-binding in the sense that an agency would not be obligated to carry out the plan.”).

    [136].    For a helpful discussion of agency options for the use of experimental and quasi-experimental research designs to learn about the impact of their regulations, see Admin. Conf. of the U.S., Learning from Regulatory Experience, supra note 117.

    [137].    Coglianese, supra note 119, at 62–63.

    [138].    Smarter Regs Act of 2015, S. 1817, 114th Cong. § 2 (2015).

    [139].    See id.

    [140].    Planning for evaluation at the outset would be consistent with the Evidence-Based Policymaking Commission Act of 2016’s goal of finding ways “to incorporate outcomes measurement . . . and rigorous impact analysis into program design.”  Evidence-Based Policymaking Commission Act of 2016, Pub. L. No. 114-140, § 4(a)(3), 130 Stat. 317, 318.

    [141].    Cf. Comm’n on Evidence-Based Policymaking, The Promise of Evidence-Based Policymaking 88 (2017), https://www.cep.gov/content/dam/cep/report/cep-final-report.pdf (“Federal departments must enhance their capacity for evidence building to support the growth of evidence-based policymaking.”).

    [142].    See Aldy, supra note 92, at 4–6 (stating that the Obama Administration focused on creating a culture of retrospective review but also left room for improvement in the quality and effectiveness of those reviews).

    [143].    The Obama Administration’s final reporting on its lookback initiative claimed “more than 800 retrospective review initiatives the agencies have identified as complete.”  Shelanski, supra note 91.  My independent count of the entries listed in the final July 2016 updates submitted by the twenty-six agencies actually yields fewer than 800 regulatory initiatives, especially because some entries were for reviews of information collection requests and other non-rule reviews.  Connor Raso has reported that, as of the July 2016 updates, twenty-two executive agencies had listed 459 planned retrospective reviews and 238 completed reviews—for a total of 697 reviews.  Connor Raso, Assessing Regulatory Retrospective Review Under the Obama Administration, Brookings (June 15, 2017), https://www.brookings.edu/research/assessing-regulatory- retrospective-review-under-the-obama-administration.

    [144].    Twenty-six executive departments and agencies filed reports indicating that they had conducted retrospective reviews.  Retrospective Review of Regulations, Office of Mgmt. & Budget, https://obamawhitehouse.archives.gov/omb/oira/regulation-reform (last visited Feb. 7, 2018).  The averages reported in the text are estimates based dividing roughly 800 reviews by twenty-six.  Raso, supra note 143.  In addition to recognizing that these numbers are at best rough estimates, it should be noted that they only apply to executive branch agencies.  Id.  Shortly after it launched its lookback initiative in 2011, the Obama Administration did initially prompt some independent agencies to submit plans for retrospective reviews.  Exec. Order 13,579, 3 C.F.R. 256 (2012), reprinted as amended in 5 U.S.C. § 601 app. at 817–18 (2012); see also Aldy, supra note 92, at 42–43, 45.  But the independent agencies were not required to submit updates or progress reports on their plans, so a compilation or other ready source of the number of retrospective reviews actually conducted by independent agencies is not available.

    [145].    See, e.g., Aldy, supra note 92, at 52 (“The vast majority of status updates on agencies’ retrospective review programs do not include evidence of formal retrospective analysis . . . .”).

    [146].    Id.  Aldy also notes the “short time-frame” the Obama lookback initiative afforded agencies to develop plans for and report on their reviews, a time-frame indicative of the cursory nature of the so-called analysis underlying agencies’ efforts.  Id. at 51.

    [147].    See Coglianese, supra note 117, at 300 (urging agencies to perform “causal attribution evaluation” for at least some regulations to understand more than actions or outcomes alone).

    [148].    Coglianese, Measuring Regulatory Performance, supra note 9, at 38–39 (highlighting statistical strategies that “can be used to estimate the counterfactual and compare it with the existing state of the world”).  See generally Jonah B. Gelbach & Jonathan Klick, Empirical Law and Economics, in 1 The Oxford Handbook of Law and Economics 29 (Francisco Parisi ed., 2017); Joshua d. Angrist & Jörn-Steffen Pischke, Mastering Metrics:  The Path from Cause to Effect (2015).

    [149].    Even if there were other sources, the pathways from fuel combustion to air levels of lead may be sufficiently well-understood, and the adverse health effects of lead so significant, that even a modest reduction from the air would still dwarf any adverse effects of a ban, making an investment in causally-oriented evaluation seem less urgent.  Gaining a better understanding of other effects of a ban on lead additives, however, would still necessitate causally-oriented evaluation, such as if it were thought meaningful to know how the ban may have affected vehicle engine design and performance, as such outcomes are almost certainly affected by other factors.

    [150].    See Coglianese, supra note 17, at 66.

    [151].    See supra notes 2–3.

    [152].    See supra notes 6–7 and accompanying text.