Evidence-based policymaking: Rules of the Road
Recommendations from the Commission on Evidence-based Policymaking – yes, there is such a thing – are making a splash in Washington, at least among those who care about or follow efforts to advance evidence-based practices in government. And though it’s never as easy as you think it’s going to be, it appears Congress is seriously considering action to enact them.
When we read this report, it became clear that the recommendations for the federal government could easily be applied to states and localities, and so we turned to Robert Shea, a participant in the Commission and a Principal at Grant Thornton where he heads up the public Sector Strategy Practice. He leads strategic and performance management engagements for federal, state and local government agencies.
Following is a well-wrought summary of the Commission on Evidence-based Policymaking report’s high points, which Shea wrote exclusively for Barrett and Greene, Inc.
By Robert Shea
When I got the call from Senator McConnell’s staff informing me they’d like to appoint me to the Commission on Evidence-Based Policymaking, I was honored and excited. I didn’t expect it, but I had been following the commission since it was a kernel of an idea of Speaker Paul Ryan’s. My background stands out from my fellow commissioners because though I’m no economist or statistician, I have been working to improve performance and evidence-based policymaking in government for decades in positions with the Congress and OMB. I had a practical view of what worked and what didn’t in the past.
Efforts to get more insight into what programs work have suffered from too little data and an unwillingness to use what little evidence we had in decision making. Better access to data we already collect, the theory goes, combined with structural changes that increase the demand for evidence, could vastly improve efforts to make programs work better. That’s what the Commission sought to address. And the commissions’ recommendations, if implemented, could make the situation better – not only for the federal government but, potentially, for states and localities as well.
After 18 months of hearings, meetings, surveys, and debate, the diverse set of 15 commissioners came to unanimous agreement on practical steps, both statutory and administrative, that would strengthen evidence gathering and use across government.
You can find our full report plus the complete set of recommendations here. But, some of the main recommendations include:
- Create a uniform process for external researchers to apply and qualify for secure access to confidential government data for evidence-building purposes while protecting privacy by carefully restricting data access to approved and qualified researchers.
- Require comprehensive risk assessments on de-identified confidential data intended for public release to improve how data are protected and risk is managed.
- Adopt modern privacy-enhancing technologies for confidential data used for evidence building to ensure that government’s capabilities to keep data secure and protect confidentiality are constantly improving. As technology evolves, we should be able to continually strengthen privacy protections.
- Assign senior officials responsibility for stewarding data within government departments. Improve leadership, coordination, and collaboration when implementing protections for the use of confidential data across government.
- Establish a new transparency and accountability portal for evidence-building activities to ensure the public is notified about how confidential data are used for evidence building and to document routine audits for compliance with rules governing privacy, confidentiality, and data access.
- Identify or establish a Chief Evaluation Officer in each department, to work in conjunction with improved coordination of other evidence-building functions within Federal departments.
- Develop learning agendas in Federal departments to support the generation and use of evidence to address the range of policymakers’ questions.
- Improve coordination of government-wide evidence building by directing OMB to facilitate improved coordination across government, and consider how a greater commitment to foundational information policy setting and coordination responsibilities can be achieved, including through any consolidation or reorganization at OMB that may be necessary.
Perhaps more important than the concrete recommendations were the principles that guided our work. About mid-way through our efforts, commissioners agreed on the need to document the fundamental concepts on which to base our recommendations. We arrived at five principles that not only frame the work of the commission, but that should also form the basis for evidence-based policymaking more broadly. The five principles we thought should guide efforts to strengthen evidence-based policymaking include:
- Privacy. Individual privacy and confidentiality must be respected in the generation and use of data and evidence.
- Rigor. Evidence should be developed using well-designed and well-implemented methods tailored to the questions being asked.
- Transparency. Those engaged in generating and using data and evidence should operate transparently, providing meaningful channels for public input and comment and ensuring that evidence produced is made publicly available.
- Humility. Care should be taken not to over-generalize from findings that may be specific to a particular study or context.
- Capacity. The capacity to generate and use data and evidence should be integrated within government institutions and adequately funded and staffed.
These principles and recommendations won’t transform government, or even evidence-based policymaking overnight. There’s a long distance between making these recommendations and realizing their impact. But it’s a start. And if we get the balance between offering greater access to data and ensuring privacy protection, we may just unleash more evidence than we’ve ever had about what programs work. Now we just need leaders who’ll use it.