Evidence Based Policy

Our Research

Between 2018 and 2022, the Evidence Based Policy Research Project (EBPRP) produced unique research on the quality of public policy making in Australia and achieved a significant breakthrough for evidence-informed policy development in NSW. Having completed its mission, the EBPRP ceasedto operate as a separate incorporated voluntary association in NSW in January 2023.

Inspired by the work of EBPRP, the Susan McKinnon Foundation drives forward a program of work to support improved governance and greater transparency and accountability on legislative policy. The immediate and ongoing priority of the Foundation’s work is to support the rigorous application of the Statement of Public Interest (SPI) in New South Wales through research, education, and advocacy.

The purpose of this website is to record the Project’s important research findings on the quality of government decision making at federal and state levels in Australia between 2018 and 2022 and to relay its success in persuading the NSW Upper House to give legal force to a Statement of Public Interest (SPI) questionnaire that all NSW Government bills must now answer.

The questions asked are – why is a public policy proposal needed, what is its objective, were alternative policies considered, if so what were their public costs and benefits, how will the proposal be executed and over what timeframe, and what stakeholders if any were consulted?

These questions are obvious, yet no other Parliament in Australia insists they be answered before a government policy requiring legislation is debated and approved.

Our Story

In November 2017 a Symposium convened by the newDemocracy Foundation agreed to challenge opinion leaders by asking them what practical, measurable steps could be taken to improve public trust in government decisions. The objective was to move from complaining about problems to offering potential solutions. The Evidence Based Policy Review Project was one of the major initiatives to emerge from the 2017 Symposium event.

Many participants – company directors, the advocacy sector, journalists, and even former MPs – lamented that “evidenced based policy making” had become an empty phrase which everyone claimed to pursue but no one knew how to quantify

Former Secretary of the NSW Treasury, Percy Allan, championed a proposal to draw on the “good practice” policy making criteria developed by Prof. Kenneth Wiltshire AO from the University of Queensland which had been used in assessing the quality of federal legislation by the Institute of Public Administration Australia (IPAA) in 2012 when Percy was its national president.

With the support of newDemocracy, it was agreed to undertake an independent benchmarking assessment of legislative initiatives from the Commonwealth and three states – New South Wales, Victoria and Queensland – rating them against the “Wiltshire criteria”.

As a result, the Institute of Public Affairs (IPA), a self-described ‘free-market’ think tank identified with the Right of politics, and Per Capita Australia, a self-labelled ‘progressive’ think tank identified with the Left, were commissioned in 2018, 2019, 2020 and 2021, in a ‘double blind’ exercise which each year saw them assess 20 policies drawn from around the country: eight from the Commonwealth and four from each of the three states. In 2022, the Blueprint Institute, which describes itself as ‘economically conservative’ took over from IPA when the latter withdrew because of other research demands.

Though these Right and Left think tanks often have opposing policy outlooks, their benchmarking results against the Wiltshire criteria were remarkably similar, across what is now a total of 100 legislative initiatives analysed. For five years running, the think
tanks have found that basic standards of evidence and consultation-based policy making are inadequately met by Australian federal and state governments.

Our Findings

The results of the 100 case studies undertaken so far over the last five years to 2022 suggest a solid process was followed in thirty-two of them by the governments involved. In twenty-five cases the ratings were well below par. In the balance of cases the process quality was mediocre. See table below

Overall Findings, 2018-2022

Policy Decision-Making Process	Think Tanks’ Average Score out of 10 Test Criteria	2018 Case Studies Number	2019 Case Studies Number	2020 Case Studies Number	2021 Case Studies Number	2022 Case Studies Number	2018-22 Total Case Studies No. & % Share
Solid (Acceptable, Sound or Excellent)	7 – 10 criteria satisfied	6	6	9	6	5	32 (32%)
Mediocre	5 – 6.5 criteria satisfied	10	6	9	9	9	43 (43%)
Unacceptable	Under 5 criteria satisfied	4	8	2	5	6	25 (25%)
Total		20	20	20	20	20	100 (100%)

The think tanks’ total scores on the ten Wiltshire criteria for the one hundred case studies to date were remarkably similar in seventy-eight cases (either identical or only one-point difference). Of the remaining twenty-two cases, nineteen had differences in scoring of two-points. See table below.

Total Score Differences	2018 Case Studies Number	2019 Case Studies Number	2020 Case Studies Number	2021 Case Studies Number	2022 Case Studies Number	2018-22 Total Case Studies No.& % Share
None	7	8	6	3	4	28 (28%)
1 point	9	7	10	9	15	50 (50%)
2 point	4	5	4	5	1	19 (19%)
3 point	0	0	0	3	0	3 (3%)
Total	20	20	20	20	20	100(100%)

Findings by Jurisdictions

Averaging the two think tanks’ total scores (out of 10) for each of the 100 cases done over the last five years (20 each in 2018, 2019, 2020, 2021, and 2022) gives the following average score for each jurisdiction

Jurisdiction	2018-2022 Number of Case Studies	2018-2022 Average Score for all Case Studies
Federal	40	5.4
NSW	20	5.6
Vic	20	6.0
Qld	20	6.4
Weighted Mean Average	100	5.8

Table Legend:

Excellent: 9.0 – 10.0
Sound: 8.0 – 8.5
Acceptable: 7.0 – 7.5
Mediocre: 5.0 – 6.5
Unacceptable: below 5.0

Our Proposal for Australian Parliaments

The remedy is for a more rigorous process of explanation, justification and consultation (such as the traditional but now-rarely used, two-stage Green and White paper approach) to apply to any legislation that deals with a contentious public policy issue. What that involves is explained here.

At the very least, all bills introduced in a parliament should be accompanied by a Statement of Public Interest (SPI) that answers six fundamental questions at the core of the Wiltshire criteria, namely:

Need
Why is the policy needed based on factual evidence and stakeholder input?
Objectives
What is the policy’s objective couched in terms of the public interest?
Options
What alternative policies and mechanisms were considered in advance of the bill?
Analysis
What were the pros/cons and benefits/costs of each option considered?
Pathway
What are the timetable and steps for the policy’s rollout and who will administer it?
Consultation
Were the views of affected stakeholders sought and considered in making the policy?

An SPI questionnaire would take only a few pages for a bill’s proponent to answer in the affirmative or negative so would not be onerous to prepare. Every member of Parliament and interested citizen is entitled to have this information before a bill is considered.

An SPI would encourage public servants, ministers and private members who develop and submit bills to address the fundamental steps of good policy making.

Also, it would assist a parliament to decide whether a bill should be debated and voted upon without further action or referred to a standing committee of the parliament for closer scrutiny.

In May 2022, the NSW Legislative Council adopted a sessional order for an SPI for all NSW Government bills which was trialled for six months. In December 2022, it was decided to make the requirement a permanent standing order, a first for an Australian parliament.

Our gratitude to Labor for moving the motion, the Greens, and One Nation, for strongly endorsing it and for the NSW Government for not only reversing its opposition to the motion but embracing it with enthusiasm. Our three-year campaign ending in this breakthrough is described here