Author: Bruce Stoddart
Last saved by: Karen Walker

Scottish Public Pensions Agency
Update on management of PS Pensions project

Prepared by Audit Scotland
June 2019

Auditor General for Scotland
The Auditor General’s role is to:
•	appoint auditors to Scotland’s central government and NHS bodies
•	examine how public bodies spend public money
•	help them to manage their finances to the highest standards 
•	check whether they achieve value for money. 

The Auditor General is independent and reports to the Scottish Parliament 
on the performance of:
•	directorates of the Scottish Government  
•	government agencies, eg the Scottish Prison Service, 
Historic Environment Scotland 
•	NHS bodies
•	further education colleges 
•	Scottish Water 
•	NDPBs and others, eg Scottish Police Authority, Scottish Fire and 
Rescue Service.

You can find out more about the work of the Auditor General on our website: www.audit-scotland.gov.uk/about-us/auditor-general 

Audit Scotland is a statutory body set up in April 2000 under the Public Finance and Accountability (Scotland) Act 2000. We help the Auditor General for Scotland and the Accounts Commission check that organisations spending public money use it properly, efficiently and effectively.

Summary
Key messages
1.	SPPA did not prepare a clear business case for its new integrated pension administration and payment system (PS Pensions). It set an unrealistic 18-month timescale while SPPA was going through significant change. It did not adequately scrutinise the winning tender for the project which it identified as being abnormally low cost.
2.	Changes in the leadership of SPPA and the management of the project made it more difficult for SPPA to manage the supplier and hold it to account. SPPA’s governance and project assurance arrangements did not support effective scrutiny of the project.
3.	Capita was not able to provide a working system and did not achieve any of the project milestones. This was a main contributor to the project failure. Capita paid SPPA £0.7 million in November 2018 following the conclusion of a legal process. 
4.	SPPA spent £6.3 million on the project and a further £2.4 million extending contracts with existing suppliers when the project failed to meet the original timetable. The failure of the project means SPPA has been unable to progress strategic, business and workforce plans as originally intended. As a consequence SPPA requires additional revenue budget of £9.8 million between 2019/20 and 2022/23, and capital allocations of £13.6 million over the next five years. SPPA failed to achieve value for money in this project.  
Background
1.	The Scottish Public Pensions Agency (SPPA) is an executive agency of the Scottish Government. Its principal role is to administer and pay the pensions for over half a million members, deferred members and pensioners of the NHS, teachers’, police and firefighters’ pension schemes in Scotland. It also provides pension policy support to the Scottish Government and local government pension schemes and has responsibility for developing the regulations for each of these schemes. 
2.	In December 2013, SPPA introduced its Business Transformation Programme 2013-18. This included a project to develop an integrated pension administration and payment system designed to make efficiencies and help meet the additional requirements of the Public Sector Pensions Act 2013.
3.	In October 2015, SPPA awarded a contract to Capita Employee Solutions (Capita) to deliver a unified pension administration and payment system, known as ‘Project 17’ but subsequently renamed ‘PS Pensions’. The aims of the project included replacing existing systems to improve business efficiency and service quality for members, and to make financial savings in the longer term. The new system was to be operational by March 2017.
4.	SPPA’s Accountable Officer is its Chief Executive who is supported by a Management Advisory Board (MAB). The MAB is SPPA’s main strategic advisory body and in line with executive agency governance, as set out in ‘On Board’, it is not designed to be a decision-making body. The MAB is composed of the Accountable Officer, non-executive members and the ‘Fraser figure’ sponsor from the Scottish Government. SPPA also has an Audit and Risk Committee (ARC). 
5.	SPPA held extraordinary ARC and MAB meetings in February 2018. Following these meetings SPPA’s Accountable Officer decided to close the project immediately. The Auditor General reported on the implications for SPPA of the closure of the project in the Section 22 report The 2017/18 audit of the Scottish Public Pensions Agency: Management of PS Pensions project published in October 2018. The Auditor General did not comment on SPPA’s management of the project due to an ongoing legal process between SPPA and Capita. This was resolved in November 2018, with Capita making a payment of £0.7 million to SPPA. 
6.	This report sets out the history of the PS Pensions project and considers SPPA’s management of the project. Our audit work included:
•	interviews with the Accountable Officer, current and former SPPA officers and non-executive members
•	reviewing project documentation and project board minutes
•	detailing a timeline of project events (Appendix 1).

Part 1
Business case and procurement process

SPPA did not prepare a clear business case for its new integrated pension administration and payment system
7.	SPPA introduced its Business Transformation Programme for 2013 to 2018 in December 2013. This included a project to develop an integrated pension administration and payment system designed to make efficiencies and help meet the additional requirements of the Public Sector Pensions Act 2013. This Act introduced career average revalued earnings (CARE) pension schemes and new governance arrangements for public pension schemes.  
8.	The auditor of SPPA found no evidence of a clear business case setting out why SPPA needed an integrated system, nor evidence that it had fully considered other options. Officers prepared a paper for the senior management team (SMT) to consider in September 2014, outlining three options for providing pension administration and payment services:
•	integrated pension administration, pensions payroll and member web services
•	combined pension administration and pensions payroll but separate member web services solution
•	separate systems for pension administration, pensions payroll and member web services.
9.	The SMT paper set out the advantages and disadvantages of each option but did not include any analysis of costs and benefits or any financial information. The SMT approved the recommendation to procure a single integrated system. The paper was not presented to SPPA’s MAB for consideration. The MAB should have been given the opportunity to advise the Accountable Officer as part of the decision-making process. The paper was presented to the project board, which sat below the MAB in SPPA’s governance structure, in October 2014. The board suggested some minor wording changes to the paper.
SPPA set an unrealistic 18-month timescale
10.	The Scottish Government procurement directorate advised SPPA to undertake a procurement exercise for the new system, which was to be operational by March 2017. SPPA was advised that it could not extend contracts with existing suppliers due to the amount of time since it last undertook a procurement exercise. SPPA started the procurement process in 2013 supported by the Scottish Government procurement directorate and led by members of SPPA’s senior leadership team. SPPA agreed a budget in the region of £9 million for the project but did not set any detailed scope or objectives. This was a significant omission and contributed to the overall failure of the project. SPPA did not have adequate resources to progress the procurement exercise. This led to it issuing the invitation to tender later than planned, in November 2014.
11.	SPPA extended contracts with some existing suppliers of pensions systems which were due to expire before the new system was implemented. It extended the contracts for its payroll and customer engagement systems to the end of 2017. It did not extend the contract for its pension administration system, which was due to expire in March 2017. SPPA judged that extending the pension administration system did not represent value for money in the expectation that the new system would be operational before this contract expired. It was therefore left with 18 months to develop and implement a new integrated system, based on the contract being awarded in November 2015. This was a high-risk approach with an unrealistic timetable.
SPPA did not adequately scrutinise Capita’s tender prior to awarding it the contract, in spite of it being identified as an abnormally low cost submission
12.	SPPA issued the invitation to tender (ITT) for the new system in November 2014. This set out its requirement for an integrated software solution to support its key objective of transforming its customers’ experience. It stated that the solution must have the necessary functionality to cover the entire pension life cycle, which includes pensions administration and payroll and digital services for scheme members, deferred members and pensioners. The selection panels were made up of SPPA staff from across all grades.
13.	Technical and commercial evaluations were undertaken by different panels with training provided to panel members. The bids were evaluated and ranked using a ratio of 70 per cent quality and 30 per cent costs. Five suppliers submitted bids to SPPA and these were assessed using a three-step process: 
•	the tender was checked for compliance against the ITT, all were found to be compliant
•	the quality requirements in the statement of responsibilities were evaluated and the commercial evaluation was undertaken
•	a presentation was completed by each compliant bidder. 
14.	Capita scored second for quality, first for cost, and first overall. The quality scores were relatively similar for three of the firms who tendered for the contract, but Capita’s bid was significantly cheaper. Under the Public Contracts (Scotland) Regulations 2012 this was classed as an abnormally low cost tender, which casts doubt on whether the contract can be fully delivered.
15.	As SPPA classed Capita’s bid as abnormally low, it was required to investigate whether it was a valid bid. SPPA queried elements of the bid with Capita and discussed Capita’s responses with the Scottish Government Legal Department, who advised that more in-depth questions should be asked to fully assess the bid. SPPA informed the Scottish Government Legal Department that it did not have the skills to further probe the tender. SPPA took no further action and accepted the bid and reassurances from Capita over its ability to deliver. SPPA did not seek further support from the Scottish Government or other external specialists.
16.	One of the unsuccessful suppliers contacted SPPA during the standstill period, before formal award of the contract, to highlight concerns with the decision to award the £5.6 million contract to Capita. The firm issued a legal challenge to SPPA’s decision, but this was subsequently withdrawn. The contract was awarded to Capita in October 2015.
17.	In November 2014, when the ITT was published, the Scottish Government Programme and Project Management Centre of Expertise completed a high-level technical assurance review of the programme of projects, including PS Pensions, under way in SPPA. Reviews are rated using a traffic light system:
•	Green – Successful delivery of the project/programme to time, cost and quality appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly.
•	Amber/Green – Successful delivery appears probable however constant attention will be needed to ensure risks do not materialise into major issues threatening delivery.
•	Amber – Successful delivery appears feasible but significant issues already exist requiring management attention. These appear resolvable at this stage and if addressed promptly, should not present a cost/schedule overrun.
•	Amber/Red – Successful delivery of the project/programme is in doubt with major risks or issues apparent in a number of key areas. Urgent action is needed to ensure these are addressed, and whether resolution is feasible.
•	Red – Successful delivery of the project/programme appears to be unachievable. There are major issues on project/programme definition, schedule, budget required, quality or benefits delivery, which at this stage do not appear to be manageable or resolvable. The project/programme may need re-baselining and/or overall viability re-assessed.
18.	The review gave an overall rating of 'amber/green' but highlighted that 'careful attention needs to be paid to the procurement process, which faces a number of challenges around requirement definition, resources and its timeline'. 

Part 2
Project governance and resources

Changes in SPPA’s leadership and management of the project made it more difficult to manage the supplier and hold it to account
19.	SPPA agreed a procurement strategy before the tender process started. A draft project initiation document (PID) was prepared in September 2015. This outlined the project approach, deliverables, governance arrangements, roles and responsibilities and milestones. The SPPA project board agreed the PID in January 2016, after the project had already been awarded to Capita. SPPA revised the PID to reflect updated roles and responsibilities and governance arrangements and it was finalised in June 2016.
20.	Governance arrangements included a project board, senior responsible officer and project manager. The people in these key roles changed over the course of the project (Exhibit 1).
21.	The changes in Chief Executive and Senior Responsible Officer had a major impact on the management of the project. The lack of continuity made it more difficult for SPPA to adequately scrutinise and challenge Capita at key stages of the project and affected decision-making.
SPPA’s governance and project assurance arrangements did not support effective scrutiny of the project
22.	SPPA set up a project board to review progress and assess and manage risks and issues. The project board was chaired by the Chief Executive and consisted of senior representatives from SPPA and Capita, and non-executive members as observers. From a review of project board papers, there was a lack of finance information taken to the project board. We note that members of the project board requested more budget and cost information. Board minutes stated that the members also wanted clearer reporting on the progress of the project, which was not addressed during the project. Timely and accurate reporting of financial information is a key component of good project management and governance. 
23.	SPPA asked its internal auditors from the Scottish Government Internal Audit Directorate to highlight areas of the PS Pensions project that it would be able to review and provide assurances on. Internal audit identified, in a progress update paper to the ARC in January 2016, that it could:
•	review the management and delivery of the contract
•	review the procurement contract
•	provide assurance to SPPA over the process.
SPPA decided not to direct internal audit to undertake this work. Internal audit did not undertake any work on the project until September 2017 (paragraph 37).
24.	Officers provided updates on the project to the MAB, but the MAB was not provided with enough information to enable it to scrutinise progress of this key project and did not receive any financial information on the project. SPPA increased the level of reporting to the ARC and MAB from early 2017 and scheduled extraordinary meetings of the ARC and MAB to help members better scrutinise key decisions and support the Accountable Officer.
SPPA did not have the resources to manage a project of this scale
25.	During the project development phase, SPPA was working alongside Capita. This involved Capita developing the product and SPPA undertaking tasks such as user acceptance testing for the project. 
26.	SPPA primarily resourced the project from its existing staff with a small number of external contractors used. This meant that SPPA had to balance its business as usual work with supporting the development of the new system.

Part 3
Contract delivery and external reviews

Capita was not able to provide a working system and did not achieve any of the project milestones
27.	In February 2016, Capita began delivery of the project with the first milestone due in June 2016. In May 2016, Capita advised SPPA that it was not going to achieve this deadline. In June 2016, a major tranche of awards calculations failed user testing. SPPA identified both issues with the design of the system and deficiencies in the communication it had received from Capita. Several changes in the project managers employed by Capita contributed to these issues. SPPA highlighted its concerns to Capita and provided Capita with dedicated facilities in SPPA’s offices to base its staff in the hope that this would improve both communication between the two organisations and the quality of project delivery. Capita was unable to resolve the issues with the system and in October 2016 it informed SPPA that it would not achieve the March 2017 implementation date.
28.	In November 2016, SPPA’s Chief Executive requested an Office of the Chief Information Officer (OCIO) external review, to determine if the project could be brought back on track and to identify recommendations to help SPPA monitor the project. The OCIO rated the project as ‘amber’. The report found that there had been a significant failure by Capita. The OCIO also found that the procurement was conducted in a robust and professional manner, consistent with due process. We have not seen evidence to support this conclusion. In our judgement, SPPA did not have enough assurances over Capita’s ability to deliver the project before it awarded it the contract.
29.	SPPA and Capita agreed a revised go live date of October 2017 at a project board meeting in January 2017. Due to this delay to the completion of the project, SPPA needed to negotiate contract extensions with its existing suppliers to maintain business continuity. SPPA prepared an outline business case for the January 2017 ARC and MAB. This identified six options for pensions administration. The ARC and MAB recommended that SPPA extend its contract with its existing pensions administration supplier to ensure continuity of service and continue to work with Capita to develop the new system. SPPA extended the contract for the pensions administration system for three years at a total additional and unplanned cost of £4.0 million. Capita prepared a re-plan for the project in early 2017 and SPPA agreed a twelve-week proving period until mid-July 2017 for Capita to demonstrate its ability to develop the system.
30.	The OCIO carried out a second review of the project in July 2017. Its assessment was ‘amber/red’. The OCIO acknowledged that Capita had improved working practices over the proving period but that further delivery issues were experienced. It recommended that SPPA continue to work with Capita but consider other options for delivering the project.  
31.	SPPA considered the review’s findings at extraordinary ARC and MAB meetings in September 2017, where an options appraisal with four options was considered. The ARC and MAB advised the newly appointed Chief Executive that remaining with Capita was the preferred option. The current Chief Executive had to decide within days of her appointment whether to continue with the contract. She decided to continue to work with Capita. 
32.	SPPA agreed a contract variation with Capita in September 2017. In doing so, SPPA sought to strengthen its ability to recover costs or damages from Capita in the event of further problems with the project. This included increasing the contract retention rate and doubling the maximum penalty for material default by Capita. Capita agreed to provide a minimum viable product (MVP) by October 2018 and the rest of the functionality by August 2019. 
33.	The OCIO completed a third project review in January 2018. Its assessment was ‘red’ and it recommended that the project should be stopped. The review team did not believe that delivery of an MVP could be achieved by October 2018 due to Capita’s failure to meet a further project milestone and ongoing high uncertainty Capita would meet future milestones.
34.	In September 2017, SPPA asked internal audit to review the governance arrangements and key decision-making processes surrounding the PS Pensions project, including the procurement and implementation phases. Internal audit’s key findings included:
•	SPPA’s engagements with external assurance providers to identify lessons learned were not fully documented 
•	there was a lack of documented audit trails relating to key decisions and processes
•	the roles of the implementation project board were not defined until March 2016
•	the project was not managed to the standards of best practise documented in the Prince2 methodology which SPPA initially adopted
•	information reported to the project board to inform decision-making should have been more clearly aligned to best practice
•	Office of Government Commerce best practice was not followed
•	the recommendations of the various reviews of the project were not taken forward in a timely manner.

Part 4
Project closure

SPPA failed to achieve value for money in this project. The project failure means SPPA has been unable to progress its strategic, business and workforce plans as originally intended
35.	SPPA held extraordinary ARC and MAB meetings in February 2018. The Chief Executive informed the ARC and MAB that she proposed to terminate the contract with Capita based on:
•	the January 2018 OCIO report 
•	Capita’s poor performance
•	the failure of Capita to meet the expectations of SPPA.
Following the meetings, SPPA formally closed the PS Pensions project.
36.	Officers advised the ARC that SPPA has spent approximately £6.3 million on the project. This includes £0.8 million paid to Capita. The remaining expenditure covers contract extensions with incumbent suppliers, internal staff costs, software license costs and specialist project support. SPPA has written off £1.6 million in capitalised assets that will no longer be used. SPPA and Capita reached a financial settlement in November 2018 with Capita making a payment of £0.7 million to SPPA.
37.	SPPA has reached an agreement to extend its contracts with existing suppliers to March 2024 to ensure it is able to continue business as usual. SPPA is undertaking a new project to develop an integrated system and has completed an exercise to learn lessons from the closure of the PS Pensions project, including:
•	being clear about what it means by an integrated product
•	ensuring it has the required resources in place
•	communicating with the Scottish Government at an early stage to highlight any areas where additional support is required.
38.	Audit Scotland published a report in May 2017 on Principles for a digital future: Lessons learned from public sector ICT projects. The report outlines the issues that have been identified in previous ICT projects and has combined the themes identified into a set of core principles. Senior management within SPPA made an assessment against the five principles of a digital future as part of the lessons learned exercise and has highlighted areas where it can improve on for the new project. Appendix 2 outlines an assessment of SPPA against the five principles. 
39.	The new project involves improving the current system for administration and payroll, and implementing member and employer web portals to allow customers to access and update their pension information online. SPPA estimates it needs capital allocations of £13.6 million from the Scottish Government over the next five years to achieve this. SPPA also requires additional revenue budget of
£9.8 million from 2019/20 to 2022/23.
40.	SPPA briefed Scottish ministers on the funding shortfall before signing a contract extension with its existing pension administration software supplier. The Cabinet Secretary for Finance, Economy and Fair Work authorised SPPA’s Chief Executive to make the legal commitment.
41.	The failure of the project means that SPPA has not been able to progress its strategic, business and workforce plans as originally intended. This includes its Target Operating Model (TOM) to deliver future services more efficiently. 

Appendix 1
Timeline of project events
The following outlines the timeline of events in relation to the PS Pensions project and the plans for a new project.

Appendix 2
Principles for a digital future
We have assessed SPPA against the five principles for successful ICT projects set out in Audit Scotland’s May 2017 report Principles for a digital future: Lessons learned from public sector ICT projects. 
Principles for a digital future | Assessment of SPPA

Comprehensive planning setting out what you want to achieve and how you will do it

The auditor of SPPA found no evidence of a clear business case setting out why SPPA needed an integrated system, nor evidence that it had fully considered other options (paragraph 8).
SPPA judged that extending the pension administration system did not represent value for money in the expectation that the new system would be operational before this contract expired. It was therefore left with 18 months to develop and implement a new integrated system, based on the contract being awarded in November 2015. This was a high-risk approach with an unrealistic timetable (paragraph 11).

Active governance providing appropriate control and oversight

There was a lack of finance information taken to the project board. We note that members of the project board requested more budget and cost information (paragraph 22).
Officers provided updates on the project to the MAB, but the MAB was not provided with enough information to enable it to scrutinise progress of this key project and did not receive any financial information on the project. SPPA increased the level of reporting to the ARC and MAB from early 2017 and scheduled extraordinary meetings of the ARC and MAB to help members better scrutinise key decisions and support the Accountable Officer (paragraph 24).

Putting users at the heart of the project

During the project development phase SPPA was working alongside Capita. This involved Capita developing the product and SPPA undertaking tasks such as user acceptance testing for the project (paragraph 25).
The Chief Executive informed the ARC and MAB that she proposed to terminate the contract with Capita based on:
• the January 2018 OCIO report 
• Capita's poor performance
• the failure of Capita to meet the expectations of SPPA (paragraph 35).

Clear leadership that sets the tone and culture and provides accountability | The changes in Chief Executive and Senior Responsible Officer had a major impact on the management of the project. The lack of continuity made it more difficult for SPPA to adequately scrutinise and challenge Capita at key stages of the project and affected decision-making (paragraph 21).

Individual projects set in a central framework of strategic oversight and assurance

SPPA informed the Scottish Government Legal Department that it did not have the skills to further probe the tender. SPPA took no further action and accepted the bid and reassurances from Capita over its ability to deliver. SPPA did not seek further support from the Scottish Government or other external specialists 
(paragraph 15).
The OCIO completed a third project review in January 2018. Its assessment was 'red' and it recommended that the project should be stopped. The review team did not believe that delivery of an MVP could be achieved by October 2018 due to Capita's failure to meet a further project milestone and ongoing high uncertainty Capita would meet future milestones
(paragraph 33).
In September 2017, SPPA asked internal audit to review the governance arrangements and key decision-making processes surrounding the PS Pensions project, including the procurement and implementation phases (paragraph 34).