Degradation to Service

Incident Report for Form3

Postmortem

An internal service experienced a degradation due to a resource constraint from 08:58 to 12:15 BST.

A small number of FPS direct payment submissions were delayed during this time, but were processed successfully once the degradation was resolved. In addition, a small number of FPS direct reversals were received and processed during this period

We have increased the resource capacity of the impacted service to ensure this issue does not recur and are exploring additional monitoring of resourcing limits to mitigate further degradation.

Posted Oct 16, 2019 - 09:22 UTC

Resolved

Following extensive monitoring and investigation we have confirmed the service has recovered and processing as normal since 12:15 BST. 

This degradation was a result of resource constraints on the affected service. We have not seen any further issues, but will increase allocated resources and continue to monitor. We have completed processing of the small number of payments that were affected.

Posted Oct 15, 2019 - 12:59 UTC

Monitoring

We have now addressed the service degradation, and normal service has resumed.

We saw a brief period of increased error rates from an internal service and have now restarted this service and do not expect any further disruption. We will continue to monitor closely.

Posted Oct 15, 2019 - 11:45 UTC

Investigating

We are currently investigating a degradation to our service which is causing a small number of reversals on our FPS Direct service. No current impact has been seen to payments on our other services, however we continue to investigate.

Our engineers are working to return to normal service as soon as possible and we will provide an update as the investigation continues.

Posted Oct 15, 2019 - 11:04 UTC

This incident affected: _Archived Components (FPS RT - Indirect - Starling, FPS RT - Direct, Account Management Services, Bacs - Direct Participant).