Payment Services Degradation
Incident Report for Form3
Postmortem

We experienced a brief degradation of our payment services between 20:47 and 20:53 BST that resulted in a small number of submission failures which can be identified by a status reason with ‘Error during payment delivery’.
This was due to timeouts during the rolling upgrade of a back-end service taking longer than our payment retry period. We are investigating the cause of the delay, removing the dependency on this service, and increasing the retry period to prevent future issues from occurring.

Posted Apr 03, 2019 - 12:43 UTC

Resolved
After a period of monitoring we have not seen any further service degradation. This is now resolved.
Posted Apr 02, 2019 - 21:28 UTC
Monitoring
A routine deployment affected a small number of submissions between 20:47 - 20:53 BST. The issue has been resolved and service has resumed as normal. We are monitoring the situation closely. Thank you.
Posted Apr 02, 2019 - 20:17 UTC
This incident affected: _Archived Components (FPS RT - Indirect - Starling, FPS RT - Direct, Account Management Services, Bacs - Direct Participant).