We identified a misconfiguration in the network settings of our connectivity provider. This led to increased and fluctuating latency when conveying messages to and from RT1. The misconfiguration was only present in production, and as such had not been detected in test environments. The network configuration was changed at 02:41 UTC, and controlled tests in the production environment have shown consistent, low latency, in line with expectations.
We continued to monitor through the day to ensure the service was handling the new configuration without further issue. All tests showed successful processing without latency issues.
We continue to work with our provider to ensure full and complete remediation is in place to prevent this issue from impacting customers in the future.