Heads up: Azure Services outage
Incident Report for BankTech
Resolved
This incident has been resolved.
Posted Mar 14, 2024 - 21:18 CAT
Update
To our valued clients,

Please note the below status from Microsoft, this message was last updated at 14:23 UTC on 14 March 2024

Current Workstreams: We are exploring the following mitigation workstreams

Working with our local network partner to fix defective links.
Exploring traffic redirection and optimization on our network


We estimate that these workstreams will take up to 12 hours to complete.

Customers who are experiencing severe disruption to their operations may want to consider their BCDR options.

The next update will be provided in 4 hours or as events warrant.
Posted Mar 14, 2024 - 17:20 CAT
Update
To our valued clients,

Please note the below status from Microsoft.

Network latencies - South Africa North and South Africa West.

Impact Statement: Starting at 10:30 UTC on 14 Mar 2024, customers using Azure Services in South Africa North and South Africa West may experience increased network latency or packet drops when accessing their resources.

Current Status: We have determined that multiple fibre cables on the West Coast of Africa — WACS, MainOne, SAT3, ACE — have been impacted which reduced the total capacity supporting our Regions in South Africa.

In addition to these cable impacts, the ongoing cable cuts in the Red Sea — EIG, Seacom, AAE-1 — are also impacting capacity on the East Coast of Africa.

This combination of incidents has impacted all Africa capacity – including other Cloud providers and public Internet as well.

Current Workstreams: We are exploring the following mitigation workstreams

Working with our local network partner to fix defective links.
Exploring traffic redirection and optimization on our network
The next update will be provided in 60 minutes, or as events warrant

This message was last updated at 13:22 UTC on 14 March 2024
Posted Mar 14, 2024 - 16:15 CAT
Update
To our valued clients,

Further to our communication, please note that the services have been restored and our team is running BCM restoration procedures to ensure your data integrity. Kindly inform your account manager of any issues experienced.

We appreciate your patience during this time.

Kind regards
BankTech Incident Response Team
Posted Mar 14, 2024 - 16:11 CAT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 14, 2024 - 16:10 CAT
Identified
To our valued clients,

Further to our major outage notification, please note that the Infrastructure Support Engineering Team has identified the issue and confirmed it with the infrastructure provider.

Once connectivity has been restored, we will ensure no data loss and communicate once the services have been restored.
Posted Mar 14, 2024 - 14:47 CAT
Update
We are continuing to investigate this issue.
Posted Mar 14, 2024 - 13:23 CAT
Investigating
To our valued clients,


Please be advised that Microsoft Azure infrastructure services are currently experiencing a major outage at 12:50 pm 2024/03/14.

Due to Banktech's use of these services, this will affect our platform.

We are awaiting further information regarding the outage and will provide updates as they are received.

We appreciate your patience during this time.

Kind regards
BankTech team
Posted Mar 14, 2024 - 13:23 CAT
This incident affected: BankTech Platform (BankTech API, BankTech Web App), Bank Integrations (Standard Bank Host - Submission Service, Standard Bank Host - Results Service, Standard Bank API - DebiCheck Authentication Service, ABSA Host - Submission Service, ABSA Host - Result Service, ABSA CAPI - DebiCheck Authentication Service, Nedbank Host - Submission Service, Nedbank Host - Result Service, Nedbank API - DebiCheck Authentication Service), BankTech Platform (non-production) (BankTech Web App Sandbox, BankTech API Sandbox), and Data Services (Bank Account Verification, RSA ID Verification).