Service Issue in Platform- Voice call issues in WDC
Incident Report for Vonage API
Postmortem

Date/Time Impacted

Start Time: 04-Oct-2021 18:17 UTC
End Time: 04-Oct-2021 22:29 UTC

Services Impacted

All voice services

Summary

Some customer API requests may have experienced issues when reaching out to our US Data Centers. This issue impacted across all API voice services on Vonage platform.

Root Cause

An influx in traffic caused by the recent outage at the Facebook/Whatsapp platform caused excessive load in one of our US Data Centers. This resulted in a relatively small volume of customer API requests encountering service failures.

Restoration Action

The load balancer stabilised independently.

Next Steps

Enhanced monitoring set up for load balancers.
Improve the performance of load balancers handling higher volume traffic.
Review and document the failover strategy for our services in US Data Center.

Posted Oct 20, 2021 - 16:24 UTC

Resolved
The service issue affecting voice traffic has been resolved.

The following services were impacted from 04-Oct-2021 18:17 UTC until 04-Oct-2021 20:35 UTC.
All types of voice traffic.

SMS APIs have not been impacted.

All services have been restored.

Please let support@nexmo.com know if you continue to experience problems with this.
A post-mortem will be published in the next few days.
Posted Oct 04, 2021 - 22:11 UTC
Monitoring
We have implemented a fix for the service issue affecting the voice platform.

The list of services that were impacted have been confirmed as:
All Voice Services

We will continue to monitor the service during the next few hours and post updates should anything change.
Posted Oct 04, 2021 - 20:49 UTC
Update
We are continuing to investigate this issue.
Posted Oct 04, 2021 - 19:42 UTC
Investigating
Our monitoring has alerted us to a potential service issue with our platform. Although It is not yet certain whether there is any customer impact, we are alerting customers immediately while our engineering teams investigate.

If there is a customer impact, it is likely to be affecting voice calls via our platform.

We aim to provide an update on our investigation within 1 hour.
Posted Oct 04, 2021 - 18:48 UTC
This incident affected: Voice APIs (See https://goo.gl/2WoAtx for Current known problems) (TTS API, SIP, Voice API).