Text-to-Speech API Degradation
Incident Report for Nexmo
Postmortem

What happened

At 10:38 UTC 6th November 2018, an elevated number of calls failing with Error 0 was identified for calls originating in our North American Data Center. All traffic was switched to the South East Asia Data Center at 11:57. The issue was resolved and normal serviced returned at 11:58 UTC.

Causes

An issue with the underlying server processing the TTS API requests caused the server to be unable to resolve DNS records and route the calls correctly.

Preventive Actions

A program of work has been identified that includes:

  • Enhancing monitoring and reports
  • Improvements in the underlying infrastructure
Posted 2 months ago. Nov 21, 2018 - 09:46 UTC

Resolved
The Text-to-Speech API Degradation problem has been resolved.

The following services were impacted until 06-Nov-2018 11:59 UTC.

Voice API and Verify SMS have not been impacted.

All services have been restored.
Please let support@nexmo.com know if you continue to experience problems with this.
A post-mortem will be published in the next few days.
Posted 3 months ago. Nov 06, 2018 - 15:46 UTC
Monitoring
We have implemented a resolution for the TTS API degradation.

We will continue to monitor the service during the next few hours and post updates should anything change.
Posted 3 months ago. Nov 06, 2018 - 12:03 UTC
Identified
Customers may have experienced problems impacting Text-to-Speech API. The issue has been mitigated and calls should now be operating as normal. You may have sees error 0 for some failed calls.

The following services have been impacted:
TTS API
Verify API

Voice API and SMS sent as part of the Verify API flow have not been impacted.

We will update this status as soon as we have more information on this issue.
Posted 3 months ago. Nov 06, 2018 - 11:31 UTC
This incident affected: Voice APIs (See https://goo.gl/2WoAtx for Current known problems) (TTS API) and Verify API and Verify SDK.