Voice API and Inbound SIP Calls - US Data Center - Degradation
Incident Report for Nexmo
Postmortem

What happened

On 6th February 2019 at 13:03:59 UTC, inbound and outbound Voice API calls in the Washington DC data center began to fail and call control was not possible. The issue was quickly identified, and service restored at 13:12 UTC. Customers experienced inbound calls being terminated without answering, and outbound calls failing immediately after being answered.

Causes

Following routine maintenance of the Nexmo Voice platform, an internal DNS issue was identified and manifested across the Washington DC data center. This issue stopped calls being routed correctly and API requests for call control not to be routed correctly. The issue was caught by the monitoring of the platform.

Preventive Actions

A program of work has been identified that includes:

Enhancing monitoring and logging

Enhanced proactive tests to capture issues quickly

Improvements in the underlying infrastructure

Posted about 2 months ago. Mar 04, 2019 - 15:54 UTC

Resolved
An issue with the Voice products in our US data center occurred between 13:05 and 13:13 UTC on 6th February 2019. The issue caused all new outbound Voice API calls to fail, as well as inbound SIP calls. This did not affect outbound SIP or Verify products.

All services have been restored. Please contact support@nexmo.com if you continue to experience problems.

A post-mortem will be published in the next few days.
Posted 2 months ago. Feb 06, 2019 - 13:13 UTC
This incident affected: Voice APIs (See https://goo.gl/2WoAtx for Current known problems) (SIP, Voice API).