DNS-related network degradation
Incident Report for Nexmo
Postmortem

What happened

August 6th 2019, from 15:37:39 to 15:55:39 UTC we experienced an issue that impacted HTTP, SIP and SMPP requests to hosts within the nexmo.com domain, including all API end-points (both customer and supplier facing) and our web sites. Customers may have experienced failures when making API requests by hostname to api.nexmo.com or rest.nexmo.com end-points. Some inbound SMS and voice calls would also have failed.

Causes

Our DNS provider experienced an outage during maintenance on their side which caused our public DNS (Domain Name System) resolutions to fail intermittently. The DNS TTL (time to live) was increased to reduce the impact, until the DNS provider outage was fully resolved. DNS resolution services were fully restored when our provider implemented a fix. The DNS TTL was then decreased back to original values to allow for seamless failover in the future.

Preventive Actions

A program of preventive work has been identified including improved High Availability of our public DNS structure via elimination of single points of failure.

Posted 13 days ago. Aug 08, 2019 - 12:44 UTC

Resolved
This incident has been resolved.
Posted 14 days ago. Aug 07, 2019 - 07:12 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted 15 days ago. Aug 06, 2019 - 16:28 UTC
Identified
We are currently investigating an issue that impacted HTTP, SIP and SMPP requests to hosts with the nexmo.com domain, including all API end-points and our web sites both customer and supplier facing.

You may have experienced failures when making API requests by hostname to api.nexmo.com or rest.nexmo.com end-points. Inbound SMS and voice calls would also have failed.

Timeframe: 06-Aug-2019, 15:37:39 - 15:55:39 UTC

We are no longer observe any issues and we are still monitoring.

We are continuing to investigate to fully understand the impact of this issue and will post updates should anything change.
Posted 15 days ago. Aug 06, 2019 - 16:26 UTC
This incident affected: [Early Adopter] Conversation ([Early Adopter] Conversation API), [Early Adopter] In-App IP communications ([Early Adopter] In-App IP Messaging, [Early Adopter] In-App IP Voice, [Early Adopter] In-App IP Video), SMS API (Outbound SMS, Inbound SMS, SMPP, Europe traffic, North America (US, Canada & Caribbean) traffic, South & Central America traffic, Africa traffic, Asia & Oceania traffic), Voice APIs (See https://goo.gl/2WoAtx for Current known problems) (TTS API, SIP, Voice API, Europe Traffic, North America (US, Canada & Caribbean) traffic, South & Central America traffic, Africa traffic, Asia & Oceania traffic), and Number Insight API, Verify API and Verify SDK, Developer API, Nexmo Dashboard, Website, Billing and Payments, Reports API, [Beta] Messages API, [Beta] Dispatch API.