[US-Midwest-2] Service Interruption Affecting a Subset of Cloudhost Services
Incident Report for Server Status | Nexcess
Update
On Tuesday, June 28, 2022, between 9:00 PM - 12:00 AM ET, our Network Engineers will be performing emergency maintenance in US-MIDWEST-2 (DC3) to improve recent issues with stability. We anticipate a brief disruption of services throughout the maintenance window as rolling reboots are initiated on networking devices.

For questions, please contact the support team via support@nexcess.net, client portal ticket, live chat, or toll-free at 866-639-2377 or +1.517.322.0434 (international).
Posted Jun 28, 2022 - 12:17 EDT
Update
The work of our Engineering team continues to make progress towards implementing a durable solution to the identified sources of the internal communications issues within our US-MIDWEST-2 OpenStack environment in collaboration with engineers from our networking equipment vendor. This remains our team's highest priority and there have been no further disruptions related to this event since June 21st.

We are continuing to monitor performance closely and will post additional updates as soon as new information becomes available.
Posted Jun 27, 2022 - 15:08 EDT
Update
Our Engineering team continues to make progress in implementing a durable solution to the identified sources of the internal communications issues within our US-MIDWEST-2 OpenStack environment in collaboration with engineers from our networking equipment vendor. This remains our team's highest priority and there have been no further disruptions related to this event since June 21st.

We are continuing to monitor performance closely and will post additional updates as soon as new information becomes available.
Posted Jun 25, 2022 - 20:58 EDT
Update
Our Engineering team is still working on the identified sources of the internal communications issues within our US-MIDWEST-2 OpenStack environment in collaboration with engineers from our networking equipment vendor. This remains our team's highest priority and there have been no further disruptions related to this event since June 21st.

We are continuing to monitor performance closely and will post additional updates as soon as new information becomes available.
Posted Jun 24, 2022 - 21:05 EDT
Update
Our Engineering team has continued to work through the identified the sources of the internal communications issues within our US-MIDWEST-2 OpenStack environment in collaboration with engineers from our networking equipment vendor. This is our team's highest priority and there have been no further disruptions related to this event since June 21st.

We are continuing to monitor performance closely and will post additional updates as soon as new information becomes available.
Posted Jun 23, 2022 - 19:04 EDT
Update
Our Engineering team has identified the sources of the internal communications issues within our US-MIDWEST-2 OpenStack environment and continues to work through them systematically to eliminate potential problems in the future.

Between 9:00-10:00 PM ET on June 21, 2022, our engineers performed emergency maintenance to correct one of the outstanding issues and contributing factors impacting the environment. There was no downtime associated with the maintenance.

We anticipate no further unscheduled downtime. We will provide additional information and full details once the issue is fully resolved. In the meantime, we appreciate your continued patience while we work as quickly as possible to resolve the issue.
Posted Jun 22, 2022 - 07:35 EDT
Update
All services have been restored at this time and additional investigation is underway.
Posted Jun 21, 2022 - 14:58 EDT
Identified
Networking confirms that we have experienced a disruption in US-MIDWEST-2 and the team is actively working to bring all services back online at this time.
Posted Jun 21, 2022 - 14:46 EDT
Update
Our Engineering team continues to work methodically through the DC to ensure all contributing factors have been resolved. There have been no further disruptions related to this event since June 15, 2022. However, we continue to monitor performance closely and will post additional updates as new information becomes available.
Posted Jun 21, 2022 - 09:07 EDT
Update
We continue to monitor performance closely as our Engineering team progresses towards full resolution. Additional updates will be provided as new information becomes available.
Posted Jun 20, 2022 - 09:26 EDT
Monitoring
Active monitoring is still in place and our Engineering teams continue to make progress towards a full and durable resolution. Services remain stable at this time. Additional updates will be provided as new information becomes available.
Posted Jun 19, 2022 - 04:18 EDT
Update
Services have been stable again throughout the day on Friday June 17th. Engineering teams continued to make progress in identifying the root cause of the issue. Active monitoring remains in place as well. We appreciate your patience as our teams work to ensure this issue is fully resolved. We will provide additional updates as new information becomes available.
Posted Jun 17, 2022 - 19:19 EDT
Update
Performance remained stable throughout the night. Our Engineering team continues to monitor closely as they methodically move the investigation forward to ensure this issue is fully resolved.

Additional updates will be provided as new information becomes available.
Posted Jun 17, 2022 - 09:11 EDT
Update
Services have been stable throughout the day and Engineering teams have made progress isolating the issue. Our teams are still working to ensure this issue is fully resolved and active monitoring is still in place. We will provide additional updates as new information becomes available.
Posted Jun 16, 2022 - 19:13 EDT
Update
We are continuing to investigate this issue.
Posted Jun 16, 2022 - 19:09 EDT
Update
Performance has been stable throughout the night and our engineers continue to work as quickly as possible to isolate and fully resolve the root cause of the issue. Additional updates will be provided as new information becomes available.
Posted Jun 16, 2022 - 10:06 EDT
Update
Our Engineering team is continuing with their investigation of this issue. Progress has been made with improving stability, but we are still monitoring the situation, in order to completely mitigate this interruption of service.

If you have any questions, please contact a member of our support team via ticket or by phone at +1-866-639-2377 (US) or +1-517-322-0434 (international).
Posted Jun 16, 2022 - 05:13 EDT
Update
We are continuing to investigate this issue but have seen better stability as we've worked on possible causes. Our engineering team is still currently investigating the root cause behind the issue to fully stabilize the environment. This investigation continues to be high priority and our teams are actively monitoring and available to act quickly, if needed. We will update this status page again as further information becomes available.

If you have any questions, please contact a member of our support team via ticket or by phone at +1-866-639-2377 (US) or +1-517-322-0434 (international).
Posted Jun 15, 2022 - 19:52 EDT
Investigating
Our Engineers are still working to resolve an interruption in service affecting a subset of cloudhost services in our us-midwest-2 data center. We'll provide further updates as soon as they become available. This is a continuation of https://status.nexcess.net/incidents/7nfrbnv9vjx1 which was inadvertently resolved.

If you have any questions, please contact a member of our support team via ticket or by phone at +1-866-639-2377 (US) or +1-517-322-0434 (international).
Posted Jun 15, 2022 - 14:38 EDT
This incident affects: Nexcess Cloud (us-midwest-2).