RSS Feed
Latest Updates
Nov
16
Resolved - Emergency Maintenance for row B4.
Posted by Anthony Kolka on 16 November 2016 09:19 AM

2016-11-16 08:42AM MST B-4 PDU unit failure:

At 08:42 MST the Power Distrobution unit for row B-4 failed. We are working to replace it and while that is happenning we are relocating hosts utilizing that PDU to other equipment in order to restore services as rapidly as possible.

 

2016-11-16 10:10AM: All affected servers have returned to service. We thank you for your patience and apologize for the inconvenience.


Read more »



Oct
19
Network Maintenance - Friday, October 21 9:00PM - 2:00AM
Posted by Jay Sudowski on 19 October 2016 11:08 AM

9:15PM - This maintenance window is closed.  The upgrade to 20Gbps was successful.  No noticeable customer impact was detected.

 

Date: October 21, 2016
Time: 9:00PM - 2:00AM

Description of Work:
We will be upgrading the connectivity between our downtown Denver data center and our Denver Tech Center data center from 10Gbps to 20Gbps.  Additionally, we will be making some modifications to lower the load on the routing engine of our core routers.

Impact of Work:
During this work, brief periods of packet loss and latency should be expected.


Read more »



Oct
14
Resolved - Connectivity issues
Posted by Tyler Molamphy on 14 October 2016 06:51 PM

We are experiencing issues with inbound connectivity and are working diligently to resolve them. We will coninute to post updates here as they are available. 

Update 2016-10-14 19:07: At approximately 18:22 mountain time, network connectivity was disrupted. We believe this is due to a faulty MIC card in one of our core routers. The card was isolated at approximately 18:45. We are currently replacing it with a spare and will be bringing the links back up shortly. 

Update 2016-10-14 19:23: The replacement MIC is working as expected and we are re-establishing connections with the affected providers now. 

Update 2016-10-14 19:34: Links have been restored and all systems appear to be functioning normally. If you're still experiencing issues, please open a ticket with us at support@handynetworks.com 


Read more »



Oct
12
Rolling CDP maintenence
Posted by Anthony Francis on 12 October 2016 11:46 AM

We will be performing rolling CDP maintenance on our backup hosts. THis will start with CDP1 and end with CDP5. We will add an update here when each server is brought down for maintenance and when it is brought back up. During each maintenance the CDP server will be unavailable for performing backups or restores.

Purpose:  Resolve disk safe fragmentation issues that have been leading to very long backup times.

Timeline:

  • CDP1 - offline on 10/12 at 12:00PM MDT
  • CDP1 - Maintenance continues, defragmentation of 13TB of data takes a long time.
  • CDP1 - Finished on 10/18 at 19:11 MDT
  • CDP2 - offline on 10/19 at 21:00PM MDT
  • CDP2 - back online 10/28 at 9:30 AM. Excessive fragmentation caused a very lengthy maintenance time. We apologize for this inconvenience.

At this time we will not be taking other servers down. We are working on another strategy to resolve backup performance issues.


Read more »



Sep
30
Phone systems are down
Posted by Anthony Francis on 30 September 2016 01:32 PM

2016-09-31: This issue is resolved.

2016-09-30: Our telco circuit has been interrupted and the LEC is investigating. We have escalated this issue and at this time we do not have an ETA. Please communicate with us electronically.


Read more »