tag:status.cloudfloordns.com,2005:/historyCloudfloorDNS Status - Incident History2024-03-28T12:51:41ZCloudfloorDNStag:status.cloudfloordns.com,2005:Incident/200001542024-02-16T13:30:00Z2024-02-16T14:03:53ZUnicast DNS Attack<p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>13:30</var> UTC</small><br><strong>Resolved</strong> - Our Unicast DNS platform was targeted by a short DDoS attack that degraded the performance across the network. It has subsided now and everything is operating normally again. THis took place between 13:20 UTC and 13:42 UTC.</p>tag:status.cloudfloordns.com,2005:Incident/110646282022-09-23T10:00:24Z2022-09-23T10:00:24ZLAX - Upstream Network Maintenance<p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>14:06</var> UTC</small><br><strong>Scheduled</strong> - One of our IP transit upstream providers in Los Angeles will be performing planned network maintenance. During this time, anycast traffic will be diverted to other POPs and customers may observe changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/110646192022-09-20T10:00:29Z2022-09-20T10:00:29ZMIA - Upstream Network Maintenance<p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>14:05</var> UTC</small><br><strong>Scheduled</strong> - One of our IP transit upstream providers in Miami will be performing planned network maintenance. During this time, anycast traffic will be diverted to other POPs and customers may observe changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/110642972022-09-16T10:00:16Z2022-09-16T10:00:16ZCHI - Upstream Network Maintenance<p><small>Sep <var data-var='date'>16</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>16</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>13:21</var> UTC</small><br><strong>Scheduled</strong> - One of our IP transit upstream providers in Chicago will be performing planned network maintenance. During this time, anycast traffic will be diverted to other POPs and customers may observe changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/110797262022-09-08T07:00:00Z2022-09-08T19:41:01ZDNS DDoS Attack<p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>Resolved</strong> - CloudfloorDNS was hit with a large DDoS attack against a domain name on the platform starting at approximately 07:00 UTC. Once this was identified and confirmed we worked to limit the attack surface and mitigate the traffic against this domain. DNS resolution was delayed and may not have been available for short periods of time in some regions due to BGP route changes during the mitigation process. At approximately 09:12 UTC DNS resolution returned to normal response times.</p>tag:status.cloudfloordns.com,2005:Incident/110106632022-09-01T11:00:32Z2022-09-01T11:00:32ZDFW - Upstream Network Maintenance<p><small>Sep <var data-var='date'> 1</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'> 1</var>, <var data-var='time'>07:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>29</var>, <var data-var='time'>14:11</var> UTC</small><br><strong>Scheduled</strong> - One of our IP transit upstream providers in Dallas will be performing planned network maintenance. During this time, some DNS traffic will be diverted to other POPs and customers may observe changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/105516952022-08-02T08:00:30Z2022-08-02T08:00:30ZMIA - Upstream Network Maintenance<p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>04:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>12</var>, <var data-var='time'>13:43</var> UTC</small><br><strong>Scheduled</strong> - One of our IP transit upstream providers in Miami will be performing planned network maintenance. During this time, some anycast traffic will be diverted to other POPs and customers may observe changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/106758192022-07-23T14:42:30Z2022-07-23T14:42:30ZDNS Attack<p><small>Jul <var data-var='date'>23</var>, <var data-var='time'>14:42</var> UTC</small><br><strong>Resolved</strong> - Between 14:03 and 14:30 UTC a massive dictionary scan came in against our Anycast DNS Network. These scans are particularly bad because they are querying hosts within a zone that don't exist at extremely high query rates. This end up exhausting resources and response speed is delayed. In some cases traffic was diverted from some locations to other locations.<br /><br />We are continuing to monitor the situation.</p><p><small>Jul <var data-var='date'>23</var>, <var data-var='time'>14:29</var> UTC</small><br><strong>Investigating</strong> - We are currently under a massive dictionary scan attack which is overloading our network. We are working to mitigate this as quickly as possible.</p>tag:status.cloudfloordns.com,2005:Incident/102039772022-07-13T11:00:30Z2022-07-13T11:00:30ZLAX - Upstream Maintenance<p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:30</var> UTC</small><br><strong>Scheduled</strong> - One of our upstream carriers will be performing scheduled network maintenance in Los Angeles. During this time, customers may see traffic re-routed, and some anycast traffic diverted to other POPs.</p>tag:status.cloudfloordns.com,2005:Incident/102039482022-06-29T07:00:32Z2022-06-29T07:00:32ZSEA - Network Maintenance<p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>06:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:29</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102039322022-06-28T19:00:35Z2022-06-28T19:00:35ZHKG - Network Maintenance<p><small>Jun <var data-var='date'>28</var>, <var data-var='time'>19:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>28</var>, <var data-var='time'>18:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:28</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102039192022-06-25T00:00:34Z2022-06-25T00:00:34ZFRA - Network Maintenance<p><small>Jun <var data-var='date'>25</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>24</var>, <var data-var='time'>23:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:26</var> UTC</small><br><strong>Scheduled</strong> - Partners engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102039152022-06-24T00:00:30Z2022-06-24T00:00:30ZCDG - Network Maintenance<p><small>Jun <var data-var='date'>24</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>23</var>, <var data-var='time'>23:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:25</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102038852022-06-22T16:00:34Z2022-06-22T16:00:34ZSYD - Network Maintenance<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>16:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>15:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:25</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102038732022-06-21T00:00:30Z2022-06-21T00:00:30ZLHR - Network Maintenance<p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>23:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:24</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102038692022-06-20T07:00:36Z2022-06-20T07:00:36ZLAX - Network Maintenance<p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>06:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:23</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance in our LAX POP to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/102038562022-06-16T22:00:38Z2022-06-16T22:00:38ZAMS - Network Maintenance<p><small>Jun <var data-var='date'>16</var>, <var data-var='time'>22:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>16</var>, <var data-var='time'>21:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>16:21</var> UTC</small><br><strong>Scheduled</strong> - Partner engineers will be performing a network maintenance in our AMS POP to improve network resiliency. Customers may notice increased latency and routing path changes during the maintenance window.</p>tag:status.cloudfloordns.com,2005:Incident/96057372022-03-22T21:49:28Z2022-03-22T21:53:31ZNetmon - Additional agent IP Addresses added<p><small>Mar <var data-var='date'>22</var>, <var data-var='time'>21:49</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>22</var>, <var data-var='time'>21:49</var> UTC</small><br><strong>Scheduled</strong> - Thursday, March 17th we added some IP Addresses to our Netmon monitoring platform.<br />ATL - Atlanta - 139.177.203.44 and 45.79.215.100<br />LAX - Los Angels - 45.56.94.254<br /><br />Please ensure to update any ACL's with the additional IP Addresses if required.</p>tag:status.cloudfloordns.com,2005:Incident/89119122021-12-22T18:18:22Z2021-12-22T18:18:29ZDNS Replication Delays<p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>18:18</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.<br />One of our DNS replication masters in AWS had gone offline and needed to be restored. Once restored, replication to CDG (Paris) and LHR (London) was also restored and the Anycast DNS network was brought back online at those locations.</p><p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>18:14</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>13:10</var> UTC</small><br><strong>Update</strong> - CDG (Paris) and LHR (London) have been removed from the Anycast DNS network until the database replication issue is resolved.</p><p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>12:59</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating an issue with DNS replication delays in 2 Data Centers, CDG (Paris) and LHR (London). As we find out more information we will send updates.</p>tag:status.cloudfloordns.com,2005:Incident/76645322021-08-02T16:45:32Z2021-08-02T16:45:32ZWeb Forwarding Server offline<p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>16:45</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.<br />One of our hosting providers had an issue on their network where the web forwarding server is hosted and is now fully operational.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>15:24</var> UTC</small><br><strong>Monitoring</strong> - Web forwarding is now operational. We are waiting to hear from the hosting provider as to what happened.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>14:57</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating an issue with our web forwarding server which has become non-responsive. As we know more information will update this status.</p>tag:status.cloudfloordns.com,2005:Incident/73144972021-07-21T06:42:29Z2021-07-21T06:42:29ZCHI - Power Maintenance<p><small>Jul <var data-var='date'>21</var>, <var data-var='time'>06:42</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>16:42</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'>23</var>, <var data-var='time'>16:45</var> UTC</small><br><strong>Scheduled</strong> - Reason for Work: UPS3A cutover to new UPS<br /><br />Risk Mitigation: Best practices and procedures will be followed during this event<br /><br />Description: Data center operations will be transferring MSB-1 and 2 to generator. They will then transfer UPS 3A and 3B to static bypass. Then all RPPs will be cutover to the new UPS 1A. Once everything has been cut over the UPS will be restored to normal and MSB1 and 2 will be transferred back to utility.</p>tag:status.cloudfloordns.com,2005:Incident/74935042021-07-13T15:47:32Z2021-07-13T15:47:32ZManagement Panel Access<p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>15:47</var> UTC</small><br><strong>Resolved</strong> - We've been informed by Amazon that this issue has been resolved at the AWS EU-CENTRAL-1 location. <br />All services are now operational.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>14:13</var> UTC</small><br><strong>Monitoring</strong> - The issue has been isolated to our AWS EU-CENTRAL-1 location. AWS is aware of the problem and is actively restoring services in that region. We are continuing to monitor service status and will post updates as they are available.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>13:20</var> UTC</small><br><strong>Identified</strong> - An access issue with our management panel has been identified. There appears to be route break between our Web Application Firewall and the origin server. Until the issue is resolved, we have bypassed this location for management panel access.</p>tag:status.cloudfloordns.com,2005:Incident/73055832021-07-07T07:00:35Z2021-07-07T07:00:35ZIAD - Upstream transit maintenance<p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>04:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>15:24</var> UTC</small><br><strong>Scheduled</strong> - One of our primary upstream IP transit providers will be conducting network maintenance for a software upgrade. During this time, some traffic will be diverted away from our Ashburn (IAD) POP and customers may see brief disruptions due to changes in routing.</p>tag:status.cloudfloordns.com,2005:Incident/71662382021-06-10T04:00:32Z2021-06-10T04:00:32ZMiami - Upstream network maintenance<p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>04:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>23:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 4</var>, <var data-var='time'>12:55</var> UTC</small><br><strong>Scheduled</strong> - Affected Infrastructure<br />Components: BGP Anycast, Transit & Transport<br />Locations: MIA - Miami, FL<br /><br />One of our local transport providers in Miami will be conducting maintenance on a fiber path. During this window, we will be removing traffic from one of our upstream transit providers, which will result in some Anycast traffic being diverted to other POPs. Customers may see some routes shift but no further impact is expected.</p>tag:status.cloudfloordns.com,2005:Incident/69382742021-05-11T04:01:08Z2021-05-11T04:01:08ZCDG - Upstream network maintenance<p><small>May <var data-var='date'>11</var>, <var data-var='time'>04:01</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'>11</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 6</var>, <var data-var='time'>19:45</var> UTC</small><br><strong>Scheduled</strong> - One of our upstream fiber providers in Paris will be conducting maintenance to perform a software upgrade. During this time, traffic will be rerouted on alternate paths. Customers may notice increased latency and some Anycast traffic may be diverted to alternate POPs.</p>