Oracle Cloud, Netsuite, and Azure go down, hard, Down Under

Updated Oracle, Netsuite, and Microsoft's clouds person gone down, hard, successful nan Sydney, Australia, region apt owed to an rumor astatine a datacenter supplier successful which some are tenants.

The Big Red Cloud first advised customers of an outage astatine 2129 Sydney clip (1229 UTC) connected Wednesday, and 29 minutes later wrote to pass customers that nan outage had started earlier than its first emailed advisory – astatine 1015 UTC.

Oracle’s 2nd email delivered nan mixed connection that: “We are still investigating an rumor successful nan Australia East (Sydney) region that is impacting aggregate OCI services. We person identified guidelines origin of work failures and are moving to mitigate nan issue.”

The snafu took retired a swathe of compute and SaaS services, pinch Kubernetes offerings disrupted, Exadata unavailable, and cloudy networks borked. Oracle has advised nan effect of nan incident intends “some customers whitethorn beryllium incapable to usage aliases link to definite OCI services successful nan region.”

Oracle’s adjacent emailed proposal again changed nan IT giant's story, this clip referring to a “preliminary guidelines cause” and naming that arsenic related to “network connectivity.”

Microsoft, meanwhile, advised customers of its Azure unreality that arsenic of astir 0830 UTC connected August 30 it was suffering an outage owed to a cooling problem: "A inferior powerfulness surge successful nan Australia East region tripped a subset of nan cooling units offline successful 1 datacenter, wrong 1 of nan Availability Zones."

Temperatures roseate wrong nan datacenter and Microsoft "proactively powered down a mini subset of selected compute and retention standard units, to debar harm to hardware."

Microsoft said it has since restored 99 percent of retention services and 99 percent of impacted Virtual Machines, and is "actively investigating individual downstream services to corroborate their betterment position and mitigate remaining issues."

Downstream limitations are preventing afloat recovery, we're told. Oracle is besides having problem restoring each services.

Oracle-owned cloudy ERP outfit Netsuite's status page states "A lightning large wind impacted nan chiller works successful nan Sydney information center, and astir systems were temporarily unopen down to trim temperatures. The temperatures person stabilized, and nan systems are being systematically powered up."

Netsuite work is slow being restored.

At nan clip of penning Oracle Cloud’s status page listed 10 services arsenic unavailable, six arsenic experiencing work disruptions, and 12 arsenic operational – including immoderate services that were antecedently listed arsenic unavailable.

Oracle Australia told The Register it has nary remark astatine this time.

• Cisco's Duo Security suffers awesome authentication outage
• Microsoft DNS boo-boo breaks Hotmail for users astir nan globe
• UK flights disrupted by 'technical issue' pinch aerial postulation machine system
• Oracle shrinks its on-prem unreality into a azygous rack

The Register understands Oracle's networking squad is astir exercised correct now arsenic Big Red battles to extremity nan downtime, accordant pinch nan small accusation Oracle has provided mentioning customers whitethorn beryllium incapable to scope its unreality and ongoing connectivity issues.

Readers person suggested a lightning large wind that struck Sydney past nighttime could beryllium nan origin of nan instrumentality breakdown. The large wind sparked powerfulness outages successful parts of nan metropolis that big respective datacenters. Your correspondent's day was darkened by nan storms, which passed done astatine astir 1600 Sydney time, a fewer hours earlier cloudy troubles began.

We've asked nan datacenter companies known to person Microsoft and Oracle arsenic tennnts for comment.

Whatever nan guidelines cause, hyperscale clouds and nan datacenters that big them pridefulness themselves connected building capable resilience to header pinch problems. Those efforts intelligibly person not worked successful this case.

We’ll update this communicative if much info arrives. ®

Editor's note: This communicative was updated astatine 0000 UTC and again astatine 0030 UTC to bespeak caller accusation astir nan storm, Microsoft's outage, and knowledge of nan business we person learned from different sources. We updated nan communicative again astatine 01:55 UTC to adhd accusation astir Netsuite.