Lucky backup might save 100 days of data for InfluxData's GCP Belgium users

Trending 2 months ago

Updated Luck, alternatively than judgement, has fixed InfluxDB users immoderate dream of restoring 100 days of information aft nan vendor decided to unopen down its Belgium Google Cloud region. Customers depending connected nan database successful Australia are not truthful fortunate.

Earlier this week, InfluxData, nan institution down nan unfastened root clip bid database, apologized for discontinuing 2 unreality regions without sufficiently preparing customers.

Although it had written emails to those affected – successful AWS Sydney and GCP Belgium – users were still caught out. The institution later admitted it should person conducted a "scream test," wherever a work supplier turns disconnected entree to a service. Those who are locked retired "scream" and nan institution tells them nan services will beryllium turned disconnected for bully astatine a later date, giving customers clip to backmost up their information and migrate applications.

InfluxData mislaid nan information of customers utilizing its services successful Australia while users successful Belgium are struggling to fig retired if they tin reconstruct nan past 100 days arsenic a consequence of bully fortune, CTO and co-founder Paul Dix said successful a blog.

He was later quizzed connected Twitter astir why GCP information mightiness beryllium recoverable, but AWS information was not, and whether it was thing to do pinch nan unreality services themselves.

  • InfluxData apologizes for deleting unreality regions without performing 'scream test'
  • MariaDB sent dollar dip informing from NYSE
  • Databricks puts cards connected nan array format arsenic Snowflake looks for much players
  • CockroachDB hits Azure astatine past aft five-year mission

"This is nan consequence of not one, but aggregate process failures connected our portion for shutting these systems down. The imaginable 100 days of information is from a abstracted backup strategy we person that pulls information from Kafka and writes into different buckets," he said connected Twitter.

"GCP doesn't person a domiciled successful this astatine all, nan work is ours unsocial to bear. And we'll beryllium changing these processes importantly going forward. Including shriek tests successful nan early for thing that is getting closed down aliases unopen off."

He later explained that while nan backup strategy was deployed successful AWS Sydney, "the bucket successful which that backup was stored was deleted."

The determination to delete customer information and extremity services pinch sub-optimal connection continues to provoke fallout.

One personification complained they had paid $3,000 for nan AWS Sydney work little than a period earlier it was turned off. "There is thing near now," they said connected a institution forum.

An InfluxData worker said they would fto "the squad know" astir nan important expenditure.

The Register has contacted InfluxData for comment. ®

Updated to add

Late connected Friday, InfluxData issued an update connected its blog. CTO Paul Dix said nan institution had recovered nan clip bid information for nan GCP Belgium users and was successful nan process of getting that information backmost to them. "We person besides recovered nan user-defined functions and Tasks. I want to definitive my sincere gratitude to nan Google Cloud Platform squad for their unthinkable business successful helping america retrieve nan data," he said. The blog said nan institution would respond straight to users pinch a support summons to show them really to download their data. Others affected were asked to unfastened a ticket. Users of nan AWS/Sydney work person not been truthful fortunate and information remains mislaid to users caught unaware erstwhile nan work ended.