Google Cloud and Seagate has devised a machine learning (ML) system to predict when disks inside Google’s data centers might fail.
Despite their volatility and high fail rates, hard disk drives (HDD) are the preferred means of housing data inside a data center.
While newer and more reliable technologies such as flash memory are being touted as viable alternatives, IDC predicts that data centers will increase their storage capacity by 17.8% over the next few years, with HDDs leading the way.
“At Google Cloud, we know first-hand how critical it is to manage HDDs in operations and preemptively identify potential failures,” wrote Nitin Aggarwal, Technical Program Manager and Rostam Dinyari, AI Engineer, with Google Cloud, in a joint blog post announcing the collaboration with Seagate.
The duo outlined the new ML system that helps them identify potential disk failures to avoid any outages and disruptions to their cloud services – and note that it could soon be rolled out wider as well, meaning it might soon be possible to spot issues in other HDDs.
ML in production
In the past, the engineers contend that manually sifting through the terabytes of raw telemetry data generated by the millions of HDDs to identify problems isn’t feasible.
This led them to devise an automated ML system to predict the health of the HDDs.
Together with Accenture, Google Cloud helped Seagate build a proof of concept based on the two most common drive types.
The companies tested two models. One based on an AutoML Tables classifier and another that was custom-developed for the project. The former turned out to have the higher precision of the two (98%) after analyzing the forecasts and predictions for a month.
While Google Cloud is using this as an example to hawk its ML tools that help put automation into production environments, the company said it has plans to expand the system to support all Seagate drives.
Via Tom’s Hardware
