David Friend, CEO and co-founder of Wasabi Technologies, talks us through his views on data storage and the future of the ‘bottomless cloud’.
Data is at the very heart of contemporary enterprise, particularly when it comes to Big Tech i.e. Facebook, Google, Amazon. Even manufacturers that produce and distribute physical goods, such as Unilever, could not exist without a huge amount of data. The consumer goods giant has established 27 data centres to date across the world that deliver insights that drive logistics, marketing and product development.
What’s important to note is the astonishing rate at which new data points are being created and cloud-based storage is growing. It’s estimated that by 2035 we will have access to over one yottabyte of data – to put that into context, that is more than the number of visible stars in the universe.
This is not happening in a vacuum either – our ability to effectively generate and store content may become fundamental to effective law enforcement. In the US, body cam footage and surveillance video has become an indispensable asset for police as well as the judicial system. However, footage is routinely deleted after 75 days, despite the statute of limitations for prosecuting a crime is seven years on average.
The reason for this is simple: it costs too much to store. This has clear ramifications not only for effective law enforcement practices but for citizen rights and the enforcement of accountability. When we zoom out and take into account the 300 million surveillance cameras currently in operation, and the increase in video resolution from HD to 4K, the effects are compounded.
An evolving landscape
Thankfully, modern enterprise has finally started viewing data as a fundamental utility that will define the operational scope and success of their business, instead of an add on or a nice to have. After 50 years of relying on internal storage solutions, businesses need to wake up to the fact that, for most, they have outgrown their ability to effectively store their own data.
However, when data is backed up physically for example, it is especially vulnerable to threats, meaning companies open themselves up to large losses of capital. We only need to look to Sheryl Crow, who recently discovered all of her master recordings were destroyed when an archive in Los Angeles burned down in 2008. This is a wider concern for any company which has a large store of high-value assets.
Our data storage requirements are only set to increase over the coming years, at an unprecedented pace and scale. Increasingly, it will become the focal point of the majority of global businesses, from small retail outfits to large professional service corporations.
What many fail to realise however is that the proliferation of nascent technologies such as Artificial Intelligence (AI) will fuel the next generation of data storage centres even further; this will in turn rapidly increase our ability to engage in innovative analytical processes and Machine Learning driven development cycles.
Whether you operate in media, education or industry, if you fail to adopt new data practices and source effective storage solutions to meet your needs, you face being shouldered out by competition.
The physical cloud
The term ‘cloud’ can be somewhat misleading – data is not ethereal; it takes up real physical space. Not considering storage when generating data is as problematic as not considering how to store oil once it’s been successfully drilled.
Much hype surrounds AI – barely a day passes without some ‘talking head’ discussing their ability to drive real progress in medicine, business and education. On the other hand, we also have the likes of Elon Musk waxing lyrical about the existential threat it poses.
AI is often used as a catch whole term, but most people focus on improvements in algorithms that mimic intelligence. What no-one is discussing however is how development in AI is progressing – to put it simply, Artificial Intelligence is only as smart as the data that it learns from.
We couldn’t have achieved what we have without the huge swathes of data we now have available – to put this into perspective, Google alone processes a staggering 3.5 billion searches every day.
The future of the cloud is already here
With the rise of IoT and the implementation of 5G infrastructure, data generation will soon explode. Just as electric power generation moved to centralised power plants, the storage of data has and must continue to move from on-premise storage to large centralised clouds run by experts with real storage expertise.
The good news is that this will facilitate and fuel our next industrial revolution. Take one example, autonomous vehicles (AV). Currently, autonomous vehicles are trained to gather specific data about a subset of their external context.
By moving data storage off-premise, costs can be dramatically reduced, meaning engineers can more easily design algorithms that teach AVs to learn just as humans do, by capturing all the data in their surrounding environment, increasing data storage per hour from 500GB to 5TB.
As data proliferates and storage costs get cheaper, we’ll also see streaming services start designing hyper personalised customer experiences. Netflix is projected to spend US$15bn on content in 2019 and production, storylines, settings and talent are also of course driven by data.
Safeguarding your future
Just as data centres rely on an abundance of cheap, reliable, electric power, we have to start thinking about data storage in a similar way. Like electricity, the need for storage just keeps growing and data centres are finding it increasingly difficult to keep up with the demand.
Specialised cloud storage providers can operate at an enormous scale and can be far more economical than on-premises storage.
Rather than relying on huge vendors which can often charge excessive egress fees, businesses should also look to independent cloud storage providers to future proof their business – having access to storage solutions at a low price is fundamental for any scaling business that will inevitably see a significant increase in data generation.
