Global data storage and manufacturing capacity is critically low

An excerpt from The Holon Data Report, Part 5

Global data storage capacity is rapidly becoming a substantial limiting factor on our ability to retain data that might be valuable in the future. The world produced almost 100 ZB of data in 2022, yet the total available data storage in global commercial data centres will be ~4 ZB. In addition, there is approximately 8 ZB of storage in computer and smartphone devices.

What really matters however is the growth of new data storage each year that allows a small portion of the exponential growth in data to be sent for storage. While much of the new data created each year will likely be of low value, questioning its need for storage, the slow rate of growth in new storage capacity is rapidly becoming a limiting factor in the cost and ability to store data for the future (we refer to this as ‘storability’).

Let’s do some simple math to outline the estimated situation in 2022:

• Total new data creation in 2022 – 100 ZB
• Estimated new storage capacity 2022 – 1 ZB
• Volume of data that can be stored assuming no previous data deleted – 0.25% (assuming four copies are stored for data protection)

If we take this simple equation out to 2040 to estimate storability, we begin to see the problem emerge. Data supporting Holon’s estimates for the total volume of data created each year is outlined in detail in Part 4 of our Data Report. This data principally covers autonomous EVs as well as Internet-of-Things (IoT) devices, but the additional new technology platforms emerging over the next two decades will also contribute immensely to these forecasts. Our data also assumes that the global data storage capacity footprint will reach 1,000 ZB by 2040, an almost 300-fold increase on the 3.5 ZB of capacity available in global data centres in 2021. Finally, we assume that storage hardware devices, principally hard-disc drive (HDDs) and solid-state drives (SSDs), need to be replaced every 6 years to ensure data integrity remains.

Table 1 outlines our alarming findings. By 2040, storability, assuming four copies of each dataset will have fallen to just 0.09% of the total data we forecast will be generated (74,926 ZB). With the total growth of data slowing over the decade to 2040 (due to being more conservative with our forecasts), our storability improves to 0.04% in 2035 and 0.09% in 2040. However, these forecasts assume that global storage capacity will grow at almost 36% over the next 20 years to achieve this target after accounting for a 6-year replacement cycle on the existing storage capacity. The following data is from Chapter 7 in Part 4 of our Data Report.

Table 1: Global Data generation and storage forecasts

Source: Holon

One of the key challenges holding back an acceleration in storage capacity is the complete control over storage by just a handful of global companies. This was outlined in detail in Chapter 5 of this report. We estimate that the cumulative investment required to build `1,000 ZB of global capacity by 2040, even after assuming a 5% annual depreciation in storage hardware costs, will surpass US$50 trillion. The cost of hardware replacement, using a 6-year cycle, will also surpass US$1 trillion by 2032 and reach US$7 trillion by 2040. This is just for the storage hardware and does not include any costs related to connectivity or any other infrastructure which would more than double the total cost above US$100 trillion. The full analysis of our assumptions here can be found in Part 4 of our Data Report.

Today, this leaves us in the situation that AWS and Microsoft are managing new storage capacity additions at a growth rate that allows them to maintain high revenue and profits. This works against the need for a massive volume growth in capacity that is urgently required to meet the long-term storage needs for new digital technology platforms emerging over the next two decades.

This data suggests that it is too great a price to pay by even today’s Web 2.0 technology giants. Managing this data will also come at a heavy price of regulatory oversight, with enormous restrictions placed on the careful management of a dataset that’s expected to be 1,000 times larger than that which is seen today. Finally, the massive depreciation bill on storage hardware will dramatically lower the profit margins of today’s tech giants, turning them into gigantic infrastructure managers and changing their business models dramatically from their existing high-margin technology businesses.

Pressure to continually lower the cost of data storage, as well as analytics, will also drive down revenue per unit of data dramatically as volume is added.

Siting back on these enormous issues, Holon’s conclusion is that if alternate business models such as Web 3.0 decentralised storage emerge and can offer equivalent data security and capacity, to today’s Web 2.0 models at a cheaper price they will help solve part of the problem.

The Filecoin model, which incentivises decentralised storage providers to add new capacity to the global storage network is one alternative emerging. By lowering the barrier of entry for a Storage Provider to offer their data services. Filecoin opens a model not to dissimilar to Airbnb. Filecoin can enable storage providers globally to operate on a unified platform and together compete with the services of the leading cloud providers of today.

Another limiting factor likely to dramatically impact the growth of global data storage capacity is the limited manufacturing capacity in HDD and SSD hardware manufacturers. These are outlined in detail in Part 3 of our Data Report.

Samsung Electronics of Korea, the world’s largest SSD manufacturer, currently holds approximately 30% global market share. With a market capitalisation of US$300 billion, it likely has the necessary balance sheet to take on this enormous challenge to grow its annual SSD capacity above 30% per year over multiple decades. However, this is an enormous assumption, and needs to also consider its commitment to growing its semiconductor manufacturing capacity.

Interestingly, they announced in March 2023 that they are spending US$228 billion on the world’s largest chip facility, partly to compete with Taiwan Semiconductor (TSMC) in 3 nm (nanometre) semi chips. Additional investment into 6G telecommunications hardware will focus on building a new mobile application processor chip. Little was discussed in its five-year plan specifically on its storage capacity investments, suggesting it is less central to its long-term growth plans.

In HDDs, the leading manufacturer is Seagate Technologies, who held 43% market share in 2021. However, its small market capitalisation of US$10.7 billion, as well as the expected shift in demand towards SSD storage by 2026, suggests it is currently too small to meet the expected demand surge over the next two decades. Western Digital, with 36% global market share in HDDs, also remains relatively small at a US$10 billion market capitalisation.

Either new storage hardware manufacturers will need to enter the market to meet this demand, or existing producers will need to raise enormous amounts of capital to fund the opportunity, suggesting substantial dilution will occur for existing shareholders. Finding a solution to solve the manufacturing shortfall outlook remains extremely uncertain at present and could be an enormous limiting factor on meeting the exponential growth demand for new data storage capacity over the next 20 years.

The Holon Data Report Part 5, in its entirety, can be found here.

Disclaimer: This Article has been prepared by Holon Global Investments Limited ABN 60 129 237 592. Holon Global Innovations Pty Ltd (“HGI”) is a wholly owned subsidiary of Holon Global Investments Limited (together “Holon”). HGI is a Filecoin (FIL) Storage Provider and is positioned as a major player in the FIL decentralised data storage arena. FIL Storage Providers are rewarded in FIL for the provision of data storage capacity. Holon, its officers, employees and agents believe that the information in this material and the sources on which the information is based (which may be sourced from third parties) are correct as at the date of publication. While every care has been taken in the preparation of this material, no warranty of accuracy or reliability is given and no responsibility for this information is accepted by Holon, its officers, employees or agents. Except where contrary to law, Holon excludes all liability for this information.