American tech conglomerate Meta (formerly Facebook) on Monday joined the list of tech companies like Microsoft and Nvidia who have built an AI supercomputer, with the recent announcement of its new Artificial Intelligence (AI) Research SuperCluster (RSC). Meta claims RSC is already among the fastest supercomputers and, when completed in mid-2022, will be the world’s fastest.
Supercomputers are high-performance computers that perform computationally intensive (high-speed) tasks such as weather forecasting, climate research, and oil and gas exploration. AI supercomputers are an improved version of supercomputers that can carry out more calculations per second than regular supercomputers, using the same hardware.
“RSC will help Meta’s AI researchers build better AI models that can learn from trillions of examples; work across hundreds of different languages; seamlessly analyse text, images and video together; develop new augmented reality tools; and more,” the company explained in a blog post.
With more than 2.6 billion monthly active users, Meta’s Facebook is the most popular social media in the world. In Africa, Meta’s suite of social messaging tools—Facebook Messenger, Instagram and WhatsApp—serve as the internet for hundreds of millions of people. Businesses and consumers depend heavily on these messaging platforms because its use is ubiquitous and access to these apps and sites are free or discounted on many African telecom networks.
RSC is expected to aid Meta in identifying harmful content on Facebook and Instagram; processing larger and longer videos with higher sampling rates; and in developing new augmented reality tools. It’s a valuable tool that’ll aid Meta in accomplishing its plans to move billions of users into the Metaverse in the near future.
Before this announcement, Meta has been using the first generation of the supercomputer designed in 2017, which could design 22,000 NVIDIA V100 Tensor Core GPUs in a single cluster that performs 35,000 training jobs a day.
Realising the need to accelerate progress, work on the RSC began in early 2020, with Meta engineers designing and building a new AI supercomputer system from ground up.
“RSC today comprises a total of 760 NVIDIA DGX A100 systems as its compute nodes, for a total of 6,080 GPUs—with each A100 GPU being more powerful than the V100 used in the previous system,” the company said.
Compared to the first generation of Meta’s supercomputer, early benchmarks on RSC show that it “runs computer vision workflows up to 20 times faster, runs the NVIDIA Collective Communication Library (NCCL) more than nine times faster, and trains large-scale NLP models three times faster. That means a model with tens of billions of parameters can finish training in three weeks, compared with nine weeks before,” Meta said.
Aware of privacy concerns, the company added that RSC has been designed from the ground up with privacy and security in mind. Meta’s researchers can safely train models using encrypted user-generated data that is not decrypted until right before training.
