Large complex data need advanced computer systems, often know as supercomputing. Answering questions in metaphysical queries, finding the origins of the universe, to discovering cancer-fighting drugs all need high-performance computing for their complexity and large questions.
Current framework for storage platforms requires users to choose between customization of features or high availability. Reliance on computers intaking, processing, and analyzing huge amounts of data gives supercomputing unprecedent speeds. As of now, the best systems only operate at a quadrillion calculations per second, or a petaflop.
So, researchers at Virginia Technical Institute are aiming to tackle this issue by giving high-performance computing (HPC) data systems the flexibility to thrive with an advanced framework called BespoKV. The goal of BespoKV is to someday perform at the exascale, or 1 billion billion calculations per second.
The main component of the new framework is key value (KV) systems which store and retrieve important data from fast memory-based storage. In multiple computers aiming to solve a problem, KV systems are used in today's high-performance applications. The support of KV systems by of BespoKV give it an innovative edge.
"I got interested in key value systems because this very fundamental and simple storage platform has not been exploited in high-performance computing systems where it can provide a lot of benefits," says first author of the study, Ali Anwar. "BespoKV is a novel framework that can enable HPC systems to provide a lot of flexibility and performance and not be chained to rigid storage design."
The study is critical for industries that process large amounts of data such as space-hogging, intense visual graphics of movie streaming sites, millions of financial transactions, or user-generated content at social media outlets. The new framework enables new HPC services for the anticipation of future workloads.
"Developers from large companies can really sink their teeth into designing innovative HPC storage systems with BespoKV," explains professor of computer science, Ali Butt. "Data-access performance is a major limitation in HPC storage systems and generally employs a mix of solutions to provide flexibility along with performance, which is cumbersome. We have created a way to significantly accelerate the system behavior to comply with desired performance, consistency, and reliability levels."
Source: Virginia Tech