A Trillion Edge Graph on a Single Accelerated Node

A Trillion Edge Graph on a Single Accelerated Node

By | April 29th, 2017
No Comments on A Trillion Edge Graph on a Single Accelerated Node

There are many paths to processing trillions of edges efficiently and with

high performance as demonstrated by companies like Facebook with its

distributed trillion-edge scaling effort across 200 nodes in 2015 and

Microsoft with a similar feat as well.


														
							

Efficiently and quickly chewing through one trillion edges of a complex graph is no longer in itself a standalone achievement, but doing so on a single node, albeit with some acceleration and ultra-fast storage, is definitely worth noting.

A team from Georgia Tech has demonstrated Facebook and Microsoft-class capabilities for graph processing, but without the heavy-handed cluster approach via MOSAIC, a graph processing engine that exploits all the hardware resources available in a standard Xeon host processor, Xeon Phi coprocessors, NVMe, and a fast interconnect.

As one of the leads, Changwoo Min, tells The Next Platform, even though terascale graph processing is no longer unheard of, finding ways to make it efficient and high performance is where the real research and implementation challenge lies.

He points to the Microsoft work that spanned one trillion edges across a 64-node Infiniband connected cluster, but says with their single node on the same sized graph, the performance difference was only 1-2X slower on single-machine versus high-end cluster.

Read more

Google
Nisheeth Bhakuni