Training Graph Neural Networks (GNNs) on large graphs is challenging due to the conflict between the high memory demand and limited GPU memory. Recently, distributed full-graph GNN training has been widely adopted to tackle this problem. However, the substantial inter-GPU communication overhead can cause severe throughput degradation. Existing communication compression techniques mainly focus on traditional DNN training, whose bottleneck lies in synchronizing gradients and parameters. We find they do not work well in distributed GNN training as the barrier is the layer-wise communication of features during the forward pass & feature gradients during the backward pass. To this end, we propose an efficient distributed GNN training framework Sylvie, which employs one-bit quantization technique in GNNs and further pipelines the curtailed communication with computation to enormously shrink the overhead while maintaining the model quality. In detail, Sylvie provides a lightweight Low-bit Module to quantize the sent data and dequantize the received data back to full precision values in each layer. Additionally, we propose a Bounded Staleness Adaptor to control the introduced staleness to achieve further performance enhancement. We conduct theoretical convergence analysis and extensive experiments on various models & datasets to demonstrate Sylvie can considerably boost the training throughput by up to 28.1x.

The game's lethal storm circle is tightening around the combat zone, a sleepy beach town with a bubblegum-pink ice cream parlor, and the handful of remaining squads are duking it out for survival. My three teammates, who are all children, are taking intense fire. One squares off with an especially ruthless competitor and is promptly dispatched. "Watch out, that kid is sweaty," he warns. Another falls to a grenade burst with a cry of "I'm knocked!"

Sylvie's work sits at the intersection of law and ethics, with a particular interest in data and machine ethics. As a Mozilla Fellow, her research will focus on the design of computer systems meant for morally-loaded contexts. "My work seeks to reverse the current top-down, fire-brigade approach to data governance," Sylvie says. "Instead of seeking to minimise the increasingly-evident yet poorly-understood harms that result from the systematic collection of personal data, its aim is to build a'people empowering' toolkit." Sylvie is also considering the potential inherent in "bottom-up" Data Trusts as a mechanism to address power imbalances between data-subjects and data-controllers.

The age of the bot has begun. We just have barely begun to notice. It's only been 10 years, and smartphones have fundamentally changed our world, the bot revolution will come quicker, and have an even greater impact on us. The bots cometh in all shapes and sizes, hardware and software, automations and AI and robot fried and foe. But most likely our most common interaction with them in the very near future will be in the form of messenger/personal assistant and customer service bots.

Later this month Valerie will go on duty behind the reception desk at Carnegie Mellon University's School of Computer Sciences. Besides doling out information and directions, she'll chat about her ever-changing personal life. If you introduce yourself, she'll remember you. If you ask about the weather, when she meets you again she may bring up the subject. Valerie, in case you haven't guessed, is a robot - one in a long line of increasingly sophisticated machines.