DePT induces a cone-shaped spatial-temporal attention prior,which injects theinformation propagation and aggregation principles and enables a global view. With physical constraint inductive bias baked into its design, our DePT is ready to plug and play for a broad class of multi-agent systems. The experimental results on one of the most challenging CPS - network-scale traffic signal control system in the open world - show that our model outperformed the state-of-the-art expert methods on synthetic and real-world datasets.

Furthermore, we develop a method to eliminate bias in cases where blindly assuming IID is expected to yield a significantly biased estimate.

While largescale pre-trained language models such asBERT[5]haveachieved greatsuccess onvariousnatural language understanding tasks,howtoefficiently and effectively incorporate them into sequence-to-sequence models and the corresponding text generation tasks remains a non-trivial problem.

Spectral embedding is a powerful graph embedding technique that has received a lot of attention recently due to its effectiveness on Graph Transformers.

Many aspects of human reasoning, including language, require learning rules from very little data. Humans can do this, often learning systematic rules from veryfewexamples,andcombining theserulestoformcompositional rule-based systems.

Wefurthermore derive, with mathematical proof and extensive numerical evidence, the scalinglawexponents inallofthese phases, inparticular computing theoptimal modelparameter-count as a function of floating point operation budget.