SoC-Driven Inference Datacenters Becoming New Reality

The mighty SoC is coming for the datacenter with inference as a prime target, especially given cost and power limitations. With multiple form factors stretching from edge to server, any company that provide a seamless jump from devices in the field to large-scale datacenter processing is ready for the future. This is the new reality Israeli startup, NeuReality, is preparing for, as are others in the AI ASIC and systems space who want to seamlessly let users jump from the edge to close compute with multiple form factors and efficiency-tuned software stacks. What is often missing from those conversations are some of the most important system elements, networking capabilities in particular. From (very) early glance, NeuReality seem to have their networking and SoC priorities straight--and serious networking and chip design pedigree to bring it together with the announcement of their first prototype inference system for the datacenter.