The increasing demand for generative AI as Large Language Models (LLMs) services has driven the need for specialized hardware architectures that optimize computational efficiency and energy consumption. This paper evaluates the performance of the Tenstorrent Grayskull e75 RISC-V accelerator for basic linear algebra kernels at reduced numerical precision, a fundamental operation in LLM computations. We present a detailed characterization of Grayskull's execution model, grid size, matrix dimensions, data formats, and numerical precision impact on computational efficiency. Furthermore, we compare Grayskull's performance against state-of-the-art architectures with tensor acceleration, including Intel Sapphire Rapids processors and two NVIDIA GPUs (V100 and A100). Whilst NVIDIA GPUs dominate raw performance, Grayskull demonstrates a competitive trade-off between power consumption and computational throughput, reaching a peak of 1.55 TFLOPs/Watt with BF16.

The Japan government-backed research group developing semiconductors will partner with U.S. startup Tenstorrent on the design of its first advanced AI chip. Tenstorrent, led by Tesla and Apple veteran Jim Keller, will license its design for part of Japan's artificial intelligence accelerator and also co-design the overall chip, the U.S. company said at a joint event in Tokyo on Tuesday. Working with the open-source RISC-V standard, Tenstorrent aims to provide customers with an alternative to the leaders Nvidia and Arm, who have their own so-called instruction sets to communicate between hardware and software. The Japanese government is funding a range of projects from research to advanced chip manufacturing, making an ambitious 67 billion bid to reclaim a central role in the semiconductor industry. The Tenstorrent agreement has the potential to advance those efforts, with the goal of producing the jointly designed AI chips at the government-backed startup Rapidus.

Now led by Jim Keller, the company has built a new leadership team and a new strategy. It is hard to believe the difference a year makes. In 2021, there were over 100 public and venture-backed startups with the same mission, to compete with NVIDIA in producing the fast chips needed to create and run artificial intelligence (AI). Fast forward to 2023, and now many companies are struggling to gain market traction or acquire enough capital to keep going. Part of the problem is undoubtedly the global economy; many AI adopters and investors do not have the resources or courage to give new chips a chance.

Tenstorrent, a hardware start-up developing next generation computers, announces the addition of Matthew Mattina as VP of Machine Learning. "Tenstorrent's mission is to advance the field of AI, which includes enabling entirely new categories of ML models which are larger and more intelligent than the industry uses today. Success requires mastery of the entire stack – from models, through compilers and runtime software, to the hardware it all relies on," said Ljubisa Bajic, Tenstorrent's CEO. "Matt is one of very few people in the world with deep expertise in all layers of the stack and his addition to the team is a significant step towards achieving our goals and moving AI forward." As Vice President of Machine Learning, Mattina will lead Tenstorrent's efforts to enable industry-leading performance and scalability of modern neural networks and Software 2.0.

It describes the technology behind the processor as: "The first conditional execution architecture for artificial intelligence facilitating scalable deep learning. Tenstorrent has taken an approach that dynamically eliminates unnecessary computation, thus breaking the direct link between model size growth and compute/memory bandwidth requirements." "Conditional computation enables adaptation to both inference and training of a model to the exact input that was presented, like adjusting NLP model computations to the exact length of the text presented, and dynamically pruning portions of the model based on input characteristics," is how the company describes it. It has eight channels of LPDDR4 for supporting up to 16Gbyte of external DRAM and 16 lanes of PCI-E Gen 4. The Tensix cores have a packet processor, a programmable SIMD and maths computation block, five single-issue RISC cores and 1Mbyte of ram. "The array of Tensix cores is stitched together with a double 2D torus network-on-chip, which facilitates multi-cast flexibility, along with minimal software burden for scheduling coarse-grain data transfers," according to the company.

That year, numerous experienced computer chip designers set out on their own to design novel kinds of parts to improve the performance of artificial intelligence. It's taken a few years, but the world is finally seeing what those young hopefuls have been working on. The new chips coming out suggest, as ZDNet has reported in past, that AI is totally changing the nature of computing. It also suggests that changes in computing are going to have an effect on how artificial intelligence programs, such as deep learning neural networks, are designed. Case in point, startup Tenstorrent, founded in 2016 and headquartered in Toronto, Canada, on Thursday unveiled its first chip, "Grayskull," at a microprocessor conference run by the legendary computer chip analysis firm The Linley Group.