Lead authors (author list ordered alphabetically) 38th Conference on Neural Information Processing Systems (NeurIPS 2024).

--It is shown that the port behavior of a resistor-diode network corresponds to the solution of a ReLU monotone operator equilibrium network (a neural network in the limit of infinite depth), giving a parsimonious construction of a neural network in analog hardware. We furthermore show that the gradient of such a circuit can be computed directly in hardware, using a procedure we call hardware linearization . This allows the network to be trained in hardware, which we demonstrate with a device-level circuit simulation. We extend the results to cascades of resistor-diode networks, which can be used to implement feedforward and other asymmetric networks. We finally show that different nonlinear elements give rise to different activation functions, and introduce the novel diode ReLU which is induced by a non-ideal diode model. The idea of building a neural network in analog hardware is classical [1]-[5]. Since the discovery of semiconductor devices with memristive properties [6], and in light of the growing energy intensiveness of machine learning systems, there has been a resurgence of interest in building devices which incorporate analog memristive components and are specially suited for deep learning applications [7], [8]. One of the primary advantages of such devices is that memristors, and similar elements such as phase change memory, act as both memory and computational units. This allows the transport delay between memory and computation to be circumvented. A particularly successful design is to arrange a number of memristors in a crossbar array, which can be used to perform matrix-vector calculation in a single operation [9]-[12].

Recent advances in large language models (LLMs) have shown impressive performance in mathematical reasoning and code generation. However, LLMs still struggle in the simulation domain, particularly in generating Simulink models, which are essential tools in engineering and scientific research. Our preliminary experiments indicate that LLM agents often fail to produce reliable and complete Simulink simulation code from text-only inputs, likely due to the lack of Simulink-specific data in their pretraining. To address this challenge, we propose SimuGen, a multimodal agent-based framework that automatically generates accurate Simulink simulation code by leveraging both the visual Simulink diagram and domain knowledge. SimuGen coordinates several specialized agents, including an investigator, unit test reviewer, code generator, executor, debug locator, and report writer, supported by a domain-specific knowledge base. This collaborative and modular design enables interpretable, robust, and reproducible Simulink simulation generation. Our source code is publicly available at https://github.com/renxinxing123/SimuGen_beta.

Large language models (LLMs) have the potential to revolutionize various fields, including code development, robotics, finance, and education, due to their extensive prior knowledge and rapid advancements. This paper investigates how LLMs can be leveraged in engineering education. Specifically, we benchmark the capabilities of different LLMs, including GPT-3.5 Turbo, GPT-4o, and Llama 3 70B, in assessing homework for an undergraduate-level circuit analysis course. We have developed a novel dataset consisting of official reference solutions and real student solutions to problems from various topics in circuit analysis. To overcome the limitations of image recognition in current state-of-the-art LLMs, the solutions in the dataset are converted to LaTeX format. Using this dataset, a prompt template is designed to test five metrics of student solutions: completeness, method, final answer, arithmetic error, and units. The results show that GPT-4o and Llama 3 70B perform significantly better than GPT-3.5 Turbo across all five metrics, with GPT-4o and Llama 3 70B each having distinct advantages in different evaluation aspects. Additionally, we present insights into the limitations of current LLMs in several aspects of circuit analysis. Given the paramount importance of ensuring reliability in LLM-generated homework assessment to avoid misleading students, our results establish benchmarks and offer valuable insights for the development of a reliable, personalized tutor for circuit analysis -- a focus of our future work. Furthermore, the proposed evaluation methods can be generalized to a broader range of courses for engineering education in the future.

Integrating Retrieval Augmented Generation (RAG) with Large Language Models (LLMs) has shown the potential to provide precise, contextually relevant responses in knowledge intensive domains. This study investigates th e ap - plication of RAG for ABB circuit breakers, focusing on accuracy, reliability, and contextual relevance in high - stakes engineering environments. By leveraging tailored datasets, advanced embedding models, and optimized chunking strategies, the research addresses challenges in data retrieval and contextual alignment unique to engineering documentation. Key contributions include the development of a domain - specific dataset for ABB circuit breakers and the evaluation of three RAG pipelines: OpenAI GPT4o, C ohere, and Anthropic Claude. Advanced chunking methods, such as paragraph - based and title - aware segmentation, are assessed for their impact on retrieval accuracy and response generation. Results demonstrate that while certain configurations achieve high pr ecision and relevancy, limitations persist in ensuring factual faithfulness and completeness, critical in engineering contexts. This work underscores the need for iterative improvements in RAG systems to meet the stringent demands of electrical engineering tasks, including design, troubleshooting, and operational decision - making. The findings in this paper help advance research of AI in highly technical domains such as electrical engineering. Electrical engineering is a cornerstone of modern infrastructure, underpin n ing systems that power cities, enable communication, and drive technological innovation. From power generation and distribution to the design of advanced electronic systems, electrical engineering plays a vital role in ensuring the reliability, efficiency, and safety of critical infrastructure [1]. Mistakes or inaccuracies in the design, operation, or maintenance of e lectrical systems can have far - reaching consequences, including equipment failure, financial losses, and risks to public safety. In such high - stakes environments, precision and reliability in accessing accurate technical information are paramount [2]. Sim ilarly, in medicine, iterative retrieval methods have been proposed to enhance the accuracy of RAG systems. Xiong et al. [3] introduced the i - MedRAG system, which dynamically generates follow - up queries to refine responses. This approach improved retrieval accuracy and generalizability, although it incurred higher computational costs.

-- Energy-based learning algorithms are alternatives to backpropagation and are well-suited to distributed implementations in analog electronic devices. However, a rigorous theory of convergence is lacking. We make a first step in this direction by analysing a particular energy-based learning algorithm, Contrastive Learning, applied to a network of linear adjustable resistors. It is shown that, in this setup, Contrastive Learning is equivalent to projected gradient descent on a convex function, for any step size, giving a guarantee of convergence for the algorithm. Backpropagation is the most popular method of training artificial neural networks. However, while artificial neural networks are inspired by biological nervous systems, it has long been observed that backpropagation is not biologically plausible [1]-[3]. Several biologically plausible alternatives to backpropagation have been proposed in the literature, among them so-called energy-based learning algorithms [4]- [11]. These algorithms apply to energy-based models, which come equipped with some generalized notion of energy, and associate to each input a minimum of this energy. The basic idea is to probe the system in two states, one free and one clamped, or dictated by the training data, and use the energy difference between these states as a cost function. An iterative procedure is then applied to minimise this cost function. Several clamping mechanisms and iterative procedures have been defined, among them Contrastive Learning [4], [5], [12], Equilibrium Propagation [7], Coupled Learning [9] and Temporal Contrastive Learning [13]. These algorithms all resemble gradient descent, where the gradient of the cost function is replaced by a gradient-like quantity which may be computed in a distributed manner across a network. The energy-based learning paradigm is particularly suited to learning in analog electronic devices, as they have a natural notion of generalized energy: the heat dissipated by electrical resistance (in this case, a power rather than energy). M. A. Huijzer, B. Besselink, and H.J. van Waarde are with the Bernoulli Institute for Mathematics, Computer Science, and Artificial Intelligence, University of Groningen, Groningen, The Netherlands; email: m.a.huijzer@rug.nl; Chaffey was with the Control Group, Department of Engineering, University of Cambridge, UK, and is now with the School of Electrical and Computer Engineering, University of Sydney, Australia; email: thomas.chaffey@sydney.edu.au. This is, in part, due to the ability of analog circuits to perform inference many times faster than conventional neural networks [20]-[22].

In this work, we propose a high-voltage, high-frequency control circuit for the untethered applications of dielectric elastomer actuators (DEAs). The circuit board leverages low-voltage resistive components connected in series to control voltages of up to 1.8 kV within a compact size, suitable for frequencies ranging from 0 to 1 kHz. A single-channel control board weighs only 2.5 g. We tested the performance of the control circuit under different load conditions and power supplies. Based on this control circuit, along with a commercial miniature high-voltage power converter, we construct an untethered crawling robot driven by a cylindrical DEA. The 42-g untethered robots successfully obtained crawling locomotion on a bench and within a pipeline at a driving frequency of 15 Hz, while simultaneously transmitting real-time video data via an onboard camera and antenna. Our work provides a practical way to use low-voltage control electronics to achieve the untethered driving of DEAs, and therefore portable and wearable devices.

Recent advancements in large language models (LLMs) have led to significant successes across various applications, where the most noticeable is to a series of emerging capabilities, particularly in the areas of In-Context Learning (ICL) and Chain-of-Thought (CoT). To better understand and control model performance, many studies have begun investigating the underlying causes of these phenomena and their impact on task outcomes. However, existing explanatory frameworks predominantly focus on isolating and explaining ICL and CoT independently, leading to an incomplete understanding of their combined influence on model performance. To address this gap, we propose the Electronic Circuit Model (ECM), which provides a foundation for developing scalable, learnable policies and improving the management of AI-generated content. Specifically, ECM conceptualizes model behavior as an electronic circuit: ICL is represented as semantic magnetic field to providing an additional voltage following Faraday's Law, while CoT is modeled as series resistors to constrain the model output performance following Ohm's Law. Experimental results demonstrate that the ECM effectively predicts and explains LLM performance across a variety of prompting strategies. Furthermore, we apply ECM to advanced reasoning strategy optimization on a series of tasks, such as the International Olympiad in Informatics (IOI) and the International Mathematical Olympiad (IMO), achieving competitive performance that surpasses nearly 80% of top human competitors.

We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is to design an appropriate electric circuit whose continuous-time dynamics converge to the solution of the optimization problem at hand. Then, the second stage is an automated, computer-assisted discretization of the continuous-time dynamics, yielding a provably convergent discrete-time algorithm. Our methodology recovers many classical (distributed) optimization algorithms and enables users to quickly design and explore a wide range of new algorithms with convergence guarantees.