LLM-based agents have demonstrated great potential in generating and managing code within complex codebases. In this paper, we introduce WebGen-Bench, a novel benchmark designed to measure an LLM-based agent's ability to create multifile website codebases from scratch. It contains diverse instructions for website generation, created through the combined efforts of human annotators and GPT4o. These instructions span three major categories and thirteen minor categories, encompassing nearly all important types of web applications. To assess the quality of the generated websites, we generate test cases targeting each functionality described in the instructions. These test cases are then manually filtered, refined, and organized to ensure accuracy, resulting in a total of 647 test cases. Each test case specifies an operation to be performed on the website and the expected outcome of the operation. To automate testing and improve reproducibility, we employ a powerful web-navigation agent to execute test cases on the generated websites and determine whether the observed responses align with the expected results. We evaluate three high-performance code-agent frameworks--Bolt.diy,

In this paper, we presentDIB-R, a differentiable rendering framework which allows gradients to be analytically computed for allpixels inanimage.

Inparticular,theability to visually hallucinate future frames has enabled applications in robotics [8] and healthcare [26].

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

We Asked Audio Pros to Blind Test Headphones. When it comes to choosing a good pair of headphones, what happens when you take features, design, and brand awareness out of the equation, and leave it all up to the sound? All products featured on WIRED are independently selected by our editors. However, we may receive compensation from retailers and/or from purchases of products through these links. What makes a really great pair of headphones? The basic answer used to be sound quality, but modern headphones offer so much more than just audio chops.

Conformal prediction can be used to construct prediction sets that cover the true outcome with a desired probability, but can sometimes lead to large prediction sets that are costly in practice. The most useful outcome is a singleton prediction-an unambiguous decision-yet existing efficiency-oriented methods primarily optimize average set size. Motivated by this, we propose a new nonconformity score that aims to minimize the probability of producing non-singleton sets. Starting from a non-convex constrained optimization problem as a motivation, we provide a geometric reformulation and associated algorithm for computing the nonconformity score and associated split conformal prediction sets in O(K) time for K-class problems. Using this score in split conformal prediction leads to our proposed Singleton-Optimized Conformal Prediction (SOCOP) method. We evaluate our method in experiments on image classification and LLM multiple-choice question-answering, comparing with standard nonconformity scores such as the (negative) label probability estimates and their cumulative distribution function; both of which are motivated by optimizing length. The results show that SOCOP increases singleton frequency (sometimes by over 20%) compared to the above scores, with minimal impact on average set size.

This supplemental material provides the proofs for the Theorems and Lemmas presented in our paper. For all the datasets, we use interpolation to create sequences of the same length. For SPDNet and SPDNetBN, we compute a covariance matrix to represent an input sequence as in [20]. Our networks are implemented with Tensorflow framework. The number of frames in each sequence is set to 100.

This paper presents a recurrent convolutional neural network for semantic image segmentation to encode and take advantage of contextual relationships. The method is basically a combination of [13] where a very similar RCNN is used for object recognition, and [4] from which the multi-scale pipeline is inspired. Thus, technically, the paper is not very novel (Sec. However, the paper is well executed, very easy and clear to read, largely well written and provides a seemingly fair evaluation to state-of-the-art. Some recent works or interesting evaluations could be added, see below.