Large language models have shown an impressive societal impact owing to their excellent understanding and logical reasoning skills. However, such strong ability relies on a huge amount of computing resources, which makes it difficult to deploy LLMs on computing resource-constrained platforms. Currently, LLMs process each token equivalently, but we argue that not every word is equally important. Some words should not be allocated excessive computing resources, particularly for dispensable terms in simple questions. In this paper, we propose a novel dynamic inference paradigm for LLMs, namely D-LLMs, which adaptively allocate computing resources in token processing. We design a dynamic decision module for each transformer layer that decides whether a network unit should be executed or skipped. Moreover, we tackle the issue of adapting D-LLMs to real-world applications, specifically concerning the missing KV -cache when layers are skipped. To overcome this, we propose a simple yet effective eviction policy to exclude the skipped layers from subsequent attention calculations. The eviction policy not only enables D-LLMs to be compatible with prevalent applications but also reduces considerable storage resources.

Figure 1: Training the proposed Patch Diffusion Inverse Solver (PaDIS) method. Different sized patches are used in each training iteration.

The authors use Greedy Coordinate Gradient (GCG) to generate discrete adversarial suffixes in natural language.

In this paper, we propose an autonomous information seeking visual question answering framework, A VIS. Our method leverages a Large Language Model (LLM)

Numerous real-world systems, ranging from healthcare to energy grids, involve users competing for finite and potentially scarce resources.

Numerous real-world systems, ranging from healthcare to energy grids, involve users competing for finite and potentially scarce resources.

We prepare 1K real-world wildfire scenarios for testing the AI models. We also propose a deep transformer, S2R-FireTr, which excels in considering the multi-modal environmental information for forecasting and backtracking the wildfire.