Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN

We also discover that the confusing samples The above theory serves as a mathematical guarantee of a DNN, which are represented by non-generalizable to let AND-OR interactions in the logical model be roughly interactions, are determined by its low-layer parameters. In considered as primitive inference patterns equivalently used comparison, other factors, such as high-layer parameters by the DNN for inference. For example, as Figure 1 shows, and network architecture, have much less impact on the given an input prompt x ="A red apple falls to the ground composition of confusing samples. Two DNNs with different because of the pull of," the LLM generates the next token low-layer parameters usually have fully different sets of "gravity," and its inference score of token generation can confusing samples, even though they have similar performance.