bpda
Supplementary material for: The balancing principle for parameter choice in distance-regularized domain adaptation
The main criterion used to define the balancing principle is as follows. Using the instantiation bound of the balancing principle in Eq. (1) further implies that null ε Figure 1 provides a helpful illustration for the last two steps. Our main theorem is stated as follows. Eq. (1) and the same monotonicity argument as used in the proof of Lemma 1, see also Figure 1. The average count of images of DomainNet in each class, and across all domains is approx.
On Efficiently Representing Regular Languages as RNNs
Svete, Anej, Chan, Robin Shing Moon, Cotterell, Ryan
Recent work by Hewitt et al. (2020) provides an interpretation of the empirical success of recurrent neural networks (RNNs) as language models (LMs). It shows that RNNs can efficiently represent bounded hierarchical structures that are prevalent in human language. This suggests that RNNs' success might be linked to their ability to model hierarchy. However, a closer inspection of Hewitt et al.'s (2020) construction shows that it is not inherently limited to hierarchical structures. This poses a natural question: What other classes of LMs can RNNs efficiently represent? To this end, we generalize Hewitt et al.'s (2020) construction and show that RNNs can efficiently represent a larger class of LMs than previously claimed -- specifically, those that can be represented by a pushdown automaton with a bounded stack and a specific stack update function. Altogether, the efficiency of representing this diverse class of LMs with RNN LMs suggests novel interpretations of their inductive bias.