Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation

Before describing our procedure for sampling deterministic FSTs, we briefly establish notation. An FST is a tuple Q, Σ, Γ, I, F,, where Q is a finite set of states, Σ is the input alphabet, Γ is the output alphabet, I Q is a set of initial states, F Q is a set of final states and Q (Σ {ϵ}) (Γ {ϵ}) Q are the transitions. We assume Σ = Γ and call it V for vocabulary. For technical reasons, we exclude the three characters [, ] and \ from the vocabulary as they are interpreted as special characters by OpenFST, which we use for constructing and representing FSTs. In addition to the shorthand for identity transitions (id), we also have shorthands for converting upper case to lower case and vice-versa (lower-to-upper, upper-to-lower). We describe our procedure to generate a deterministic FST with pseudocode in Algorithm 1.