The reversal curse--a language model's (LM) inability to infer an unseen fact "B is A " from a learned fact "A is B"--is widely considered a fundamental limitation. We show that this is not an inherent failure but an artifact of how models encode knowledge. By training LMs from scratch on a synthetic dataset of relational knowledge graphs, we demonstrate that bilinear relational structure emerges in their hidden representations. Crucially, we also find that this bilinear structure plays a key role in consistent model editing. When a fact is updated in a LM with this structure, the edit correctly propagates to its reverse and other logically dependent facts. In contrast, models lacking this representation not only suffer from the reversal curse but also fail to generalize edits, further introducing logical inconsistencies. Our results establish that training on a relational knowledge dataset induces the emergence of bilinear internal representations, which in turn enable LMs to behave in a logically consistent manner after editing. This implies that the success of model editing depends critically not just on editing algorithms but on the underlying representational geometry of the knowledge being modified. Language models (LMs) have become powerful tools for knowledge-intensive tasks, yet their reasoning capabilities often fall short of human-level logical consistency (Berglund et al., 2024; Allen-Zhu & Li, 2025); a prominent example is the reversal curse: a model trained on "A is the parent of B" frequently fails to infer the reverse fact, "B is the child of A." This failure suggests that LMs learn shallow, directional associations rather than robust, symmetrical relationships, undermining their reliability. Ensuring logical consistency is particularly challenging in model editing, which seeks to update factual knowledge in a trained model without costly retraining from scratch.