Rethinking Encoder-Decoder Flow Through Shared Structures