Bridging the Divide: End-to-End Sequence-Graph Learning