Multi-Granularity Guided Fusion-in-Decoder