Sharing Key Semantics in Transformer Makes Efficient Image Restoration