Towards Redundancy-Free Sub-networks in Continual Learning