A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality