Optimization for Amortized Inverse Problems