Combining Flow Matching and Transformers for Efficient Solution of Bayesian Inverse Problems