Embracing the chaos: analysis and diagnosis of numerical instability in variational flows