Anatomy Might Be All You Need: Forecasting What to Do During Surgery