Query lower bounds for log-concave sampling