Efficient Serving of LLM Applications with Probabilistic Demand Modeling