Adaptive Policies for Sequential Sampling under Incomplete Information and a Cost Constraint

In this paper we consider the problem of sequential sampling from k independent statistical populations with unknown distributions. The objective is to maximize the expected outcome per period achieved over infinite horizon, under a constraint that the expected sampling cost per period does not exceed an upper bound. The introduction of a sampling cost introduces a new dimension in the standard tradeoff between experimentation and profit maximization faced in problems of control under incomplete information. The sampling cost may prohibit using populations with high mean outcomes because their sampling cost may be too high. Instead, the decision maker must identify the subset of populations with the best combination of outcome versus cost and allocate the sampling effort among them in an optimal manner. 1 From the mathematical point of view, this class of problems incorporates statistical methodologies into mathematical programming problems.