Decision Making Over Combinatorially-Structured Domains