Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning