Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding