A Results

A.1 Central Limit Theorem for SA Statements in this part are adapted from [4, Chapter 2 and 3]. We can now state the central limit theorem. To guarantee Assumption 2 and Assumption 3, we make the following assumption. Q-learning even when equipped with neural networks [33]. The proof of this lemma is deferred to the next section.