A Proof of Proposition

A.1 Problem Definition We consider two binary classification tasks, with Y The task labels are drawn from two different probabilities. For simplicity, we assume the probability to sample the two label value is balanced, i.e., P (Y = 1) = P (Y = 1) = 0.5. Our conclusion could be extended to unbalanced distribution. In this paper, we mainly study the spurious correlation between task labels. We first consider the setting that we're given infinite samples. If we assume there's no traditional factor-label spurious correlation in single task learning, the bayes optimal classifier will only take each task's causal factor as feature, and assign zero weights to non-causal factors.