r/MachineLearning - [D] Finding optimal threshold for F1 evaluation

Train the model k time based on a k-fold cross validation, predict the out of folds, concatenate them, and find the optimal threshold for it. This currently works the best, but I can also see that I can either get lucky on the threshold, or not and sometimes the optimal test threshold is quite far away. Also, the different models have sometimes quite different optimal thresholds on their own folds. There seems to be some form of "winner's curse" [1].