How Wish A/B tests percentiles

And Jᵢ is the number of page loading times lower than the sample P95 for user i in one bucket. Note that all of these summary statistics are aggregated at the experiment bucket level, making the data pipeline much simpler. With regard to the tuning parameter, we found 0.25 led to satisfactory results for percentiles like P50 and P95. It may require more careful tuning when it comes to extreme percentiles like P99 or P99.9.