In the realm of scientific exploration and technological advancement, the use of Artificial Intelligence (AI) has catalyzed breakthroughs across various scientific and technological domains. One such domain that has witnessed significant transformation is materials science, where AI-driven approaches is believed to have the potential to revolutionize the search for novel materials with desired properties: towards this aim, data quality remains key in determining reliability of AI-models. Clearly, the quality of data is a multifaceted issue, as it is linked to disparate aspects in data generation including the accuracy by which materials properties are either measured or computed by simulations, the state of knowledge and/or ability to control operating parameters during experiments, the different adopted protocols and metrological approaches etc. In this work, we focus on a few special aspects of data quality that - to the best of our knowledge - have been poorly discussed in the literature, despite their possible detrimental impact on the ability of AI-based models to serve as platforms for material discovery. One of these aspects pertains to data bias, and while it has been previously mentioned in other works [1, 2], we believe that we still lack quantitative detection and assessment tools. We therefore present here a potential quantitative approach to assess it. Other overlooked aspects, namely possible hidden variables and disparate data age, are also discussed.