Metadata Representations for Queryable ML Model Zoos

The potential of model zoos is currently hindered by the lack of a structured, queryable metadata format. Current Machine learning (ML) practitioners and organizations repositories include a wide range of information, in a form are building model zoos of pre-trained of a model card (Mitchell et al., 2019), but such information models, containing metadata describing properties is mostly for human consumption, making it hard for automatic of the ML models and datasets that are useful extension or management. At the same time, the level for reporting, auditing, reproducibility, and interpretability of the detail remains coarse-grained: for instance, Amazon purposes. The metatada is currently SageMaker, AzureML, MLflow (Zaharia et al., 2018) do not not standardised; its expressivity is limited; and require mandatory reporting of the related metadata, except there is no interoperable way to store and query for model name and version. Practitioners have to search on it. Consequently, model search, reuse, comparison, external websites for further metadata information such as and composition are hindered. In this paper, the data instances, and they even have to evaluate the model we advocate for standardized ML model metadata at hand in order to assess its performance.