Media
Universal Captioner: Long-Tail Vision-and-Language Model Training through Content-Style Separation
Cornia, Marcella, Baraldi, Lorenzo, Fiameni, Giuseppe, Cucchiara, Rita
While captioning models have obtained compelling results in describing natural images, they still do not cover the entire long-tail distribution of real-world concepts. In this paper, we address the task of generating human-like descriptions with in-the-wild concepts by training on web-scale automatically collected datasets. To this end, we propose a model which can exploit noisy image-caption pairs while maintaining the descriptive style of traditional human-annotated datasets like COCO. Our model separates content from style through the usage of keywords and stylistic tokens, employing a single objective of prompt language modeling and being simpler than other recent proposals. Experimentally, our model consistently outperforms existing methods in terms of caption quality and capability of describing long-tail concepts, also in zero-shot settings. According to the CIDEr metric, we obtain a new state of the art on both COCO and nocaps when using external data.
Senior Data Analyst, Business Operations (Remote)
About Us The Athletic is a direct-to-consumer digital sports media company committed to helping subscribers experience storytelling in a whole new way. Founded in 2016 and headquartered in San Francisco, The Athletic has over 500 full-time employees and covers more than 250 professional sports and collegiate teams in the US, Canada and the UK. The Athletic's newsroom has produced thousands of in-depth articles along with more than 120 podcasts and premium video content. The Athletic is a remote-friendly company as we have offices in San Francisco, Los Angeles, London, and Melbourne. About the Role We are looking for a senior pricing and business operation analyst who will leverage data to guide and develop our short-term and long-term pricing strategy, and support performance driven product development and marketing budgeting processes.