Why is the winner the best?
Eisenmann, Matthias, Reinke, Annika, Weru, Vivienn, Tizabi, Minu Dietlinde, Isensee, Fabian, Adler, Tim J., Ali, Sharib, Andrearczyk, Vincent, Aubreville, Marc, Baid, Ujjwal, Bakas, Spyridon, Balu, Niranjan, Bano, Sophia, Bernal, Jorge, Bodenstedt, Sebastian, Casella, Alessandro, Cheplygina, Veronika, Daum, Marie, de Bruijne, Marleen, Depeursinge, Adrien, Dorent, Reuben, Egger, Jan, Ellis, David G., Engelhardt, Sandy, Ganz, Melanie, Ghatwary, Noha, Girard, Gabriel, Godau, Patrick, Gupta, Anubha, Hansen, Lasse, Harada, Kanako, Heinrich, Mattias, Heller, Nicholas, Hering, Alessa, Huaulmé, Arnaud, Jannin, Pierre, Kavur, Ali Emre, Kodym, Oldřich, Kozubek, Michal, Li, Jianning, Li, Hongwei, Ma, Jun, Martín-Isla, Carlos, Menze, Bjoern, Noble, Alison, Oreiller, Valentin, Padoy, Nicolas, Pati, Sarthak, Payette, Kelly, Rädsch, Tim, Rafael-Patiño, Jonathan, Bawa, Vivek Singh, Speidel, Stefanie, Sudre, Carole H., van Wijnen, Kimberlin, Wagner, Martin, Wei, Donglai, Yamlahi, Amine, Yap, Moi Hoon, Yuan, Chun, Zenk, Maximilian, Zia, Aneeq, Zimmerer, David, Aydogan, Dogu Baran, Bhattarai, Binod, Bloch, Louise, Brüngel, Raphael, Cho, Jihoon, Choi, Chanyeol, Dou, Qi, Ezhov, Ivan, Friedrich, Christoph M., Fuller, Clifton, Gaire, Rebati Raman, Galdran, Adrian, Faura, Álvaro García, Grammatikopoulou, Maria, Hong, SeulGi, Jahanifar, Mostafa, Jang, Ikbeom, Kadkhodamohammadi, Abdolrahim, Kang, Inha, Kofler, Florian, Kondo, Satoshi, Kuijf, Hugo, Li, Mingxing, Luu, Minh Huan, Martinčič, Tomaž, Morais, Pedro, Naser, Mohamed A., Oliveira, Bruno, Owen, David, Pang, Subeen, Park, Jinah, Park, Sung-Hong, Płotka, Szymon, Puybareau, Elodie, Rajpoot, Nasir, Ryu, Kanghyun, Saeed, Numan, Shephard, Adam, Shi, Pengcheng, Štepec, Dejan, Subedi, Ronast, Tochon, Guillaume, Torres, Helena R., Urien, Helene, Vilaça, João L., Wahid, Kareem Abdul, Wang, Haojie, Wang, Jiacheng, Wang, Liansheng, Wang, Xiyue, Wiestler, Benedikt, Wodzinski, Marek, Xia, Fangfang, Xie, Juanying, Xiong, Zhiwei, Yang, Sen, Yang, Yanwu, Zhao, Zixuan, Maier-Hein, Klaus, Jäger, Paul F., Kopp-Schneider, Annette, Maier-Hein, Lena
–arXiv.org Artificial Intelligence
International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.
arXiv.org Artificial Intelligence
Mar-30-2023
- Country:
- Oceania > Australia
- South Australia > Adelaide (0.04)
- North America
- United States
- Hawaii (0.04)
- Texas > Harris County
- Houston (0.04)
- Minnesota > Hennepin County
- Minneapolis (0.14)
- Pennsylvania > Philadelphia County
- Philadelphia (0.14)
- Nebraska > Douglas County
- Omaha (0.04)
- California > Santa Clara County
- Sunnyvale (0.04)
- Illinois > Cook County
- Washington > King County
- Seattle (0.14)
- Massachusetts
- Suffolk County > Boston (0.04)
- Middlesex County > Cambridge (0.04)
- Canada > Ontario
- Toronto (0.14)
- United States
- Europe
- Austria > Vienna (0.14)
- Norway (0.04)
- Belgium > Flanders (0.04)
- Slovenia > Central Slovenia
- Municipality of Ljubljana > Ljubljana (0.04)
- Germany
- Saxony > Dresden (0.04)
- North Rhine-Westphalia
- Arnsberg Region > Dortmund (0.04)
- Upper Bavaria > Munich (0.04)
- Bavaria > Upper Bavaria
- Munich (0.05)
- Ingolstadt (0.04)
- Baden-Württemberg > Karlsruhe Region
- Heidelberg (0.05)
- Netherlands
- North Holland > Amsterdam (0.04)
- South Holland > Rotterdam (0.04)
- Gelderland > Nijmegen (0.04)
- Denmark > Capital Region
- Copenhagen (0.04)
- Switzerland
- United Kingdom > England
- Oxfordshire > Oxford (0.14)
- Greater London > London (0.05)
- West Yorkshire > Leeds (0.04)
- West Midlands > Coventry (0.04)
- Greater Manchester > Manchester (0.04)
- France
- Île-de-France > Paris
- Paris (0.04)
- Grand Est > Bas-Rhin
- Strasbourg (0.04)
- Brittany > Ille-et-Vilaine
- Rennes (0.04)
- Île-de-France > Paris
- Czechia > South Moravian Region
- Brno (0.04)
- Italy
- Spain > Catalonia
- Barcelona Province > Barcelona (0.04)
- Finland > Northern Savo
- Kuopio (0.04)
- Portugal > Braga
- Braga (0.04)
- Poland > Lesser Poland Province
- Kraków (0.14)
- Asia
- Nepal (0.04)
- South Korea
- Middle East > UAE
- Abu Dhabi Emirate > Abu Dhabi (0.14)
- Japan
- Hokkaidō (0.04)
- Honshū > Kantō
- Tokyo Metropolis Prefecture > Tokyo (0.14)
- India > NCT
- Delhi (0.04)
- China
- Hong Kong (0.04)
- Guangdong Province > Shenzhen (0.04)
- Fujian Province > Xiamen (0.04)
- Sichuan Province > Chengdu (0.04)
- Shaanxi Province > Xi'an (0.04)
- Heilongjiang Province > Harbin (0.04)
- Anhui Province > Hefei (0.04)
- Africa > Middle East
- Egypt > Cairo Governorate > Cairo (0.04)
- Oceania > Australia
- Genre:
- Research Report
- Experimental Study (1.00)
- New Finding (0.88)
- Research Report
- Industry:
- Education (1.00)
- Government > Regional Government (0.68)
- Health & Medicine
- Nuclear Medicine (1.00)
- Health Care Technology (1.00)
- Diagnostic Medicine > Imaging (1.00)
- Health Care Providers & Services (0.94)
- Therapeutic Area
- Technology: