Performance Analysis
ILETIA: An AI-enhanced method for individualized trigger-oocyte pickup interval estimation of progestin-primed ovarian stimulation protocol
Wu, Binjian, Li, Qian, Kuang, Zhe, Gao, Hongyuan, Liu, Xinyi, Guo, Haiyan, Chen, Qiuju, Liu, Xinyi, Jiang, Yangruizhe, Zhang, Yuqi, Zha, Jinyin, Li, Mingyu, Ren, Qiuhan, Feng, Sishuo, Zhang, Haicang, Lu, Xuefeng, Zhang, Jian
In vitro fertilization-embryo transfer (IVF-ET) stands as one of the most prevalent treatments for infertility. During an IVF-ET cycle, the time interval between trigger shot and oocyte pickup (OPU) is a pivotal period for follicular maturation, which determines mature oocytes yields and impacts the success of subsequent procedures. However, accurately predicting this interval is severely hindered by the variability of clinicians'experience that often leads to suboptimal oocyte retrieval rate. To address this challenge, we propose ILETIA, the first machine learning-based method that could predict the optimal trigger-OPU interval for patients receiving progestin-primed ovarian stimulation (PPOS) protocol. Specifically, ILETIA leverages a Transformer to learn representations from clinical tabular data, and then employs gradient-boosted trees for interval prediction. For model training and evaluating, we compiled a dataset PPOS-DS of nearly ten thousand patients receiving PPOS protocol, the largest such dataset to our knowledge. Experimental results demonstrate that our method achieves strong performance (AUROC = 0.889), outperforming both clinicians and other widely used computational models. Moreover, ILETIA also supports premature ovulation risk prediction in a specific OPU time (AUROC = 0.838). Collectively, by enabling more precise and individualized decisions, ILETIA has the potential to improve clinical outcomes and lay the foundation for future IVF-ET research.
Review for NeurIPS paper: Estimating weighted areas under the ROC curve
One contribution seems to have been in defining a surrogate functional g (line 166) that replaces the \mu(0) term in a denominator term with an arbitrary parameter c and then using a uniform convergence bound over values of c to ensure that estimation does take place even if c is replaced with its actual value of \mu(0). Another contribution seems to be in fine tuning the proof technique used to prove Proposition 5. The main contribution is a proof for obtaining generalization bound for weighted areas under the ROC curve for Lipschitz weight functions.
Review for NeurIPS paper: Estimating weighted areas under the ROC curve
This is a theoretical paper that has received relatively good reviews. However, two of the reviewers only increased their scores from 5 to 6 in order to reduce the divergence and help form a consensus (in the discussions), but neither was really convinced about the quality of the paper. Unfortunately, the highest scoring reviewer was also the least confident. I read the paper myself and I find that it has some merits --- it seems theoretically solid, but I have a slight tendency towards saying that it may be a better fit at ALT/AISTATS/COLT, and it is unclear if the NeurIPS community will benefit from knowing these results. Nevertheless, regardless of the final outcome, the authors are encouraged to improve the readability of their paper through (it is currently somewhat dense for the average reader).
Review for NeurIPS paper: Bootstrapping neural processes
Weaknesses: Given the paper's current state, I have following major comments: - The proposed method's motivation is to tackle the issue of model-data mismatch by modeling the context representation uncertainty. However the notion of the model-data mismatch is loosely defined. It would be more interesting if the paper's formulation would fomulate this problem in a principled way, e.g. the model-data mismatch problem can be framed in a more principled way, e.g. The combined objective of two models with/without bootstraps is somewhat questionable. The computation of residuals would influence a lot to the input hence the convergence of the full model.
Review for NeurIPS paper: Bootstrapping neural processes
This is an important paper on uncertainty quantification. However as the reviewers noted the main concerns are competitiveness with reespect to GPs and also an analysis (perrhaps with intuitions) of when the method underperforms would be useful. Overall, this paper might pave the way for really interesting follow-ups which will build on top of it.
Facies Classification with Copula Entropy
Facies are the type of rocks with similar characteristics given by geologists and facies classification is of very significance in geological tasks, such as formation evaluation, reservoir characterization. As the geological data accumulates, there are growing interests in facies classification with machine learning methods [1, 2, 3, 4, 5, 6, 7, 8, 9]. There are two issues with the existing works on facies classification. First, the machine learning models are built without variable selection or with only very primary method, such as cross-validation, which makes the classifiers with useless variable as inputs and therefore with low performance. Second, most of the models for facies classification are block-box, such as deep learning [5, 10, 11], Boostings or SVMs[7], which are un-interpretable to geologists. Variable selection is a common task that selects a subset from all the available variables for machine learning models. By this, the accuracy of the predictive models built with the selected variables can be improved compared with those built without selection. The traditional method for variable selection are mainly based on likelihoods, such as AIC, BIC, or accuracy, such as LASSO [12], or correlation, such as HSIC [13], distance correlation [14], and copula entropy [15]. Copula Entropy (CE) is a recently proposed rigorous mathematical concept for measuring multivariate statistical independence and is proved to be equivalent to mutual information in information theory [16].
Prompt-Based Cost-Effective Evaluation and Operation of ChatGPT as a Computer Programming Teaching Assistant
Ballestero-Ribรณ, Marc, Ortiz-Martรญnez, Daniel
The dream of achieving a student-teacher ratio of 1:1 is closer than ever thanks to the emergence of large language models (LLMs). One potential application of these models in the educational field would be to provide feedback to students in university introductory programming courses, so that a student struggling to solve a basic implementation problem could seek help from an LLM available 24/7. This article focuses on studying three aspects related to such an application. First, the performance of two well-known models, GPT-3.5T and GPT-4T, in providing feedback to students is evaluated. The empirical results showed that GPT-4T performs much better than GPT-3.5T, however, it is not yet ready for use in a real-world scenario. This is due to the possibility of generating incorrect information that potential users may not always be able to detect. Second, the article proposes a carefully designed prompt using in-context learning techniques that allows automating important parts of the evaluation process, as well as providing a lower bound for the fraction of feedbacks containing incorrect information, saving time and effort. This was possible because the resulting feedback has a programmatically analyzable structure that incorporates diagnostic information about the LLM's performance in solving the requested task. Third, the article also suggests a possible strategy for implementing a practical learning tool based on LLMs, which is rooted on the proposed prompting techniques. This strategy opens up a whole range of interesting possibilities from a pedagogical perspective.
Principal Graph Encoder Embedding and Principal Community Detection
Shen, Cencheng, Dong, Yuexiao, Priebe, Carey E., Larson, Jonathan, Trinh, Ha, Park, Youngser
In this paper, we introduce the concept of principal communities and propose a principal graph encoder embedding method that concurrently detects these communities and achieves vertex embedding. Given a graph adjacency matrix with vertex labels, the method computes a sample community score for each community, ranking them to measure community importance and estimate a set of principal communities. The method then produces a vertex embedding by retaining only the dimensions corresponding to these principal communities. Theoretically, we define the population version of the encoder embedding and the community score based on a random Bernoulli graph distribution. We prove that the population principal graph encoder embedding preserves the conditional density of the vertex labels and that the population community score successfully distinguishes the principal communities. We conduct a variety of simulations to demonstrate the finite-sample accuracy in detecting ground-truth principal communities, as well as the advantages in embedding visualization and subsequent vertex classification. The method is further applied to a set of real-world graphs, showcasing its numerical advantages, including robustness to label noise and computational scalability.
Statistical Verification of Linear Classifiers
Zhiyanov, Anton, Shklyaev, Alexander, Galatenko, Alexey, Galatenko, Vladimir, Tonevitsky, Alexander
We propose a homogeneity test closely related to the concept of linear separability between two samples. Using the test one can answer the question whether a linear classifier is merely ``random'' or effectively captures differences between two classes. We focus on establishing upper bounds for the test's \emph{p}-value when applied to two-dimensional samples. Specifically, for normally distributed samples we experimentally demonstrate that the upper bound is highly accurate. Using this bound, we evaluate classifiers designed to detect ER-positive breast cancer recurrence based on gene pair expression. Our findings confirm significance of IGFBP6 and ELOVL5 genes in this process.
Utilizing Graph Neural Networks for Effective Link Prediction in Microservice Architectures
Khodabandeh, Ghazal, Ezaz, Alireza, Babaei, Majid, Ezzati-Jivan, Naser
Managing microservice architectures in distributed systems is complex and resource intensive due to the high frequency and dynamic nature of inter service interactions. Accurate prediction of these future interactions can enhance adaptive monitoring, enabling proactive maintenance and resolution of potential performance issues before they escalate. This study introduces a Graph Neural Network GNN based approach, specifically using a Graph Attention Network GAT, for link prediction in microservice Call Graphs. Unlike social networks, where interactions tend to occur sporadically and are often less frequent, microservice Call Graphs involve highly frequent and time sensitive interactions that are essential to operational performance. Our approach leverages temporal segmentation, advanced negative sampling, and GATs attention mechanisms to model these complex interactions accurately. Using real world data, we evaluate our model across performance metrics such as AUC, Precision, Recall, and F1 Score, demonstrating its high accuracy and robustness in predicting microservice interactions. Our findings support the potential of GNNs for proactive monitoring in distributed systems, paving the way for applications in adaptive resource management and performance optimization.