ROUGE-N scores are the overlapping of n-grams [2] between the generated and referencesequence. Those scores are then averaged overthe whole corpus toreach anoverall quality. For both proposed MMMMI attack methods, shadow models are indispensable. The first hidden layer in the attack model has 256 units and the second hidden layer has20units, bothactivatedbyReLU function. We used resnet-LSTM architecture as the target model architecture.

Compared with the existing membership inference against machine learning classifiers, we focus on the problem that the input and output of the multi-modal models are in different modalities, such as image captioning.

Neural architecture search (NAS) has demonstrated impressive performance in automatically designing high-performance neural networks. The power ofdeep neural networks is to be unleashed for analyzing a large volume of data (e.g.

We investigate whether post-hoc model explanations are effective for diagnosing model errors-model debugging. In response to the challenge of explaining a model's prediction, a vast array of explanation methods have been proposed. Despite increasing use, it is unclear if they are effective. To start, we categorizebugs,based on their source, into: data, model, and test-timecontamination bugs.