With the popularity of automatic code generation tools, such as Copilot, the study of the potential hazards of these tools is gaining importance. In this work, we explore the social bias problem in pre-trained code generation models. We propose a new paradigm to construct code prompts and successfully uncover social biases in code generation models. To quantify the severity of social biases in generated code, we develop a dataset along with three metrics to evaluate the overall social bias and fine-grained unfairness across different demographics. Experimental results on three pre-trained code generation models (Codex, InCoder, and CodeGen) with varying sizes, reveal severe social biases. Moreover, we conduct analysis to provide useful insights for further choice of code generation models with low social bias1.

This material may not be published, broadcast, rewritten, or redistributed. Quotes displayed in real-time or delayed by at least 15 minutes. Market data provided by Factset . Powered and implemented by FactSet Digital Solutions . Mutual Fund and ETF data provided by LSEG .

For both15 modalities, essential information such as geographic coordinates, elevation, and precise time-stamps16 is available. In this section, we provide a comprehensive explanation of the encoding process for each17 feature and conclude by presenting the hyperparameters of the model.18 For each time point, we have access to the following time19 features: The year, the month, the day, the hour and the minute at which the measurement was made.20 We use a cyclical embedding to encode these time features discarding the year. For a time feature x,21 its corresponding embedding can be expressed as:22 sin 2πx ω(x),cos 2πx ω(x) (1) Submitted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023).