Goto

Collaborating Authors

 Technology


e8dbeb1c947a30576c699e7f5c73d3e3-Paper-Conference.pdf

Neural Information Processing Systems

However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables andstrictly Gaussian posterior approximations.








7298332f04ac004a0ca44cc69ecf6f6b-AuthorFeedback.pdf

Neural Information Processing Systems

Withthesechallenges,thedatasetisalreadydifficult,andwebelievethisisagood14 initial step to tackle the problem of combining natural language and test cases in a more complex setting than any15 previouswork.



ExploringtheLimitsofDomain-AdaptiveTrainingfor DetoxifyingLarge-ScaleLanguageModels

Neural Information Processing Systems

Wethen comprehensively study detoxifying LMswithparameter sizesranging from126Mupto530B(3 largerthanGPT3), a scale that has never been studied before. We find thati) large LMs have similar toxicity levels as smaller ones given the same pre-training corpus, and ii) large LMs require more endeavor to unlearn the toxic content seen at pretraining. Wealso explore parameter-efficient training methods fordetoxification.