Learning from Bad Data via Generation