In the absence of abundant reliable annotations for challenging tasks and contexts, how can we expand the frontier of LLM capabilities with potentially wrong answers? We focus on two research questions: (1) Can LLMs generate reliable preferences among wrong options? And if so, (2) Would alignment with such wrong-over-wrong preferences be helpful? We employ methods based on self-consistency, token probabilities, and LLM-as-a-judge to elicit wrong-over-wrong preferences, and fine-tune language models with preference optimization approaches using these synthesized preferences. Extensive experiments with seven LLMs and eight datasets demonstrate that (1) LLMs do have preliminary capability in distinguishing various shades of wrong, achieving up to 20.9% higher performance than random guess; (2) Alignment with wrong-over-wrong preferences helps LLMs to produce less wrong and sometimes even outright correct answers, while overall improving model calibration.

As the planet's largest economies struggle to forge consensus on the future of artificial intelligence, Brazil's Luiz Inacio Lula da Silva wants to ensure the developing world isn't left out of the debate. The Brazilian leader has added AI to his list of priorities for his country's presidency of the Group of 20 nations this year, seizing on the position to try to shape regulatory discussions that are raging from Europe to Asia to the United Nations, where the technology is expected to be a major theme of this week's General Assembly. Already seeking reforms to global institutions like the U.N. Security Council, Lula wants to use November's G20 leaders summit to craft a governance framework that includes the interests of Global South nations and forces AI superpowers China and the U.S. to the table, according to two people familiar with his views.

Every other day, my WhatsApp bursts with messages from friends in Brazil and abroad expressing equal parts of excitement and apprehension as Sunday's Brazilian presidential elections approach. On Wednesday, my best friend who lives in the country's capital, Brasília, texted to say she was scared of wearing red clothes to go vote this weekend because red is the color associated with the Worker's Party of former President Luiz Inácio Lula da Silva. Lula, the current front-runner, has a real, if slim, chance to beat far-right incumbent President Jair Bolsonaro in the first round by getting more than 50 percent of valid votes. "The mood is terrible," she wrote, later adding that in the last 48 hours, four instances of political violence had been recorded across the country. My friend's worries are justified.