Codebooks -- documents that operationalize constructs and outline annotation procedures -- are used almost universally by social scientists when coding unstructured political texts. Recently, to reduce manual annotation costs, political scientists have looked to generative large language models (LLMs) to label and analyze text data. However, previous work using LLMs for classification has implicitly relied on the universal label assumption -- correct classification of documents is possible using only a class label or minimal definition and the information that the LLM inductively learns during its pre-training. In contrast, we argue that political scientists who care about valid measurement should instead make a codebook-construct label assumption -- an LLM should follow the definition and exclusion criteria of a construct/label provided in a codebook. In this work, we collect and curate three political science datasets and their original codebooks and conduct a set of experiments to understand whether LLMs comply with codebook instructions, whether rewriting codebooks improves performance, and whether instruction-tuning LLMs on codebook-document-label tuples improves performance over zero-shot classification. Using Mistral 7B Instruct as our LLM, we find re-structuring the original codebooks gives modest gains in zero-shot performance but the model still struggles to comply with the constraints of the codebooks. Optimistically, instruction-tuning Mistral on one of our datasets gives significant gains over zero-shot inference (0.76 versus 0.53 micro F1). We hope our conceptualization of the codebook-specific task, assumptions, and instruction-tuning pipeline as well our semi-structured LLM codebook format will help political scientists readily adapt to the LLM era.

QUETTA, PAKISTAN – The family of a driver who was killed alongside Taliban chief Mullah Akhtar Mansour in a U.S. drone strike in Pakistan has filed a case against U.S. officials, seeking to press murder charges, police said Sunday. Mansour had entered Pakistan from Iran using a false name and fake Pakistani identity documents on May 21, when his car was targeted by a U.S. drone. The driver, who was also killed, was later identified as Mohammed Azam. The police filed a case on behalf of Azam's family, police official Abdul Wakil Mengal said. It was not immediately clear what legal avenues the family can realistically pursue.

A Pakistani police officer and paramedics stand beside two dead bodies reportedly killed in a U.S. drone strike in the Ahmad Wal area in Baluchistan province, Pakistan, at a hopsital in Quetta, Pakistan, Sunday, May 22, 2016. A senior commander of the Afghan Taliban confirmed on Sunday that the extremist group's leader, Mullah Mohammad Akhtar Mansour, had been killed in the strike.

If you look at the record, America has been concerned about not alienating Pakistan for the last 10 years or more. But the Afghans are clear about this, and the Americans know that one of the reasons the Taliban insurgency has been resilient is that they have sanctuaries across the border in Pakistan, in Balochistan. I think the Americans were thinking that Pakistan would probably deliver the Taliban, because they had influence over them. Pakistan claims they have control, but then they say, "well, we don't have full control, we can bring them to the table but we can't make them do things unless you promise something concrete."

A U.S. drone strike targeting the Afghan Taliban's commander, Mullah Akhtar Mansour, led to the leadership council meeting Sunday to discuss succession, two Taliban sources told Reuters. This has been the strongest indication by the group of its acceptance of Mansour's death. Pakistani local residents gather around a destroyed vehicle hit by a drone strike, in which Afghan Taliban Chief Mullah Akhtar Mansour was believed to be travelling, in the remote town of Ahmad Wal in Balochistan, around 100 miles west of Quetta, May 21, 2016. President Barack Obama, according to ABC, has released a statement confirming Mansour's death. In the statement, Obama called Mansour's death "an important milestone in our longstanding effort to bring peace and prosperity to Afghanistan."

Balochistan, PAKISTAN/KABUL/WASHINGTON – Taliban supremo Mullah Akhtar Mansour was killed in a U.S. drone attack in Pakistan, senior militant sources told AFP Sunday, adding that an insurgent assembly was underway to decide on his successor. Saturday's bombing raid, the first known U.S. assault on a top Afghan Taliban leader on Pakistani soil, marks a major blow to the militant movement, which saw a new resurgence under Mansour. The elimination of Mansour, who rose to the rank of leader nine months earlier after a bitter internal leadership struggle, could also scupper any immediate prospect of peace talks. "I can say with good authority that Mullah Mansour is no more," a senior Taliban source told AFP. Mansour's death, which risks igniting new succession battles within the fractious group, was confirmed by two other senior figures who said its top leaders were gathering in Quetta to name their future chief.

Afghan Taliban leader Mullah Akhtar Mansour has been killed, Afghanistan's National Directorate of Security confirmed Sunday afternoon after several hours of uncertainty. The Afghan intelligence agency said Mansour, who was officially named the group's leader last year, was killed in an "airstrike" in a remote area in Balochistan in southwestern Pakistan Saturday. Prior to that, Afghan President Ashraf Ghani also confirmed that Mansour had been targeted in a drone strike by the United States. "Mullah Akhtar Mansour refused to answer repeated calls by the people and Government of Afghanistan to end the war and violence in the country. While sheltering himself in hideouts outside Afghanistan, he was also involved in deception, concealment of facts, maiming and killing innocent Afghans, terrorism, intimidation, drug smuggling as well as obstruction of development and progress in Afghanistan as he obstinately insisted on continuing the war," the Afghan president's office said in a statement Sunday.