While large language models (LLMs) excel at understanding and generating plain text, they are not specifically tailored to handle hierarchical text structures. Extracting the task-desired property from their natural language responses typically necessitates additional processing steps. In fact, selectively comprehending the hierarchical structure of large-scale text is pivotal to understanding its substance. Aligning LLMs more closely with the classification or regression values of specific task through prompting also remains challenging. To this end, we propose a novel framework called Tree-Based Hard Attention with Self-Motivation for Large Language Models (TEAROOM). TEAROOM incorporates a tree-based hard attention mechanism for LLMs to process hierarchically structured text inputs. By leveraging prompting, it enables a frozen LLM to selectively focus on relevant leaves in relation to the root, generating a tailored symbolic representation of their relationship. Moreover, TEAROOM comprises a self-motivation strategy for another LLM equipped with a trainable adapter and a linear layer. The selected symbolic outcomes are integrated into another prompt, along with the predictive value of the task. We iteratively feed output values back into the prompt, enabling the trainable LLM to progressively approximate the golden truth. TEAROOM outperforms existing state-of-the-art methods in experimental evaluations across three benchmark datasets, showing its effectiveness in estimating task-specific properties. Through comprehensive experiments and analysis, we have validated the ability of TEAROOM to gradually approach the underlying golden truth through multiple inferences.

In Mansfield, Ohio, 1962, police set up hidden cameras in a public bathroom to record consensual sexual activity between men. An artist named William E Jones, who was born in Ohio that same year, later found the footage online, edited out a voiceover that he described as "as illiterate and hateful a text as I have ever heard committed to film", and released the result in 2007 as a "found footage" documentary called Tearoom (US slang for a public bathroom in which men meet to have anonymous sex). The footage reveals the men involved were diverse in appearance – and presumably background – but all were wary. And with good reason: many of them were later arrested. Public bathrooms have long been a battlefield where LGBT people are targeted by the law.