Sarne, David
Generating Effective Ensembles for Sentiment Analysis
Etelis, Itay, Rosenfeld, Avi, Weinberg, Abraham Itzhak, Sarne, David
In recent years, transformer models have revolutionized Natural Language Processing (NLP), achieving exceptional results across various tasks, including Sentiment Analysis (SA). As such, current state-of-the-art approaches for SA predominantly rely on transformer models alone, achieving impressive accuracy levels on benchmark datasets. In this paper, we show that the key for further improving the accuracy of such ensembles for SA is to include not only transformers, but also traditional NLP models, despite the inferiority of the latter compared to transformer models. However, as we empirically show, this necessitates a change in how the ensemble is constructed, specifically relying on the Hierarchical Ensemble Construction (HEC) algorithm we present. Our empirical studies across eight canonical SA datasets reveal that ensembles incorporating a mix of model types, structured via HEC, significantly outperform traditional ensembles. Finally, we provide a comparative analysis of the performance of the HEC and GPT-4, demonstrating that while GPT-4 closely approaches state-of-the-art SA methods, it remains outperformed by our proposed ensemble strategy.
Understanding Over Participation in Simple Contests
Levy, Priel (Bar Ilan University) | Sarne, David (Bar Ilan University)
One key motivation for using contests in real-life is the substantial evidence reported in empirical contest-design literature for people's tendency to act more competitively in contests than predicted by the Nash Equilibrium. This phenomenon has been traditionally explained by people's eagerness to win and maximize their relative (rather than absolute) payoffs. In this paper we make use of "simple contests," where contestants only need to strategize on whether to participate in the contest or not, as an infrastructure for studying whether indeed more effort is exerted in contests due to competitiveness, or perhaps this can be attributed to other factors that hold also in non-competitive settings. The experimental methodology we use compares contestants' participation decisions in eight contest settings differing in the nature of the contest used, the number of contestants used and the theoretical participation predictions to those obtained (whenever applicable) by subjects facing equivalent non-competitive decision situations in the form of a lottery. We show that indeed people tend to over-participate in contests compared to the theoretical predictions, yet the same phenomenon holds (to a similar extent) also in the equivalent non-competitive settings. Meaning that many of the contests used nowadays as a means for inducing extra human effort, that are often complex to organize and manage, can be replaced by a simpler non-competitive mechanism that uses probabilistic prizes.
Strategic Signaling and Free Information Disclosure in Auctions
Alkoby, Shani (Bar-Ilan University) | Sarne, David (Bar-Ilan University) | Milchtaich, Igal (Bar-Ilan University)
With the increasing interest in the role information providers play in multi-agent systems, much effort has been dedicated to analyzing strategic information disclosure and signaling by such agents. This paper analyzes the problem in the context of auctions (specifically for second-price auctions). It provides an equilibrium analysis to the case where the information provider can use signaling according to some pre-committed scheme before introducing its regular (costly) information selling offering. The signal provided, publicly discloses (for free) some of the information held by the information provider. Providing the signaling is thus somehow counter intuitive as the information provider ultimately attempts to maximize her gain from selling the information she holds. Still, we show that such signaling capability can be highly beneficial for the information provider and even improve social welfare. Furthermore, the examples provided demonstrate various possible other beneficial behaviors available to the different players as well as to a market designer, such as paying the information provider to leave the system or commit to a specific signaling scheme. Finally, the paper provides an extension of the underlying model, related to the use of mixed signaling strategies.
Nurturing Group-Beneficial Information-Gathering Behaviors Through Above-Threshold Criteria Setting
Rochlin, Igor (The College of Management Academic Studies) | Sarne, David (Bar-Ilan University) | Bremer, Maytal (The College of Management Academic Studies) | Grynhaus, Ben (The College of Management Academic Studies)
This paper studies a criteria-based mechanism for nurturing and enhancing agents' group-benefiting individual efforts whenever the agents are self-interested. The idea is that only those agents that meet the criteria get to benefit from the group effort, giving an incentive to contribute even when it is otherwise individually irrational. Specifically, the paper provides a comprehensive equilibrium analysis of a threshold-based criteria mechanism for the common cooperative information gathering application, where the criteria is set such that only those whose contribution to the group is above some pre-specified threshold can benefit from the contributions of others. The analysis results in a closed form solution for the strategies to be used in equilibrium and facilitates the numerical investigation of different model properties as well as a comparison to the dual mechanism according to only an agent whose contribution is below the specified threshold gets to benefit from the contributions of others. One important contribution enabled through the analysis provided is in showing that, counter-intuitively, for some settings the use of the above-threshold criteria is outperformed by the use of the below-threshold criteria as far as collective and individual performance is concerned.
The Benefit in Free Information Disclosure When Selling Information to People
Alkoby, Shani (Bar-Ilan University) | Sarne, David (Bar-Ilan University)
This paper studies the benefit for information providers in free public information disclosure in settings where the prospective information buyers are people. The underlying model, which applies to numerous real-life situations, considers a standard decision making setting where the decision maker is uncertain about the outcomes of her decision. The information provider can fully disambiguate this uncertainty and wish to maximize her profit from selling such information. We use a series of AMT-based experiments with people to test the benefit for the information provider from reducing some of the uncertainty associated with the decision maker's problem, for free. Free information disclosure of this kind can be proved to be ineffective when the buyer is a fully-rational agent. Yet, when it comes to people we manage to demonstrate that a substantial improvement in the information provider's profit can be achieved with such an approach. The analysis of the results reveals that the primary reason for this phenomena is people's failure to consider the strategic nature of the interaction with the information provider. Peoples' inability to properly calculate the value of information is found to be secondary in its influence.
Extending Workers' Attention Span Through Dummy Events
Elmalech, Avshalom (Harvard University) | Sarne, David (Bar Ilan University) | David, Esther (Ashkelon Academic College) | Hajaj, Chen (Vanderbilt University)
This paper studies a new paradigm for improving the attention span of workers in tasks that heavily rely on user's attention to the occurrence of rare events. Such tasks are highly common, ranging from crime monitoring to controlling autonomous complex machines, and many of them are ideal for crowdsourcing.ย The underlying idea in our approach is to dynamically augment the task with some dummy (artificial) events at different times throughout the task, rewarding the worker upon identifying and reporting them.ย This, as an alternative to the traditional approach of exclusively relying on rewarding the worker for successfully identifying the event of interest itself.ย We propose three methods for timing the dummy events throughout the task. Two of these methods are static and determine the timing of the dummy events at random or uniformly throughout the task. The third method is dynamic and uses the identification (or misidentification) of dummy events as a signal for the worker's attention to the task, adjusting the rate of dummy events generation accordingly. We use extensive experimentation to compare the methods with the traditional approach of inducing attention through rewarding the identification of the event of interest and within the three. The analysis of the results indicates that with the use of dummy events a substantially more favorable tradeoff between the detection (of the event of interest) probability and the expected expense can be achieved, and that among the three proposed method the one that decides on dummy events on the fly is (by far) the best.
Intelligent Advice Provisioning for Repeated Interaction
Levy, Priel (Bar Ilan University) | Sarne, David (Bar Ilan University)
This paper studies two suboptimal advice provisioning methods ("advisors") as an alternative to providing optimal advice in repeated advising settings. Providing users with suboptimal advice has been reported to be highly advantageous whenever the optimal advice is non-intuitive, hence might not be accepted by the user. Alas, prior methods that rely on suboptimal advice generation were designed primarily for a single-shot advice provisioning setting, hence their performance in repeated settings is questionable. Our methods, on the other hand, are tailored to the repeated interaction case. Comprehensive evaluation of the proposed methods, involving hundreds of human participants, reveals that both methods meet their primary design goal (either an increased user profit or an increased user satisfaction from the advisor), while performing at least as good with the alternative goal, compared to having people perform with: (a) no advisor at all; (b) an advisor providing the theoretic-optimal advice; and (c) an effective suboptimal-advice-based advisor designed for the non-repeated variant of our experimental framework.
Strategy-Proof and Efficient Kidney Exchange Using a Credit Mechanism
Hajaj, Chen (Bar-Ilan University) | Dickerson, John P. (Carnegie Mellon University) | Hassidim, Avinatan (Bar-Ilan University) | Sandholm, Tuomas (Carnegie Mellon University) | Sarne, David (Bar-Ilan University)
We present a credit-based matching mechanism for dynamic barter markets โ and kidney exchange in particular โ that is both strategy proof and efficient, that is, it guarantees truthful disclosure of donor-patient pairs from the transplant centers and results in the maximum global matching. Furthermore, the mechanism is individually rational in the sense that, in the long run, it guarantees each transplant center more matches than the center could have achieved alone. The mechanism does not require assumptions about the underlying distribution of compatibility graphs โ a nuance that has previously produced conflicting results in other aspects of theoretical kidney exchange. Our results apply not only to matching via 2-cycles: the matchings can also include cycles of any length and altruist-initiated chains, which is important at least in kidney exchanges. The mechanism can also be adjusted to guarantee immediate individual rationality at the expense of economic efficiency, while preserving strategy proofness via the credits. This circumvents a well-known impossibility result in static kidney exchange concerning the existence of an individually rational, strategy-proof, and maximal mechanism. We show empirically that the mechanism results in significant gains on data from a national kidney exchange that includes 59% of all US transplant centers.
When Suboptimal Rules
Elmalech, Avshalom (Bar Ilan University) | Sarne, David (Bar Ilan University) | Rosenfeld, Avi (Jerusalem College of Technology) | Erez, Eden Shalom (Independent Researcher)
This paper represents a paradigm shift in what advice agents should provide people. Contrary to what was previously thought, we empirically show that agents that dispense optimal advice will not necessary facilitate the best improvement in people's strategies. Instead, we claim that agents should at times suboptimally advise. We provide results demonstrating the effectiveness of a suboptimal advising approach in extensive experiments in two canonical mixed agent-human advice-giving domains. Our proposed guideline for suboptimal advising is to rely on the level of intuitiveness of the optimal advice as a measure for how much the suboptimal advice presented to the user should drift from the optimal value.
Can Agent Development Affect Developer's Strategy?
Elmalech, Avshalom (Bar Ilan University) | Sarne, David (Bar Ilan University) | Agmon, Noa (Bar Ilan University)
Peer Designed Agents (PDAs), computer agents developed by non-experts, is an emerging technology, widely advocated in recent literature for the purpose of replacing people in simulations and investigating human behavior. Its main premise is that strategies programmed into these agents reliably reflect, to some extent, the behavior used by their programmers in real life. In this paper we show that PDA development has an important side effect that has not been addressed to date -- the process that merely attempts to capture one's strategy is also likely to affect the developer's strategy. The phenomenon is demonstrated experimentally, using several performance measures. This result has many implications concerning the appropriate design of PDA-based simulations, and the validity of using PDAs for studying individual decision making. Furthermore, we obtain that PDA development actually improved the developer's strategy according to all performance measures. Therefore, PDA development can be suggested as a means for improving people's problem solving skills.