Institutional review boards (IRBs) play a crucial role in ensuring the ethical conduct of human subjects research, but face challenges including inconsistency, delays, and inefficiencies. We propose the development and implementation of application-specific large language models (LLMs) to facilitate IRB review processes. These IRB-specific LLMs would be fine-tuned on IRB-specific literature and institutional datasets, and equipped with retrieval capabilities to access up-to-date, context-relevant information. We outline potential applications, including pre-review screening, preliminary analysis, consistency checking, and decision support. While addressing concerns about accuracy, context sensitivity, and human oversight, we acknowledge remaining challenges such as over-reliance on AI and the need for transparency. By enhancing the efficiency and quality of ethical review while maintaining human judgment in critical decisions, IRB-specific LLMs offer a promising tool to improve research oversight. We call for pilot studies to evaluate the feasibility and impact of this approach.

While ethical challenges are widely discussed in HCI, far less is reported about the ethical processes that researchers routinely navigate. We reflect on a multispecies project that negotiated an especially complex ethical approval process. Cat Royale was an artist-led exploration of creating an artwork to engage audiences in exploring trust in autonomous systems. The artwork took the form of a robot that played with three cats. Gaining ethical approval required an extensive dialogue with three Institutional Review Boards (IRBs) covering computer science, veterinary science and animal welfare, raising tensions around the welfare of the cats, perceived benefits and appropriate methods, and reputational risk to the University. To reveal these tensions we introduce beneficiary-epistemology space, that makes explicit who benefits from research (humans or animals) and underlying epistemologies. Positioning projects and IRBs in this space can help clarify tensions and highlight opportunities to recruit additional expertise.

Transfer learning on edge is challenging due to on-device limited resources. Existing work addresses this issue by training a subset of parameters or adding model patches. Developed with inference in mind, Inverted Residual Blocks (IRBs) split a convolutional layer into depthwise and pointwise convolutions, leading to more stacking layers, e.g., convolution, normalization, and activation layers. Though they are efficient for inference, IRBs require that additional activation maps are stored in memory for training weights for convolution layers and scales for normalization layers. As a result, their high memory cost prohibits training IRBs on resource-limited edge devices, and making them unsuitable in the context of transfer learning. To address this issue, we present MobileTL, a memory and computationally efficient on-device transfer learning method for models built with IRBs. MobileTL trains the shifts for internal normalization layers to avoid storing activation maps for the backward pass. Also, MobileTL approximates the backward computation of the activation layer (e.g., Hard-Swish and ReLU6) as a signed function which enables storing a binary mask instead of activation maps for the backward pass. MobileTL fine-tunes a few top blocks (close to output) rather than propagating the gradient through the whole network to reduce the computation cost. Our method reduces memory usage by 46% and 53% for MobileNetV2 and V3 IRBs, respectively. For MobileNetV3, we observe a 36% reduction in floating-point operations (FLOPs) when fine-tuning 5 blocks, while only incurring a 0.6% accuracy reduction on CIFAR10. Extensive experiments on multiple datasets demonstrate that our method is Pareto-optimal (best accuracy under given hardware constraints) compared to prior work in transfer learning for edge devices.

To understand why this matters, we need to examine the traditional ethical gateway, the Institutional Review Board (IRB), which is typically synonymous with'research ethics' in the United States. The primary difference between the ESR and IRBs is that IRBs are expressly disallowed from addressing concerns about harms to society and focus instead on harms to human participants in research only. A large proportion of AI research does not directly engage human subjects, which means many IRBs decline to review AI research, and a significant number of projects never undergoes any ethical review.

Conversations around AI and ethics may have started as a preoccupation of activists and academics, but now -- prompted by the increasing frequency of headlines of biased algorithms, black box models, and privacy violations -- boards, C-suites, and data and AI leaders have realized it's an issue for which they need a strategic approach. A solution is hiding in plain sight. Other industries have already found ways to deal with complex ethical quandaries quickly, effectively, and in a way that can be easily replicated. Instead of trying to reinvent this process, companies need to adopt and customize one of health care's greatest inventions: the Institutional Review Board, or IRB. Most discussions of AI ethics follow the same flawed formula, consisting of three moves, each of which is problematic from the perspective of an organization that wants to mitigate the ethical risks associated with AI. Here's how these conversations tend to go. First, companies move to identify AI ethics with "fairness" in AI, or sometimes more generally, "fairness, equity, and inclusion."

The disproportionate impact of COVID-19 on certain populations, such as Black, Latinx, and Indigenous populations in the United States, has focused attention on inequalities in health and on the need to increase enrollment of racial and ethnic minorities and other underrepresented groups in biomedical research ([ 1 ][1]). Yet too often, in the United States and globally, participant enrollment in research has not reflected the demographic composition of the general population, those affected by the health conditions being studied, or those for whom the investigational product is intended ([ 2 ][2]), with racial and ethnic minorities and the young and the elderly, among others, being consistently underrepresented ([ 3 ][3]). Underlying causes for this underrepresentation have been described ([ 4 ][4], [ 5 ][5]), but change has been slow. Notwithstanding the roles of other stakeholders in addressing this issue, we maintain that the specific value of institutional review boards (IRBs) and research ethics committees (RECs) in promoting diversity has been underrecognized and their authority underutilized. Here, we substantiate the role of and outline practical steps for the IRB and REC (hereafter “IRB”) to help achieve greater diversity in clinical research. The appropriate inclusion of diverse populations in clinical research is necessary if we are to understand how biological variability and social determinants of health contribute to disease prevalence, transmission, course, experience of illness, and treatment outcome. The inclusion of understudied and underserved groups informs clinical decision-making and health policy and can serve efforts to address mistrust of research and health care ([ 6 ][6], [ 7 ][7]). Responsibility to the goals of diversity lies with all stakeholders in the clinical research enterprise ([ 6 ][6]), and a commitment to diversity, individually and collaboratively, by research sponsors, funders, academic institutions, contract research organizations, study sites, investigators, and IRBs is necessary. Most regulated clinical research undergoes obligate review and approval by an IRB. IRBs are charged with safeguarding the rights and well-being of human participants in accordance with the foundational tenets of respect for persons, beneficence, and justice, as described in the Belmont Report ([ 8 ][8]). An IRB's ethical responsibilities with regard to diversity derive from these and other principles, guidelines, and standards ([ 9 ][9], [ 10 ][10]). The discussion of justice in Belmont cites “moral requirements that there be fair procedures and outcomes in the selection of research subjects.” As Belmont and other codes of ethics emerged from a historical backdrop of abuse and injustice in research, “fair procedures” have been applied by IRBs largely (and, we believe, too narrowly) to ensure that subjects are not exploited and enrolled as a matter of convenience. The idea of justice within the Belmont Report also includes the notion of access to the benefits of research (i.e., knowledge gained); this has direct implications for populations that have been understudied, whether incidentally or systematically. Subject selection cannot be equitable, and the requirements of justice cannot be met, when there is de facto exclusion of understudied populations. This notion of justice is supported by the World Health Organization's International Ethical Guidelines for Health-related Research Involving Humans, Guideline 3, which states, “In cases where the underrepresentation of particular groups results in or perpetuates health disparities, equity may require special efforts to include members of those populations in research” ([ 9 ][9]), and by the World Medical Association Declaration of Helsinki, which states, “Groups that are underrepresented in medical research should be provided appropriate access to participation in research” ([ 10 ][10]). Therefore, consideration of diversity is essential to the question of fairness in subject selection and to IRB review. Diversity in clinical research is responsive to the principle of beneficence, which places priority on the welfare of research participants and creates the obligation that research presents a favorable balance of benefit to risk, after risks and burdens have been minimized. In calling for “maximization of benefits” in the research, Belmont directs attention to both individual benefit and to the broader value of research to society. A clinical research enterprise that is not inclusive does not adequately address the health needs of a diverse society. Group differences in susceptibility to disease and in treatment outcome can only be identified when those groups are studied. It is the obligation of an IRB to maximize benefits through the inclusion of understudied groups in a manner that is consistent with the study aims and does not introduce unacceptable harm or burden. Belmont describes two ethical convictions in relation to respect for persons, self-determination, and decision-making: the obligations to treat individuals as autonomous agents and to protect those with diminished autonomy. IRBs provide additional safeguards for research involving participants with compromised voluntariness (e.g., prisoners) or impaired comprehension. With regard to the inclusion of diverse populations, respect for persons demands efforts to foster informed and autonomous decision-making and, therefore, to address common barriers posed by age, language, culture, and educational disadvantage. Respect for persons requires the identification of opportunities and resources to engage understudied populations and to enhance awareness, access, and inclusion in research ([ 4 ][4], [ 6 ][6]). It also demands modification of those aspects of research and of consent that inadvertently limit the participation of understudied populations. For example, although inclusion of non-English speakers in a study may involve additional expenses of translation and/or interpreters, it strengthens the commitment to autonomy and justice. The ethical positions presented above compel attention to inclusion of diverse populations in clinical research and define a specific duty for the IRB. In a 2019 survey ([ 11 ][11]), a majority of IRB chairs, IRB administrators, and investigators agreed that “IRBs should play a key role in ensuring diversity among participants in terms of gender, ethnicity, and language.” Despite this, there has been scant regulatory consideration, and little formal discussion within the field, as to whether diversity falls within the IRB's remit. There are also little data as to whether and when IRBs exercise this authority, but the observed underrepresentation in completed studies suggests that IRBs do not consistently attend to this responsibility. Further questions relate to recent U.S. regulation and policy requiring a single, designated IRB to serve as the IRB of record for multicenter research and whether this will offer benefit in consistency and reach with regard to diversity and inclusion. In the face of the persistent problem of underrepresentation in clinical research, institutions should establish policies and provide necessary resources at all institutional levels to ensure that reviewing IRBs fulfill this obligation. The specific approaches we outline here will serve to help incorporate the ethical oversight of diversity in IRB procedures, deliberations, and expectations (see the box). An IRB has authority to require that a research protocol details study elements relevant to considerations of diversity. A description and justification by the investigator of the demographics of the intended study sample (e.g., by age, sex, race, ethnicity, social determinants of health) and a description of either the demographics of the condition or those using or intended to use the product in the general population permit the IRB to make an assessment of the appropriateness of the recruitment plan. When the makeup of the proposed sample deviates substantially from that of the demographics of the condition being studied in the general population or for whom the intervention is intended, and no valid scientific justification is offered, the IRB can require modification of the study to recruit a more representative sample. Such requirements are tailored to the nature and phase of the study, the study's specific aims, and the study location, as discussed further below. #### Institutional review board oversight: Points to consider by reviewers ##### Initial Review ###### Study aims and subject selection ###### Criteria for inclusion and exclusion ###### Recruitment ###### Study conduct ###### Payment ###### Return of results ##### Continuing Review Note that inclusion of a demographically diverse study population does not imply that statistical conclusions regarding heterogeneity of treatment outcome will be possible, but it may allow directional assessments of efficacy and safety that can then be further investigated. Inclusion will, at a minimum, address the equitable selection of participants and the principle of justice in research. During review, an IRB should consider the feasibility of study methods that seek to identify, recruit, and retain underrepresented populations. Research team partnerships with patients and their families, advocacy groups, and community representatives have been shown to be effective in informing recruitment and retention strategies ([ 12 ][12]) as well as in providing input on study questions and participant-relevant endpoints, study conduct, and culturally and linguistically appropriate communications. The IRB should require a statement in the study proposal summarizing the nature, process, input, and impact of such patient and community engagement and how this information has shaped the study itself and the recruitment plan; simply asking the question will prompt consideration by investigators. The IRB can review and provide specific feedback to facilitate successful recruitment of specific populations, including language use, translation, placement of advertisements, and workforce characteristics. The IRB should also ensure that all study materials adhere to health literacy principles and that user-testing is utilized where indicated. The IRB should identify factors, such as excessive time commitment, restricted clinic hours, the costs of travel, and inadequate compensation, that have a foreseeable and negative impact on the enrollment of an appropriately representative sample ([ 13 ][13]). IRBs should require investigators to detail study inclusion and exclusion criteria and, when not self-evident, to provide a rationale for exclusion. Review of eligibility criteria should ensure that understudied populations are not inadvertently or unnecessarily excluded and that criteria are only as restrictive as necessary for safety and to minimize harm. For example, the exclusion of older populations with a specific age criterion might be revised to exclude individuals with specific health concerns who would be at increased risk, regardless of age. When laboratory measures serve as the basis for eligibility criteria, they should be adapted to reflect known sex-, age-, race-, or ancestry-specific normal values, when failure to do so would unnecessarily decrease eligibility of some individuals ([ 14 ][14]). IRBs should identify common practices that limit enrollment of immigrant or minority language speakers in multilingual communities, restrict the participation of women of child-bearing potential (when requiring appropriate contraception would suffice), and introduce bias in participant selection by using overly subjective criteria (such as “investigator discretion”). In exercising the regulatory requirement for continuing oversight of ongoing research, the IRB should periodically review the demographic breakdown of the accrued sample by age, race, ethnicity, sex, and social determinants of health where applicable to the research. Along with these data, IRBs should require an explanation of any meaningful departures from the recruitment plan and request, review, and approve proposed corrective action when indicated. Ongoing tracking of accrual by the IRB, as well as dialogue between the IRB and investigator, communicates the importance of diversity, promotes transparency with regard to progress or lack of progress, provides a measure of accountability, and, ultimately, will change behavior. IRB requirements with regard to study demographics should be flexibly tailored to individual study purpose, phase, setting, and size. For example, for some research (e.g., phase 3 studies, comparative effectiveness research), an IRB may adopt the principle, as a rebuttable presumption, that a study sample should reflect the demographic makeup of the condition being studied or for whom the intervention is intended. Other studies, such as small exploratory, proof-of-concept, early phase studies, or research that seeks to learn about specific communities, would not be expected to be representative of those affected by the condition. Similarly, a local site in a multisite study may be selected because it proposes to recruit a specific racial or ethnic group to diversify a larger study population. Equitable subject selection requires the balancing of inclusion and protection and, like all aspects of research with human participants, grounding in good science. When a study proposes to recruit a sample that is composed predominantly or solely of a racial, ethnic, or other minority, the IRB might reasonably ask why the selection of this sample is scientifically necessary, how the findings are generalizable, and whether an alternative recruitment strategy might yield a more diverse or less burdened, stigmatized, or disadvantaged population. Flexibly adapting requirements to specific study types will encourage dialogue between investigators and the IRB. When an investigator faces particular challenges in the recruitment and retention of specific populations, the IRB could offer guidance or consultation on protocol revision. The IRB itself should be diverse in composition, with membership and input reflecting the demographic compositions of the communities and populations studied in the research it reviews through the use of ad hoc consultants and by appointing members with experience working with diverse communities. However, a recent study showed that 87% of IRB members and 91% of IRB chairs were white ([ 11 ][11]). A diverse IRB will be better attuned to the experience and needs of participants and better able to offer input from the perspective of varied populations. At a minimum, training in cultural competence and implicit bias should become part of required ethics education for all IRB members and staff. Finally, IRBs should develop expertise in providing concrete recommendations for investigators in methods and tools to achieve greater diversity ([ 6 ][6]). Impediments to inclusion of underrepresented and underserved populations in research are numerous and complex. There are no specific regulatory mandates of the kind that typically drive accountability in clinical research. Institutional commitment to diversity is uneven, the research workforce itself is inadequately diverse, and resistance from the research community to any additional oversight is likely. Further, expertise in the engagement and study of hard-to-reach populations is variable and related infrastructure is limited. Finally, in the United States, some question whether Belmont or the Common Rule are appropriately applied to matters of social justice. Institutions should support, educate, and resource IRBs, investigators and their study teams, and others in research so that they can give necessary attention to diversity as a fundamental value in the ethical conduct of research. The application of diversity to research review is neither simple nor without risk, but we do not believe the requirement fundamentally differs from other components of IRB review. Overly prescriptive approaches by the IRB and REC, specific mandates, or the application of quotas to study samples will not serve the interests of science and would not be justifiable or palatable to the research community. Drawing attention to diversity and inclusion as a goal and setting reasonable expectations as a condition of study approval, however, will give rise to necessary discussion and collaboration between IRBs and among investigators and the evolution of best practices in the field. Of course, the obligation to promote diversity in clinical research does not rest solely on the IRB or REC or the investigators. Sponsors, regulators, research and academic institutions, funders, patients and patient advocates, and others must build capacity and infrastructure in what, in the end, must be a collaborative enterprise. As entities that hold investigators accountable, IRBs are themselves accountable to their ethical and regulatory mandates and ultimately to those who serve as participants in research. The duty of IRBs to view subject enrollment and retention beyond the lens of “protection,” to deliberate on the benefits and risks of greater inclusion, and to exercise their authority to promote diversity should be recognized and actively implemented as a matter of justice. 1. [↵][15]1. D. B. Chastain et al ., N. Engl. J. Med. 383, e59 (2020). [OpenUrl][16][PubMed][17] 2. [↵][18]1. T. C. Knepper, 2. H. L. McLeod , Nature 557, 157 (2018). [OpenUrl][19][CrossRef][20] 3. [↵][21]U.S. Food and Drug Administration (FDA), “2015-2019 drug trials snapshots summary report: Five-year summary and analysis of clinical trial participation and demographics” (FDA, 2020); [www.fda.gov/media/143592/download][22]. 4. [↵][23]1. S. George, 2. N. Duran, 3. K. Norris , Am. J. Public Health 104, e16 (2014). [OpenUrl][24][CrossRef][25][PubMed][26] 5. [↵][27]1. L. T. Clark et al ., Curr. Probl. Cardiol. 44, 148 (2019). [OpenUrl][28] 6. [↵][29]1. B. E. Bierer et al ., “Achieving diversity, inclusion, and equity in clinical research,” version 1.0 (Multi-Regional Clinical Trials Center of Brigham and Women's Hospital and Harvard, 2020); . 7. [↵][30]1. R. C. Warren, 2. M. G. Shedlin, 3. E. Alema-Mensah, 4. C. Obasaju, 5. D. A. Hodge Sr. , Ethics Med. Public Health 10, 128 (2019). [OpenUrl][31] 8. [↵][32]The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, “The Belmont Report: Ethical principles and guidelines for the protection of human subjects of research” (U.S. Department of Health, Education, and Welfare, 1979); [www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html][33]. 9. [↵][34]World Health Organization, Council for International Organizations of Medical Sciences (CIOMS), “International ethical guidelines for health-related research involving humans” (CIOMS, ed. 4, 2016); . 10. [↵][35]World Medical Association, “WMA declaration of Helsinki—Ethical principles for medical research involving human subjects,” 9 July 2018; [www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-humansubjects/][36]. 11. [↵][37]1. S. H. Berry et al ., “Profile of institutional review board characteristics prior to the 2019 implementation of the revised common rule” (RAND Coproration, 2019); [www.rand.org/pubs/research_reports/RR2648.html][38]. 12. [↵][39]1. T. Greenhalgh et al ., Health Expect. 22, 785 (2019). [OpenUrl][40][CrossRef][41][PubMed][42] 13. [↵][43]1. L. Gelinas et al ., N. Engl. J. Med. 378, 766 (2018). [OpenUrl][44][CrossRef][45][PubMed][17] 14. [↵][46]1. D. A. Vyas, 2. L. G. Eisenstein, 3. D. S. Jones , N. Engl. J. Med. 383, 874 (2020). [OpenUrl][47][CrossRef][48][PubMed][17] Acknowledgments: D.H.S. received consulting fees as a member of the Takeda Pharmaceuticals Bioethics Advisory Council. [1]: #ref-1 [2]: #ref-2 [3]: #ref-3 [4]: #ref-4 [5]: #ref-5 [6]: #ref-6 [7]: #ref-7 [8]: #ref-8 [9]: #ref-9 [10]: #ref-10 [11]: #ref-11 [12]: #ref-12 [13]: #ref-13 [14]: #ref-14 [15]: #xref-ref-1-1 "View reference 1 in text" [16]: {openurl}?query=rft.jtitle%253DN.%2BEngl.%2BJ.%2BMed.%26rft.volume%253D383%26rft.spage%253De59%26rft_id%253Dinfo%253Apmid%252Fhttp%253A%252F%252Fwww.n%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [17]: /lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsci%2F371%2F6535%2F1209.atom [18]: #xref-ref-2-1 "View reference 2 in text" [19]: {openurl}?query=rft.jtitle%253DNature%26rft.volume%253D557%26rft.spage%253D157%26rft_id%253Dinfo%253Adoi%252F10.1038%252Fd41586-018-05049-5%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [20]: /lookup/external-ref?access_num=10.1038/d41586-018-05049-5&link_type=DOI [21]: #xref-ref-3-1 "View reference 3 in text" [22]: http://www.fda.gov/media/143592/download [23]: #xref-ref-4-1 "View reference 4 in text" [24]: {openurl}?query=rft.jtitle%253DAm.%2BJ.%2BPublic%2BHealth%26rft.volume%253D104%26rft.spage%253De16%26rft_id%253Dinfo%253Adoi%252F10.2105%252FAJPH.2013.301706%26rft_id%253Dinfo%253Apmid%252F24328648%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [25]: /lookup/external-ref?access_num=10.2105/AJPH.2013.301706&link_type=DOI [26]: /lookup/external-ref?access_num=24328648&link_type=MED&atom=%2Fsci%2F371%2F6535%2F1209.atom [27]: #xref-ref-5-1 "View reference 5 in text" [28]: {openurl}?query=rft.jtitle%253DCurr.%2BProbl.%2BCardiol.%26rft.volume%253D44%26rft.spage%253D148%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [29]: #xref-ref-6-1 "View reference 6 in text" [30]: #xref-ref-7-1 "View reference 7 in text" [31]: {openurl}?query=rft.jtitle%253DEthics%2BMed.%2BPublic%2BHealth%26rft.volume%253D10%26rft.spage%253D128%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [32]: #xref-ref-8-1 "View reference 8 in text" [33]: http://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html [34]: #xref-ref-9-1 "View reference 9 in text" [35]: #xref-ref-10-1 "View reference 10 in text" [36]: http://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-humansubjects/ [37]: #xref-ref-11-1 "View reference 11 in text" [38]: http://www.rand.org/pubs/research_reports/RR2648.html [39]: #xref-ref-12-1 "View reference 12 in text" [40]: {openurl}?query=rft.jtitle%253DHealth%2BExpect.%26rft.volume%253D22%26rft.spage%253D785%26rft_id%253Dinfo%253Adoi%252F10.1111%252Fhex.12888%26rft_id%253Dinfo%253Apmid%252F31012259%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [41]: /lookup/external-ref?access_num=10.1111/hex.12888&link_type=DOI [42]: /lookup/external-ref?access_num=31012259&link_type=MED&atom=%2Fsci%2F371%2F6535%2F1209.atom [43]: #xref-ref-13-1 "View reference 13 in text" [44]: {openurl}?query=rft.jtitle%253DN.%2BEngl.%2BJ.%2BMed.%26rft.volume%253D378%26rft.spage%253D766%26rft_id%253Dinfo%253Adoi%252F10.1056%252FNEJMsb1710591%26rft_id%253Dinfo%253Apmid%252Fhttp%253A%252F%252Fwww.n%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [45]: /lookup/external-ref?access_num=10.1056/NEJMsb1710591&link_type=DOI [46]: #xref-ref-14-1 "View reference 14 in text" [47]: {openurl}?query=rft.jtitle%253DN.%2BEngl.%2BJ.%2BMed.%26rft.volume%253D383%26rft.spage%253D874%26rft_id%253Dinfo%253Adoi%252F10.1056%252Fnejmms2004740%26rft_id%253Dinfo%253Apmid%252Fhttp%253A%252F%252Fwww.n%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [48]: /lookup/external-ref?access_num=10.1056/nejmms2004740&link_type=DOI

With humans increasingly serving as computational elements in distributed information processing systems and in consideration of the profit-driven motives and potential inequities that might accompany the emerging thinking economy[1], we recognize the need for establishing a set of related ethics to ensure the fair treatment and wellbeing of online cognitive laborers and the conscientious use of the capabilities to which they contribute. Toward this end, we first describe human-in-the-loop computing in context of the new concerns it raises that are not addressed by traditional ethical research standards. We then describe shortcomings in the traditional approach to ethical review and introduce a dynamic approach for sustaining an ethical framework that can continue to evolve within the rapidly shifting context of disruptive new technologies.