Goto

Collaborating Authors

 Law


AI & Law: Legal Stockpiling

#artificialintelligence

Artificial Intelligence (AI) is gradually and inexorably entering into the legal profession. There is the use of Natural Language Processing (NLP), which we already experience in everyday ordinary interaction with Alexa and Siri and has been increasingly added into various LegalTech systems such as used for contract management, e-Discovery, and the like. Another avenue of AI consists of Machine Learning and Deep Learning. These computational pattern matching techniques are being used to predict court rulings and are also employed to ferret out prior relevant cases amongst a large-scale corpus of online court records. One of the most fascinating and likely law-disruptive AI technologies involves AI-based legal reasoning systems. The notion is that the AI simulates the legal argumentation precepts of human attorneys and essentially carries out a limited form of legal reasoning. Initially, these AI-based legal reasoners would be used as an aid for lawyers and jurists seeking to craft legal arguments. In this semi-autonomous mode, the AI works hand-in-hand with the human legal expert and they jointly establish a robust legal argument or legal posture. Some assert that this capability by the AI will inevitably be further advanced and we will have available fully autonomous AI-based legal reasoning systems that can act in lieu of needing any human legal guidance.


The role of the arts and humanities in thinking about artificial intelligence (AI)

#artificialintelligence

What is the contribution that the arts and humanities can make to our engagement with the increasingly pervasive technology of artificial intelligence? My aim in this short article is to sketch some of these potential contributions. Perhaps the most fundamental contribution of the arts and humanities is to make vivid the fact that the development of AI is not a matter of destiny, but instead involves successive waves of highly consequential human choices. It's important to identify the choices, to frame them in the right way, and to raise the question: who gets to make them and how? This is important because AI, and digital technology generally, has become the latest focus of the historicist myth that social evolution is preordained, that our social world is determined by independent variables over which we, as individuals or societies, are able to exert little control. So we either go with the flow, or go under.


DiscoverText

#artificialintelligence

Data scientists working on text analytics know cleaning data can be time consuming. Users of DiscoverText build reusable custom machine classifiers or "sifters" to find the most (or least) relevant items before using other classifiers for sorting items into topic, sentiment, and other categories. DiscoverText combines hybrid data science methods (measurement, adjudication, iteration, replication) along with established e-discovery text analytics tools, to shorten a process that used to last weeks or months when words get sorted in spreadsheets. Our machine-learning sifters are created in hours or just a few minutes using crowdsourcing. We offer an API and support technical integrations with Twitter and SurveyMonkey.


Live facial recognition technology creates 'supercharged CCTV' that could be used recklessly, Information Commission warns

The Independent - Tech

Plans to allow CCTV cameras to recognise people's faces in realtime could be used "inappropriately, excessively or even recklessly", the Information Commissioner has warned. In recent years, authorities have been rolling out new kinds of facial recognition, with the promise that it would be able to spot dangerous people in real-time. But privacy activists and others have warned that it is a vast invasion of privacy, could be used to create watchlists of people, might falsely accuse people because of racism and other biases and unfair practices. There is still time for authorities to change their mind and avoid the vast dangers that the technology could produce, the head of the watchdog warned. "We're at a crossroads right now, we in the UK and other countries around the world see the deployment of live facial recognition and I think it's still at an early enough stage that it's not too late to put the genie back in the bottle," Commissioner Elizabeth Denham told the PA news agency.


FICO Launches Executive LinkedIn Live Video Series on Operationalizing Analytics and Artificial Intelligence

#artificialintelligence

Global analytics software provider, FICO, today announced its upcoming executive LinkedIn Live video series, "Coffee with Claus" and "Expect the Unexpected." Hosted by FICO Executive Vice President and Chief Technology Officer, Claus Moldt, "Coffee with Claus" will discuss the role of analytics and artificial intelligence in digital transformation, while "Expect the Unexpected" will feature FICO Chief Analytics Officer Scott Zoldi exploring a range of AI topics, such as ethics, governance, diversity, and regulation, with executive leaders. Many of today's enterprises rely on data, and further AI, to deliver a constant stream of intelligence and insight that can be applied to help them pivot in constantly changing business environments as well as address pressing everyday challenges. "With the COVID-19 pandemic accelerating countless digital transformation journeys, our goal is to ensure enterprises are deploying the data at their disposal in the most beneficial ways, some of which include a need to adopt AI to make robust and informed digital decisions," said Claus Moldt, EVP and CTO at FICO. The first episode of "Coffee with Claus," What is an AI Platform?, airs Tuesday, June 22, 2021 at noon EST and features Forrester Analyst Mike Gualtieri.


It's FLAN time! Summing feature-wise latent representations for interpretability

arXiv.org Machine Learning

Interpretability has become a necessary feature for machine learning models deployed in critical scenarios, e.g. legal systems, healthcare. In these situations, algorithmic decisions may have (potentially negative) long-lasting effects on the end-user affected by the decision. In many cases, the representational power of deep learning models is not needed, therefore simple and interpretable models (e.g. linear models) should be preferred. However, in high-dimensional and/or complex domains (e.g. computer vision), the universal approximation capabilities of neural networks is required. Inspired by linear models and the Kolmogorov-Arnol representation theorem, we propose a novel class of structurally-constrained neural networks, which we call FLANs (Feature-wise Latent Additive Networks). Crucially, FLANs process each input feature separately, computing for each of them a representation in a common latent space. These feature-wise latent representations are then simply summed, and the aggregated representation is used for prediction. These constraints (which are at the core of the interpretability of linear models) allow an user to estimate the effect of each individual feature independently from the others, enhancing interpretability. In a set of experiments across different domains, we show how without compromising excessively the test performance, the structural constraints proposed in FLANs indeed increase the interpretability of deep learning models.


On the Connections between Counterfactual Explanations and Adversarial Examples

arXiv.org Artificial Intelligence

Counterfactual explanations and adversarial examples have emerged as critical research areas for addressing the explainability and robustness goals of machine learning (ML). While counterfactual explanations were developed with the goal of providing recourse to individuals adversely impacted by algorithmic decisions, adversarial examples were designed to expose the vulnerabilities of ML models. While prior research has hinted at the commonalities between these frameworks, there has been little to no work on systematically exploring the connections between the literature on counterfactual explanations and adversarial examples. In this work, we make one of the first attempts at formalizing the connections between counterfactual explanations and adversarial examples. More specifically, we theoretically analyze salient counterfactual explanation and adversarial example generation methods, and highlight the conditions under which they behave similarly. Our analysis demonstrates that several popular counterfactual explanation and adversarial example generation methods such as the ones proposed by Wachter et. al. and Carlini and Wagner (with mean squared error loss), and C-CHVAE and natural adversarial examples by Zhao et. al. are equivalent. We also bound the distance between counterfactual explanations and adversarial examples generated by Wachter et. al. and DeepFool methods for linear models. Finally, we empirically validate our theoretical findings using extensive experimentation with synthetic and real world datasets.


Google Artificial Intelligence Team Draws From Critical Race Theory, Internal Document Shows

#artificialintelligence

Google's artificial intelligence (AI) work draws from Critical Race Theory, a philosophical framework that posits that nearly every interaction should be seen as a racial power struggle and seeks to "disrupt" American society which it views as immutably racist, according to a company document obtained by The Daily Wire. A screenshot of an internal company page, obtained by The Daily Wire, says under the header "Ethical AI": We focus on AI at the intersection of Machine Learning and society, developing projects that inform the general public; bringing the complexities of individual identity into the development of human-centric AI; and creating ways to measure different kinds of biases and stereotypes. Out [sic] work includes lessons from gender studies, critical race theory, computational linguistics, computer vision, engineering education, and beyond! Google's Ethical AI team appears intent on encoding far-left ideology into its algorithms even after previous leaders of the team plunged the section into chaos over their insistence on overlaying progressive politics onto mathematics. Until recently, the team was co-led by Timnit Gebru, who cofounded a "Black in AI" racial affinity group and in 2018 coauthored a paper saying facial recognition technology was less accurate at recognizing women and minorities.


Chan Zuckerberg Initiative Invests in Duke Team's Work to Improve Cryo-EM Images

#artificialintelligence

Duke's expertise in Cryo-EM microscopy has attracted a nearly $700,000 grant from the Chan Zuckerberg Initiative (CZI) that will support an effort to make this Nobel-prize winning technology do even more. Cryo-EM is a powerful way to visualize the shapes and configurations of individual proteins, said Alberto Bartesaghi, an associate professor of computer science, biochemistry, and electrical and computer engineering, who spearheads the new effort. After taking thousands of pictures of purified protein, the system uses software to assemble a view of the protein's shape. Duke scientists however have begun working with a 3-D approach called tomography, in which the protein sample is rotated. "It's like a CT scan," Bartesaghi said.


Planned relocation: Pluralistic and integrated science and governance

Science

Although relocation of human populations is nothing new, global environmental changes such as climate change, sea level rise, and land use change are increasing the likelihood of relocation for potentially millions of people, especially in coastal regions. Globally, sea level rise alone could place 340 million people on land projected to be below annual flood levels by 2050 ([ 1 ][1]). The need for relocation will increase because of such risks, the lack of funding for protection and accommodation strategies, and/or the reality that sea walls and other measures will eventually be ineffective. Thus, current approaches to planned relocation such as buyouts for individual households are likely to be “woefully inadequate” in the future ([ 2 ][2]). We discuss how science, governance, and their interactions need to evolve to make planned relocation a strategic option that leaves people, communities, and the environment better off. The starting point is to acknowledge that relocation involves a physical transition away from locations exposed to global change hazards, as well as the need for transformation of institutions, social networks, cultural associations, economic relationships, and other aspects of a community's way of life. Given that relocation is a life-altering change, organizations such as the United Nations (UN) High Commission on Refugees mandate that it needs to be planned and implemented with meaningful engagement of affected parties and carried out to improve (or at least maintain) their quality of life. To ensure responsiveness to changing conditions and preferences, relocation should be part of a flexible, nested, and interconnected set of adaptation strategies that also include coping (reactive, short-term risk-reduction measures) and incremental adjustments (measures to increase resistance and/or resilience) ([ 3 ][3]). How to combine these different measures into a strategic portfolio of policies and actions places demands on science and governance to support open-ended adaptive planning processes that manage trade-offs across interests, uncertainties in knowledge, and institutional ambiguity created by overlapping jurisdictions, authorities, and expertise. Planned relocation is a complex social dilemma that involves many structural, perceptual, economic, and interpersonal dynamics that discourage collective action. It will involve resolving fraught questions such as what decision processes are used, who relocates (and when), how are they compensated, where will they move, what assistance is provided (and to whom) in receiving communities, how abandoned wastes and environmental legacies are remediated, and how agreements are monitored and enforced. There is no single best approach to move a community—stakeholders with conflicting objectives will see it differently even when they share basic world views. The interaction of social and environmental triggers and lack of a preferred pathway make planned retreat similar to other global change dilemmas. But the potential scope, existential character of needed transformations, and complexity of governance challenges make it especially demanding. Despite the immensity of the challenge, it is vital now to constructively engage science and governance to plan physical transitions and socioeconomic transformations that reduce risk and make people, communities, and the environment better off. Here, we offer several ideas for improving governance partnerships in developing strategies for planned relocation. ### Eliminate perverse incentives and establish inclusive governance Existing institutions and processes of governance will be stretched to address the challenges of planning and implementing relocation in a way that meets basic humanitarian principles and good practices. This is because current mixes of policies, institutions, and relationships are responsible for producing the prevailing distribution of privilege and vulnerability in society. Although climate change plays a role, it amplifies present challenges that are an amalgam of past governance, entrenched inequities, and norms. The sheer potential scale of relocation globally is beyond anything our modern global society has experienced. For example, the megacity Jakarta is actively considering relocation because of growing climate hazards, aquifer subsidence, and the density of a highly vulnerable low-income population. These challenges are not limited to the developing world, as evidenced by the mounting annual damages and recovery costs of climate extremes on populations in the United States. Improving governance will require addressing structural inequalities and many perverse incentives and behavioral dynamics that continue to drive people to settle in areas exposed to hazards. Innovations are needed to address organizational silos, poor planning and risk communication, psychological attachments to place, and dependence on continued occupation for tax revenues. These challenges can be exacerbated with well-intentioned coping strategies (e.g., the “levee effect” that reduces accurate perception of risk). In the United States, for example, federal programs including subsidization of beach nourishment, the National Flood Insurance Program, and the federalization of natural disaster recovery encourage settlement of risky areas. Planned relocation toolkits ([ 4 ][4]) are beginning to emerge that orient the challenge within domestic legal frameworks and international organizations (e.g., the UN Office for Disaster Risk Reduction) and the experiences garnered from existing national efforts (e.g., Fiji's efforts to move 46 villages). Making and implementing decisions in which communities voluntarily relocate will require inclusive, deliberative processes that emphasize transparency, engagement, trust building, accountability, and an interactive approach for engaging with science. Policy or legislative frameworks are critical to defining long-term targets and providing credible commitments to maintain the continuity of objectives across institutions and political mandates ([ 5 ][5]). Strategies will need to accommodate changing circumstances (new scientific evidence, technological change, new preferences) and the management of implementation tactics based on expert advice, monitoring and reporting, and accountability. In most countries, new institutions and funding are required to improve access to expert advice, coordination, and consultation. Governance frameworks for relocation will need to include periodic communication about future risks, engagement with private sector and civil society, and oversight mechanisms to monitor and enforce the implementation of agreed plans. ### Diverse perspectives in problem framing Defining the problem and its context is the central challenge posed by planned relocation. Framing a problem establishes what is prioritized (and what is treated as unimportant), what the objectives are, and what questions will be asked and answered. Framing is often contested, and to avoid marginalizing communities, it needs to incorporate diverse perspectives, start from the specific local context of ongoing systemic challenges, enhance stakeholders' agency, and bring together diverse sources of knowledge ([ 6 ][6], [ 7 ][7]). It is particularly challenging to carefully analyze the diverse stakeholders and the types of knowledge that are pivotal to understanding and framing planned relocation (e.g., capturing perspectives from the relocating, receiving, and remaining populations). Problem framing could consider the need for expertise, tactical engagement, and sustained advocacy to catalyze plans into transformative actions ([ 6 ][6], [ 8 ][8]). In addition, emerging innovations in computational social science and “coproduction” of research (in which stakeholder communities are involved in different aspects of the scientific process) offer opportunities for formalizing stakeholder analysis. Analyses could improve stakeholder identification, categorization, and relationship (power) mapping. ### Account for power dynamics Decades of research in planning, public administration, sustainability science, and science and technology studies have examined how to improve the relevance and effectiveness of science to inform planning and policy for a wide range of social, environmental, and sustainability challenges. Several prominent strands of this work focus on coproduction as being more than a means to produce science, providing a mechanism to generate public goods, services, and institutions ([ 7 ][7]). Accordingly, the design of coproduction processes is not just about how the interactions of policy-makers, stakeholders, and scientists affect the usability of science. It is also about the process of social change—how epistemologies, social and cultural norms, institutions and policies, and power relationships among communities and stakeholders interact to determine who is involved in the process, which types of knowledge are seen as legitimate, what is produced, and what outcomes result. For challenges as fraught as planned relocation, this more expansive approach provides a foundation for codeveloping knowledge and action. It requires engaging multiple perspectives on values and knowledge where the actors involved in coproduction of planned retreat must work together to explore normative and political differences inherent in their different visions of the future ([ 6 ][6]). A critique of coproduction processes is that they can depoliticize discourse by using scientific arguments to evoke universalized ideas of what is “best.” They can be structured as if all participants have an equal role when in fact governments, large nongovernmental organizations, and economic interests have disproportionate power and greater opportunities for participation ([ 7 ][7]). This is not just a process issue but can also affect the outcomes of coproduction—for example, favoring the use of narrow cost-benefit framings that conclude that protective measures such as beach nourishment or construction of sea walls are economically justified only for high-value assets. Empirically informed awareness of the diverse roles and dimensions of power in coproduction and social change offers an avenue for rebalancing problematic relationships that lead to inequality or exclusion, or at least avoiding their unintended consequences ([ 7 ][7]). Modest steps such as providing funding to enable underserved communities to participate in coproduction, or formalizing the participation of Indigenous advisory councils, can also help level the playing field ([ 9 ][9]). ### Diversify knowledge sources and types To support planned relocation, science needs to deliver not just technical solutions but also knowledge of how to relocate and transform communities, including the willingness and capacities of different groups and institutions to support fundamental change over time ([ 6 ][6]). Providing this knowledge will require a transdisciplinary approach to research that broadens the array of scientific disciplines and other sources of knowledge engaged. Government bodies and stakeholders (e.g., real estate interests, businesses, community-based organizations) will need to be integrated into research not just as “users” but as knowledge holders and experts in community needs, preferences, norms, and evolving capacity to implement solutions. When relocation involves Indigenous communities, rather than integrating traditional knowledge into Western science, scientists involved in coproduction arrangements should foster mutual respect on the multiple ways of knowing, by engaging in tribal avenues, such as regional newsletters and talking circles at tribal meetings ([ 9 ][9], [ 10 ][10]). Informing social and economic transformation will require research into the capacities and values of different populations and institutions. This requires understanding issues such as what will motivate people to make changes, the capacity of individuals and institutions to act on their preferences, and how current conditions and path dependencies affect the viability of future options ([ 6 ][6]). It will be necessary to “think critically about outcomes as well as processes, about institutional and process designs, [and] about power and performance” ([ 11 ][11]). ### Sample from a range of plausible futures to evaluate decision options Science can better inform action if it stops trying to predict inherently unpredictable phenomena. Currently, many decision-makers frame their questions to scientists as “what will happen,” and scientists respond with “projections” (possibilities based on assumptions about future radiative forcing), which are often used as predictions. This framing, in addition to putting science in the dangerous position of speculating, is not necessarily as helpful to decision-makers as “what if” questions about the consequences of options under many plausible futures. Science can be more useful by changing the objective of collaboration from “predict then act” to the exploration of hypothetical questions about what short-term actions would be consistent with long-term objectives and perform well under a diverse range of plausible futures ([ 12 ][12]). As a specific example, the State of Louisiana has been confronting sea level rise, land subsidence, accelerating losses of coastal lands, and increasing risks from storm surge. The state has initiated an innovative and collaborative planning process that budgets $50 billion in a portfolio of projects to be adaptively implemented over the next 50 years ([ 13 ][13]). Unlike traditional cost-benefit–driven risk planning efforts based on a specific expected future (“what will happen”), the Louisiana master plan has engaged broad stakeholder participation to map what project investments hold immediate benefits while providing flexibility to confront a broad range of plausible future scenarios that could reshape their investment priorities as well as future stakeholder needs (“what if” planning). This approach recognizes that many types of uncertainty will impede judgment and decision-making ([ 12 ][12]). The natural stressors that can trigger the need for evacuation are uncertain because they are emergent, compounding, and cascading outcomes of complex human–environment interactions. But the implications of changes in future values and behaviors are also uncertain and arguably just as important for evaluating decision options. Even in well-documented historical instances of relocation, it is difficult to understand how outcomes emerged from the actions taken—let alone anticipate with any certainty how desired outcomes arise from future actions ([ 14 ][14]). One important opportunity is to more widely apply decision-making under deep uncertainty (DMDU) methods ([ 12 ][12]). These exploratory approaches draw on local-scale stakeholders' knowledge of the key factors and dynamics (human and natural) and provide a promising mechanism for informing planned relocation. Models and scenarios serve as focal points to build shared understanding about the potential implications of the different values and options preferred by stakeholders. ### Social learning to build local capacity Relocation is a complex process that will benefit from expanding the range of intermediaries and services available to facilitate production and application of knowledge. Those involved will need to know not only what scientifically robust sources of information are available for the hazards they face, but also how this information should be used to assess vulnerability, revise flood maps or zoning, evaluate financial risks to reset insurance rates and bond ratings, redesign infrastructure systems, update capital improvement and other plans, or establish thresholds and monitoring systems to trigger the next phase of agreed measures. Much attention has focused on providing climate scenarios and data, but to meet the needs of relocation, the range of services must be expanded. Needed services include not only identifying good practices in engineering, financial risk, and other technical analyses but also supporting transformation, capacity building, and establishment of standards for different types of deliberative and analytic processes. Research, case studies, and pilot projects are testing approaches to meet these challenges, and a useful next step is to organize evaluation and social learning to establish good practices and technical guidance. One option is to incorporate evaluation into assessments such as the Intergovernmental Panel on Climate Change and the US National Climate Assessment to establish a knowledge foundation for climate services. This would create standards for services delivered through international organizations, the private sector, academia, and public agencies (to ensure availability of services for underserved, low-income communities) ([ 15 ][15]). Another is an open-source wiki for climate solutions that would enable a more diverse range of knowledge holders to interact and curate guidance on good practices on an ongoing basis, emphasizing sources of credible information. Another opportunity is to expand the use of intermediaries—individuals and institutions that facilitate interactions between stakeholders and experts ([ 8 ][8]). Many intermediary skillsets are necessary for the different stages of deliberative planning, financing, tactical implementation, and ex-post monitoring of relocation actions. Given the potential for contested needs and values, it is important that intermediaries be aware of how they can unintentionally affect power relationships or outcomes—for example, by using types of knowledge, analysis metrics, or visualizations that favor the perspectives of one group or another. A “critical pragmatic approach” highlights the importance of this awareness and of designing and critically evaluating deliberative processes where conflicts between parties are not reduced to simple consensus-driven debates ([ 11 ][11]). A variety of measures are needed to increase the number and efficacy of intermediaries, including professional certification; greater recognition, including in promotion and tenure processes; and increased funding. ### Harness emerging innovations in community science and data analytics Innovations in community science, sensing, and data analytics hold great promise in providing insights for planned relocation if privacy, equity, and other concerns such as maladaptive applications of generic algorithmic or sensing tools are addressed ([ 15 ][15]). Combining these innovations with monitoring investments in socioeconomic data offers the potential to better capture the interdependent evolution of human and natural systems that shape the experiences and prospects of populations facing relocation. For example, high-resolution models of flooding magnitude and extent might be available for an area, but data are missing on how inequities in agency and justice interact with exposure to hazards to shape the prospects of using planned relocation to improve people's lives. These innovations will increase the utility of standard modes of multidisciplinary scientific research that combine hazard predictions, engineering, financial, and other analyses to inform technical solutions that contribute to physical transitions. Additional methodological advances that have not yet been fully exploited include improved projections of hazards at various spatial scales; research on coastal habitat loss and nature-based solutions; new data sources, indicator-based assessments, and demographic modeling to identify vulnerable populations; and practice standards for using global change risk analytics in engineering and other professions. This contextualized technical knowledge can provide insights for sequencing transitional risk reduction and protection measures. Revolutionizing the role of science to focus on conditions that will affect the ability of society to identify just relocation pathways, build agency, and implement strategies under uncertainty will require a “pluralistic and integrated approach to action-oriented knowledge” ([ 6 ][6]). Such an approach will increase confidence in the ability of communities to successfully navigate planned relocation on the massive scales at which it is likely to be required. It must build a more ethical and responsible approach that serves those affected. 1. [↵][16]1. S. A. Kulp, 2. B. H. Strauss , Nat. Commun. 10, 4844 (2019). [OpenUrl][17] 2. [↵][18]1. J. Carey , Proc. Natl. Acad. Sci. U.S.A. 117, 13182 (2020). [OpenUrl][19][FREE Full Text][20] 3. [↵][21]1. N. Chhetri, 2. M. Stuhlmacher, 3. A. Ishtiaque , Environ. Res. Commun. 1, 015001 (2019). [OpenUrl][22] 4. [↵][23]1. E. Ferris , Int. Organ. Migr. (2017). 5. [↵][24]World Bank, “World Bank Reference Guide to Climate Change Framework Legislation” (Washington, DC 2020); . 6. [↵][25]1. G. Caniglia et al ., Nat. Sustain. 4, 93 (2021). [OpenUrl][26] 7. [↵][27]1. C. Wyborn et al ., Annu. Rev. Environ. Resour. 44, 319 (2019). [OpenUrl][28][CrossRef][29] 8. [↵][30]1. P. Kivimaa, 2. W. Boon, 3. S. Hyysalo, 4. L. Klerkx , Res. Policy 48, 1062 (2019). [OpenUrl][31] 9. [↵][32]1. J. K. Maldonado, 2. B. Colombi, 3. R. Pandya 1. P. Cochran et al ., in Climate Change and Indigenous Peoples in the United States: Impacts, Experiences and Actions, J. K. Maldonado, B. Colombi, R. Pandya, Eds. (Springer, 2014; ), pp. 49–59. 10. [↵][33]1. N. Latulippe, 2. N. Klenk , Curr. Opin. Environ. Sustain. 42, 7 (2020). [OpenUrl][34] 11. [↵][35]1. J. Forester , Plann. Theory 12, 5 (2013). [OpenUrl][36] 12. [↵][37]1. V. A. W. J. Marchau, 2. W. E. Walker, 3. P. J. T. M. Bloemen, 4. S. W. Popper , Decision Making under Deep Uncertainty: From Theory to Practice (Springer Nature, 2019); . 13. [↵][38]1. J. R. Fischbach, 2. D. R. Johnson, 3. D. G. Groves , Environ. Res. Commun. 1, 111001 (2019). [OpenUrl][39] 14. [↵][40]1. K. de Koning, 2. T. Filatova , Environ. Res. Lett. 15, 034008 (2020). [OpenUrl][41] 15. [↵][42]1. R. H. Moss et al ., Weather Clim. Soc. 11, 465 (2019). [OpenUrl][43] [1]: #ref-1 [2]: #ref-2 [3]: #ref-3 [4]: #ref-4 [5]: #ref-5 [6]: #ref-6 [7]: #ref-7 [8]: #ref-8 [9]: #ref-9 [10]: #ref-10 [11]: #ref-11 [12]: #ref-12 [13]: #ref-13 [14]: #ref-14 [15]: #ref-15 [16]: #xref-ref-1-1 "View reference 1 in text" [17]: {openurl}?query=rft.jtitle%253DNat.%2BCommun.%26rft.volume%253D10%26rft.spage%253D4844%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [18]: #xref-ref-2-1 "View reference 2 in text" [19]: {openurl}?query=rft.jtitle%253DProc.%2BNatl.%2BAcad.%2BSci.%2BU.S.A.%26rft_id%253Dinfo%253Adoi%252F10.1073%252Fpnas.2008198117%26rft_id%253Dinfo%253Apmid%252F32461355%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [20]: /lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMjoiMTE3LzI0LzEzMTgyIjtzOjQ6ImF0b20iO3M6MjM6Ii9zY2kvMzcyLzY1NDgvMTI3Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30= [21]: #xref-ref-3-1 "View reference 3 in text" [22]: {openurl}?query=rft.jtitle%253DEnviron.%2BRes.%2BCommun.%26rft.volume%253D1%26rft.spage%253D015001%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [23]: #xref-ref-4-1 "View reference 4 in text" [24]: #xref-ref-5-1 "View reference 5 in text" [25]: #xref-ref-6-1 "View reference 6 in text" [26]: {openurl}?query=rft.jtitle%253DNat.%2BSustain.%26rft.volume%253D4%26rft.spage%253D93%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [27]: #xref-ref-7-1 "View reference 7 in text" [28]: {openurl}?query=rft.jtitle%253DAnnu.%2BRev.%2BEnviron.%2BResour.%26rft.volume%253D44%26rft.spage%253D319%26rft_id%253Dinfo%253Adoi%252F10.1146%252Fannurev-environ-101718-033103%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [29]: /lookup/external-ref?access_num=10.1146/annurev-environ-101718-033103&link_type=DOI [30]: #xref-ref-8-1 "View reference 8 in text" [31]: {openurl}?query=rft.jtitle%253DRes.%2BPolicy%26rft.volume%253D48%26rft.spage%253D1062%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [32]: #xref-ref-9-1 "View reference 9 in text" [33]: #xref-ref-10-1 "View reference 10 in text" [34]: {openurl}?query=rft.jtitle%253DCurr.%2BOpin.%2BEnviron.%2BSustain.%26rft.volume%253D42%26rft.spage%253D7%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [35]: #xref-ref-11-1 "View reference 11 in text" [36]: {openurl}?query=rft.jtitle%253DPlann.%2BTheory%26rft.volume%253D12%26rft.spage%253D5%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [37]: #xref-ref-12-1 "View reference 12 in text" [38]: #xref-ref-13-1 "View reference 13 in text" [39]: {openurl}?query=rft.jtitle%253DEnviron.%2BRes.%2BCommun.%26rft.volume%253D1%26rft.spage%253D111001%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [40]: #xref-ref-14-1 "View reference 14 in text" [41]: {openurl}?query=rft.jtitle%253DEnviron.%2BRes.%2BLett.%26rft.volume%253D15%26rft.spage%253D034008%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [42]: #xref-ref-15-1 "View reference 15 in text" [43]: {openurl}?query=rft.jtitle%253DWeather%2BClim.%2BSoc.%26rft.volume%253D11%26rft.spage%253D465%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx