Goto

Collaborating Authors

Spatial Reasoning


GIS, Mapping, and Spatial Analysis Specialization

#artificialintelligence

Learn the concepts, tools, and techniques to make great maps that answer geographic questions. This Specialization is for those new to mapping and GIS, as well as anyone looking to gain a better understanding of how it all works and why. You will learn practical skills that can be applied to your own work using cutting-edge software created by Esri Inc., the world's leading GIS company and our industry partner. PLEASE NOTE: THE SOFTWARE IS NO LONGER PROVIDED WITH THIS COURSE. The courses are based on the idea that you will get the most out of the learning experience if you can tailor your work to your own interests.


Analysis of multispectral imaging with the AstroPath platform informs efficacy of PD-1 blockade

Science

Immunohistochemical stains for individual markers revolutionized diagnostic pathology decades ago but cannot capture enough information to accurately predict response to immunotherapy. Newer multiplex immunofluorescent technologies provide the potential to visualize the expression patterns of many functionally relevant molecules but present numerous challenges in accurate image analysis and data handling, particularly over large tumor areas. Drawing from the field of astronomy, in which petabytes of imaging data are routinely analyzed across a wide spectral range, Berry et al. developed a platform for multispectral imaging of whole-tumor sections with high-fidelity single-cell resolution. The resultant AstroPath platform was used to develop a multiplex immunofluorescent assay highly predictive of responses and outcomes for melanoma patients receiving immunotherapy. Science , aba2609, this issue p. [eaba2609][1] ### INTRODUCTION New therapies have been designed to stimulate the host’s immune system to fight cancer. Despite these exciting, recent successes, a large proportion of patients still do not respond to anti–programmed cell death-1 (PD-1) or anti–programmed death ligand-1 (PD-L1) therapies, and thus, biomarkers for patient selection are highly desirable. The only U.S. Food and Drug Administration–approved histopathology biomarker tests for anti–PD-1 or anti–PD-L1 therapy is assessment of PD-L1 protein expression by means of immunohistochemistry. This approach is unidimensional and has limitations. Innovative characterization of the tumor microenvironment (TME) with a focus on multidimensional, spatially resolved interactions at the single-cell level will provide critical mechanistic insights into therapeutic responses and potentially identify improved biomarkers for patient selection. Using multispectral approaches to image the TME and substituting cells for stars and galaxies, we applied the methodology and infrastructure developed for astronomy to pathologic analysis of specimens from patients with melanoma. ### RATIONALE The next generation of pathologic analyses will require platforms that can characterize the coexpression of key molecules on specific cellular subsets in situ and spatial relationships between tumor cells and multiple immune elements. To that aim, we applied astronomical algorithms for high-quality imaging and the establishment of relational databases to multiplex immunofluorescence (mIF) labeling of pathology specimens, facilitating spatial analyses and immunoarchitectural characterization of the host-tumor interface. In all, we curated and coordinately mapped six markers, both individually and in combination in tumor tissue from 98 patients with melanoma receiving anti–PD-1 therapy. This dataset comprised ~127,400 image mosaics composed of more than 100 million single cells. The data outputs were linked to patient outcomes, informing in a clinically relevant way how cancer evades the immune system and potentiating biomarker assay development for precision immunotherapy. ### RESULTS The imaging protocols used in this study were used to address outstanding questions regarding the impact of high-power field sampling strategies on biomarker performance. This information was then used to develop an approach for operator-independent field selection. The image handling strategies also facilitated the robust assessment of the intensity of PD-1 and PD-L1 expression in situ (negative, low, mid, and high levels) on different cell types. Thus, with only six markers (PD-1, PD-L1, CD8, FoxP3, CD163, and Sox10/S100), we were able to develop 41 combinations of expression patterns for these molecules and map relatively rare cells such as CD8+FoxP3+ cells to the tumor stromal boundary. Moreover, a high density of CD8+FoxP3+PD-1low/mid cells was closely associated with response to PD-1 blockade. Cell types associated with a lack of response to therapy were also identified—for example, CD163+ macrophages that were PD-L1–. This latter phenotype was also found to have a negative effect on long-term survival. When these and other key cell phenotype densities were combined, they were highly predictive of objective response and stratified long-term patient outcomes after anti–PD-1–based therapies in both a discovery cohort and an independent validation cohort. ### CONCLUSION Here, we present the AstroPath platform, an end-to-end pathology workflow with rigorous quality control for creating quantitative, spatially resolved mIF datasets. Although the current effort focused on a six-plex mIF assay, the principles described here provide a general framework for the development of any multiplex assay with single-cell image resolution. Such approaches will vastly improve the standardization and scalability of these technologies, enabling cross-site and cross-study comparisons. This will be essential for multiplex imaging technologies to realize their potential as biomarker discovery platforms and ultimately as standard diagnostic tests for clinical therapeutic decision-making. ![Figure][2] Strong parallels between multispectral analyses in astronomy and emerging multiplexing platforms for pathology. The next generation of tissue-based biomarkers are likely to be identified by use of large, well-curated datasets. To that end, image analysis approaches originally developed for astronomy were applied to pathology specimens to produce trillions of pixels of robust tissue imaging data and facilitate assay and atlas development. IMAGES: (BENEATH “ASTRONOMY”) SLOAN DIGITAL SKY SURVEY; (MICROSCOPE) AKOYA BIOSCIENCE Next-generation tissue-based biomarkers for immunotherapy will likely include the simultaneous analysis of multiple cell types and their spatial interactions, as well as distinct expression patterns of immunoregulatory molecules. Here, we introduce a comprehensive platform for multispectral imaging and mapping of multiple parameters in tumor tissue sections with high-fidelity single-cell resolution. Image analysis and data handling components were drawn from the field of astronomy. Using this “AstroPath” whole-slide platform and only six markers, we identified key features in pretreatment melanoma specimens that predicted response to anti–programmed cell death-1 (PD-1)–based therapy, including CD163+PD-L1– myeloid cells and CD8+FoxP3+PD-1low/mid T cells. These features were combined to stratify long-term survival after anti–PD-1 blockade. This signature was validated in an independent cohort of patients with melanoma from a different institution. [1]: /lookup/doi/10.1126/science.aba2609 [2]: pending:yes


Geospatial Analyses & Remote Sensing : from Beginner to Pro

#artificialintelligence

Description Geospatial Data Analyses & Remote Sensing: 5 Classes in 1 Do you need to design a GIS map or satellite-imagery based map for your Remote Sensing or GIS project but you don't know how to do this? Have you heard about Remote Sensing object-based image analysis and machine learning or maybe QGIS or Google Earth Engine but did not know where to start with such analyses? Do you find Remote Sensing and GIS manuals too not practical and looking for a course that takes you by hand, teach you all the concepts, and get you started on a real-life GIS mapping project? I'm very excited that you found my Practical Geospatial Masterclass on Geospatial Data Analyses & Remote Sensing. This course provides and information that is usually delivered in 4 separate Geospatial Data Analyses & Remote Sensing courses, and thus you with learning all the necessary information to start and advance with Geospatial analysis and includes more than 9 hours of video content, plenty of practical analysis, and downloadable materials.


Spatial Data Analysis with Earth Engine Python and Colab

#artificialintelligence

One of the common problems with learning image processing is the high cost of software. In this course, I entirely use open source software including the Google Earth Engine Python API and Colab. All sample data and script will be provided to you as an added bonus throughout the course. Jump in right now and enroll.


Spatial Data Analysis with Earth Engine Python and Colab

#artificialintelligence

Description Do you want to access satellite sensors using Earth Engine Python API and Google Colab? Do you want to learn the spatial data science on the cloud? Do you want to become a geospatial data scientist? I will provide you with hands-on training with example data, sample scripts, and real-world applications. By taking this course, you be able to install Anaconda and Jupyter Notebook.


Democratizing data for a fair digital economy

MIT Technology Review

The digital revolution is here, but not everyone is benefiting equitably from it. And as Silicon Valley's ethos of "move fast and break things" spreads around the world, now is the time to pause and consider who is being left out and how we can better distribute the benefits of our new data economy. "Data is the main resource of a new digital economy," says Parminder Singh, executive director at nonprofit organization IT for Change. Global society will benefit because the economy will benefit, argues Singh, on decentralization of data and distributed digital models. Data commons--or open data sources--are vital to help build an equitable digital economy, but with that comes the challenge of data governance. "Not everybody is sharing data," says Singh. Big tech companies are holding onto the data, which stymies the growth of an open data economy, but also the growth of society, education, science, in other words, everything. According to Singh, "Data is a non-rival resource. It's not a material resource that if one uses it, other can't use it." Singh continues, "If all people can use the resource of data, obviously people can build value over it and the overall value available to the world, to a country, increases manifold because the same asset is available to everyone." One doesn't have to look very far to understand the value of non-personal data collected to help the public, consider GIS data from government satellites. Innovation plus the open access to geographic data helped not only create the Internet we know today, but those same tech companies.


Mining GIS Data to Predict Urban Sprawl

arXiv.org Artificial Intelligence

This paper addresses the interesting problem of processing and analyzing data in geographic information systems (GIS) to achieve a clear perspective on urban sprawl. The term urban sprawl refers to overgrowth and expansion of low-density areas with issues such as car dependency and segregation between residential versus commercial use. Sprawl has impacts on the environment and public health. In our work, spatiotemporal features related to real GIS data on urban sprawl such as population growth and demographics are mined to discover knowledge for decision support. We adapt data mining algorithms, Apriori for association rule mining and J4.8 for decision tree classification to geospatial analysis, deploying the ArcGIS tool for mapping. Knowledge discovered by mining this spatiotemporal data is used to implement a prototype spatial decision support system (SDSS). This SDSS predicts whether urban sprawl is likely to occur. Further, it estimates the values of pertinent variables to understand how the variables impact each other. The SDSS can help decision-makers identify problems and create solutions for avoiding future sprawl occurrence and conducting urban planning where sprawl already occurs, thus aiding sustainable development. This work falls in the broad realm of geospatial intelligence and sets the stage for designing a large scale SDSS to process big data in complex environments, which constitutes part of our future work.


Data Science: Feature Engineering with Spatial Flavour – Sp.4ML

#artificialintelligence

Imagine that we are working for a real estate agency and our role is to estimate price of apartment renting in a different parts of the New York City. In classic machine learning approach we work through those variables and build model for prediction of a price. We will consider the last example and learn how to retrieve spatial information using GeoPandas package and publicly available geographical datasets. Data for this article is shared in Kaggle. Data is also available in the blogpost repository.


Biologists revel in pinpointing active genes in tissue samples

Science

As with real estate, location matters greatly for cells. Douglas Strand confirmed that truth last year when he used a new technique to map gene activity in bladder cancers. Until recently, scientists wanting to know all the genes at work in a tissue could analyze single cells without knowing their position, or they could measure average activity levels of genes across thousands of cells. Now, an emerging technology called spatial transcriptomics combines precision and breadth, mapping the work of thousands of genes in individual cells at pinpoint locations in tissue. That, Strand says, has been a “total game changer” for his research. The virtual Advances in Genome Biology and Technology (AGBT) meeting this month was a big coming-out party for the technique, which is revealing whole new landscapes of gene expression. Strand, for example, reported finding that cells surrounding bladder tumors, though outwardly normal, display many of the same gene activity changes as the cancer. “They looked more like tumor than normal tissue,” says Strand, who works at the University of Texas Southwestern Medical Center. He found surprises within the tumors, too: hidden patterns of gene activity suggesting some of the cells are more likely than others to spread beyond the bladder. Other biologists at the meeting reported using the technique to study Alzheimer's disease, track the dynamics of different types of T cells, and study lung, heart, and other tissues in COVID-19 patients. “The field is developing very, very fast,” says Aparna Bhaduri of the University of California, Los Angeles, who uses it to examine developing human brains. Scientists studying cells have long been able to examine the activity of a few, select genes in intact tissue—for example, by engineering a gene to tack on a fluorescent tag to the protein it encodes. By 2010, traditional transcriptomics, which examines cellular activity of many, if not all, known genes by probing for the messenger RNA (mRNA) transcripts they encode, took off. But those studies require tissues to be ground up first, so the data represent the average activity of genes in millions of cells. More recently, biologists have begun to monitor all the genes of single cells, uncovering vast differences in gene activity between different cell types and variation even within types. But because those cells are extracted from tissue with enzymes or teased out with lasers, microscopic tweezers, or other methods, the influence of their precise location and neighbor cells is lost. “We could see the individual parts, but we didn't know how the parts fit together,” explains Joseph Beechem, a biophysicist at NanoString Technologies, a leading company for spatial transcriptomics and related methods. Then in 2016, Swedish researchers described in Science how they managed to keep track of cells' locations while assessing the activity of about 200 of their genes ( Science , 1 July 2016, p. [78][1]). The group put thin slices of a tissue onto slides precoated with short, known sequences of DNA, meant to act like identifiable barcodes, attached to other DNA designed to latch nonspecifically onto any mRNA nearby. The team treated the tissue with detergent to make cells leak their mRNA, which linked to the anchored, barcoded DNA, marking which cell the mRNA came from. Then, they added enzymes and DNA bases to the slice to translate each mRNA into a complementary DNA strand. Sequencing that strand along with its position-identifying barcode revealed the active parent gene and its position. Those data enabled computer programs to reconstruct the tissue locations of all the active genes. Multiple companies have begun to sell expensive machines that conduct such spatial transcriptomics analyses, making it possible to study thousands of genes in hundreds of cells in their proper places. That “can tell you a lot about how cell communication might break down in disease,” says Aviv Regev, a computation and systems biologist who heads the Genentech Research and Early Development unit of Roche. Christopher Mason, a geneticist at Weill Cornell Medicine (WCM), and colleagues have performed spatial transcriptomics on fresh or preserved tissue samples from autopsied COVID-19 patients, comparing them with lung tissue of healthy adults and people who died of other acute respiratory infections or flu. The commercial devices they used, one based on the Swedish approach, can assess lots of genes but can't completely pinpoint their activity to single cells. (Other methods are limited to far fewer genes, but specify locations better.) The team, including WCM's Robert Schwartz and Alain Borczuk and others, mapped the activity of the gene for angiotensin-converting enzyme 2, the cell-surface receptor targeted by SARS-CoV-2, and other identifying immune cells called macrophages and neutrophils. In normal lung tissue, macrophages make up less than 4% of the cells; in COVID-19 lungs, they sometimes topped 50%, Mason reported at the AGBT meeting. The lung itself changes as well, he and his colleagues discovered by looking at gene activity in these lung samples. Late in the disease, the organ's normal cellular architecture was disrupted, and cells adjacent to blood vessels had changed. The WCM group and others have done spatial analyses of gene activity for other parts of the COVID-19–ravaged body. The coronavirus seems to turn off genes in nasal cells that sense smells and causes a reorganization of the cells in the lining of the nose; those changes may contribute to the loss of smell and taste infected people often experience. The hearts of COVID-19 patients also betrayed an impact. Under the microscope they appear to have a normal number of muscle cells, Mason says, “but if you look at gene expression, it seems the cells have forgotten what they are supposed to be doing.” Stanford University neuroscientist Andrew Yang has done a similar gene activity comparison of preserved human brain tissue to understand why some people with protein deposits called amyloid plaques don't develop Alzheimer's disease and others do. In tissue from Alzheimer's patients, nonneuronal cells close to these plaques show increased activity of genes whose proteins mark nerve cell connections called synapses for destruction. Other revved up genes suggest increased action by scavenger cells called microglia, which prune synapses and cause potentially harmful inflammation. “We're beginning to understand what makes for a good or bad response to these aggregates,” Yang says. These early results only begin to address the potential of spatial transcriptomics. The current methods don't yet work robustly in all types of tissues, and analyses can take days to complete. Companies continue to upgrade their instruments, but so far, none can really quantify all the active genes in a tissue at the single-cell level. At about $300,000 each, some of the machines are also prohibitively expensive for many labs. The Broad Institute has come up with a cheaper DIY version. Called “Slide-seq,” the technique uses a layer of tiny beads, coated with pieces of barcoded DNA, on a slide to help mark the positions of mRNA from thousands of genes ( Science , 29 March 2019, p. [1463][2]). At the AGBT meeting, Broad genomicist Robert Stickels described version 2.0, which crams much more DNA onto each bead and can put up to 1 million beads on a slide, making the gene-activity mapping more precise by an order of magnitude. The entire protocol is public, Stickels says. “It really empowers other labs to do it.” For example, Abhishek Sampath Kumar, a graduate student at the Max Planck Institute for Molecular Genetics, now gets slides from Broad. “This technique is easy to apply,” says Kumar, who is studying mammalian heart development. “You don't need any special instruments compared to other methods.” Both industrial and academic labs are racing to improve spatial transcriptomics and to extend cell-by-cell mapping to other key indicators. “Soon there will be technologies that give you more and more types of data all together at the same time, spatial information, RNA, DNA, chromatin, protein, temporal information about cellular histories, metabolite profiling, you name it, at single-cell resolution,” Stickels predicts. Many biologists are thrilled at the prospects. “I think we will be rewriting the textbook on how organisms develop, and we are going to understand how the body responds to drugs in a way that nobody has been able to do before,” Beechem says. “Spatial biology is providing the next revolution in biology.” [1]: http://www.sciencemag.org/content/353/6294/78 [2]: http://www.sciencemag.org/content/363/6434/1463


Hippocampal formation-inspired probabilistic generative model

arXiv.org Artificial Intelligence

We constructed a hippocampal formation (HPF)-inspired probabilistic generative model (HPF-PGM) using the structure-constrained interface decomposition method. By modeling brain regions with PGMs, this model is positioned as a module that can be integrated as a whole-brain PGM. We discuss the relationship between simultaneous localization and mapping (SLAM) in robotics and the findings of HPF in neuroscience. Furthermore, we survey the modeling for HPF and various computational models, including brain-inspired SLAM, spatial concept formation, and deep generative models. The HPF-PGM is a computational model that is highly consistent with the anatomical structure and functions of the HPF, in contrast to typical conventional SLAM models. By referencing the brain, we suggest the importance of the integration of egocentric/allocentric information from the entorhinal cortex to the hippocampus and the use of discrete-event queues.