F ood insecurity is a significant social and public health issue that plagues many urban metropolitan areas around the world. Existing approaches to identifying food insecurity rely primarily on qualitative and quantitative survey data, which is difficult to scale. This project seeks to explore the effectiveness of using street-level images in modeling food insecurity at the census tract level. T o do so, we propose a two-step process of feature extraction and gated attention for image aggregation. W e evaluate the effectiveness of our model by comparing against other model architectures, interpreting our learned weights, and performing a case study. While our model falls slightly short in terms of its predictive power, we believe our approach still has the potential to supplement existing methods of identifying food insecurity for urban planners and policymakers.

Researchers at MIT Media Lab are using crowdsourced data to create an algorithm that determines how safe a street looks to the human eye – information that could be used to guide important urban design decisions. In 2011, researchers at the MIT Media Lab debuted Place Pulse, a website that served as a kind of "hot or not" for cities. Given two Google Street View images culled from a select few cities including New York City and Boston, the site asked users to click on the one that seemed safer, more affluent, or more unique. The result was an empirical way to measure urban aesthetics. Now, that data is being used to predict what parts of cities feel the safest.

In a sign that computers will be able to perform image analysis as fluently as text analysis, a group of Stanford-based researchers were able to make accurate predictions about neighborhood voting patterns based on millions of pictures collected from Google Street View, reports The New York Times. While other academic projects have used artificial intelligence to mine Google Street View for socioconomic insights (such as Streetchange), this project is notable because of the vast quantity of images that its AI software processed. Led by Stanford computer vision scientist Timnit Gebru, the team of researchers used software to analyze 50 million images of street scenes and location data. Their goal was to find data that could be used to predict demographic statistics at the zip code and precinct (which usually contain about 1,000 people) level. From those images, they were able to glean information, including make and model, about 22 million cars, or 8% of all cars in the country, in 3,000 zip codes and 39,000 voting districts.

From the understated opulence of a Bentley to the stalwart family minivan to the utilitarian pickup, Americans know that the car you drive is an outward statement of personality. You are what you drive, as the saying goes, and researchers at Stanford have just taken that maxim to a new level. Using computer algorithms that can see and learn, they have analyzed millions of publicly available images on Google Street View. The researchers say they can use that knowledge to determine the political leanings of a given neighborhood just by looking at the cars on the streets. "Using easily obtainable visual data, we can learn so much about our communities, on par with some information that takes billions of dollars to obtain via census surveys. More importantly, this research opens up more possibilities of virtually continuous study of our society using sometimes cheaply available visual data," said Fei-Fei Li, an associate professor of computer science at Stanford and director of the Stanford Artificial Intelligence Lab and the Stanford Vision Lab, where the work was done.

Google Street View images are filled with cars. That is a simple and pedestrian truth, and one which artificial intelligence researchers have taken advantage of to do something surprising. By analyzing car type, they were able to make predictions about the demographic information of the people in the cities they studied. For example, the team, largely from Stanford University, analyzed whether they saw more pickups trucks or sedans in a given city. With a greater number of pickup trucks, the urban area had an 82 percent chance of voting Republican, and with more sedans, there was an 88 percent chance it voted Democrat. Artificial intelligence systems shine when crunching staggeringly large amounts of data and then making predictions about what they see in it.

Each year, the U.S. Census Bureau spends $1 billion surveying the population. These surveys are designed to tease apart the demographic makeup of the country by asking a representative group of people about their race, gender, education, occupation, and so on. This is an important exercise because it gives a crucial bird's-eye view of the population and how it is changing. For a start, the data is relatively large scale--the Census Bureau's main survey, the American Community Survey, gives results for all cities and counties with a population greater than 65,000. What's more, surveying the population is a time-consuming exercise; so much so that some data can be five years old by the time it is published.

Targeted socio-economic policies require an accurate understanding of a country’s demographic makeup. To that end, the United States spends more than 1 billion dollars a year gathering census data such as race, gender, education, occupation and unemployment rates. Compared to the traditional method of collecting surveys across many years which is costly and labor intensive, data-driven, machine learning-driven approaches are cheaper and faster—with the potential ability to detect trends in close to real time. In this work, we leverage the ubiquity of Google Street View images and develop a computer vision pipeline to predict income, per capita carbon emission, crime rates and other city attributes from a single source of publicly available visual data. We first detect cars in 50 million images across 200 of the largest US cities and train a model to predict demographic attributes using the detected cars. To facilitate our work, we have collected the largest and most challenging fine-grained dataset reported to date consisting of over 2600 classes of cars comprised of images from Google Street View and other web sources, classified by car experts to account for even the most subtle of visual differences. We use this data to construct the largest scale fine-grained detection system reported to date. Our prediction results correlate well with ground truth income data (r=0.82), Massachusetts department of vehicle registration, and sources investigating crime rates, income segregation, per capita carbon emission, and other market research. Finally, we learn interesting relationships between cars and neighborhoods allowing us to perform the first large scale sociological analysis of cities using computer vision techniques.

SA Forum is an invited essay from experts on topical issues in science and technology. It has become fashionable to bad-mouth big data. In recent weeks the New York Times, Financial Times, Wired and other outlets have all run pieces bashing this new technological movement. To be fair, many of the critiques have a point: There has been a lot of hype about big data and it is important not to inflate our expectations about what it can do. But little of this hype has come from the actual people working with large data sets.