It's easy to forget that even with the fanciest of machine learning models, we still need humans in the trenches cleaning input data. Descartes Labs, a startup that combines satellite imagery with data about our planet to produce insights and forecasts, knows this all too well. The company ended up building its own cloud-based parallel computing infrastructure to clean and process its massive corpus of satellite imagery. Companies like Descartes Labs cannot just throw raw satellite imagery into machine learning models to extract insights. Images captured contain clouds, cloud shadows and other atmospheric aberrations that make it impossible to compare images taken at different times.

Using analogical model formulation with sketches to solve Bennett Mechanical Comprehension Test problems.

Spatial Visualization Using R: One of the less understood aspects of R is in spatial data visualization. The below article will outline two case studies on using R to spatially visualize data. Our first step is figuring out how to use the Census API within R. Given below are the key data Source Details from the Census ACS Data We use the acs.lookup function & use the keywords to find the required data across all ACS tables. For example, the following are the search results for the keywords owner, occupied, and median. Using the Choroplethr package make it really easy to create thematic maps in R.

Everyone knows businesses today are grappling with the explosion of data. What's less well-known is the value of location data and the intelligence that geospatial analysis can provide business decision makers, beyond those that are GIS (geographic information system) professionals. Location data is already and will continue to be a growing component of all business data. Smartphone penetration is on the rise around the world, location infrastructure--such as cell towers, beacons, RFID and GPS--is proliferating, and the Internet of Things is poised to go mainstream within a few years. However, rather than adding to the complexity of the data landscape, location has the power to bring order to it.

Airbnb boasts almost two million listings in 34,000 cities, and according to data from Inside Airbnb, a independent data analysis website, listed about 36000 apartments in New York as of July 5, 2016. This data exploration sets out to visualize how Airbnb operates in New York City. Airbnb's presence in NYC has been clouded in controversy from the beginning, with law makers arguing that Airbnb drive up rents for New York residents, as well as facilitating a lot of illegal hosting activities, all the while not paying any of the fees hotels are subjected to. Rent is drived up when landlords decide to rather rent apartments to short-term guests at higher rates, compared to signing up tenants for yearlong leases. In a study conducted in 2014, The New York State Attorney General concluded that 72%of all units used as private short-term rentals on Airbnb during 2010 through mid-2014 appeared to violate both state and local New York laws.

This paper presents Generalized Correspondence-LDA (GC-LDA), a generalization of the Correspondence-LDA model that allows for variable spatial representations to be associated with topics, and increased flexibility in terms of the strength of the correspondence between data types induced by the model. We present three variants of GC-LDA, each of which associates topics with a different spatial representation, and apply them to a corpus of neuroimaging data. In the context of this dataset, each topic corresponds to a functional brain region, where the region's spatial extent is captured by a probability distribution over neural activity, and the region's cognitive function is captured by a probability distribution over linguistic terms. We illustrate the qualitative improvements offered by GC-LDA in terms of the types of topics extracted with alternative spatial representations, as well as the model's ability to incorporate a-priori knowledge from the neuroimaging literature. We furthermore demonstrate that the novel features of GC-LDA improve predictions for missing data.

Spatial Visualization Using R: One of the less understood aspects of R is in spatial data visualization. The below article will outline two case studies on using R to spatially visualize data. Our first step is figuring out how to use the Census API within R. Given below are the key data Source Details from the Census ACS Data We use the acs.lookup function & use the keywords to find the required data across all ACS tables. For example, the following are the search results for the keywords owner, occupied, and median. Using the Choroplethr package make it really easy to create thematic maps in R.

He takes the NYC Data Science Academy 12 week full time Data Science Bootcamp program from July 5th to September 22nd, 2016. This post is based on their first class project - the Exploratory Data Analysis Visualization Project, due on the 2nd week of the program. You can find the original article here. Airbnb boasts almost two million listings in 34,000 cities, and according to data from Inside Airbnb, a independent data analysis website, listed about 36000 apartments in New York as of July 5, 2016. This data exploration sets out to visualize how Airbnb operates in New York City.

Codified narrative is the product of converting human-friendly narrative into computer-friendly code. In past blogs, I discussed my own approach towards this process of codification. Here, I will be covering the idea of spatial, temporal, and contextual distribution of codified narrative. I have never suggested that narrative can or should be used in place of quantitative data. However, I have reflected on how the quantitative regime has tended to dominate discourse; this has perhaps led to data being contextually constrained or deprived. Geography is a type of context that can shape the extent to which people interact with the world. Space provides a medium to distribute resources. It can be involved in forced confinement. An office full of cubicles demonstrates control and dominance over space.