Information Fusion
Semantic Image Fusion
Image fusion methods and metrics for their evaluation have conventionally used pixel-based or low-level features. However, for many applications, the aim of image fusion is to effectively combine the semantic content of the input images. This paper proposes a novel system for the semantic combination of visual content using pre-trained CNN network architectures. Our proposed semantic fusion is initiated through the fusion of the top layer feature map outputs (for each input image)through gradient updating of the fused image input (so-called image optimisation). Simple "choose maximum" and "local majority" filter based fusion rules are utilised for feature map fusion. This provides a simple method to combine layer outputs and thus a unique framework to fuse single-channel and colour images within a decomposition pre-trained for classification and therefore aligned with semantic fusion. Furthermore, class activation mappings of each input image are used to combine semantic information at a higher level. The developed methods are able to give equivalent low-level fusion performance to state of the art methods while providing a unique architecture to combine semantic information from multiple images.
From SLAM to Situational Awareness: Challenges and Survey
Bavle, Hriday, Sanchez-Lopez, Jose Luis, Schmidt, Eduardo F., Voos, Holger
The knowledge that an intelligent and autonomous mobile robot has and is able to acquire of itself and the environment, namely the situation, limits its reasoning, decision-making, and execution skills to efficiently and safely perform complex missions. Situational awareness is a basic capability of humans that has been deeply studied in fields like Psychology, Military, Aerospace, Education, etc., but it has barely been considered in robotics, which has focused on ideas such as sensing, perception, sensor fusion, state estimation, localization and mapping, spatial AI, etc. In our research, we connected the broad multidisciplinary existing knowledge on situational awareness with its counterpart in mobile robotics. In this paper, we survey the state-of-the-art robotics algorithms, we analyze the situational awareness aspects that have been covered by them, and we discuss their missing points. We found out that the existing robotics algorithms are still missing manifold important aspects of situational awareness. As a consequence, we conclude that these missing features are limiting the performance of robotic situational awareness, and further research is needed to overcome this challenge. We see this as an opportunity, and provide our vision for future research on robotic situational awareness.
Using Automation in AI with Recent Enterprise Tools
Data Science (DS) and Machine Learning (ML) are the spines of today's data-driven business decision-making. From a human viewpoint, ML often consists of multiple phases: from gathering requirements and datasets to deploying a model, and to support human decision-making--we refer to these stages together as DS/ML Lifecycle. There are also various personas in the DS/ML team and these personas must coordinate across the lifecycle: stakeholders set requirements, data scientists define a plan, and data engineers and ML engineers support with data cleaning and model building. Later, stakeholders verify the model, and domain experts use model inferences in decision making, and so on. Throughout the lifecycle, refinements may be performed at various stages, as needed. It is such a complex and time-consuming activity that there are not enough DS/ML professionals to fill the job demands, and as much as 80% of their time is spent on low-level activities such as tweaking data or trying out various algorithmic options and model tuning. These two challenges -- the dearth of data scientists, and time-consuming low-level activities -- have stimulated AI researchers and system builders to explore an automated solution for DS/ML work: Automated Data Science (AutoML). Several AutoML algorithms and systems have been built to automate the various stages of the DS/ML lifecycle. For example, the ETL (extract/transform/load) task has been applied to the data readiness, pre-processing & cleaning stage, and has attracted research attention.
Belief Evolution Network: Probability Transformation of Basic Belief Assignment and Fusion Conflict Probability
Zhou, Qianli, Huang, Yusheng, Deng, Yong
We give a new interpretation of basic belief assignment transformation into probability distribution, and use directed acyclic network called belief evolution network to describe the causality between the focal elements of a BBA. On this basis, a new probability transformations method called full causality probability transformation is proposed, and this method is superior to all previous method after verification from the process and the result. In addition, using this method combined with disjunctive combination rule, we propose a new probabilistic combination rule called disjunctive transformation combination rule. It has an excellent ability to merge conflicts and an interesting pseudo-Matthew effect, which offer a new idea to information fusion besides the combination rule of Dempster.
Neural Dependency Coding inspired Multimodal Fusion
Information integration from different modalities is an active area of research. Human beings and, in general, biological neural systems are quite adept at using a multitude of signals from different sensory perceptive fields to interact with the environment and each other. Recent work in deep fusion models via neural networks has led to substantial improvements over unimodal approaches in areas like speech recognition, emotion recognition and analysis, captioning and image description. However, such research has mostly focused on architectural changes allowing for fusion of different modalities while keeping the model complexity manageable. Inspired by recent neuroscience ideas about multisensory integration and processing, we investigate the effect of synergy maximizing loss functions. Experiments on multimodal sentiment analysis tasks: CMU-MOSI and CMU-MOSEI with different models show that our approach provides a consistent performance boost.
KISS the 288 View of Your Customer
Much has been written about the power of our massive data collections to enable the 360 view of our customers, our business, our employees, and our processes. When our numerous disparate heterogeneous data collections are aggregated and joined in our data lake or data cloud or data fabric or wherever, with appropriate data tagging, data discovery and data integration tools in place, then we can reach for that ideal: the 360 view of our domain! But is the "360 view" really the right goal? It is definitely a good target and we should incentivize productive work toward that ambition, but should we go all the way to achieving that full 360 view in all projects, at all times? Most of us have probably learned by now the truth in the statement "the perfect is the enemy of good enough."
Data Science- The basic necessity of Business Intelligence
Data Science, as the name suggests is the science that handles data, large amounts, and increasing volumes of data and focuses on using it to draw conclusive patterns to presenting the overall results. This multidisciplinary field encompasses multiple levels of data analysis and data structuring that would give out insightful patterns and help Business Intelligence (BI) take technical expertise to the upper notch through its data-driven decision-making. How Data Science works in Business Intelligence? It also uses SQL for data mining and data integration purposes. The data is modified according to defined sets of mapping rules using Extract Transform & Load (ETL) tools to manipulate data and get on with data structuring.
Fivetran Raises $565 Million, Buys CDC Vendor HVR
Fivetran took a big step into the world of enterprise data integration today when it announced an Andreessen Horowitz-led $565 million round of financing and plans to acquire change data capture (CDC) vendor HVR for $700 million. The move positions the up-and-coming ETL company to further access exabytes of data stored in on-prem databases and ERP systems on behalf of its customers. The Series C round and acquisition position Fivetran to be at the forefront of the next generation of data integration and extract, transform, and load (ETL) capabilities. The nine-year-old, Oakland, California company has made its mark by simplifying the process of setting up pipelines that extract data from source systems–primarily SaaS applications running on clouds–and load it into cloud-based data warehouses. Today's news will help to expand Fivetran's footprint with on-prem systems, including the ERP applications at the heart of established enterprises.
Multimodal Classification: Current Landscape, Taxonomy and Future Directions
Sleeman, William C. IV, Kapoor, Rishabh, Ghosh, Preetam
Multimodal classification research has been gaining popularity in many domains that collect more data from multiple sources including satellite imagery, biometrics, and medicine. However, the lack of consistent terminology and architectural descriptions makes it difficult to compare different existing solutions. We address these challenges by proposing a new taxonomy for describing such systems based on trends found in recent publications on multimodal classification. Many of the most difficult aspects of unimodal classification have not yet been fully addressed for multimodal datasets including big data, class imbalance, and instance level difficulty. We also provide a discussion of these challenges and future directions.
Multimodal Data Fusion in High-Dimensional Heterogeneous Datasets via Generative Models
Yilmaz, Yasin, Aktukmak, Mehmet, Hero, Alfred O.
The commonly used latent space embedding techniques, such as Principal Component Analysis, Factor Analysis, and manifold learning techniques, are typically used for learning effective representations of homogeneous data. However, they do not readily extend to heterogeneous data that are a combination of numerical and categorical variables, e.g., arising from linked GPS and text data. In this paper, we are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. The learned generative model provides latent unified representations that capture the factors common to the multiple dimensions of the data, and thus enable fusing multimodal data for various machine learning tasks. Following a Bayesian approach, we propose a general framework that combines disparate data types through the natural parameterization of the exponential family of distributions. To scale the model inference to millions of instances with thousands of features, we use the Laplace-Bernstein approximation for posterior computations involving nonlinear link functions. The proposed algorithm is presented in detail for the commonly encountered heterogeneous datasets with real-valued (Gaussian) and categorical (multinomial) features. Experiments on two high-dimensional and heterogeneous datasets (NYC Taxi and MovieLens-10M) demonstrate the scalability and competitive performance of the proposed algorithm on different machine learning tasks such as anomaly detection, data imputation, and recommender systems.