fixation point
DIJIT: A Robotic Head for an Active Observer
Tabrizi, Mostafa Kamali, Chi, Mingshi, Dey, Bir Bikram, Yuan, Yu Qing, Solbach, Markus D., Liu, Yiqian, Jenkin, Michael, Tsotsos, John K.
We present DIJIT, a novel binocular robotic head expressly designed for mobile agents that behave as active observers. DIJIT's unique breadth of functionality enables active vision research and the study of human-like eye and head-neck motions, their interrelationships, and how each contributes to visual ability. DIJIT is also being used to explore the differences between how human vision employs eye/head movements to solve visual tasks and current computer vision methods. DIJIT's design features nine mechanical degrees of freedom, while the cameras and lenses provide an additional four optical degrees of freedom. The ranges and speeds of the mechanical design are comparable to human performance. Our design includes the ranges of motion required for convergent stereo, namely, vergence, version, and cyclotorsion. The exploration of the utility of these to both human and machine vision is ongoing. Here, we present the design of DIJIT and evaluate aspects of its performance. We present a new method for saccadic camera movements. In this method, a direct relationship between camera orientation and motor values is developed. The resulting saccadic camera movements are close to human movements in terms of their accuracy.
Appendix: A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains Minkyu Choi
Fig. S1 displays the full set of region labels, corresponding to Regions including significant voxels from Fig.3(a) in the main text are As detailed in Section 3.1 of the main text, our model underwent a three-stage training process. After this stage, we conducted a fine-tuning process using the learned fixations from the WhereCNN. In this stage, the WhereCNN, after the pre-training in Stage 1, was incorporated to guide the WhatCNN's fixations. The model samples fixations from the predicted saliency maps from WhereCNN. As indicated in Section 3.1 of the main text, we utilized For All Stages All training stages were conducted using four NVIDIA A40 GPUs. Figure S2: Process of determining the next fixation point given the current fixation.
Saccadic Vision for Fine-Grained Visual Classification
Schmidt, Johann, Stober, Sebastian, Denzler, Joachim, Bodesheim, Paul
Fine-grained visual classification (FGVC) requires distinguishing between visually similar categories through subtle, localized features - a task that remains challenging due to high intra-class variability and limited inter-class differences. Existing part-based methods often rely on complex localization networks that learn mappings from pixel to sample space, requiring a deep understanding of image content while limiting feature utility for downstream tasks. In addition, sampled points frequently suffer from high spatial redundancy, making it difficult to quantify the optimal number of required parts. Inspired by human saccadic vision, we propose a two-stage process that first extracts peripheral features (coarse view) and generates a sample map, from which fixation patches are sampled and encoded in parallel using a weight-shared encoder. We employ contextualized selective attention to weigh the impact of each fixation patch before fusing peripheral and focus representations. To prevent spatial collapse - a common issue in part-based methods - we utilize non-maximum suppression during fixation sampling to eliminate redundancy. Comprehensive evaluation on standard FGVC benchmarks (CUB-200-2011, NABirds, Food-101 and Stanford-Dogs) and challenging insect datasets (EU-Moths, Ecuador-Moths and AMI-Moths) demonstrates that our method achieves comparable performance to state-of-the-art approaches while consistently outperforming our baseline encoder.
Review for NeurIPS paper: Biologically Inspired Mechanisms for Adversarial Robustness
Summary and Contributions: The paper tries to understand the mechanisms that potentially make human vision robust to test-time adversarial attacks. Two biologically plausible mechanisms are considered: 1) Retinal fixation: The first mechanism models the non-uniform sampling of the image performed by the retina due to uneven distribution of cones. This mechanism essentially involves subsampling and upsampling the pixels of the image. The density of sampling is highest at a fixation point on an image and decreases with distance from the fixation point. The final predicted output is the average of predicted output for subsampled images with different fixation points.