Understanding the neural basis of behavior is a fundamental goal in neuroscience. Current research in large-scale neuro-behavioral data analysis often relies on decoding models, which quantify behavioral information in neural data but lack details on behavior encoding. This raises an intriguing scientific question: how can we enable in-depth exploration of neural representations in behavioral tasks, revealing interpretable neural dynamics associated with behaviors. However, addressing this issue is challenging due to the varied behavioral encoding across different brain regions and mixed selectivity at the population level. To tackle this limitation, our approach, named (BeNeDiff), first identifies a fine-grained and disentangled neural subspace using a behavior-informed latent variable model. It then employs state-of-the-art generative diffusion models to synthesize behavior videos that interpret the neural dynamics of each latent factor.

Kevin Durant is expected to sit out Game 4 with a sprained left ankle and bone bruise, as the Rockets face elimination against the Lakers on Sunday night.

China's economy was just starting to open up in the late 1980s when a determined high school dropout made his way to Beijing with 600 pairs of shoes. Ding Shizhong had them made in a relative's factory and now he was going to sell them. The money he earned paid for his first workshop where he began making footwear for other companies. The 17-year-old was one of China's many newly minted entrepreneurs as capitalism took off under the watchful eye of its Communist Party rulers. But, as it turns out, Ding had much bigger plans.

We chose the Google Search corpus [Luo et al., 2021] for our question-answering system as it provides good coverage of the knowledge needed and is publicly available. However, as noted by the authors of RA-VQA, additional knowledge bases may be required to answer some questions correctly. Future work may address the issue by improving the quality and expanding the coverage of knowledge. We do not perceive any immediate ethical concerns associated with the misuse of our proposed system. There is a possibility that the trained KB-VQA system might generate inappropriate or biased content as a result of the training data biases during LLM and LMM pre-training and fine-tuning.

Knowledge-based Visual Question Answering (KB-VQA) requires VQA systems to utilize knowledge from external knowledge bases to answer visually-grounded questions. Retrieval-Augmented Visual Question Answering (RA-VQA), a strong framework to tackle KB-VQA, first retrieves related documents with Dense Passage Retrieval (DPR) and then uses them to answer questions. This paper proposes Fine-grained Late-interaction Multi-modal Retrieval (FLMR) which significantly improves knowledge retrieval in RA-VQA. FLMR addresses two major limitations in RA-VQA's retriever: (1) the image representations obtained via image-to-text transforms can be incomplete and inaccurate and (2) relevance scores between queries and documents are computed with one-dimensional embeddings, which can be insensitive to finer-grained relevance. FLMR overcomes these limitations by obtaining image representations that complement those from the image-totext transforms using a vision model aligned with an existing text-based retriever through a simple alignment network. FLMR also encodes images and questions using multi-dimensional embeddings to capture finer-grained relevance between queries and documents. FLMR significantly improves the original RA-VQA retriever's PRRecall@5 by approximately 8%. Finally, we equipped RA-VQA with two state-of-the-art large multi-modal/language models to achieve 61% VQA score in the OK-VQA dataset.

Lebanon's Ministry of Health has said Israeli strikes on the country on Sunday killed 14 people, including two children and two women, and injured 37. An Israel Defense Forces (IDF) spokesperson had earlier issued evacuation warnings for several villages in southern Lebanon, writing that residents must evacuate immediately, and that staying would be endangering their life. The IDF later said it had carried out artillery and aerial strikes targeting Hezbollah operatives and sites in southern Lebanon that it claims were used to advance attacks against IDF soldiers. It also said a 19-year-old IDF soldier had been killed and six others injured by a Hezbollah drone attack in Lebanon. Separately, Hezbollah launched three drones towards Israel, the IDF reported, which it said were intercepted by Israel's air force before they crossed the border.

Provably efficient Model-Based Reinforcement Learning (MBRL) based on optimism or posterior sampling (PSRL) is ensured to attain the global optimality asymptotically by introducing the complexity measure of the model. However, the complexity might grow exponentially for the simplest nonlinear models, where global convergence is impossible within finite iterations. When the model suffers a large generalization error, which is quantitatively measured by the model complexity, the uncertainty can be large. The sampled model that current policy is greedily optimized upon will thus be unsettled, resulting in aggressive policy updates and over-exploration. In this work, we propose Conservative Dual Policy Optimization (CDPO) that involves a Referential Update and a Conservative Update. The policy is first optimized under a reference model, which imitates the mechanism of PSRL while offering more stability. A conservative range of randomness is guaranteed by maximizing the expectation of model value. Without harmful sampling procedures, CDPO can still achieve the same regret as PSRL. More importantly, CDPO enjoys monotonic policy improvement and global optimality simultaneously.

Mainstream approaches for unsupervised domain adaptation (UDA) learn domaininvariant representations to narrow the domain shift, which are empirically effective but theoretically challenged by the hardness or impossibility theorems. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. In this paper, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains.

Our default model has 3RSTB-D downsampling blocks, 2RSTB-B bottleneck blocks and 3RSTB-U upsampling blocks with 3 6 12 attention heads in the D/U blocks and 24 attention heads in the bottleneck block. For Swin Transformers layers, the window size is 8 for all methods and MLP ratio (hidden_dim/input_dim) of 2 is used. Each RSTB block consists of 2 STLs with embedding dimension of 66. For HUMUS-Net-L, we increase the embedding dimension to 96. We use 8cascades of unrolling with a U-Net as sensitivity map estimator (same as in E2E-VarNet) with 16channels.