We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs $X \in \mathbb R^{d \times N}$ and outputs $Y \in \mathbb R^{m \times N}$, while capturing the correlation structure among the $Y$. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify and discard irrelevant features, thereby promoting sparsity in the model. Algorithmically, it iteratively updates both the precision matrix and the relationship between $Y$ and the refined inputs. To mitigate the computational inefficiencies of the $\mathcal O(m^3 + d^3)$ cost per iteration, we introduce Sequential NARD, which evaluates features sequentially, and a Surrogate Function Method, leveraging an efficient approximation of the marginal likelihood and simplifying the calculation of determinant and inverse of an intermediate matrix. Combining the Sequential update with the Surrogate Function method further reduces computational costs. The computational complexity per iteration for these three methods is reduced to $\mathcal O(m^3+p^3)$, $\mathcal O(m^3 + d^2)$, $\mathcal O(m^3+p^2)$, respectively, where $p \ll d$ is the final number of features in the model. Our methods demonstrate significant improvements in computational efficiency with comparable performance on both synthetic and real-world datasets.

In my experience, when you ask a person what their favorite film is, you'll often be told the title of the film that made the most impression on them at a most impressionable age. For some, it was "The Monster Squad" (1987). A pastiche of "The Goonies," featuring a group of suburban American kids up against the classic Universal monsters, instead of a pirate, the film mixes horror and comedy in the style of those Abbott & Costello Universal film spoofs (some of which are very good). "The Monster Squad" was not very well received by the critics (me included) or the public, which stayed away in droves to quote Sam Goldwyn ("The Lost Boys" preceded it by two weeks). But a funny thing happened.