Compressing Deep Neural Network (DNN) models to alleviate the storage and computation requirements is essential for practical applications, especially for resource limited devices. Although capable of reducing a reasonable amount of model parameters, previous unstructured or structured weight pruning methods can hardly truly accelerate inference, either due to the poor hardware compatibility of the unstructured sparsity or due to the low sparse rate of the structurally pruned network. Aiming at reducing both storage and computation, as well as preserving the original task performance, we propose a generalized weight unification framework at a hardware compatible micro-structured level to achieve high amount of compression and acceleration. Weight coefficients of a selected micro-structured block are unified to reduce the storage and computation of the block without changing the neuron connections, which turns to a micro-structured pruning special case when all unified coefficients are set to zero, where neuron connections (hence storage and computation) are completely removed. In addition, we developed an effective training framework based on the alternating direction method of multipliers (ADMM), which converts our complex constrained optimization into separately solvable subproblems. Through iteratively optimizing the subproblems, the desired micro-structure can be ensured with high compression ratio and low performance degradation. We extensively evaluated our method using a variety of benchmark models and datasets for different applications. Experimental results demonstrate state-of-the-art performance.

This paper is intended to solve the nonconvex $\ell_{p}$-ball constrained nonlinear optimization problems. An iteratively reweighted method is proposed, which solves a sequence of weighted $\ell_{1}$-ball projection subproblems. At each iteration, the next iterate is obtained by moving along the negative gradient with a stepsize and then projecting the resulted point onto the weighted $\ell_{1}$ ball to approximate the $\ell_{p}$ ball. Specifically, if the current iterate is in the interior of the feasible set, then the weighted $\ell_{1}$ ball is formed by linearizing the $\ell_{p}$ norm at the current iterate. If the current iterate is on the boundary of the feasible set, then the weighted $\ell_{1}$ ball is formed differently by keeping those zero components in the current iterate still zero. In our analysis, we prove that the generated iterates converge to a first-order stationary point. Numerical experiments demonstrate the effectiveness of the proposed method.

In this paper, we revisit the problem of smoothed online learning, in which the online learner suffers both a hitting cost and a switching cost, and target two performance metrics: competitive ratio and dynamic regret with switching cost. To bound the competitive ratio, we assume the hitting cost is known to the learner in each round, and investigate the greedy algorithm which simply minimizes the weighted sum of the hitting cost and the switching cost. Our theoretical analysis shows that the greedy algorithm, although straightforward, is $1+ \frac{2}{\alpha}$-competitive for $\alpha$-polyhedral functions, $1+O(\frac{1}{\lambda})$-competitive for $\lambda$-quadratic growth functions, and $1 + \frac{2}{\sqrt{\lambda}}$-competitive for convex and $\lambda$-quadratic growth functions. To bound the dynamic regret with switching cost, we follow the standard setting of online convex optimization, in which the hitting cost is convex but hidden from the learner before making predictions. We modify Ader, an existing algorithm designed for dynamic regret, slightly to take into account the switching cost when measuring the performance. The proposed algorithm, named as Smoothed Ader, attains an optimal $O(\sqrt{T(1+P_T)})$ bound for dynamic regret with switching cost, where $P_T$ is the path-length of the comparator sequence. Furthermore, if the hitting cost is accessible in the beginning of each round, we obtain a similar guarantee without the bounded gradient condition.

Opportunistic relay selection (ORS) has been recognized as a simple but efficient method for mobile nodes to achieve cooperative diversity in slow fading channels. With the proliferation of high-mobility applications and the adoption of higher frequency bands in 5G and beyond systems, the problem of outdated CSI will become more serious. Therefore, the design of a novel cooperative method that is applicable to not only slow fading but also fast fading is increasingly of importance. To this end, we develop and analyze a deep-learning-aided cooperative method coined predictive relay selection (PRS) in this article. It can remarkably improve the quality of CSI through fading channel prediction while retaining the simplicity of ORS by selecting a single opportunistic relay so as to avoid the complexity of multi-relay coordination and synchronization. Information-theoretic analysis and numerical results in terms of outage probability and channel capacity reveal that PRS achieves full diversity gain in slow fading wireless environments and substantially outperforms the existing schemes in fast fading channels. N wireless communications [1], diversity is an important and essential technique, which can effectively combat the effect of multi-path channel fading by means of transmitting redundant signals over independent channels and then combining multiple faded copies at the receiver. Spatial diversity is particularly attractive as it can be easily combined with other forms of diversity and achieve higher diversity order by simply installing more antennas. Because of the constraint on power supply, hardware size, and cost, it is difficult for mobile terminals in cellular systems or wireless nodes in ad hoc networks to exploit spatial diversity at sub-6GHz carrier frequencies. W. Jiang is with German Research Centre for Artificial Intelligence (DFKI), Kaiserslautern, Germany, and is also with the University of Kaiserslautern, Germany, (e-mail: wei.jiang@dfki.de). H. D. Schotten is with German Research Centre for Artificial Intelligence (DFKI), Kaiserslautern, Germany, and is also with the University of Kaiserslautern, Germany, (e-mail: schotten@eit.uni-kl.de). In such a cooperative network, when a node sends a signal, its neighboring nodes could act as relays to decode-and-forward (DF) or amplify-and-forward (AF) this signal. By combining multiple copied versions of the original signal at the destination, the network achieves cooperative diversity that is equivalent to spatial diversity gained from co-located multi-antenna systems [4].