Convolutional neural networks (CNNs) are deep learning fra meworks which are well-known for their notable performance in classification tasks. Hence, many skeleton-based action recognition and segmentation (SBAR S) algorithms benefit from them in their designs. However, a shortcoming of such applications is the general lack of spatial relationships between the input features in such data types. Besides, nonuniform temporal scalings is a common i ssue in skeleton-based data streams which leads to having different input siz es even within one specific action category. In this work, we propose a novel dee p-aligned convolu-tional neural network (DACNN) to tackle the above challenge s for the particular problem of SBARS. Our network is designed by introducing a ne w type of filters in the context of CNNs which are trained based on their alignm ents to the local subsequences in the inputs. These filters result in efficient predictions as well as learning interpretable patterns in the data. W e empiricall y evaluate our framework on real-world benchmarks showing that the proposed DACNN al gorithm obtains a competitive performance compared to the state-of-the-ar t while benefiting from a less complicated yet more interpretable model.