Efficient In-Memory Acceleration of Sparse Block Diagonal LLMs