$\texttt{LM}^\texttt{2}$: A Simple Society of Language Models Solves Complex Reasoning