Compressing Large Language Models using Low Rank and Low Precision Decomposition