Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models