Are Long-LLMs A Necessity For Long-Context Tasks?