What Limits LLM-based Human Simulation: LLMs or Our Design?