LLM enhanced graph inference for long-term disease progression modelling