O-MAPL: Offline Multi-agent Preference Learning