Context is Key for Agent Security