Align on the Fly: Adapting Chatbot Behavior to Established Norms