In the last few years, three major topics received increased interest: deep learning, NLP and conversational agents. Bringing these three topics together to create an amazing digital customer experience and indeed deploy in production and solve real-world problems is something innovative and disruptive. We introduce a new Portuguese financial domain language representation model called BERTa\'u. BERTa\'u is an uncased BERT-base trained from scratch with data from the Ita\'u virtual assistant chatbot solution. Our novel contribution is that BERTa\'u pretrained language model requires less data, reached state-of-the-art performance in three NLP tasks, and generates a smaller and lighter model that makes the deployment feasible. We developed three tasks to validate our model: information retrieval with Frequently Asked Questions (FAQ) from Ita\'u bank, sentiment analysis from our virtual assistant data, and a NER solution. All proposed tasks are real-world solutions in production on our environment and the usage of a specialist model proved to be effective when compared to Google BERT multilingual and the DPRQuestionEncoder from Facebook, available at Hugging Face. The BERTa\'u improves the performance in 22% of FAQ Retrieval MRR metric, 2.1% in Sentiment Analysis F1 score, 4.4% in NER F1 score and can also represent the same sequence in up to 66% fewer tokens when compared to "shelf models".

Has digital customer service gone a step too far? We haven't mastered Artificial Intelligence, yet for some reason, it seems like robots and automation are primed to be the next trends for customer service. Automation can save you money in the short run, compared to traditional methods (like human power), because of its increased efficiency. But not all customers are happy with all automated, all the time. While automation and digital customer service are tools you should be using (they save resources and allow you to serve customers faster), making sure your customer service strategy still includes a human element is massively important.

Note: All DigitalGov University events take place in Eastern Time. Join us for an afternoon with innovative public servants and industry leaders and learn how automated services powered by government data and new advances in artificial intelligence and machine learning, including chatbots, can help improve and expand digital customer service initiatives, and make our public services more open, responsive, informative and accessible. For more information, or to request that your organization participate, please contact Justin Herman at justin.herman@gsa.gov.