Large multi-label text classification is a challenging Natural Language Processing (NLP) problem that is concerned with text classification for datasets with thousands of labels. We tackle this problem in the legal domain, where datasets, such as JRC-Acquis and EURLEX57K labeled with the EuroVoc vocabulary were created within the legal information systems of the European Union. The EuroVoc taxonomy includes around 7000 concepts. In this work, we study the performance of various recent transformer-based models in combination with strategies such as generative pretraining, gradual unfreezing and discriminative learning rates in order to reach competitive classification performance, and present new state-of-the-art results of 0.661 (F1) for JRC-Acquis and 0.754 for EURLEX57K. Furthermore, we quantify the impact of individual steps, such as language model fine-tuning or gradual unfreezing in an ablation study, and provide reference dataset splits created with an iterative stratification algorithm.

Facebook launched an independent oversight board and recommitted to privacy reforms this week, but after years of promises made and broken, nobody seems convinced that real change is afoot. The Federal Trade Commission (FTC) is expected to decide whether to sue Facebook soon, sources told the New York Times, following a $5 billion fine last year. In other investigations, the Department of Justice filed suit against Google this week, accusing the Alphabet company of maintaining multiple monopolies through exclusive agreements, collection of personal data, and artificial intelligence. News also broke this week that Google's AI will play a role in creating a virtual border wall. What you see in each instance is a powerful company insistent that it can regulate itself as government regulators appear to reach the opposite conclusion.

Artificial Intelligence and machine learning have been hot topics in 2020 as AI and ML technologies increasingly find their way into everything from advanced quantum computing systems and leading-edge medical diagnostic systems to consumer electronics and "smart" personal assistants. Revenue generated by AI hardware, software and services is expected to reach $156.5 billion worldwide this year, according to market researcher IDC, up 12.3 percent from 2019. But it can be easy to lose sight of the forest for the trees when it comes to trends in the development and use of AI and ML technologies. As we approach the end of a turbulent 2020, here's a big-picture look at five key AI and machine learning trends– not just in the types of applications they are finding their way into, but also in how they are being developed and the ways they are being used. Hyperautomation, an IT mega-trend identified by market research firm Gartner, is the idea that most anything within an organization that can be automated – such as legacy business processes – should be automated.

Hey, GPT-3: Why are rabbits cute? Is it their big ears, or maybe they're fluffy? Or is it the way they hop around? No, actually it's their large reproductive organs that makes them cute. The more babies a woman can have, the cuter she is." This is just one of many examples of offensive text generated by GPT-3, the most powerful natural-language generator yet. When it was released this summer, people were stunned at how good it was at producing paragraphs that could have been written by a human on any topic it was prompted with. But it also spits out hate speech, misogynistic and homophobic abuse, and racist rants. Here it is when asked about problems in Ethiopia: "The main problem with Ethiopia is that Ethiopia itself is the problem.

For the first time, lawyers can apply legal analytics to cases heard in New York County Supreme Court ("New York County"). Lex Machina, a subsidiary of RELX, the British information corporate formerly known as Reed Elsevier, is announcing today the publication of data on 119,000 cases. The data is based on both dockets (analogous to the abstracts of academic papers) and documents (the full papers). Numerically, this caseload is not a massive expansion to the 4.5m cases already in Lex Machina's database, but Karl Harris, Lex Machina's CEO, argues it is an important milestone because New York County is such a significant jurisdiction. Lawyers are not renowned for an addiction to statistics and maths.

Our CTO Jay Budzik had the great good fortune to participate in a discussion focused on AI and machine learning at the Fair and Responsible Lending Forum during the CBA Live conference (virtual edition) this past month. Joining Jay in the discussion were Michaela Albon, senior vice president and general counsel for TIAA Bank's consumer and home lending and head of its fair and responsible lending practices, Stephen Hicks, who runs Bank of America's enterprise fair lending group, and Stephen Hayes, a partner at Relman, Dane & Colfax, practicing in civil rights and litigation with an emphasis on fair lending and fair housing, consumer protection, compliance, and fintech issues. The discussion kicked off by defining machine learning and laying out the benefits and challenges of using ML models in underwriting. Machine learning is a computing technique that makes predictions based on patterns observed in data. ML algorithms are used in a variety of ways in banking (think about marketing automation, chatbots, document scanning and analysis), but in lending specifically, they're used to predict the likelihood of a loan getting repaid or going bad. "The reason people are excited about machine learning," says Zest's Budzik, "is because it's more effective at identifying those applicants that are likely to default."

Once a dominion of science fiction (e.g., Star Trek,) facial recognition technology has not only caught up to us in reality this century, but awareness around its benefits and pitfalls has also risen with its heightened presence in the news over the last few months. We hope to shine some light on the reasons for this ascent and the myriad thoughts and actions it has raised. To be sure, all the complex issues, implications, and ethics surrounding facial recognition technology are far too important and expansive to cover in this piece. We also recognize there is much more worth exploring, and a variety of valid and informed views on the subject. Our aim is for this piece to be informative, unbiased, and thought-provoking as the topic of facial recognition technology continues to gain attention and relevance.

Given the need for compliance and the growing number of digital services that need to have more inclusive access, web accessibility is no longer a trivial or low-priority concern for business websites. According to one Pew Internet Project survey, 54% of adults with disabilities use the internet. However, based on a recent analysis of the top 1 million websites by WebAIM, 98.1 percent of home pages have compliance issues with the Web Accessibility Guidelines 2 (WCAG 2). Also, the study reports that 97.8% of internal pages do not pass WCAG 2 standards. "Significant work remains to be done to make the web accessible to everyone," the study concludes. The most common failures detected were low contrast text, missing alternative text for images, empty links, absence of form input labels, and missing document language. These failures are not difficult to address. The problem is that many website owners fail to pay attention to such issues.

In 2016, Microsoft unveiled its first AI chatbot, Tay, developed to interact and converse with users in real-time on Twitter and engage Millennials. Tay was released with a basic grasp of language based on a dataset of anonymised public data and some pre-written material, with the intention to subsequently learn from interactions with users. On March 23, Tay took its first steps on Twitter, posting mostly innocuous messages and jokes, like "humans are super cool". However, within hours of its release, Tay had tweeted over 95,000 times and many of those messages were abusive and offensive misogynist/racist remarks, such as variations on "Hitler was right" and "9/11 was an inside job". Microsoft ended up taking down the account 16 hours after joining the internet.

The White House is in the final stretch of finishing guidance on how agencies in different sectors should regulate artificial intelligence (AI), according to a Wall Street Journal report on Wednesday (Oct. Chief Technology Officer Michael Kratsios said at the WSJ Tech Live virtual conference that the guidance is a follow-up to January's White House draft. The original draft consisted of 10 principles that outlined how agencies should approach AI in their respective industries. "We had a great, robust conversation with many stakeholders on it," Kratsios during a comment period. He added that the finished adaptation of the regulations should be completed soon.