Since the increasing popularity of large language model (LLM) backend systems, it is common and necessary to deploy stable serverless serving of LLM on multi-GPU clusters with autoscaling. However, there exist challenges because the diversity and co-location of applications in multi-GPU clusters will lead to low service quality and GPU utilization. To address them, we build ENOVA, a deployment, monitoring and autoscaling service towards serverless LLM serving. ENOVA deconstructs the execution process of LLM service comprehensively, based on which ENOVA designs a configuration recommendation module for automatic deployment on any GPU clusters and a performance detection module for autoscaling. On top of them, ENOVA implements a deployment execution engine for multi-GPU cluster scheduling. The experiment results show that ENOVA significantly outperforms other state-of-the-art methods and is suitable for wide deployment in large online systems.

Joe DeCosmo has 25 years of experience in technology, machine learning and AI. He is Chief Analytics and Technology Officer at Enova. Technological change accelerated during the pandemic, leading many people to adopt new ways to complete everyday tasks. Online tools and mobile applications have exploded for everything from shopping to food delivery and even financial services. Fintechs have led the way in providing working people with online access to financial services regardless of where they live, what they look like or whether they have an imperfect credit history. Doing so requires technical innovation.

Enova is currently accepting candidates for remote positions in the following eligible states: AL, AK, AR, AZ, CT, GA, IA, ID, IL, IN, KY, LA, MA, ME, MD, MI, MN, MO, MS, NC, ND, NE, NH, NV, NJ, NM, OH, OK, OR, PA, RI, SC, SD, TN, UT, VT, WI, WV, WY. The Data Services Team effectively and sustainably builds data strategy and provides data solutions and tools across the organization. We integrate, transform, and improve volumes of data at the project or enterprise level for streamlined processes, greater efficiencies, and smarter, more informed decision-making. This team is high-energy, dynamic and in a business-critical domain space. This role is an opportunity to make a real difference in the data space, and we need confident, experienced people eager to bring in solutions, with a demonstrated ability to learn fast and make that happen.

The online lender Enova, which has been using artificial intelligence to make credit decisions for years, has lately been expanding its use of AI to handle additional tasks: spotting fraud, determining who should receive product offers, and projecting possible losses once a loan is booked. Most recently, it has also been been having AI scour paper documents to uncover false information, verify income and employment, and conduct know-your-customer checks. It also sponsored a survey, conducted by Harvard Business Review and set to be released Thursday, that benchmarks how businesses are using AI. Generally speaking, it finds companies are adopting AI slowly: though 68% of executives say AI will be a competitive differentiator within the next year and 64% are investigating or piloting AI projects, only 15% of organizations have AI-powered analytics in place due to technical and cultural challenges. In its back office, Enova has been training AI engines to review documents such as bank statements and pay stubs and automating decisions based on the information in those documents.

With self-driving cars cruising around, robots doing backflips and helping each other open doors, computers learning how to play GO in a few days and then beating experts who spent their lives mastering the game, we are definitely witnessing an exciting era in human history. Like Enova's CTO John Higginson said in his blog post, as an analytics and technology company we want to use and even seek to extend these technologies to invent the future, but for credit. That's exactly why our executive team picked'advancing our machine learning capabilities' as one of our strategic initiatives this year. Usually when people see these amazing advancements in technology the first thing they think about is how machines are taking over our jobs. In the lending industry, however, the takeover has already happened.