number cruncher
LLMs are Not Just Next Token Predictors
Downes, Stephen M., Forber, Patrick, Grzankowski, Alex
LLMs are statistical models of language learning through stochastic gradient descent with a next token prediction objective. Prompting a popular view among AI modelers: LLMs are just next token predictors. While LLMs are engineered using next token prediction, and trained based on their success at this task, our view is that a reduction to just next token predictor sells LLMs short. Moreover, there are important explanations of LLM behavior and capabilities that are lost when we engage in this kind of reduction. In order to draw this out, we will make an analogy with a once prominent research program in biology explaining evolution and development from the genes eye view. LLMs are statistical models of language learning through stochastic gradient descent with a next token prediction objective. So, LLMs are'just next token predictors', a popular view among AI modelers, explicitly laid out by Shanahan (2024): "A great many tasks that demand intelligence in humans can be reduced to next-token prediction with a sufficiently performant model" (2024, 68), and "surely what they are doing is more than'just' next-token prediction? Well, it is an engineering fact that this is what an LLM does. The noteworthy thing is that next-token prediction is sufficient for solving previously unseen reasoning problems" (2024, 77).
Number crunchers in demand as data, AI startups see potential - Times of India
CHENNAI: With a PhD in mathematics, Bharat Ramakrishna was preparing content for school children when suddenly he found a well-paying job in the machine learning & data sciences space. No longer is a mathematics background purely academic. Maths majors are now in demand for a job in artificial intelligence and data sciences. "After graduating from the University of Utah, I was into preparing question banks for students. Now, concepts such as matrices, linear algebra and calculus are being used in artificial intelligence and it is easier for a mathematics graduate to learn coding than vice versa," said Ramakrishna, data scientist at Skillenza.
What Artificial Intelligence Means for Markets & Investing
Artificial intelligence and machine learning is suddenly all the rage, and for good reason. It is the future of this, and every other industry. If you've been paying attention to the evolution of technology over the past 2.6 million years, you knew it was coming. Wherever the bulk of the effort has been shouldered by human beings, we have always sought to replace us with technology that could do the job better, faster, more efficiently and, since the invention of capital, cheaper. It began with the most basic, brute force physical tasks and has progressively involved more nuanced, cognitive processes.
Tomorrow's accountant will be a business advisor rather than a number cruncher
The rise of artificial intelligence (AI) and the growing maturity of cloud-based business software promise to dramatically change the role of the accountant over the next five to 10 years. Savvy professionals should already be reskilling themselves in anticipation of the shift in the market. The Finance Indaba is taking place today and tomorrow – it's the perfect platform for finance professionals to gear up and learn about the changing accounting world from leaders in the industry He says that the arrival of smart software bots, paired with the affordability of cloud-based business applications, will change the way that accountants work as vividly as the first spreadsheet and accounting software packages did. "Financial software is getting smarter, more affordable and easier to use, so more and more of the admin accountants typically do for the business is becoming automated," Cohen says. "What's more, intuitive software paired with AI and other new developments, could empower small business owners do more of the tasks they used to entrust to an accountant."