Goto

Collaborating Authors

 computing science


Evaluating the Limitations of Local LLMs in Solving Complex Programming Challenges

Matotek, Kadin, Cassel, Heather, Amiruzzaman, Md, Ngo, Linh B.

arXiv.org Artificial Intelligence

This study examines the performance of today's open-source, locally hosted large-language models (LLMs) in handling complex competitive programming tasks with extended problem descriptions and contexts. Building on the original Framework for AI-driven Code Generation Evaluation (FACE), the authors retrofit the pipeline to work entirely offline through the Ollama runtime, collapsing FACE's sprawling per-problem directory tree into a handful of consolidated JSON files, and adding robust checkpointing so multi-day runs can resume after failures. The enhanced framework generates, submits, and records solutions for the full Kattis corpus of 3,589 problems across eight code-oriented models ranging from 6.7-9 billion parameters. The submission results show that the overall pass@1 accuracy is modest for the local models, with the best models performing at approximately half the acceptance rate of the proprietary models, Gemini 1.5 and ChatGPT-4. These findings expose a persistent gap between private, cost-controlled LLM deployments and state-of-the-art proprietary services, yet also highlight the rapid progress of open models and the practical benefits of an evaluation workflow that organizations can replicate on in-house hardware.


Computing Everywhere, for Everyone, at Any Level

Communications of the ACM

Over the past few decades, computing has emerged as a transformative discipline, significantly reshaping the world. Virtually all knowledge areas now use computing to enhance their applicability, and even everyday activities often require some computing knowledge. Recent advancements in artificial intelligence (AI) have highlighted the profound impact computing has on our lives to everyone. Therefore, it is desirable that students of all levels have age-appropriate familiarity with computing.5,6 Several international efforts, including frameworks and reports, have emerged to address the development of early computational skills.


Research Re: search & Re-search

Plaat, Aske

arXiv.org Artificial Intelligence

Search algorithms are often categorized by their node expansion strategy. One option is the depth-first strategy, a simple backtracking strategy that traverses the search space in the order in which successor nodes are generated. An alternative is the best-first strategy, which was designed to make it possible to use domain-specific heuristic information. By exploring promising parts of the search space first, best-first algorithms are usually more efficient than depth-first algorithms. In programs that play minimax games such as chess and checkers, the efficiency of the search is of crucial importance. Given the success of best-first algorithms in other domains, one would expect them to be used for minimax games too. However, all high-performance game-playing programs are based on a depth-first algorithm. This study takes a closer look at a depth-first algorithm, AB, and a best-first algorithm, SSS. The prevailing opinion on these algorithms is that SSS offers the potential for a more efficient search, but that its complicated formulation and exponential memory requirements render it impractical. The theoretical part of this work shows that there is a surprisingly straightforward link between the two algorithms -- for all practical purposes, SSS is a special case of AB. Subsequent empirical evidence proves the prevailing opinion on SSS to be wrong: it is not a complicated algorithm, it does not need too much memory, and it is also not more efficient than depth-first search.


ChatGPT: Handle with care and don't be fooled into thinking it's human

#artificialintelligence

People yelling at broken computers was a popular YouTube genre in the 2000s. If you think this is unique to non-digital boomers when it comes to new technologies, think again. Something similar is happening today with ChatGPT, the chatbot recently launched by OpenAI, which has already generated hype not seen since the Metaverse was considered the next big thing (seems like eons ago, doesn't it?). We are not yelling at ChatGPT, but we interact with it as if it were a person, sometimes even accusing it of lying. "And, in a way, it's natural," says Heather Yang, an Assistant Professor at Bocconi Department of Management and Technology, whose research focuses on how people interact with novel technologies and how that is changing our workplace environment.


PhD student in Computing Science with focus on responsible machine learning

#artificialintelligence

The Department of Computer Science, characterized by world-leading research in several scientific fields and a multitude of educations ranked highly in international comparison, is looking for a Doctoral student in computing science with a focus on responsible AI with learning from multiple representations. The Department of Computing science has been growing rapidly in recent years where focus on an inclusive and bottom-up driven environment are key elements in our sustainable growth. The 60 Doctoral students within the department consists of a diverse group from different nationalities, background and fields. If you work as a Doctoral student with us you receive the benefits of support in career development, networking, administrative and technical support functions along with good employment conditions. Is this interesting for you?



2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

#artificialintelligence

Lawrence Berkeley National Lab is hiring for Full Time 2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences - San Francisco, CA - a Mid-level AI/ML/Data Science role offering benefits such as Career development, Competitive pay, Equity, Relocation support


4 European universities preparing students for Industry 4.0

#artificialintelligence

According to the US Bureau of Labour Statistics, "Employment of computer and information technology occupations is projected to grow 12% from 2018 to 2028, much faster than the average for all occupations. These occupations are projected to add about 546,200 new jobs." We are at the cusp of the Fourth Industrial Revolution, where the physical, digital and biological worlds are merging in unprecedented forms and scale. Yet, despite the number of STEM jobs flourishing, less than a third (29.3%) of those employed in scientific research and development across the world in 2016 are women. Eurostat found that in 2020, of almost 73 million persons employed in science and technology in the EU, aged from 15 to 74, nearly 37.5 million were women (51.3%) and 35.5 million men (48.7%).


Postdoc Position in Artificial Intelligence - Sweden

#artificialintelligence

Applicants must have earned a PhD in Artificial Intelligence, interaction design, human-computer interaction, or similar subjects relevant for the position. The PhD degree should not be more than three years old by the application deadline, unless special circumstances exist. The candidate is expected to have an overall interest in responsible AI concepts and methods, and expertise in participatory methods and interaction design, as demonstrate by publications and other scientific output. Proficiency in English, both spoken and written, is required. Ideal candidates are research driven, organized, and would like to work on challenging problems and innovative solutions.


New AI-powered deep learning model to support medical diagnostics

#artificialintelligence

A new deep-learning model can learn to identify diseases from medical scans faster and more accurately, according to new research by a team of University of Alberta computing scientists and the U of A spinoff company MEDO. The breakthrough model is the work of a team of researchers in the Faculty of Science--including the contributions of Pouneh Gorji, a graduate student lost in Flight PS752. Deep learning is a type of machine learning--a subfield of artificial intelligence; deep learning techniques are computer algorithms that find patterns in large sets of data, producing models that can then be used to make predictions.These models work best when they learn from hundreds of thousands or even millions of examples. But the field of medical diagnostics presents a unique challenge, where researchers typically only have access to a few hundred medical scan images for reasons of privacy. "When a deep-learning model is trained with so few instances, its performance tends to be poor," said Roberto Vega, lead author of the study and graduate student in the Department of Computing Science.