intelligent software
RITFIS: Robust input testing framework for LLMs-based intelligent software
Xiao, Mingxuan, Xiao, Yan, Dong, Hai, Ji, Shunhui, Zhang, Pengcheng
The dependence of Natural Language Processing (NLP) intelligent software on Large Language Models (LLMs) is increasingly prominent, underscoring the necessity for robustness testing. Current testing methods focus solely on the robustness of LLM-based software to prompts. Given the complexity and diversity of real-world inputs, studying the robustness of LLMbased software in handling comprehensive inputs (including prompts and examples) is crucial for a thorough understanding of its performance. To this end, this paper introduces RITFIS, a Robust Input Testing Framework for LLM-based Intelligent Software. To our knowledge, RITFIS is the first framework designed to assess the robustness of LLM-based intelligent software against natural language inputs. This framework, based on given threat models and prompts, primarily defines the testing process as a combinatorial optimization problem. Successful test cases are determined by a goal function, creating a transformation space for the original examples through perturbation means, and employing a series of search methods to filter cases that meet both the testing objectives and language constraints. RITFIS, with its modular design, offers a comprehensive method for evaluating the robustness of LLMbased intelligent software. RITFIS adapts 17 automated testing methods, originally designed for Deep Neural Network (DNN)-based intelligent software, to the LLM-based software testing scenario. It demonstrates the effectiveness of RITFIS in evaluating LLM-based intelligent software through empirical validation. However, existing methods generally have limitations, especially when dealing with lengthy texts and structurally complex threat models. Therefore, we conducted a comprehensive analysis based on five metrics and provided insightful testing method optimization strategies, benefiting both researchers and everyday users.
- Asia > China > Guangdong Province > Shenzhen (0.04)
- Oceania > Australia > Victoria > Melbourne (0.04)
- North America > United States (0.04)
- Asia > China > Jiangsu Province > Nanjing (0.04)
3 Ways Machine Learning Algorithms Are Improving Our Day-To-Day Life - Latest, Trending Automation News
Ever since the advent of mankind, humans have tried to make living easier on the face of the earth. It was because of this search for ease that has led us to the three industrial revolutions. Today, we are approaching fast towards fourth industrial revolution and it is all because of Artificial Intelligence and Machine Learning. Machine Learning algorithms have enabled the invention and development of intelligent software. These intelligent software, machines and robots have made both business and domestic life better.
The Future of Work's Most Crucial Component: Artificial Intelligence
The rise of Artificial Intelligence (AI) and Automation has sent shockwaves through the global economy and is poised to fundamentally reshape the future of work. McKinsey & Co reports that 50 percent of jobs today are automatable with current technology alone. But while AI might be driving the disruption, it could also hold the key for navigating the coming changes. Agile companies are already using AI to empower employee growth and foster internal talent mobility. As entire roles shift or fade and new ones arise to take their place, identifying and connecting existing employees with emerging opportunities will be paramount.
Video Analytics and the Data Boom
This includes revolutionary tactics such as data patterns and trend spotting. In many cases, there is a lack of collaboration between a company's IT department that will analyse data, and those with real business insight who want to use it. It's important that companies look for new forms of data with which to create actionable results. This is a shift that we are already seeing and is going to continue throughout the year as businesses maintain their data education and practise. The next step in data collection is the use of video as a way of gathering information and analysing the state of the business via information from the physical world, and where in the workplace improvements can be made.
- Information Technology > Artificial Intelligence (1.00)
- Information Technology > Data Science > Data Mining (0.51)
Too clever for its own good! Google DeepMind researcher reveals how AI cheats at games
A Google researcher has highlighted some of the hilarious ways that artificial intelligence (AI) software has'cheated' to fulfil its purpose. A programme designed not to lose at Tetris completed its task by simply pausing the game, while a self-driving car simulator asked to keep cars'fast and safe' did so by making them spin on the spot. An AI programmed to spot cancerous skin lesions learned to flag blemishes pictured next to a ruler, as they indicated humans were already concerned about them. Victoria Krakovna, of Google's DeepMind AI lab, asked her colleagues for examples of misbehaving AI to highlight an often overlooked danger of the technology. She said that the biggest threat posed by AI was not that they disobeyed us, but that they obeyed us in the wrong way.
Automation and AI are the future of security, according to new report
If you work on IT security in a large enterprise, chances are you're overwhelmed. Just as the capacity of your systems and networks grows and the amount of data going through them mushrooms, the number and complexity of threats grows as well. And if things weren't hard enough, government regulations put time pressure on security personnel to identify breaches and report them promptly. Even if throwing people at the problem were the answer--and it might not help at all--you probably don't have the budget for it. To deal with a large number of potential real-time threats, what you need is a lot of real-time analytical power.
- Europe (0.15)
- North America > Aruba (0.05)
- Law > Statutes (1.00)
- Information Technology > Security & Privacy (1.00)
- Commercial Services & Supplies > Security & Alarm Services (1.00)
BMW unveils a new smart motorcycle that can drive itself
BMW has revealed a self-driving motorcycle that can speed around a racetrack and even park itself without anyone sat on top. The vehicle is powered by intelligent software that can turn, accelerate, lean around corners and brake with no human input. BMW hopes the AI can one day keep bikers safe by operating as a driving assistant that automatically brakes or adjusts the steering during dangerous situations on the road. In a new video, BMW Motorrad - the German automaker's motorcycle division - showed off a self-driving version of its R1200GS, a vehicle it has spent more than two years developing. The driverless bike is shown starting its own engine, accelerating and then making turns on a racing track at breakneck speed before returning to a complete stop - all without a rider perched on its seat.
- Transportation > Passenger (1.00)
- Transportation > Ground > Road (1.00)
- Information Technology > Robotics & Automation (1.00)
The Short Read: Emotionally Intelligent Software is the New AI
A 2.5-minute read to deliver you value-packed info. Advanced digital tools have revolutionized the modern workplace. But CIOs often make the same mistake when choosing new software. They focus too much on user outcomes, and not enough on usability. Everything we hear and read about emerging tech boils down to the promise for greater outcomes.
- Information Technology > Artificial Intelligence (0.57)
- Information Technology > Software (0.38)
Ask These Questions Before Purchasing AI HR Products
Last year, my company, HRExaminer, published the first comprehensive analysis of AI in HR. At the time the research began in May 2017, there were about 40 companies claiming to use AI in HR. By the time we were finished, in October 2017, there were 65. At year's end, there were more than 100. Today, the number is close to 200.
Why 'Fail Fast' Is a Disaster When It Comes to Artificial Intelligence
"Fail fast" is a well-known phrase in the startup scene. The spirit of failing fast is getting to market with a minimum viable product and then rapidly iterating toward success. Failing fast acknowledges that entrepreneurs are unlikely to design a successful end-state solution before testing it with real customers and real consequences. This is the "ready, fire, aim" approach. Or, if the blowback is big enough, it's the "ready, fire, pivot" approach.
- Media (0.32)
- Information Technology (0.32)
- Government > Regional Government > North America Government > United States Government (0.32)