Goto

Collaborating Authors

 Government


Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization

arXiv.org Machine Learning

Machine learning algorithms such as random forests or xgboost are gaining more importance and are increasingly incorporated into production processes in order to enable comprehensive digitization and, if possible, automation of processes. Hyperparameters of these algorithms used have to be set appropriately, which can be referred to as hyperparameter tuning or optimization. Based on the concept of tunability, this article presents an overview of theoretical and practical results for popular machine learning algorithms. This overview is accompanied by an experimental analysis of 30 hyperparameters from six relevant machine learning algorithms. In particular, it provides (i) a survey of important hyperparameters, (ii) two parameter tuning studies, and (iii) one extensive global parameter tuning study, as well as (iv) a new way, based on consensus ranking, to analyze results from multiple algorithms. The R package mlr is used as a uniform interface to the machine learning models. The R package SPOT is used to perform the actual tuning (optimization). All additional code is provided together with this paper.


Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi

arXiv.org Artificial Intelligence

Deep reinforcement learning has generated superhuman AI in competitive games such as Go and StarCraft. Can similar learning techniques create a superior AI teammate for human-machine collaborative games? Will humans prefer AI teammates that improve objective team performance or those that improve subjective metrics of trust? In this study, we perform a single-blind evaluation of teams of humans and AI agents in the cooperative card game Hanabi, with both rule-based and learning-based agents. In addition to the game score, used as an objective metric of the human-AI team performance, we also quantify subjective measures of the human's perceived performance, teamwork, interpretability, trust, and overall preference of AI teammate. We find that humans have a clear preference toward a rule-based AI teammate (SmartBot) over a state-of-the-art learning-based AI teammate (Other-Play) across nearly all subjective metrics, and generally view the learning-based agent negatively, despite no statistical difference in the game score. This result has implications for future AI design and reinforcement learning benchmarking, highlighting the need to incorporate subjective metrics of human-AI teaming rather than a singular focus on objective task performance.


Artificial Intelligence and Law Enforcement Risks

#artificialintelligence

Artificial intelligence ("AI") has shown great promise in helping to solve many of life's challenges. Advances in computing along with the availability of an endless data stream has made artificial intelligence the topic of conversations in every corner of society and within practically every industry imaginable. Consistent with AI's ubiquity, law enforcement agencies at all levels and in many parts of the world have begun to consider or have already implemented the use of artificial intelligence to assist or entirely rely upon in making law enforcement-related decisions that until recently were entirely human-based. While on the surface AI appears to be an effective tool for law enforcement agencies, the use of artificial intelligence in its current form has already exposed serious flaws with respect to human rights and related areas. This technology has produced outcomes that are very worrisome and that require immediate attention and analysis.


Futuregazing: as economies create more data, how can they manage analytics?

#artificialintelligence

But, as entire economies become more data-driven, with government-enforced tax controls demanding increasingly granular levels of transactional information, there is a growing need for analytics solutions capable of handling this data, securely and at scale. Fortunately, innovations such as artificial intelligence (AI) and machine learning mean that businesses are able to scale up their analysis like never before to ensure the right data is presented in the right format for the right audience. Tax reporting is becoming more complicated as different countries have different requirements. With increasingly strict penalties for non-compliance, businesses everywhere need to consider their analytics capabilities if they hope to keep up. In an effort to close their respective country's VAT gap, tax authorities across the world are using every tool at their disposal to collect all revenue owed to them. Real-time VAT reporting, for example, is growing in popularity, with many tax authorities employing continuous transaction controls – such as electronic invoicing and audit reporting – to insert themselves ever closer to companies' transactions.


15 free & open-source data resources for your next data science project

#artificialintelligence

There are many beginners in the field of data science since when the requirement of data scientists boosted in this pandemic. Most of the time, they have questions like where can I find datasets for machine learning/ deep learning projects? Where can I get free datasets for data science? So here I am writing a piece of useful information for every beginner from the very basic. I hope this article will be helpful to beginners as well as advanced data science professionals who were not familiar with these resources earlier.


Feds Call Operational, Organizational Maturity Keys for AI Deployment

#artificialintelligence

As artificial intelligence (AI) adoption continues to grow across the Federal government, officials said on July 15 it's important to share lessons learned across the government, and spoke about the importance of operational and organization efficiencies in the AI adoption process. At a virtual event organized by AI in Government, experts from the General Services Administration (GSA), National Science Foundation (NSF), and NASA talked about how their agencies got started with the AI journey. Bryan Lane, Director of Data and AI at GSA, said the agency made sure to take an honest account of its own organizational and operational maturity, and along those lines said it's also important to determine whether a strategy or roadmap and requirements to drive the AI journey are in place. "Do you have the right development programs to train and educate people on artificial intelligence?" "As we move through the layers of operational maturity for AI, we get into things like DevSecOps cloud and infrastructure data operations, machine learning operations, and you can have different levels of maturity across those operational areas. You can have a very mature cloud DevSecOps environment, but you may have data scientists that are still operating on local machines and testing one-shot models."


Joint Artificial Intelligence Center Press Briefing

#artificialintelligence

I'll be moderating today's press briefing. Today it's my pleasure to introduce the director of the Department of Defense [Joint] Artificial Intelligence Center (JAIC), Lieutenant General Michael Groen. Lieutenant General Groan is joined today by Dr. Jane Pinelis, who is the Chief of Test and Evaluation for the JAIC, and Ms. Alka Patel, who is the Chief of Responsible AI (Artificial Intelligence). We'll begin today's press briefing with an opening statement followed by questions. We've got people out in the line. And I think we'll be able to get to everybody today. LIEUTENANT GENERAL MICHAEL S. GROEN: Thank you, Arlo. And greetings to the members of the Defense Press Corps, really glad to be here with you today. I hope many of you got the opportunity to listen in to at least some of the AI symposium and technology exchange that we had this week. This week, it was our second annual symposium. We have over 1,400 participants in three days of virtualized content. I want to say thank you, ...


Towards a Responsible and Ethical AI

#artificialintelligence

Responsible AI, Ethical AI, AI for social good -- I am sure you must have heard these terms at some point or the other, whether you are a Data Scientist or not. "The development of full artificial intelligence could spell the end of the human race." And there my journey of understanding this critical aspect of the AI foundation started. I used to wonder how to relate ethics with AI which is just a series of algorithms, when, in fact, we have not been able to apply ethical behavior among ourselves. As per the AI index report published by the Stanford University Institute for Human-Centered AI, cybersecurity and regulatory compliance are among the top risks identified by AI/ML-oriented organizations.


Classified details of army's Challenger tank leaked via video game

The Guardian

Classified details of the British Army's main battle tank, Challenger 2, have been leaked online after a player in a tank battle video game disputed its accuracy. The player, who claimed to have been a real life Challenger 2 tank commander and gunnery instructor, disputed the design of the tank in the popular combat video game "War Thunder", arguing it needed changing. He claimed game designers had failed to "model it properly". To support his argument the player posted pages from the official Challenger 2 Army Equipment Support Publication – a manual and maintenance guide. Excerpts from the documents, some of which were heavily redacted, appeared to show some documents had "UK RESTRICTED" label crossed out and a stamp of "UNCLASSIFIED" added, the website UK Defence Journal reported.


Portuguese Daniela Braga appointed to Biden's Artificial Intelligence Task Force

#artificialintelligence

WASHINGTON - Portuguese-born Daniela Braga, founder of DefinedCrowd, a Seattle-based startup that specializes in artificial intelligence, is among the 12 members who will make up Biden administration's National Artificial Intelligence (AI) Research Resource Task Force. Announced on June 10, the new task force "will write the road map for expanding access to critical resources and educational tools that will spur AI innovation and economic prosperity nationwide," according to a White House press release. Braga, who moved to the United States in 2012 and founded DefinedCrowd three years later, said she was "truly honored and humbled" by this appointment. "The Task Force will provide recommendations for establishing and sustaining the NAIRR (National AI Research Resource), including technical capabilities, governance, administration, and assessment, as well as requirements for security, privacy, civil rights, and civil liberties," she posted to her LinkedIn account. "This is another step to have our country continuing to lead AI in the world in an ethic and unbias fashion."