Collaborating Authors


Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learning Artificial Intelligence

--In the trial-and-error mechanism of reinforcement learning (RL), a notorious contradiction arises when we expect to learn a safe policy: how to learn a safe policy without enough data and prior model about the dangerous region? Existing methods mostly use the posterior penalty for dangerous actions, which means that the agent is not penalized until experiencing danger . This fact causes that the agent cannot learn a zero-violation policy even after convergence . Otherwise, it would not receive any penalty and lose the knowledge about danger . In this paper, we propose the safe set actor-critic (SSAC) algorithm, which confines the policy update using safety-oriented energy functions, or the safety indexes . The safety index is designed to increase rapidly for potentially dangerous actions, which allow us to locate the safe set on the action space, or the control safe set . Therefore, we can identify the dangerous actions prior to taking them, and further obtain a zero constraint-violation policy after convergence. We claim that we can learn the energy function in a model-free manner similar to learning a value function. By using the energy function transition as the constraint objective, we formulate a constrained RL problem. We prove that our Lagrangian-based solutions make sure that the learned policy will converge to the constrained optimum under some assumptions. The proposed algorithm is evaluated on both the complex simulation environments and a hardware-in-loop (HIL) experiment with a real controller from the autonomous vehicle. Experimental results suggest that the converged policy in all environments achieve zero constraint violation and comparable performance with model-based baseline. EINFORCEMENT learning has drawn rapidly growing attention for its superhuman learning capabilities in many sequential decision making problems like Go [1], Atari Games [2], and Starcraft [3].

'Swapping bodies' changes a person's personality, study reveals

The Independent - Tech

Swapping bodies with another person would have a profound effect on the subject's behaviour and even their personality, a new study has revealed. Scientists at the Karolinska Institutet in Sweden discovered a way to allow people to experience the effect of swapping bodies, through a perceptual illusion, in order to understand the relationship between a person's psychological and physical sense of self. They found that when pairs of friends "switched bodies", each friend's personality became more like the other. "Body swapping is not a domain reserved for science fiction anymore," said Pawel Tacikowski, a postdoctoral researcher at the institute and lead author of the study. In order to create the illusion that the study's subjects had switched bodies, Dr Tacikowski and his team fitted them with virtual reality goggles showing live feeds of the other person's body from a first-person perspective.

Billion-dollar quantum computing and AI fund announced by US

The Independent - Tech

The US plans to invest $1 billion (£760 million) in quantum computing and artificial intelligence research, the White House has announced. The initiative will fund 12 hubs around the country and help the US in its bid to compete with China and Europe in two of the most promising next-generation technologies. US Chief Technology Officer Michael Kratsios described the investment as "unprecedented" and a "defining achievement" of the Trump administration. "Built upon the uniquely American free market approach to technological advancement, these institutes will be world-class hubs for accelerating American innovation and building the 21st century American workforce," he said in a statement. Governments around the world are investing heavily in the development of AI and quantum computers, as well as technology giants like Alphabet and Alibaba.

Mummified animals from ancient Egypt 'digitally unwrapped' to unlock mysteries

The Independent - Tech

Researchers have "digitally unwrapped" mummified animals from ancient Egypt, revealing previously undiscovered details about ritualistic mummification. Analysis of the 2,000-year-old animals – which include a bird, a kitten and a snake – was done using a technique that achieves a resolution 100-times higher than a medical CT scan. "Using micro CT we can effectively carry out a post-mortem on these animals, more than 2,000 years after they died in ancient Egypt," said Professor Richard Johnston from Swansea University, who led the research. "We were able to piece together new evidence of how they lived and died, revealing the conditions they were kept in, and possible causes of death. "These are the very latest scientific imaging techniques.

New charging standard promises full charge in less than 15 minutes

The Independent - Tech

Smartphones could soon be able to fully recharge in under 15 minutes after a new fast-charging standard was introduced. Qualcomm's Quick Charge 5 will also allow phones to charge from 0 to 50 per cent in just five minutes, as well as introduce new safety features to prevent overheating. The "world's fastest commercial charging solution" will be up to four-times faster than current charging technologies, according to Qualcomm, and will find its way into commercial devices before the end of the year. It will be compatible with more than 250 smartphones, though it is not a feature that Apple supports. This means that only Androids, not iPhones, will benefit from the technology.

'Touchless touchscreen' could fight future epidemics, researchers say

The Independent - Tech

Cambridge University researchers have developed a "no-touch touchscreen" that uses artificial intelligence to predict a user's intention before their hand reaches the display. The screen was originally designed for use in cars, but the engineers who built it claim it could also have widespread applications during a pandemic. The "predictive touch" technology can be retrofitted to existing displays and could be used to prevent the spread of pathogens on touchscreens at supermarket check-outs, ATMs and ticket terminals at railway stations. Studies have shown that coronavirus can remain on plastic and glass for anywhere between two hours and a week, meaning touchscreens in public places need to be constantly disinfected to prevent transmission. "Touchscreens and other interactive displays are something most people use multiple times per day, but they can be difficult to use while in motion, whether that's driving a car or hanging the music on your phone while you're running," said Simon Godsill from the university's department of engineering.

6G will bring 'digital twins', Samsung says – and it's two years ahead of schedule

The Independent - Tech

Samsung has revealed its plans for 6G technology, outlining its vision for "digital twins" of our physical selves. In a research paper published on Wednesday, the South Korean smartphone giant stated that there will be three key 6G services: Immersive extended reality (XR); high-fidelity mobile hologram; and digital replicas. "With the help of advanced sensors, AI, and communication technologies, it will be possible to replicate physical entities, including people, devices, objects, systems, and even places, in a virtual world," the white paper states. "In a 6G environment, through digital twins, users will be able to explore and monitor the reality in a virtual world, without temporal or spatial constraints. Users will be able to observe changes or detect problems remotely through the representation offered by digital twins."

Virgin internet down: Wifi and network connections stop working amid major outage

The Independent - Tech

Virgin internet has stopped working amid what appears to be a major network outage. Vast numbers of people reported issues with their wifi connections, which appeared to begin just before 10am UK time. Virgin Media's service status page, which allows users to check whether they are affected by any given outage, also appeared to be offline, according to various customers who tried to use it. Problems were reported right across the UK, according to tracking website Down Detector. But Virgin Media said the issue was affecting some parts of London, though not everyone.

Samsung Pay Card offers rival to Apple Wallet and credit card

The Independent - Tech

Samsung has partnered with London-based fintech startup Curve to launch a digital payments card later this year. The Samsung Pay Card will allow users to link all of their bank and loyalty cards to a single digital card on their phone, the tech giant said. It will also allow people with Samsung smartphones to control their money and view all their spending from a single place. The company is also working with Mastercard to launch the new feature. The announcement continues a growing trend of technology firms such as Google and Apple offering a variety of digital payment solutions on their smartphones – last year Apple launched its own credit card for the first time through its Apple Wallet payments app.

Google to start automatically deleting data as part of privacy drive

The Independent - Tech

Google will start automatically deleting users' data as part of a plan to increase privacy on its services. The new change is just one of a wide number of alterations intended to show the company's commitment to privacy, according to chief executive Sundar Pichai. It will also allow people to more easily turn on incognito mode, invest in better technology to protect privacy, and more easily perform checks that alert users if their password is caught up in large breaches. Perhaps the largest of the changes is the switch so that activity gathered about people's physical location as well as about their behaviour on the internet is automatically deleted after 18 months. That option has been available for users since last year, but has been turned off by default.