Training wide and deep neural networks (DNNs) require large amounts of storage resources such as memory because the intermediate activation data must be saved in the memory during forward propagation and then restored for backward propagation. However, state-of-the-art accelerators such as GPUs are only equipped with very limited memory capacities due to hardware design constraints, which significantly limits the maximum batch size and hence performance speedup when training large-scale DNNs. Traditional memory saving techniques either suffer from performance overhead or are constrained by limited interconnect bandwidth or specific interconnect technology. In this paper, we propose a novel memory-efficient CNN training framework (called COMET) that leverages error-bounded lossy compression to significantly reduce the memory requirement for training, to allow training larger models or to accelerate training. Different from the state-of-the-art solutions that adopt image-based lossy compressors (such as JPEG) to compress the activation data, our framework purposely adopts error-bounded lossy compression with a strict error-controlling mechanism. Specifically, we perform a theoretical analysis on the compression error propagation from the altered activation data to the gradients, and empirically investigate the impact of altered gradients over the training process. Based on these analyses, we optimize the error-bounded lossy compression and propose an adaptive error-bound control scheme for activation data compression. We evaluate our design against state-of-the-art solutions with five widely-adopted CNNs and ImageNet dataset. Experiments demonstrate that our proposed framework can significantly reduce the training memory consumption by up to 13.5X over the baseline training and 1.8X over another state-of-the-art compression-based framework, respectively, with little or no accuracy loss.

Evolution – according to the Dictionary – is "the process by which different kinds of living organisms are thought to have developed and diversified from earlier forms during the history of the earth." Other sources mention also natural selection and adaptation. Now, we are approaching a new stage in our development as species. For the first time in history, we are about to partake in a manmade evolution. We are about to become enhanced courtesy of new technologies such as Brain-Computer Interface (BCI), artificial intelligence (AI), machine learning and VR (virtual reality) and yes, genomics.

The Morrison Government is investing $10 million in innovative artificial intelligence (AI) technologies that will strengthen Defence's military capability and support highly skilled jobs in Australia's defence industry. The investment supports the Government's new Blueprint for Critical Technologies and Action Plan, released by the Prime Minister yesterday. It also contributes to the development of a sovereign critical technology capability in AI, one of the Government's nine listed critical technologies of national interest. Minister for Defence Industry and Science and Technology Melissa Price today announced 10 new Defence Innovation Hub contracts funded under the Government's two-year, $32 million COVID economic stimulus package. The package was established to support jobs growth in the defence industry while navigating the challenges posed by the pandemic.

The search for'another Earth' has been a staple of science fiction for decades, and now a group of astronomers hope to discover one on our galactic doorstep. Alpha Centauri is a triple star system just over four light years from the Earth, split into a pair of sun-like stars known as AB, and a red dwarf called Proxima Centauri. So far planets have only been found orbiting Proxima Centauri, but experts from the University of Sydney and Breakthrough Initiatives believe they will find a world orbiting the larger binary pair using a new privately funded telescope. Known as the Toliman mission, it will launch in 2023 and scan Alpha Centauri AB for worlds in the habitable zone, where liquid water can flow on the surface. The team hope to be able to say whether there are habitable worlds orbiting either or both of the binary stars by the middle of this decade.

The University of Houston-Victoria's second UHV Discovers faculty research series will feature a presentation by a faculty member and a student on different applications for artificial intelligence in the fields of medicine, energy and weather prediction. Hardik Gohel, a UHV assistant professor of computer science, will showcase his research in a presentation titled, "Applied AI for Accessible Health Care and Energy Security Transition," followed by a presentation by Pavithra Sivashanmugam, a UHV computer science graduate student from India, who will present her research "AI in Rainfall Prediction." The presentations will be from 1 to 2 p.m. Nov. 18 in Room 111 of UHV University North, 3007 N. Ben Wilson St. The presentations also will be offered through Microsoft Teams. Gohel and his students use technology to turn massive amounts of data into useful information," said Joann Olson, UHV associate provost for research and dean of graduate studies. "By exploring the applications of artificial intelligence to challenges like weather forecasting and health care delivery, their high-level research helps solve real-world problems." During his presentation, Gohel plans to show how artificial intelligence can be used in the fields of health care and energy to accumulate, organize and analyze massive amounts of data. During the pandemic, health care began to shift toward a focus on telemedicine rather than in-person doctor's visits, Gohel said. In addition, people use devices such as Fitbits that collect constant data about an individual's physical condition. Another focus of Gohel's research looks at the need for renewable energy. As more and more research is being done into energy resources, artificial intelligence can organize and filter the data to see which resources are effective and where energy is needed. "Texas is a major hub for energy and health care in the nation," Gohel said. "The state needs to prepare for shifts in these areas and others, and artificial intelligence can help the state's systems do that.

In this work, we investigate the expressiveness of the "conditional mutual information" (CMI) framework of Steinke and Zakynthinou (2020) and the prospect of using it to provide a unified framework for proving generalization bounds in the realizable setting. We first demonstrate that one can use this framework to express non-trivial (but sub-optimal) bounds for any learning algorithm that outputs hypotheses from a class of bounded VC dimension. We prove that the CMI framework yields the optimal bound on the expected risk of Support Vector Machines (SVMs) for learning halfspaces. This result is an application of our general result showing that stable compression schemes Bousquet al. (2020) of size $k$ have uniformly bounded CMI of order $O(k)$. We further show that an inherent limitation of proper learning of VC classes contradicts the existence of a proper learner with constant CMI, and it implies a negative resolution to an open problem of Steinke and Zakynthinou (2020). We further study the CMI of empirical risk minimizers (ERMs) of class $H$ and show that it is possible to output all consistent classifiers (version space) with bounded CMI if and only if $H$ has a bounded star number (Hanneke and Yang (2015)). Moreover, we prove a general reduction showing that "leave-one-out" analysis is expressible via the CMI framework. As a corollary we investigate the CMI of the one-inclusion-graph algorithm proposed by Haussler et al. (1994). More generally, we show that the CMI framework is universal in the sense that for every consistent algorithm and data distribution, the expected risk vanishes as the number of samples diverges if and only if its evaluated CMI has sublinear growth with the number of samples.

The People's Speech is a free-to-download 30,000-hour and growing supervised conversational English speech recognition dataset licensed for academic and commercial usage under CC-BY-SA (with a CC-BY subset). The data is collected via searching the Internet for appropriately licensed audio data with existing transcriptions. We describe our data collection methodology and release our data collection system under the Apache 2.0 license. We show that a model trained on this dataset achieves a 9.98% word error rate on Librispeech's test-clean test set. Finally, we discuss the legal and ethical issues surrounding the creation of a sizable machine learning corpora and plans for continued maintenance of the project under MLCommons's sponsorship.

Although artificial intelligence (AI) is solving real-world challenges and transforming industries, there are serious concerns about its ability to behave and make decisions in a responsible way. Many AI ethics principles and guidelines for responsible AI have been recently issued by governments, organisations, and enterprises. However, these AI ethics principles and guidelines are typically high-level and do not provide concrete guidance on how to design and develop responsible AI systems. To address this shortcoming, we first present an empirical study where we interviewed 21 scientists and engineers to understand the practitioners' perceptions on AI ethics principles and their implementation. We then propose a template that enables AI ethics principles to be operationalised in the form of concrete patterns and suggest a list of patterns using the newly created template. These patterns provide concrete, operationalised guidance that facilitate the development of responsible AI systems.

Randomized smoothing is currently a state-of-the-art method to construct a certifiably robust classifier from neural networks against $\ell_2$-adversarial perturbations. Under the paradigm, the robustness of a classifier is aligned with the prediction confidence, i.e., the higher confidence from a smoothed classifier implies the better robustness. This motivates us to rethink the fundamental trade-off between accuracy and robustness in terms of calibrating confidences of a smoothed classifier. In this paper, we propose a simple training scheme, coined SmoothMix, to control the robustness of smoothed classifiers via self-mixup: it trains on convex combinations of samples along the direction of adversarial perturbation for each input. The proposed procedure effectively identifies over-confident, near off-class samples as a cause of limited robustness in case of smoothed classifiers, and offers an intuitive way to adaptively set a new decision boundary between these samples for better robustness. Our experimental results demonstrate that the proposed method can significantly improve the certified $\ell_2$-robustness of smoothed classifiers compared to existing state-of-the-art robust training methods.

We introduce the GANformer2 model, an iterative object-oriented transformer, explored for the task of generative modeling. The network incorporates strong and explicit structural priors, to reflect the compositional nature of visual scenes, and synthesizes images through a sequential process. It operates in two stages: a fast and lightweight planning phase, where we draft a high-level scene layout, followed by an attention-based execution phase, where the layout is being refined, evolving into a rich and detailed picture. Our model moves away from conventional black-box GAN architectures that feature a flat and monolithic latent space towards a transparent design that encourages efficiency, controllability and interpretability. We demonstrate GANformer2's strengths and qualities through a careful evaluation over a range of datasets, from multi-object CLEVR scenes to the challenging COCO images, showing it successfully achieves state-of-the-art performance in terms of visual quality, diversity and consistency. Further experiments demonstrate the model's disentanglement and provide a deeper insight into its generative process, as it proceeds step-by-step from a rough initial sketch, to a detailed layout that accounts for objects' depths and dependencies, and up to the final high-resolution depiction of vibrant and intricate real-world scenes.