Automated machine learning has been widely explored to reduce human efforts in designing neural architectures and looking for proper hyperparameters. In the domain of neural initialization, however, similar automated techniques have rarely been studied. Most existing initialization methods are handcrafted and highly dependent on specific architectures. In this paper, we propose a differentiable quantity, named GradCoisne, with theoretical insights to evaluate the initial state of a neural network. Specifically, GradCosine is the cosine similarity of sample-wise gradients with respect to the initialized parameters.

Abstract: The sophistication of modern malware, specifically regarding communication with Command and Control (C2) servers, has rendered static blacklist - based defenses obsolete. The use of Domain Generation Algorithms (DGA) allows attackers to generate thousands of dynamic addresses daily, hindering blocking by traditional firewalls. This paper aims to propose and evaluate a method for detecting DGA domains using Deep Learning and Natural Language Processing (NLP) techniques. The methodology consisted of collecting a hybrid database containing 50,000 legitimate and 50,000 malicious domains, followed by the extraction of lexical features and the training of a Recurrent Neural Network (LSTM). Results demonstrated that while statistical entropy analysis is effective for simple DGAs, the Neural Network approach presents superiority in detecting complex patterns, reaching 97.2% accuracy and reducing the false positive rate in ambiguous lawful traffic scenarios.

Automated machine learning has been widely explored to reduce human efforts in designing neural architectures and looking for proper hyperparameters. In the domain of neural initialization, however, similar automated techniques have rarely been studied. Most existing initialization methods are handcrafted and highly dependent on specific architectures. In this paper, we propose a differentiable quantity, named GradCoisne, with theoretical insights to evaluate the initial state of a neural network. Specifically, GradCosine is the cosine similarity of sample-wise gradients with respect to the initialized parameters.

We're still waiting for the full promise of self-driving cars to be realized. After all, fleshy, unreliable humans are killing more people than ever on US roads. And now Apple fans could be waiting indefinitely as poor Project Titan has been killed off after its owner gave up on human-free control. Still, there are glimmers of hope. Just this month, Waymo secured approval from California regulators for paid robotaxi rides on San Francisco freeways and other highways in the Bay Area, meaning that autonomous trips to San Francisco International Airport will be possible.

What if, instead of seeking out a charger and plugging in for half an hour, you could swap out your depleted EV battery for a new one in just five minutes? That's the question asked by Nio, a 7-year-old Chinese electric car company with a network of more than 1,300 battery-swap stations for doing exactly that. However, the vast majority of these stations are in China, with 100 in Shanghai alone, serving customers who mostly live in apartments and lack the space to install a charger at home. Customers buy the car but lease the battery, with the monthly fee granting them access to the swap station network. By contrast, there are currently only 27 so-called Power Swap Stations in Europe, across Germany, Denmark, the Netherlands, Norway, and Sweden, but more are supposedly in the pipeline.

A large class of inverse problems for PDEs are only well-defined as mappings from operators to functions. Existing operator learning frameworks map functions to functions and need to be modified to learn inverse maps from data. We propose a novel architecture termed Neural Inverse Operators (NIOs) to solve these PDE inverse problems. Motivated by the underlying mathematical structure, NIO is based on a suitable composition of DeepONets and FNOs to approximate mappings from operators to functions. A variety of experiments are presented to demonstrate that NIOs significantly outperform baselines and solve PDE inverse problems robustly, accurately and are several orders of magnitude faster than existing direct and PDE-constrained optimization methods.

Automated machine learning has been widely explored to reduce human efforts in designing neural architectures and looking for proper hyperparameters. In the domain of neural initialization, however, similar automated techniques have rarely been studied. Most existing initialization methods are handcrafted and highly dependent on specific architectures. In this paper, we propose a differentiable quantity, named GradCosine, with theoretical insights to evaluate the initial state of a neural network. Specifically, GradCosine is the cosine similarity of sample-wise gradients with respect to the initialized parameters. By analyzing the sample-wise optimization landscape, we show that both the training and test performance of a network can be improved by maximizing GradCosine under gradient norm constraint. Based on this observation, we further propose the neural initialization optimization (NIO) algorithm. Generalized from the sample-wise analysis into the real batch setting, NIO is able to automatically look for a better initialization with negligible cost compared with the training time. With NIO, we improve the classification performance of a variety of neural architectures on CIFAR-10, CIFAR-100, and ImageNet. Moreover, we find that our method can even help to train large vision Transformer architecture without warmup.

William Li is being mobbed. At a gala dinner in Shanghai, the founder of Chinese electric carmaker Nio Inc. can barely move forward in the buffet queue before being stopped for another selfie, handshake or hug. Swapping his usual attire of jeans and a T-shirt for a tailored grey suit and blue dress shirt, the tall 46-year-old happily obliges with a smile. Li manages to spoon a small amount of fried rice and vegetables onto his plate, but he's not here for the food. Over the next three hours, Li poses for hundreds more photos, chatting with customers of the automaker he started just over six years ago and has built into a way of life -- at least for the people who buy his cars -- with clubhouses, a round-the-clock battery recharging service and even clothing, food and exercise equipment, all decked out in Nio's geometric logo. As Li works the room, a video backdrop shows six performers, each wearing a different-colored Nio hoodie, singing a self-composed song dedicated to the company.