Code pre-trained models have shown great success in various code-related tasks, such as code search, code clone detection, and code translation. Most existing code pre-trained models often treat a code snippet as a plain sequence of tokens. However, the inherent syntax and hierarchy that provide important structure and semantic information are ignored. The native derived sequence representations of them are insufficient. To this end, we propose CLSEBERT, a Contrastive Learning Framework for Syntax Enhanced Code Pre-Trained Model, to deal with various code intelligence tasks. In the pre-training stage, we consider the code syntax and hierarchy contained in the Abstract Syntax Tree (AST) and leverage the Contrastive Learning (CL) to learn noise-invariant code representations. Besides the original masked language model (MLM) objective, we also introduce two novel pre-training objectives: (1) ``AST Node Edge Prediction (NEP)'' to predict edges between nodes in the abstract syntax tree; (2) ``Code Token Type Prediction (TTP)'' to predict the types of code tokens. Extensive experiments on four code intelligence tasks demonstrate the superior performance of CLSEBERT compared to state-of-the-art at the same pre-training corpus and parameter scale.

This includes popular architectures such as ResNet-18, VGG16, GoogLeNet and ResNeXt-50. We will do something different for this project by selecting a pre-trained model that is not within the default list of Torchvision models. In particular, we will be using EfficientNet. EfficientNet is a convolutional neural network architecture and scaling method developed by Google in 2019. It has been shown to surpass state-of-the-art accuracy with up to 10 times better efficiency (i.e.

Before we go any further into text classification, we need a way to represent words numerically in a vocabulary. Because most of our ML models require numbers, not text. One way to achieve this goal is by using the one-hot encoding of word vectors, but this is not the right choice. Given the structure of one-hot encoded vectors, the similarity is always going to be 0 between different words. Word2Vec overcomes the above difficulties by providing us with a fixed-length (usually much smaller than the vocabulary size) vector representation of words.

In early August 2021, OpenAI (which is partly controlled by Elon Musk) launched Codex, a revolutionary new AI system which can automatically write code in a variety of languages, using only plaintext prompts as input.

Computer vision has a wide field of application in our modern society. One aspect of computer vision is Object Recognition, which can be divided into three sub-fields of Image Classification, Object Localization and Object Detection. We have discussed Single- and Multi-label Classification. Object Localization involves assigning bounding boxes to relevant objects in an image. Object detection combines the objectives of classification and localization.

WIRED has challenged computer scientist and Hidden Door cofounder and CEO Hilary Mason to explain machine learning to 5 different people; a child, teen, a college student, a grad student and an expert.

Abstract: We construct a Bayesian inference deep learning machine for parameter estimation of gravitational wave events of binaries of black hole coalescence. The structure of our deep Bayseian machine adopts the conditional variational autoencoder scheme by conditioning both the gravitational wave strains and the variations of amplitude spectral density of the detector noise. We show that our deep Bayesian machine is capable of yielding the posteriors compatible with the ones from the nest sampling method, and of fighting against the noise outliers. Abstract: Gravitational waves are ripples in the fabric of space-time that travel at the speed of light. The detection of gravitational waves by LIGO is a major breakthrough in the field of astronomy.

Pathologists examine histochemically stained tissue biopsy sections to make medical diagnoses. Hematoxylin and eosin (H&E) is the most widely used histochemical stain in pathology, accounting for the vast majority of human tissue biopsy stains worldwide. In many clinical cases, however, additional "special stains" are required to bring contrast and color to various tissue components and allow pathologists to obtain a clearer diagnostic image. These specific stains frequently necessitate much more tissue preparation time, as well as tedious effort and monitoring by specialist histotechnologists, all of which raise the expenses and length of time to diagnose. By computationally translating existing photos of H&E stained tissue into special stains, UCLA researchers devised a deep learning-based approach that can be used to eliminate the need for these specific stains to be prepared by human histotechnologists.

Fibre-optic Distributed Acoustic Sensing (DAS) is an emerging technology for vibration measurements with numerous applications in seismic signal analysis, including microseismicity detection, ambient noise tomography, earthquake source characterisation, and active source seismology. Using laser-pulse techniques, DAS turns (commercial) fibre-optic cables into seismic arrays with a spatial sampling density of the order of metres and a time sampling rate up to one thousand Hertz. The versatility of DAS enables dense instrumentation of traditionally inaccessible domains, such as urban, glaciated, and submarine environments. This in turn opens up novel applications such as traffic density monitoring and maritime vessel tracking. However, these new environments also introduce new challenges in handling various types of recorded noise, impeding the application of traditional data analysis workflows.

This conversation with the composer Philip Glass and me discusses an exciting project in partnership with OpenAi, in which we trained a neural net on a corpus of Glass' work. He offers commentary on the music created by "his AI", as well as insights on composition and creating art. We then talk about the different limitations and capacities of humans and Artificial Intelligence–if and how neural nets can help us create art, appreciate art, and find the same things humans find meaningful. Due to the covid-19 pandemic, this call took place over video conference in December 2020. Art and tech are both captivating to me because they frame the elevation and the limitations of being human. Art is also closely intertwined with technological advancements, as movement shifting art seems predicated on tech. For example, the photography of Martin Munkacsi from the 1920s and 1930s revolutionized the art, as he is often credited for being the first photographer to explore dynamic and candid styles. The emergence and ability of these new forms of creation coincided with the technological advancements at the time that enabled flash and faster shutters–candid and spontaneous movement shots wouldn't have been technically possible to make with the cameras that existed before. The advancements in machine learning today, likewise, excite me for the possibilities and new forms in art and creation. The goal of this project is to explore the capacities of artificial intelligence as a new medium (or instrument or tool?) for art, and to create a collaborative music composition with Philip Glass and "his AI." More details about the project can be found below. Philip: Nice to see you.