It is organized in a tensor of the shape (M,J,N), where M 10, 000 is the total ensemble size, J 32 is the spatial discretization size, and N 50 is the sampled time instants with the time step Δt 0.01. Thus the initial data for training are given as the tKdV solutions in the time window 0,0.5.

This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019. For each highlight, I summarise the main advances that took place this year, briefly state why I think it is important, and provide a short outlook to the future. Unsupervised pretraining was prevalent in NLP this year, mainly driven by BERT (Devlin et al., 2019) and other variants. A whole range of BERT variants have been applied to multimodal settings, mostly involving images and videos together with text (for an example see the figure below). Unsupervised pretraining has also made inroads into domains where supervision had previously reigned supreme.

What a time to be working in the deep learning space! Deep learning is ubiquitous right now. From the top research labs in the world to startups looking to design solutions, deep learning is at the heart of the current technological revolution. We are living in a deep learning wonderland! Whether it's Computer Vision applications or breakthroughs in the field of Natural Language Processing (NLP), organizations are looking for a piece of the deep learning pie.

Google AI has open-sourced A Lite Bert (ALBERT), a deep-learning natural language processing (NLP) model, which uses 89% fewer parameters than the state-of-the-art BERT model, with little loss of accuracy. The model can also be scaled-up to achieve new state-of-the-art performance on NLP benchmarks. The research team described the model in a paper to be presented at the International Conference on Learning Representations. ALBERT uses two optimizations to reduce model size: a factorization of the embedding layer and parameter-sharing across the hidden layers of the network. Combining these two approaches results in a baseline model with only 12M parameters, compared to BERT's 108M, while achieving an average of 80.1% accuracy on several NLP benchmarks compared with BERT's 82.3% average.

Half a year ago, I wrote my first post on Medium about my journey of routine paper reading, as part of the fulfillment of my New Year's Resolution. At that time, I concluded that this daily activity of paper-reading is crucial to keep my mind active and abreast of the latest advancement in the field of deep learning. Now on the eve of the new year of 2020, I can proudly say that I executed my 2019 new year resolution of "reading at least one new paper per week" with flying colors. In the second half of the year, I read 116 papers and kept organized notes. If you are interested, the structured notes of these papers are listed in this following Github repo.

On Tuesday, RISC-V CPU fixer SiFive announced it's working with CEVA, which licenses technology for deep learning, audio, and computer vision, to simplify the creation of processors capable of handling machine learning code without demanding too much power. RISC-V is an open, royalty-free instruction set architecture, unlike Intel's x86 chip architecture which requires a license to implement recent processor designs. SiFive provides clients with access to the necessary intellectual property licenses to create custom silicon with minimal negotiation and hassle. CEVA does much the same in a more specific set of domains. SiFive and CEVA anticipate that making it easy to design low-power SoCs tuned for AI-oriented tasks will attract hardware vendors looking to sell RISC-based hardware for applications demanding on-device neural networks, like imaging, computer vision, speech recognition, and sensor data handling.

The previous version was sent out prematurely. This version 2 will be continuously updated for a few months. The copy you receive in your email is static. Hope you had a great Christmas.

Tensor is just a multi-dimensional matrix. A tensor is usually a matrix of dimension 3 or higher. Scalar 1, a vector also known as a list or array [1,2,3], a two by two matrix [[1,2],[3,4]], tensor [ [[1,2],[3,4]], [[5,6],[7,8]] ]. A vector contains a bunch of scalars. A matrix contains a bunch of vectors. A tensor contains a bunch of matrices. You can check the data type of a variable in python using type(variable_name). In pytorch this will return the specific type of torch.tensor.

Breast cancer affects way too many of us. In the US, one in eight women will develop it in their lifetimes. But encouraging new research shows that artificial intelligence can help with early detection. An AI system developed by Google Health, Google-owned DeepMind, and several medical centers is so good at detecting breast cancer that it can outperform actual doctors, according to a paper published this week in the journal Nature. The AI analyzes mammograms -- the X-rays commonly used to check for breast cancer -- to determine whether the disease is present. Researchers found that the AI system reduced false positives by 5.7 percent for US women -- a significant improvement, when you consider how distressing it would be to be told you have cancer when you actually do not.

What if I told you none of these people actually exist? I'll let you have your moment. What you see are results from a generative deep learning model. That's right, a computer made these! These are the models which are the culprits for those fake videos you see of Obama, and Trump.