The goal is most certainly not to be "efficient." It'd be silly to try to compete with the already very robust DL libraries out there. The purpose of this course is to learn about the inner-workings of DL so that you can actually understand what you're doing rather than blindly putting blocks together and not understanding how to make it better or why it's not working. I don't claim to be an expert at really anything, and I'd rather have a teacher that didn't, but I've also never blown smoke about what we'd be doing. There are definitely some topics that I have covered that I am just "playing around" with. When that's been true, I've said so and been very clear/upfront about it.

I'm speaking as someone who is very, VERY much into Bayesian and in particular, variational modelling, but dear god can they be finicky. Some of it is just growing pains; I wrote my first VAE in plain Keras, and I remember the pains of having to reparameterize by hand and implementing my own Kullback-Leibler. Nowadays, even plain Torch and Tensorflow come with tools for that. Plus, the theory behind what you're doing is a little bit daunting, since it's less mainstream. Forgot to square the variance param?

In general agree, but in machine learning mutual information seems to be a case where approximation can help sometime rather than hurt. In another discussion this week about the Tishby information bottleneck cameldrv correctly said that the mutual information between a signal and its encrypted version should be high, but in practice no algorithm will discover this. But turn that around: when used in a complex DNN, a learning algorithm that seeks to maximize mutual information (such as today's putting-and-end-to-end-to-end) could in theory produce something like a weak encryption: the desired information is extracted, but it is in such a complex form that _another_ DNN classifier would be needed to extract it! So the fact that mutual information can only approximate can be a good thing, because this is prevented when optimizing objectives that cannot "see" complex relationships. A radical example is in the HSIC bottlneck paper where an approximation that is only monotonically related spontaneously produced on-hot classifications without any guidance.

So, I've been meaning to move away from creating new models and the whole training process for each new project or coding experiment I undertake. I've been meaning to check out a few NLP research frameworks built on top of PyTorch but in the meantime, a lot of libraries have sprung up that do a whole bunch of stuff. I was wondering, could any NLP researcher and core developer share their views on which library does what well and whether there is a single library that can do everything? I know there are a few libraries that are really good for transfer-learning, structured prediction, bayesian stuff, interpretability, transformers and pre trained language models etc. but i'm looking for a general purpose research library with modular add on components for general research in PyTorch. Would be really helpful if someone could share their views on this thing?

Iam working on a deep neural network (U-net architecture) where the input are segmentations and the output is a heatmap. The problem is that there is a lot of variation in my dataset. I noticed that when training with a batch size of 32, the output of my network is more a mean of all inputs (which makes sense). Now I did one training with a batch size of 2. Here I could observe that I, at some epochs, reach a validation loss which is much much better compared to the larger batch size.

I have tried both fine tuning a GPT-2 transformer on my own chats as well a number of academic conversation datasets and further using a GAN on the output embedding space. If you are doing this for fun, than it can be interesting and sometimes hilarious. But the overall technology is not ready for commercial applications as it lacks even basic common sense and general intelligence. It doesn't have a persistent world model so it fails quite quickly in real world applications. For example, it learned my favorite restaurant served burgers, but didn't know burgers are food. It basically outputs believable conversation that is completely disconnected from reality.

I am a professional software developer with a web development background, looking to do ML algotrading with someone as a hobby / side hustle. I am proficient in java, python, js, angular, react, some db languages, have taken Andrew Ng's course, some college AI / ML courses, minor kaggle stuff, added to an existing java ML project, have implemented some automated historical / current data downloaders, code to "save" and "load" the trained model, automatically execute the trades generated by the model on alpaca, and run this all daily on an AWS instance, etc, with an emphasis on keeping it all "free" (I know there's somewhat of a stigma against both algotrading and trying to do this sort of thing "on the cheap" (concerning timely data and execution time), but still). I'm still mediocre at machine learning and finance in general, but I learn best by doing and am quite self-motivated.

There are countless ways to add machine learning into your app development. As listed in the case study, machine learning can predict conversion KPIs, churn, in-app purchases, build personas, and prevent affiliate fraud. That's only the tip of the ML iceberg (I've been wanting to use that saying). This post will be building off that case study and show you where and how you can insert ML into your app growth. Besides branding, ideation is your the most creative step in developing your app.