OpenAI topped Bert with its own AI tool dubbed GPT2, which mimicked the writing style of humans with high accuracy. It used an astonishing 1.5 billion parameters to train its model. To address the issues of heavy resource utilization & climate toll, researchers have been actively pursuing an option where they can shrink these AI's in size, thus not only reducing their resource-intensive nature but at the same time making them more efficient. Two new research papers released recently, have come up with models that might be able to accomplish this. The first one is from researchers at Huawei Noah's Ark Lab called TinyBERT (figure below).