fundamental building block
Efficient Modular Learning through Naive LoRA Summation: Leveraging Orthogonality in High-Dimensional Models
Cao, Zhanhao, Truong, Clement, Lizarraga, Andrew
Recent advances in large language models are driven by scale, while parameter-efficient fine-tuning (PEFT) enables updating only a small fraction of parameters. Low-Rank Adaptation (LoRA) stores parameter deltas as the product of two small matrices, which makes them natural building blocks that can be composed. Motivated by the superposition principle, we hypothesize that independently trained LoRA modules on disjoint domains are approximately orthogonal and can be combined by simple addition. Using GPT-2 Small (117M) with LoRA rank 4 and alpha=64, we train adapters for three QA domains (math, medicine, finance). In pairwise tests, adding Math+Medicine adapters improves perplexity by -9.10% relative to merged-data fine-tuning, while Math+Finance and Finance+Medicine change by +4.54% and +27.56%, respectively. Across combinations, the RMS cosine similarity between LoRA deltas correlates positively and approximately linearly with the change in perplexity. Naive summation requires no additional training, can be applied in seconds, and achieves performance comparable to models trained on merged data, while clarifying when interference appears in higher-order compositions.
Implementing An Artificial Quantum Perceptron
Hathidara, Ashutosh, Pandey, Lalit
A Perceptron is a fundamental building block of a neural network. The flexibility and scalability of perceptron make it ubiquitous in building intelligent systems. Studies have shown the efficacy of a single neuron in making intelligent decisions. Here, we examined and compared two perceptrons with distinct mechanisms, and developed a quantum version of one of those perceptrons. As a part of this modeling, we implemented the quantum circuit for an artificial perception, generated a dataset, and simulated the training. Through these experiments, we show that there is an exponential growth advantage and test different qubit versions. Our findings show that this quantum model of an individual perceptron can be used as a pattern classifier. For the second type of model, we provide an understanding to design and simulate a spike-dependent quantum perceptron. Our code is available at \url{https://github.com/ashutosh1919/quantum-perceptron}
A Review of Findings from Neuroscience and Cognitive Psychology as Possible Inspiration for the Path to Artificial General Intelligence
This review aims to contribute to the quest for artificial general intelligence by examining neuroscience and cognitive psychology methods for potential inspiration. Despite the impressive advancements achieved by deep learning models in various domains, they still have shortcomings in abstract reasoning and causal understanding. Such capabilities should be ultimately integrated into artificial intelligence systems in order to surpass data-driven limitations and support decision making in a way more similar to human intelligence. This work is a vertical review that attempts a wide-ranging exploration of brain function, spanning from lower-level biological neurons, spiking neural networks, and neuronal ensembles to higher-level concepts such as brain anatomy, vector symbolic architectures, cognitive and categorization models, and cognitive architectures. The hope is that these concepts may offer insights for solutions in artificial general intelligence.
Python Classes and Their Use in Keras
Classes are one of the fundamental building blocks of the Python language, which may be applied in the development of machine learning applications. As we shall be seeing, the Python syntax for developing classes is simple, and can be applied to implement callbacks in Keras. In this tutorial, you will discover the Python classes and their functionality. Python Classes and Their Use in Keras Photo by S Migaj, some rights reserved. In object-oriented languages, such as Python, classes are one of the fundamental building blocks.
Take a 44-Hour Deep Dive Into the Algorithms & Statistical Models That Make Machine Learning & Artificial Intelligence Work
This 5-hour course is created to take you by hand and teach you how to tackle the most fundamental building blocks of practical data science: data wrangling and visualization. It will equip you to use some of the most important Python data wrangling and visualization packages such as Seaborn. You will also be able to decide which wrangling and visualization techniques are best suited to answer your research questions and applicable to your data and interpret the results. Minerva Singh is a PhD graduate from Cambridge University where she specialized in Tropical Ecology. She is also a Data Scientist on the side.
The Five Fundamental Building Blocks for Artificial Intelligence in Banking
To find success with artificial intelligence, banks and credit unions will need to cultivate new capabilities -- from machine learning to natural language processing. Computer scientist and celebrated futurist Ray Kurzweil says artificial intelligence will match human intelligence by 2029, and that by 2045, it will have multiplied the human biological machine intelligence of our civilization a billion times. Predictions like this abound amid the hype surrounding AI. Real understanding however, is less common. Many enterprises are unclear about what constitutes AI, where it can be applied, and how to prioritize its use cases within the organization.