Fine-Tuning Small Embeddings for Elevated Performance