A Novel Translation-Driven Approach to Enhance LLM Performance on Low-Resource Languages

Large Language Models (LLMs) excel in highresource languages but struggle with low-resource languages due to limited training data and insufficient representation during pre-training. This disparity creates significant barriers for deploying advanced NLP technologies across diverse linguistic communities. This paper presents TALL (Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages), a novel framework that strategically integrates an LLM with two bilingual translation models to bridge the performance gap between high and low-resource languages. TALL transforms lowresource inputs into high-resource representations through a multi-stage pipeline, leveraging the LLM's robust capabilities while preserving essential linguistic features through carefully designed dimension alignment layers and custom transformer components. The architecture addresses the challenge of integrating models with different hidden dimensions and representation spaces, enabling seamless knowledge transfer across languages. Our comprehensive experiments on Hebrew demonstrate significant improvements over several competitive baselines, including direct LLM use, naive translation approaches, finetuning strategies, and soft prompting techniques. Notably, TALL achieves up to 5. 5 9% accuracy compared to 2. 9 3% for the next best approach, representing a substantial performance gain. The architecture employs a parameter-efficient strategy, freezing large pre-trained components while training only lightweight adapter modules, effectively balancing computational efficiency with performance gains. This approach makes TALL particularly suitable for resource-constrained environments while maintaining strong cross-lingual transfer capabilities. Code is available in https://github.com/MosheOfer1/TALL