Journal Article

·2025

Optimal RAG System Design for Turkish Textbooks: A Comprehensive Evaluation and Performance Enhancement Study

Elif Nur Öner YTU , Sinem Ceyhun YTU , Muhammed Yıldız YTU , Andalib Goncharova YTU , Türkan Sena Yücel YTU , H. Toprak Kesgin YTU , Mehmet Fatih Amasyalı YTU

Abstract

Retrieval-augmented generation enables precise educational question-answering by combining retrieval with natural language generation. Limited evaluation exists for educational RAG systems, particularly for curriculum-based applications in diverse languages. We introduce systematic evaluation using Turkish Ministry of Education textbooks, comparing embedding architectures, generation models, and optimization techniques across educational questions. Results establish multilingual-e5-large-instruct and BAAI/bge-m3 as optimal embedding models, demonstrate Qwen3 and Turkish-Gemma-9b's superior generation performance, and show ensemble methods yield 3.3% improvements. This provides evidence-based recommendations for educational RAG development across different languages and curricula.

Keywords

Turkish Embedding Christian ministry Systems design Performance improvement Natural language generation Computer science Artificial intelligence Machine learning

Subject Areas

Topic Modeling ·Artificial Intelligence ·Physical Sciences
Intelligent Tutoring Systems and Adaptive Learning ·Artificial Intelligence ·Physical Sciences
Multimodal Machine Learning Applications ·Computer Vision and Pattern Recognition ·Physical Sciences