The artificial intelligence revolution is becoming increasingly localized. Pakistani developer Junaid Ahmed has officially released Qehwa, the first Large Language Model (LLM) in the world to be specific to the Pashto language, in a groundbreaking solo project.
Intended to support a population of Pashto speakers around the world (approximately 60 million), Qehwa is oriented to the Peshawari dialect, filling an extremely huge gap in natural language processing where global AI systems have traditionally been unable to handle regional languages and cultural conditions.
Ahmed was inspired by Qalb, the Urdu LLM developed by Taimoor Hassan, and started working on this project without any institutional support of funds or team of developers. The following is a detailed analysis of the construction of Qehwa, the amazing features it has, and its implications to the open-source community.
Qehwa AI Building on Qwen2.5-7B
To create Qehwa, the developer utilized Qwen2.5-7B as the foundation. Qwen is an open-source and highly competent family of large language models created by Alibaba Cloud. The 7B name is used to show that the model has 7 billion parameters- basically the brain cells in which the logic and reasoning of the model is determined.
Although the foundation Qwen model had a good grip of general logic and various languages, it did not have a localized focus that could allow it to communicate natural Pashto. Ahmed used this base and subjected it to a two-stage intensive training regime, which was particularly in the Peshawari Pashto dialect.
Qehwa AI Training ProcesS
Developing a localized LLM from scratch requires an immense amount of data processing. The training of Qehwa was executed in two critical phases:
Stage 1: Continued Pre-Training
In this first step, the model was supplied with 3.4 million Pakistani Pashto documents. This vast exposure enabled the AI to develop a rich vocabulary, comprehend complicated grammatical constructs and most of all, comprehend the cultural context of the region that the world models lack.
Ahmed used a LoRA rank of 64, which is able to handle the computational load as a single developer. LoRA (Low-Rank Adaptation) is a superior mathematical method through which developers can make AI models of enormous scale fine-tuned without supercomputers priced in the millions of dollars. Instead of re-training the entire 7 billion parameters, LoRA updates a small and highly specific subset of the neural network, which is why the training process is very efficient and cheap.
Stage 2: Fine-Tuning for Instructions
Once the model understood the language, it needed to learn how to follow commands. In the second stage, Qehwa was fine-tuned using over 100,000 Pashto instruction pairs. This training enables the chatbot to accurately answer questions, translate text, and engage in fluid conversational tasks.
Qehwa AI Setting a New Standard
Because there was no existing standard for testing Pashto AI, Ahmed had to create one. Qehwa introduces the first dedicated Pashto LLM benchmark, consisting of 150 rigorous evaluation tests spread across 15 distinct categories.
The results are highly impressive. Qehwa achieved an overall accuracy score of 85.3%.
Qehwa AI Key Benchmark Highlights:
- English to Pashto Translation: 90% accuracy.
- Urdu to Pashto Translation: 84% accuracy.
- Subject-Specific Accuracy: The model scored a stellar 90% in categories concerning Culture and History, Health and Daily Life, and Geography and Nature.
The chatbot currently supports user prompts in English, Urdu, and Pashto, while generating its responses in pure, grammatically correct Pashto.
How to Install and Run Qehwa AI
For developers looking to run the model locally, Qehwa is optimized for consumer hardware:
- Unsloth Integration: The model can be run using Unsloth, a popular open-source tool that makes fine-tuning and running LLMs significantly faster and reduces memory consumption.
- BitsAndBytes (4-bit Quantization): By utilizing 4-bit quantization (a compression technique), the model’s size is drastically shrunk. This allows users to run the 7-billion parameter model on standard consumer graphics cards (such as an 8GB gaming GPU) rather than requiring highly expensive enterprise server hardware.
With the launch of Qehwa, Pashto speakers finally have an AI tool designed explicitly for their language and culture, marking a significant milestone in Pakistan’s growing AI landscape.
What is Qehwa AI?
Qehwa AI is the world’s first Large Language Model (LLM) and chatbot built specifically for the Pashto language. Developed by Pakistani developer Junaid Ahmed, it focuses on the Peshawari dialect and is designed to accurately understand Pashto grammar, vocabulary, and cultural nuances.
How is Qehwa different from global AI models like ChatGPT?
While global AI models support many languages, they often struggle with the cultural context and localized grammar of regional languages like Pashto. Qehwa was trained specifically on 3.4 million Pakistani Pashto documents, making its responses highly accurate and culturally relevant for Pashto speakers.
Can Qehwa translate other languages into Pashto?
Yes. Qehwa accepts user prompts in English, Urdu, and Pashto. It is highly proficient in translation, achieving a 90% accuracy rate for English-to-Pashto translation and an 84% accuracy rate for Urdu-to-Pashto translation in benchmark tests.
Do I need a supercomputer to run Qehwa AI?
No. Because the developer utilized a compression technique called 4-bit quantization, the model’s size has been significantly reduced. This allows developers and tech enthusiasts to run Qehwa locally on standard consumer-grade computers with regular graphics cards (like an 8GB gaming GPU).
Is Qehwa AI free to use?
Yes, Qehwa was developed as a completely free and open-source project. Researchers, students, and developers can access the model, explore its code, and integrate it into their own applications without any cost.
