The Readability Level of The Advance Arabic Textbook Based on the Cloze Test

Research article
Issue: № 9 (45), 2023


One of the sources that determine the success of the educational process are textbooks. However, we often come across Arabic textbooks that are difficult for students to understand. The content of textbooks is related to the ease or complexity of the material, known as readability. The importance of readability cannot be overestimated, as it directly affects the effectiveness of written texts. The content of the text must be clear, effective and adapted to the abilities of the students so that the learning objectives are achieved, and their language skills are improved. Therefore, this study aims to determine the level of readability of the Extended Arabic Language Textbook (BAL) for graduate students from one of the Malaysian higher education institutions at the Islamic University of Sultan Ahmad Shah Pahang (UnIPSAS), Malaysia. A quantitative survey design using the Close test was conducted among 169 graduate students from UnIPSAS. The data, which were analyzed using descriptive analysis, shows that the level of readability of the BAL textbook as a whole is at an unsatisfactory level, which does not correspond to the goals of teaching and learning. Textbook authors should review the texts selected as academic texts for the course (BAL). It is hoped that the results of this study can be used by Arabic language textbook authors to improve the quality of students' reading skills and understanding of the language.

1. Introduction

Learning Arabic at the higher education level is considered a necessity, especially for students who specialize in the field of Islamic studies because this language is a medium for understanding the Quran and Hadith. There are also some students who choose Arabic as an additional subject in order to increase communication skills other than using their mother tongue. However, the teaching and learning atmosphere of Arabic is still not satisfactory. There are issues regarding teaching and learning of Arabic in Malaysia such as teacher-centred learning, teaching Arabic based on translation, lack of exercises and questions, lack in the use of teaching aids and additional reading materials, lack of activity, passive attitude of students, and lack of communication in Arabic


Arabic textbooks play a big role in the education system by providing students with essential knowledge and skills to master the Arabic language. Good textbooks will provide effective information for students. However, the importance of readability of the textbooks is often neglected. According to Salwa

, teachers frequently enhance or adjust the materials they use in the classroom to match their students' developmental stage, but these adjustments are often based on intuition and previous experience. Many students find that the texts are difficult to understand because of complex sentence structures, unfamiliar vocabulary, and tight formatting. As educators and curriculum developers, it's important to address these challenges and connect the gap between the content and learners. That is where readability comes in. Readability is the measure of how easy or difficult it is for a reader to understand a piece of a written text. Readability refers to how easy it is to read and understand a text, depending on its unique features. When a lecturer selects a text to use as teaching material, one of the most significant issues to consider is the text's readability level
. Teachers and educators should select texts that are relevant to their students’ proficiency or developmental level
.  Readability takes into several factors including text presentation (font choice, spacing, or colours) and context (the sentences and words on the page). Other factors that affect readability were sentence length, sentence structure, and syllable count per word. The importance of readability cannot be overstated, as it directly impacts the effectiveness of the written content.

2. Readability

Readability is defined as the “compliance of a book or text with the readers' reading ability” Nurbayan and Ihwan Rahman

. Marohaini (1999), in Zamri Arifin
, defined readability as a concept that refers to the level or degree of ease of reading a material. According to Khadijah (1987), in Rahman
, readability refers to the ability of a reading material that allows easy understanding of its meaning. Zulazhan
found that reading materials with a high level of readability are classified as a difficult text, while a low level of readability is classified as an easy-to-understand text. In the realm of Arabic readability, past studies have concentrated on various aspects, and they have been conducted in multiple countries. One noteworthy study was by Aisyah and Maimun
, which stands out in this regard.

Several previous studies have measured the readability level of Arabic textbooks. Some of the studies had used the qualitative method, like Nurbayan and Ihwan Rahman (6) had examined a study entitled "The Readability of Arabic Textbook for Students in Senior High School" in Indonesia by Zakiyah Arifah & Nadia Afidati. Whereas, some of them used quantitative method such as Aisyah and Maimun

found that the average score in the Cloze test for the Maharat al-Qira'ah textbook was 65%, which is deemed to be low score and suggests its limited suitability for effective teaching and learning in the classroom context.

Therefore, it can be concluded that the scope of the study is focused on the same issue, which is to evaluate the difficulty level of a reading text to determine its suitability for the target reader by considering two factors, namely the reading text and the reader. This present study’s intended to determine the difficulty level (readability) of Arabic reading materials for non-Arabic speakers and its suitability for the target reader by focusing on the textual factor.

3. Methodology

The discussion on methodology intends to explain the approach and nature of this study. According to Kothari

the purpose of a research design is to allow for the acquisition of important evidence with minimal expenditure of effort, time and money. In order to achieve the purpose of this study, the identified research design that can measure the level of readability of textbooks is a quantitative descriptive approach that employs the readability test for reading texts in the Advanced Arabic Textbook.

The research tool used is the Cloze test, which is the best alternative that provides an indication about the suitability of reading materials with a readability level suitable for a particular individual Kamarulzaman

since there are no other suitable methods for measuring the level of readability of Arabic reading materials for non-Arabic speakers. This Cloze test uses selected texts in the Advanced Arabic Textbook only and does not include other textbooks. The procedure used to build and conduct this test is based on the guidelines and recommendations presented by several scholars (Rye
) and Kamarulzaman
with some changes based on the needs and focus of this study. The procedure is as follows:

Identify three reading texts from 14 topics in the Advanced Arabic Textbook. This test is divided into three main texts. Which are text A containing text from the beginning and has 54 blank spaces. Text A was taken from the beginning of the textbook. The next one is the middle text or text B, which is selected from the fifth topic in the textbook. This text has 44 blank spaces. Finally, the last topic in the textbook is text C, which has 41 blank spaces. The title of each text is shown in Table 1.

Table 1 - The Readability texts

Discourse position

Discourse Title

Text A (Beginning)

أَنَا أُحِبُّ اللُّغَةَ العَرَبِيَّةَ

Text B (Middle)

الْفَحْصُ الطِّبِّيُّ

Text C (End)

العِبَادَةُ فِي الإِسْلاَمِ

ii. All these texts are arranged exactly as in the original textbook. Students are required to accurately fill in the blanks in the text. Students are given three words as answer options, and they are required to choose the appropriate word whether it is the first, second, or third word. Students only need to choose the correct answer.

iii. As for foreign language texts, one in every seven words was omitted, regardless of a sentence form or word type, Thuaimah

. Deletion of every seventh word is preferable when the student's language proficiency is weak or the text is quite difficult.

iv. Based on the study of the Cloze test, the multiple-choice answers technique is used because it is suitable and can reflect the actual level of the text's readability. For each gap, three answers are provided, but only one is correct.

v. A brief instruction of the test is included on the cover page, in addition to the verbal instructions, to ensure that students truly understand the purpose and response required in this written test.

iv. The scoring system in this study only accepts the correct answer based on the original word as it appears in the Advanced Arabic Textbook to be the correct answer. The correct answer is then converted into a percentage by dividing it by the number of omitted words.

4. Legitimacy And Reliability

After the instrument built, evaluation of the experts assessments pertaining to the Arabic language was carried out. Results indicated that several items did not measure what was required to be measured in this study. Thus, several items from texts A, B and C were considered for removal. Then, a pilot study was carried out on 30 respondents to determine the reliability of the instrument. The results found that all the texts tested to measure the level of readability of the Advanced Arabic Textbook indicated a high level of reliability, which exceeded the value of 0.90, as suggested by Linacre

. This means that the instrument is valid and reliable for use in collecting actual data for identifying the level of readability of the Advanced Arabic Textbook.

4.1. Population And Sampling

The purposive sampling method was used in this study, as it is a form of a non-probability sampling. From a population number of 273 students enrolled in the Advanced Arabic course, only 169 students were selected based on the sampling design suggested by Krejcie & Morgan


4.2. Data Collection

The data collection process used in this study to determine the level of readability of an Advanced Arabic Textbook is described as follows

1. The researcher contacted the Academic Management Office at UnIPSAS to obtain the list of names of diploma students enrolled in the Advanced Arabic course and to determine the location and actual number of study samples.

2. The researcher obtained approval from the Vice Chancellor of Academic Affairs, UnIPSAS, various Deans of faculties and coordinators of courses to conduct studies involving the students.

3. The prospective respondents involved were informed through the "WhatsApp" application by the lecturers who were teaching and providing information about the time, day and place of the test.

4. The researcher had a meeting with the respondents through the “Google Meet” application.

5. Before the test started, the researcher gave a briefing regarding the purpose of the Cloze test and the rules to answer the test questions.

6. The test commenced after the briefing was completed and the respondent understood the purpose and intention of the test. The Cloze test lasted for one hour and 30 minutes.

7. The instructions and questions were moved to “Google Form” in order to make it easier for students to answer online, and the answers were recorded and saved in Microsoft Excel form.

4.3. Data Analysis

Findings were analysed by determining the percentage of correct responses for each text separately and as a whole, in example all three texts at once. The percentage of the respondent's score was compared to the Readability Score Measurement Table (9), as shown in the table 2:

Table 2 - The Readability Score Measurement Table

Readability Level

Score Range, %







Note: source: [9]

5. Results & Discussions

1) Text A

Table 3 - Readability Level of Text A based on Rye's Readability Score Measure (1982)

Readability Score












Text A is a text from Chapter One with the title أَنَا أُحِبُّ اللُّغَةَ العَرَبِيَّةَ. According to Table 3, 34 students filled in all the blank spaces in the readability test correctly, which is equal to 20.1% of the sample. These are the students at the independent level, where they can read and understand the Advanced Arabic Textbook without their teacher's help or supervision. The number of students in the instructional level was 21, which is equal to 12.4% of the total sample. The students at this level need guidance from teachers to understand what they read. The number of students at the frustrational level was 114, which is equal to 67.5% of the total sample. At this level, the students are unable to read or understand the Arabic language book, even with the help or supervision of their teachers. Overall, the readability level of Text A shows that this text is at the frustrational level, whereby it is difficult for students to read the text, and it is unsuitable for use in teaching and learning.

2) Text B

Table 4 - Readability Level of Text B based on Rye's Readability Score Measure (1982)

Readability Score












Text B is taken from chapter five, with the title الْفَحْصُ الطِّبِّيُّ. The majority of 90 students (53.3%) indicated that this text was at the frustrational level. While 52 students were seen to be at the independent level, (30.8 %) and only 27 students experienced the instructional level of the tested text (16%). Hence, based on the percentage stated above, it can be concluded that Text B is also at the frustrational level with a Rye’s Readability Score Measure of 0-74.

3) Text C

Table 5 - Readability Level of Text A based on Rye's Readability Score Measure (1982)

Readability Score












Text C is taken from chapter nine, with the title العِبَادَةُ فِي الإِسْلاَمِ. Table 5 shows the findings on the readability level of Text C, 29 students (17.2%) at the independent level, 31 students (18.3%) indicating the instructional level but 109 students (64.5%) at the frustrational level.

Based on the table 1,2 and 3 we can conclude that text A shows the frustrational level with 67.5%. Likewise, texts B and C are also in the frustrational level with 53.3% and 64.5% respectively. After carrying out an analysis of the entire text, the study found that texts A, B and C for the Advanced Arabic Textbook were at the frustrational level.

According to Aisyah (2014), her findings on the readability of Arabic texts at the tertiary level and also Arabic books used in the university, namely Maharat al-Qiraah, are consistent with the findings of this present study. Aisyah (2014) conducted the Cloze Test on 162 students and found that the average score was 65%, which was interpreted to be at a low level and unsuitable for teaching and learning purposes in class.

The results of this study show that the scores obtained by students for the Cloze Test were at the frustrational level (62%). This means that the respondents have a poor comprehension of the text, slow and stuttering speech, and use many words to analyse. However, this result cannot be extended to students from other universities because this study focused only on one group.

Therefore, it is anticipated that the Cloze Test can be used to measure difficulty in reading texts used for learning purposes in higher education institutions in Malaysia, in addition to measuring students’ level of comprehension.        

6. Conclusion

Therefore, it can be concluded that the level of readability of Advanced Arabic textbooks is in the frustrational level, which exceeds the ability of the target readers, namely first year diploma students. In this regard, it is hoped that the authors of the books and publishers can re-assess the level of readability of Arabic texts using reasonable procedures, such as those used in this study, to ensure that the selected texts are at a teaching level which is appropriate for the target learner level.

Until now, there are still no research on the readability of Arabic textbooks at the diploma level has been carried out in Malaysia. Studies on the readability of Arabic textbooks at the advanced level are very important for evaluating, improving and strengthening these Arabic textbooks used at UnIPSAS considering the students' capabilities. Although this study is limited only to Advanced Arabic texts used in UnIPSAS, this type of study can be extended to all higher education institutions in Malaysia in a variety of broader areas to strengthen the curriculum used in line with the Malaysian education system today. Textbooks play a very important role in the teaching and learning process of Arabic subjects. Readability is an important factor in determining the difficulty or ease of the text used, and can also help certain particular peoples or groups in creating an acceptable textbook in the future. The results of this study are not only important to UnIPSAS, but can serve as a benchmark for all higher education institutions in Malaysia. They are very useful for evaluating, improving and designing a new curriculum to meet current needs. The findings can also serve as a guide for students and teachers involved in Arabic language either in primary level education or Arabic classes in government schools, parents, textbook writers, policymakers and curriculum developers, or the Ministry of Education of Malaysia. Inadequate teaching and learning or inaccurate assessment identified in Arabic textbooks can be improved to enhance the quality of education in the country in the future.

Article metrics