Question analysis in indonesians’ new curriculum secondary mathematics textbook

Doi: 10.24042/djm.v5i3.14590 This study analyzed questions (examples and exercises) in the Numbers chapter on Indonesians’ new curriculum (Kurikulum Merdeka) mathematics textbooks for grade 7. An analytical framework developed for mathematics textbooks question analysis had seven dimensions: Mathematical Activity, Problem Complexity, Answer Type, Contextual Situation, Response Type, Mathematical Questions, and Data Problem. The result showed that the Indonesians’ new curriculum of secondary mathematics textbooks contains more balanced question types. In Mathematical Activity question types, only 46.73% of questions were about counting and using count operations. The questions about interpretation were 28.97% and questions that needed an argument or logical reasoning were 15.58%. In question type analysis, 22.12% were open-ended questions and 77.88% were close-ended questions. We also found that there were no multiple choice question types in Indonesians’ new curriculum (Kurikulum Merdeka) mathematics textbooks for grade 7. On the other hand, we also found questions with insufficient data (12.15%) and extraneous data (7.48%).


INTRODUCTION
Mathematics textbooks are the main teaching aids in mathematics teaching and learning activity in class (Fan et al., 2013).Mathematics textbooks provide students different learning opportunities and directly affect students' learning outcomes (Reys et al., 2010;D.-C. Yang et al., 2010)).Moreover, as textbooks are a key component of the intended curriculum, they also, to a certain degree, reflect the educational philosophy and pedagogical values of the textbook developers and the decision makers of textbook selection, and have substantial influence on teachers' teaching and students' learning (Zhu & Fan, 2006).It should be noted that textbooks are not the only curriculum materials and not the only factor influencing students' mathematics achievement.
Nevertheless, since textbooks are important materials for predicting students' performance in mathematics, some studies emphasized that the analyses of textbooks can provide an important means for explaining the differences in student achievement (Reys et al., 2004).
Trends in International Mathematics and Science Study (TIMSS), which has done research on hundreds of textbooks from 50 nations, has made the function of textbooks in the learning of mathematics its primary concern and emphasis at the moment (Shield & Dole, 2013).Previous researchers have also conducted problem analysis of mathematics textbooks in Indonesia from the 1994 curriculum to the 2013 curriculum (2017 revision).Based on the study results, the questions in Indonesian textbooks generally have no significant changes even though the curriculum has changed from 1994 to 2017.The types of questions presented in mathematics textbooks in Indonesia still use various arithmetic operations, applying direct knowledge or basic skills without any daily life context.The existing questions are also the closed answer types; the questions only require direct answers without reason and a single procedure (Raditya, Iskandar, & Suwarno, 2020).Previously, Indonesia adopted the 2013 curriculum (revised) as a solution to improve the deficiency of the previous 2013 curriculum ( Ratu Sarah Fauziah Iskandar et al., 2021) This research was conducted to analyze the Indonesian new curriculum secondary mathematics textbook and focused on the Numbers chapter.The study addressed the following research questions: What types of questions occurred in this textbook?To answer that question, this research used a modified framework from (Glasnovic Gracin, 2018;Zhu & Fan, 2006;Li, 2000) then there were 7 dimensions for textbook analysis.Those dimensions are Mathematical Activity (A), Problem complexity (B), Answer type (C), Contextual situation (D), Response type (E), Mathematical Procedure (F), and Data Problem (G).
The ability of students to describe or model, count objects or utilize different counting procedures, understand data, and offer justifications or arguments makes up the dimension of mathematical activity.While the direct application of fundamental information or abilities (reproduction), creating or forming links (connections), and applying reflective knowledge make up the complexity level dimension (reflection).The sort of answer (answer form) is then either closed, open, or provides a range of options.Questions with multiple possible solutions are said to have open answers, whereas those with just one right response are said to have closed answers.The contextual situation (contextual characteristics) is a dimension that examines how questions relate to the context of everyday life.Questions without context, realistic or fictitious contexts, and real-world situations make up this dimension.The four dimensions above are modifications of the framework developed by Gracin (Glasnovic Gracin, 2018).
The response type and the type of mathematical features make up the dimension that modifies Li's (Li, 2000) framework.The response type refers to the type of response provided by students in order to address the issue, and it includes answers with and without justifications.The relationship between the number of procedures utilized and the supplied problem will next be examined using the mathematical features on this dimension.A single technique and a layered procedure for resolving a specific problem will be seen in this dimension.
The data problem features are a further dimension that modifies the framework created by Zhu & Fan (Zhu & Fan, 2006).The data problem, also known as an excessive data problem, is a sort of problem that has more than enough data or conditions to solve it.If a problem's Desimal, 5 (3), 2022 -355 Aji Raditya, Nisvu Nanda Saputra information is fundamentally insufficient to yield a solution and it is neither expected nor feasible for the problem solver to fill in the gaps, the problem is deemed to have insufficient data.The remaining issues are classified as sufficient data issues because the information is precise enough for a problem solver to address them.
This research was conducted to analyze and map question types in Indonesian new curriculum (Kurikulum Merdeka) secondary mathematics textbook and focused on the Numbers chapter.

METHOD
This study using qualitative method that based on a content analysis of secondary mathematics textbooks on Numbers chapter.Those textbooks were published by the Indonesian Ministry of Education using a new curriculum (Kurikulum Merdeka) and printed in 2022.In this study, examined items consist of exercises and examples that are included in the Numbers chapter.
The framework in this study is modified from (Glasnovic Gracin, 2018), (Zhu & Fan, 2006) and (Li, 2000), there were 7 dimensions for textbook analysis.Those dimensions are Mathematical Activity (A), Problem complexity (B), Answer type (C), Contextual situation (D), Response type (E), Mathematical Procedure (F), and Data Problem (G).With this framework, the researcher will classify and convert the questions in the mathematics textbook into a coding system (Table 1).Two researchers served as the raters for checking the coding reliability.These mathematics educators reviewed the methods used for coding the content in the student textbooks and then independently coded the material.This study applied the measures recommended by (D. Yang & Chiang, 2017) and obtained a P value of mutual agreement of 0.866, and a value of reliability of 0.928.The average mutual agreements between the two raters are 0.890 and 0.850 for those textbooks.The P value and reliability were calculated by the following methods: (1) Mutual agreement for two raters (Pi): M represents the total item numbers for mutual agreement N1 + N2 represents the total item numbers for coding (2).The average P value for mutual agreement (P) N represents the total number of raters (3) Reliability (R) n represents the total number of raters (Yang D, Reys R, Wu, 2010).

RESULTS AND DISCUSSION
The research indicated that, in general, there was a bit of a balance between the types of questions in the new curriculum (Kurikulum Merdeka) Indonesian mathematics textbooks for the numbers topic presented.Based on the dimensions of mathematical activity (dimension A), questions on the textbook are mainly in the form of counting or presenting various counting operations (A2).In the dimension of problem complexity (dimension B), most of the questions are still in the form of application of direct knowledge or basic skills (B1).Furthermore, in the answer types (dimension C), those mathematics textbooks contain closed answer (C1) questions.In the contextual situation dimension (dimension D), the type of question without context (D1) dominates the question in those textbooks.In the dimension of response type (dimension E), short-answer (no reason) problems (E1) are dominant.In the dimension of the mathematical problems (dimension F), the types of problems curriculum mathematics textbooks are dominated mainly by single procedure (F1) type of problems.In the dimension of the data problems (dimension G), the types of problems in those mathematics textbooks are mostly questions with sufficient data problems.Related to the dimension of mathematical activity (dimension A), the questions in those mathematics textbooks with the highest percentage of 46.7% are the sub-dimensions of representing or modeling (A1), while 28.9% of interpretation (A3) subdimensions.There is still less argument or logical reason (A4) sub-dimension (15.5%) and representing or modeling (A1) sub-dimensions (8.7%).Related to the dimension of problem complexity (dimension B), the problems in those mathematics textbooks with the highest percentage of 62.6% are the subdimensions of application of direct knowledge or basic skills (B1), while only 18.6% each of making connection subdimensions (B2) and Apply reflective knowledge (B3).The comparison of dimension A and dimension B could also be seen in Figure 1.In problem complexity (B) dimension, the percentage of questions that include in making connection (B2) sub-dimension could be increased.Ausubel, Bruner (Mundiri, Suwangsih, Erna, Tiurlina, 2006) explains "In mathematics, every concept is related to another concept.Neither with others, such as theorem and theorem, between theory and theory, between topics and topics ".Mathematical connections required students to study several mathematical topics that are interconnected with one another.If a topic is presented individually, the lesson will lose a valuable moment in the effort to improve students' mathematics achievement in general.Students will struggle to learn mathematics if they do not have a mathematical connection ability.
Related to the dimension of answer type (dimension C), the questions in those mathematics textbooks with the highest percentage of 77.9% were the subdimensions of closed answer (C1), while only 22.1% of open answer (C2) questions and not a single multiple choice answer (C3) questions.Related to the dimension of contextual situation (dimension D), most of the questions in those mathematics textbooks were questions without context (57.3%) or D1 subdimension, while only 38.9% questions were question with real-world contexts (D3) and only 3.7% questions were problem with context of fiction (D2) The comparison of dimension C and dimension D could also be seen in Figure 2. In Answer type (C) dimension, there were only 22.1% of questions with open answer (C3) sub-dimension.Open-ended problems can provide students with a sense of accomplishment and fulfillment because they allow students with limited mathematical ability to propose their own solutions within their own capabilities.Furthermore, it allows students to experience what it is like to be a true mathematics learner by allowing them to create their own problems (Kwon et al., 2006).On the other hand, questions with open answer or open-ended question will lead to an open-ended learning (OEL) model that can support student creativity thinking skill in learning mathematics (Kartikasari et al., 2022).
In the Contextual situation (D) dimension, questions with real-world context (D3) dimension were already in great percentages compared to percentage in previous Indonesian curriculum.According (Raditya et al., 2020) percentage for questions with realworld context dimension were 30% in KBK curriculum, 6.67% in KTSP curriculum and 5.68% in K13 curriculum compared to 38.9% in new curriculum (Kurikulum Merdeka).Research found that learners found it easier to work with calculations using the contextual rules than to engage in reasoning about the rules.A common tendency of providing their everyday reasoning instead of contextual reasoning was identified.It was also found that learners' engagement in contextual reasoning was hampered when they did not understand some of the contextual language (Bansilal & Debba, 2012).
The dimension of response type (dimension E), the questions in those mathematics textbooks with the highest percentage of 78.5% were the subdimensions of answer only (no reason) (E1), while only 21.5% were the subdimension of answer using reason (E3) sub-dimensions and 0% questions were the sub-dimension of reason only (E2).Related to the dimension of mathematical procedure (dimension F), there were 68.8% of questions from sub-dimension single procedure (F1) and only 31.2% of questions were layered procedure (F2) sub-dimension.Finally on data problem (dimension G), 80.4% of questions were sufficient data problem (G1) questions.The extraneous data problem (G2) and insufficient data problem (G3) were only 12.1% and 7.4% respectively.The comparison of response type (E) dimension, mathematical procedure (F) dimension, and data problem (G) dimension could also be seen in Figure 3.The study found that most problems in these textbooks provided just enough information for questions asked (80.4%) compared to questions with insufficient and extraneous information (7.4% and 12.1%).Problems containing either extraneous or insufficient information would likely cause students to have an impression that problems always had exact information.In fact, researchers have shown that many students attempted to all of the numbers presented in their solutions no matter whether those numbers were necessary in the solutions or not (e.g., Carpenter et al., 1980).However, such an impression contradicts the reality of the situation.In reality, people must gather information, evaluate the quality of the information, and then select the information required to solve problems, which is a necessary procedure (Zhu & Fan, 2006).

CONCLUSIONS AND SUGGESTIONS
In general, this research found Indonesian new curriculum (Kurikulum Merdeka) mathematics textbooks is in the right direction to become high quality textbooks.Compared with Indonesian textbooks from the previous curriculum (KBK, KTSP and K-13), these Indonesian new curriculum (Kurikulum Merdeka) mathematics textbooks have more variety and balanced questions.Although the questions still dominated with closeended questions (C1) these mathematics textbooks started to introduce openended questions (open answer of dimension C2) to 7th grade students.Based on the findings, these textbooks could include more questions in realworld context (D3).Researchers believed that increasing the number of real-life or application problems could increase the variety of problems, thereby providing students with a learning environment conducive to a higher level of understanding.
From the findings of this study, we also think it would be helpful if there were more problems with extraneous data (G2) and those with insufficient data (G3) in the textbooks.Using these types of questions, forced students to analyze and selectively choose which data was important to solve certain problems.A comparison of question types in KTSP mathematics textbooks and new curriculum (Kurikulum Merdeka) mathematics textbooks would be an interesting next research project.It would also be interesting to examine question types in new textbooks using the PISA or TIMMS frameworks.

Figure 1 .
Figure 1.Percentage for Dimension A and B

Figure 2 .
Figure 2. Percentage for Dimension C and D

Figure 3 .
Figure 3. Percentage for Dimension E, F, and G

Table 2 .
Result Dimension and Sub-Dimension

Table 3 .
Example of questions analysis in textbooks