The effect of hawgent dynamic mathematics software on mathematics achievement: A meta-analysis

Abstract


INTRODUCTION
Since technological developments are increasingly advanced, the internet, computers, and tools related to technology also develop and help facilitate social and encourage social progress (Aixia et al., 2020). Technological developments unconsciously continue to influence education and reform education (Depaepe et al., 2007;KPMG, 2016;Maeng et al., 2013). Many learning media, such as videos, animations, sounds, and so on, come from technological developments (Weinhandl et al., 2021;Wijaya, Zhou, et al., 2021). This development can change students' views to better appreciate mathematics (Bouma et al., 2010;Imansari et al., 2019;Wijaya, Ying, et al., 2020). For teachers, ICT development helps to transfer mathematics material to students (Shute & Emihovich, 2018;Yildiz Durak, 2019). Dynamic mathematics software, as a product of the development of technology and information, can efficiently enter teaching and learning activities and help teachers visualize measure, and animate abstract mathematical topics to be easier to understand (Wijaya, Tang, et al., 2021;Wijaya, Zhou, et al., 2021). Many studies have proven that dynamic mathematics software can incr ease students' interest in learning, make students more active, and make learning more effective and efficient (P. Jerito, Shiwei Tan & Purnama, 2020;Wijaya, 2021;. Dynamic mathematical software has been developed for more than two decades (Martinovic & Manizade, 2020;Sablić et al., 2021). Software that is often used globally and internationally is Geogebra and Geometer Sketchpad Rohaeti & Bernard, 2018). Geogebra and Geometry Sketchpad-based learning media is dynamic and can help mathematics learning (Listiawan et al., 2018;Rockinson et al.,2013). Hawgent Inc. in Guangzhou, China, analyzed the shortcomings of the two dynamic mathematics software and saw that Geogebra and Geometer Sketchpad are paid software to use all of the features in teaching and learning activities in class Suan et al., 2020). Also, GeoGebra and Geometer Sketchpad do not have strong analytical geometry and cannot meet the needs of teaching algebra and statistics. GeoGebra and Geometer Sketchpad are more difficult for novice teachers (Kaǧizmanli et al., 2011;Reis, 2010). From the background analysis, Hawgent emerged and was developed in 2016 in Guangzhou, China, which was modified from the shortcomings of the previous dynamic mathematics software. Hawgent is easier to use and can visualize mathematical contexts such as geometry, algebra, statistics, calculus, and probability. The software is very light and free .
After Hawgent was launched in 2016, teachers were very enthusiastic and gave a positive impression of Hawgent, although it has not been launched too long ago. Furthermore, when Hawgent was introduced to Indonesia, it received enthusiasm from teachers in West Java and Riau, Indonesia . Some previous Hawgent research has proven that Hawgent can increase students' learning interest, learning performance, mathematical achievement, and other mathematical abilities (Wijaya, 2021;Wijaya, Jianlan, et al., 2020;Wijaya, Sukma, et al., 2020;Wijaya, Tang, et al., 2021;Wijaya, Zhou, et al., 2021). Hawgent can be easily installed anywhere, both on tablets and laptops, in minutes. Hawgent dynamic mathematics software is also widely combined with micro-lectures or interesting learning videos in mathematics.
After five years, there is little information about the effect size of Hawgent on mathematics achievement and other mathematics abilities. In the search of "皓骏(Hawgent)" on CNKI China, there were only 12 empirical research papers explaining Hawgent's influence on mathematical achievements. In the Web of Science, Science Direct, Engineering Village, Google Scholar, There are 206 articles about Hawgent were retrieved, but only a few pointed out Hawgent's influence on mathematical achievements. It is also unknown under what conditions Hawgent has a good effect size. Therefore, a meta-analysis study by summarizing the effect size of Hawgent based on further research should be conducted.
Meta-analysis is a quantitative method that has been widely used in the medical world (Choi, 2017;Coles, 2019). In recent years, meta-analysis has also been used to solve problems in the social sciences (Lalian, 2018;Taranto & Arzarello, 2020). Glass GV first coined metaanalysis in 1981 (Glass, G. V., MacGaw, B., & Smith, 1981). Meta-analysis can summarize and see the effect size specifically to determine the effectiveness of an object (Chiang et al., 2022;Page et al., 2021). The meta-analysis method was used to analyze Hawgent's effectiveness on mathematics achievement in the last five years in this research.
There are three focus questions in this research: First, does Hawgent dynamic mathematics software have any effect on mathematics achievement?; Second, how strong is the effect of size Hawgent on mathematics achievement?; Third, is there a difference in the effect size of Hawgent on moderator effects such as publication year, number of students, grade level, gender of the teacher, duration of use, and mathematics topics?.
There are three innovations in this research: First, our study is the first to propose a metaanalysis of Hawgent; Second, make a qualitative visit to some authors we are familiar with to explore the conclusions and outlook suggestions; Third, the results of this study will inform Hawgent managers and the graduate students who are studying Hawgent to help them to further repair the existing research.

METHODS
To answer the three research questions of this research, the researchers employed a meta-analysis technique to explore the overall impact of Hawgent on mathematics achievement.t By drawing on the researchers' research methods (Field A P, Gillett R., 2010;Hedges L V., 1992), we total the steps applicable to this study.The steps of the analysis technique are 1) searching Hawgent data from a published article, 2) identifying and selecting the article, 3) performing data coding, 4) testing the reliability, and 5) testing the effect size and analyzing the results.

Data Search Strategy
The researchers used several databases to search for a research article that discussed the use of Hawgent dynamic mathematics software in mathematics. The databases used in this metaanalysis research are: •

Identifying and Selecting the Article Criteria for the Selected Article
The criteria for the articles that can be used to analyze the Hawgent effect size on mathematics achievement using the CMA software can be seen specifically below: • The research was conducted from 2016 to 2021.
• The object of research is from elementary school to high school • The type of research is quantitative, empirical research that uses two class groups, experimental and control. • The article or thesis has been published, which means that the quality of the article has gone through a blind review, and the validity is recognized. • The article contains statistical information such as the average value, standard deviation, sample size, and the length of the period of using Hawgent. These data are reference data to measure the effect size of Hawgent dynamic mathematics software.

Coding Process
Because of the need to code articles in Chinese and English, we chose two graduate students with mathematics education direction and good Mandarin and English skills, they independently coded and checked the data in each article. If the agreement is not obtained from coder 1 and coder 2, then coder 3 is chosen to help decide the data assessment. We took four steps for the selection of documents (Moher et al., 2009), as shown in Figure  1. Of the 378 articles, 78 articles remained after excluding the literature that did not meet the rules based on the abstracts. Then, by reading the full-text catalog, 28 articles (26 Chinese version graduation theses and 2 English version empirical articles) were screened and met the inclusion criteria. Among them, six of the Chinese version of the graduation thesis contained several experimental sets. After a detailed review of the full text, the data was divided into 46 experimental data (including 2,070 students in the experimental class and 2,067 students in the control class, with 4137 students).

Moderating Effects
In addition to studying whether Hawgent has an impact on student Mathematics Achievement, we can also explore which factors of the study subjects will influence Mathematics Achievement. In this study, by combing the literature and referring to the existing studies, these regulatory variables were explored: publication year, number of students, grade level, gender of the teacher, duration of use, and mathematics topics.
Hawgent dynamic mathematics software was launched in 2016, and new empirical research was carried out in 2018. Furthermore, a new version of Hawgent 3.0 was launched in 2020, so there is no empirical research in 2022. Based on this background, We want to study the effect of Hawgent on mathematical achievement under different versions, the researcher divided research articles on Hawgent into two groups, namely 2018-2019 and 2020-2021, and analyzed the effect size difference between the two-year groups (C, 2013). The main principle in the development of ICT is to provide learning media according to education. Students at each level of education have different needs and cognitive levels (Piaget) regarding ICT (C, 2013). Many meta-analytical studies divide education levels into three groups: elementary school, junior high school, and senior high school. Based on this reason, this research divided education levels into three groups, namely grades 1-6 (elementary school), grades 7-9 (junior high school), and grades 10-12 (senior high school).
Previous studies have shown that the sample size (Choi, 2017;Coles, 2019) or the number of students in one class will affect a learning media. Studies with less than 50 students are considered a small sample and are often used in many other meta-analytical studies. In China, the average of a class averaged about 50, so, to facilitate study, This research divided the sample of students into two classes, classes with students under 50 and classes with students above 50. Many studies have examined that the gender of teachers affects students' mathematical abilities. Previous research found that male teachers have better technological pedagogical knowledge than female teachers (Garg, 2021;Möller et al., 2020). Therefore, this research intended to determine whether Hawgent had a higher effect size when used by male teachers than by female teachers.
In meta-analysis research, the experimental time of using instructional media is included as an important moderating factor (Chen & Yang, 2019;Tutal & Yazar, 2021). Usually, the treatment duration is divided into several groups, including "1-4 weeks", "5-8 weeks", and "8-11 weeks". This research analyzed the effect size differences of the use of Hawgent on students' mathematical achievement within the three periods. Dynamic mathematics software aims to help teachers to teach mathematics material. Mathematical material is mostly divided into several major groups, namely numbers, algebra, geometrics, statistics, and probability . Therefore, this research analyzed the effect size of the use of Hawgent in each topic of mathematics lessons.

Reliability Test
The coding done in Excel was inputted into SPSS to perform the Cohen Kappa test. A reliability test was conducted to see the compatibility of the two coders when coding before further analyzing the five moderating variables that affect the Hawgent effect size on students' mathematics achievement. The reliability test result obtained was 0.89, which was higher than the lower limit of 0.7. Therefore, the coding results of the two coders were homogeneous.
This research deployed two team members to code the original literature and produce an Excel data worksheet after identifying the original literature. The (Viera & Garrett, 2005) consistency of the coding results of the two team members was calculated to be 0.89, which proved that the coding results were reliable. Then, according to the statistical requirements, the Cohen kappa value was greater than 0.7 (Viera & Garrett, 2005).

Data Analysis
The meta-analysis in this research used Comprehensive Meta-Analysis 3.0 to analyze the Hawgent dynamic mathematics software data. Standardized Mean Difference (SMD) was used as the effect size measure for the continuous variable. Furthermore, the Risk Ratio and 95% confidence interval were used as the statistical effect analysis for the dichotomous variable. According to the researchers' conclusions (Jia Z, Zhang X, Kang S, et al., 2013;Rozental A, Bennett S, Forsström D, et al., 2018). This research used a test to determine the heterogeneity of each study. If (P=0.00<0,1, I^2>50%) is obtained, then the random-effects model is used as a reference for effect size. 2 . If there is statistical heterogeneity between the results of each study (P=0.00<0.1, I^2>50%), a random effect model is used, referred to as Random or REM for the meta-analysis.

Publication Bias Analysis
A publication bias test was conducted to prove that all studies used in this meta-analysis did not have publication bias before further analyzing the overall effect size of the Hawgent and moderating effects (Lalian, 2018;Taranto & Arzarello, 2020). Publication bias can affect the effect size in meta-analysis research. The effect size does not match the actual effect size (Frianti., 2019) and affects the meta-analysis study's quality and accuracy. Publication bias is considered to occur when the published research literature is not systematically representative of the research population carried out in the field . This research used a funnel plot by Hedges's g and safe N files for publication bias testing (Garzón & Acevedo, 2019;Ulum, 2021).
In the funnel plot, publication bias can be analyzed by seeing whether the bubbles on the right are equal and balanced with the bubbles on the left (Liu et al., 2013;Villena-Taranilla et al., 2022). Figure 2 shows that the vertical axis is the standard error, and the horizontal axis is the effect size with a 95% confidence interval. Therefore, this research has no publication bias. Numerous previous studies state that the publication bias test is not sufficient only with funnel plots. Therefore, to further ensure the resistancy of publication bias, the researchers employed the safe N file test as supporting evidence. According to the CMA software analysis results, the N value is 3841, and according to the mullen formula (Boone et al., 2014), if N/(5K+10) is greater than 1, then it can be determined that there is no publication bias. It can be seen in table 1 that N = 3841 with K=46, so N/(5K+10)=3841/(46* 5+10)=16.00>1. Therefore, the result can be interpreted that there was no publication bias in the studies used in this meta-analysis.

Overall Hawgent Effect Size on Mathematics Achievement
A Forest plot was used to determine the overall effect of Hawgent on mathematics achievement specifically.   Table 2 displays that Hawgent has a significant effect on mathematics achievement of 2.11 (P = 0.00 < 0.1). According to Cohen (Cohen, J., 1988, ES<0.2 is in the small effect category, and the effect size of 0.2<ES<0.8 is in the moderate category. Furthermore, the effect size of ES>0.8 can be interpreted as a large effect category. Thus, it can be said that Hawgent has a great effect on students' mathematics achievement. Hawgent's overall effect size is bigger than augmented reality (Garzón & Acevedo, 2019), and bigger than blended learning (Bernard et al., 2014).

The Analysis of Moderating Variables on the Hawgent's Effect on
Mathematics Achievement This research divided the effect size according to different years, school period, class size, teacher gender, experimental period, and teaching to discover more about the Hawgent effect of different moderating variables. Specifically, the results can be seen in Table 3.

The Year Factor on Hawgent's Effect Size
In both the 2018-2019 period and the 2020-2021 period, Hawgent significantly affected mathematics achievement (P<0.05). There is a significant difference in effect size (Qn=101,477, P<0.001) between the 2018-2019 and 2020-2021 periods. In 2020-2021, the Hawgent effect size was greater (1.76) than in the 2018-2019 period, only 0.99. This finding can be explained for two reasons. First, in 2020, Hawgent released the Hawgent version 3.0 with beneficial features which are more comfortable and easier-to-use interface for novice teachers. In 2018-2019, teacher training and mentoring continued to be held during the summer and winter holidays so that teachers can adapt to Hawgent when teaching in their classes.

Education Level Factor on Hawgent's Effect Size
At each level of education, Hawgent had a high effect size (more than 0.8). Hawgent had the highest effect (1.56) at the senior high school level and had the lowest effect (1.05) at the elementary school level. Previous research believes that the dynamic mathematics software effects are different at each level of education. Next,  research shows that dynamic mathematics software achieves maximum results at the junior high school level and has the smallest effect at the senior high school level. Besides, Hawgent effect size at junior high school level is higher than VR and AR effect size which is only 0.006 (Avcı et al., 2019). This meta-analysis showed that Hawgent had a better effect at the senior high school level and the smallest effect at the elementary school level. It might be caused by senior high school students already having more mature spatial imagination skills. Therefore, Hawgent helps them visualize abstract mathematical forms and produce maximum effects.

Sample Size Factor on Hawgent's Effect Size
It is interesting that while other dynamic mathematics software is more effective in small classes, Hawgent can be used effectively in small and large scale classes. While a previous meta-analysis study showed that ICT-based learning media were more effective when used in small classes, the researchers found no difference in the Hawgent effect size in both small classes and classes with a sample of over 50 students. Hawgent has the same effect size, which is classified as high with the effect values of 1.37 and 1.41, respectively.

Gender Factor on Hawgent's Effect Size
Hawgent dynamic mathematics software had high effect sizes on male and female teachers (effect size > 0.8). However, this meta-analysis research found that female teachers produced larger effect sizes on mathematics achievement than male teachers. This finding may be explained by the fact that female teachers always prepare lessons and have high expectations of the learning media. Thus, Hawgent becomes better and unique for both students.

Hawgent Usage Duration Factor on Mathematics Achievement
The duration factor for the Hawgent effect was 1-4 weeks (effect size = 1.91, P<0.001), 5-8 weeks (effect size=1.19, P<0.001), and 8-11 weeks (effect size = 1.14, P<0.001). It can be seen that the 1-4 weeks duration had the highest effect. Furthermore, the Hawgent effect on mathematics achievement declined in the following week. Therefore, every dynamic mathematics software had the same effect. It is better used in the short term than in the long term. It made the students view that mathematics software was ordinary and not interesting anymore.
Students' curiosity about new things in the classroom stimulates their curiosity, significantly improving their performance and achievement.

Mathematics Topic Factor on the Hawgent's Effect Size
The hawgent effect is different in each mathematics topic. Besides statistics and probability material which showed no Hawgent effect on mathematics achievement, Hawgent had a high effect on every topic of mathematics subject (greater than 0.8). The detail can be seen in table 3.
In analyzing geometric and algebraic materials, higher spatial imagination and mathematical logic skills are required. Hawgent helps teachers visualize graphs and quantitative relationships between geometry and algebra and helps students visualize geometric shapes related to algebra.
As an example, proving (a + b) 2 = a 2 + 2ab + b 2 using a cube in geometric form (see figure 4). There was no Hawgent effect on mathematics achievement in statistics and probability due to several factors. First, Hawgent made a breakthrough in making dynamic mathematics software that can help with statistics and probability topics, so its development is not yet perfect. Second, research on the Hawgent effect on statistical and probability material has not been done much, so publication bias may occur, making the effect size unable to be analyzed properly. The researchers support other researchers to use Hawgent and test the effect of Hawgent on probability and statistical materials. After several years, the Hawgen effect on probability and statistics can be analyzed using the meta-analysis technique.
Every research always has shortcomings, including this research. Researchers believe that these shortcomings will be a new idea for further research. This meta-analysis only focused on the Hawgent effect on mathematics achievement and did not analyze the Hawgent effect on student effectiveness. Therefore, further research can analyze the effect of dynamic mathematics software on students' effectiveness rather than only focusing on students' cognitive domain. Furthermore, the article used in this meta-analysis study is limited to only Chinese and English language articles; however, the researchers believe that Hawgent has been widely used in other countries, including Indonesia.

CONCLUSIONS
Researchers used a meta-analysis technique to comprehensively evaluate the impact of Hawgent's dynamic mathematics software on mathematics achievement from primary to high school level. Furthermore, the researcher analyzed the effect of the moderating variable in 46 studies (4137 students) to find out under what conditions the Hawgent effect was greater. Based on the results of the analysis, it can be concluded that the Hawgent effect is significant on students' mathematics learning achievement. The results of the moderating effect analysis show that the Hawgent effect is high in geometry and algebra lessons. Furthermore, the effect of Hawgent is very high if it is used for less than four weeks. This condition must be considered when teachers use dynamic mathematics software to teach mathematics. In mathematics teaching, teachers should deeply understand the key and difficult points of content themes and teaching requirements, and design class activities according to different learning conditions and different teaching content instead of blindly using dynamic information technology. For example, in "geometry and algebra lessons," you can use more of Hawgent's technology to help students understand abstract geometric numbers.

AUTHOR CONTRIBUTIONS STATEMENT
XL: analyzed and interpreted the data; wrote-review and editing. TTW: conceived and designed the experiments; analyzed and interpreted the data; wrote the original draft. YZ: supervising; validation; writing-review and editing; project administration.