What is the Most Impressive Treatment to Foster Students’ Creative Thinking Skills? A Meta-Analysis and Bibliometric Review

: Research related to learning treatment to train creative thinking skills has been frequently done. However, this has become a concern for teachers to assess which ‘learning treatment’ is better in developing students’ creative thinking skills. This study aims to analyze the most effective treatment in improving students’ creative thinking skills on science and mathematics concepts, based on data sources from international journal articles. The meta-analysis was conducted using CMA software and involving 21 published papers in international journal indexed by Scopus and or WOS. Data analysis was carried out by comparing the effect size values from the research results. According to the results of the meta-analysis, the average effect size data obtained with the learning model reached 1.075, the learning approach reached 1.627


INTRODUCTION
Creative thinking, often mentioned as one of the most needed skills in this century (Akpur, 2020;Al-Momani, 2019), is one of the higher order thinking skills related to produce something original, new, different or unique, effective, meaningful, or exceeding the target limit (Beghetto, 2019;Gingl & Mingesz, 2019). However, it is not only limited to 'producing' activities, but also has a broader meaning. Refering to the creative thinking framework developed by OECD (2021), creative thinking activities are not only limited to generate different and creative 'brilliant' ideas, but also include evaluating, developing, and improving an idea.
Creative thinking is one of the ideal skills which are expected to be mastered by students during the learning process (Rusimamto et al., 2021;Saregar et al., 2020). This ability is very important for students to master, especially to learn abstract concepts (Gardiner, 2020;Sun et al., 2020), science and mathematics that contain many abstract concepts. For example, in studying physics or in understanding the concept of physics, students must be able to use their imagination well so that the concept of physics can be correctly depicted in their minds. Therefore, mastery of creative thinking skills is very important in learning physics (Batlolona, 2019;Wicaksono et al., 2017).
Because of the importance of creative thinking skills to be mastered, it is necessary to find solutions to practice creative thinking skills in everyday learning. As such, many researchers do experiments to train creative thinking skills. This is proven based on data from Scopus page, in the 2016-2021 range, there were 3,486 studies from various countries using Creative Thinking keywords.
There are several learning treatments that are often used to train creative thinking skills, such as problembased learning models (Rahman et al., 2020), STEM learning (Parno et al., 2019;Wannapiroon & Petsangsri, 2020), or as simple as applying technology as a learning media (Aldalalah, 2020;Park & Kim, 2021). Based on the results of bibliographic analysis using Vosviewer, in each keyword, several learning treatments are often associated with creative thinking skills based on 3,486 Scopus articles as shown in Figure 1. Based on the results of the bibliography, there are several research variables that are often related to creative thinking such as Problem-Based Learning models, project-based learning, design thinking models, technology-based learning, learning approaches like STEM/AM and so on.
The map also shows that just like other thinking skills, creative thinking skills can also be trained through learning (Lucas & Spencer, 2017;Tabieh et al., 2020). Thus, many educators and educational researchers try to apply varied 'treatments' in learning to train students' creative thinking skills (Batlolona et al., 2019;Saregar et al., 2020;Suryandari et al., 2018).
Many studies have proven that learning can train students' creative thinking skills. As the results, in the last three years, there have been many studies related to 'learning treatment' to train creative thinking skills. As pointed out by Suyidno et al. (2017), they found that the creative responsibility-based learning model was effective in improving students' scientific creativity skills. Saregar et al. (2020) stated that the application of the STEM approach positively affects students' creative thinking skills. The results of other studies also show that students' creative thinking skills can be improved by implementing the problem-based learning model (Batlolona et al., 2019;Nuswowati & Taufiq, 2015;Wartono et al., 2018). Shabrina & Kuswanto in 2018 have also proven that creative thinking skills can be increased through the application of Android-based learning multimedia.
From a bunch of studies previously mentioned, it tiggers a question for us regarding what is exactly the most effective learning treatment for training students' creative thinking skills? To answer such question, the researcher made a research design to analyze what learning models, approaches, and media had the most impressive results in training students' creative thinking skills. The analysis was carried out in two stages, namely the first was the bibliographic analysis with Vosviewer software, and the second was statistical analysis through the Comprehensive Meta-Analysis software. (Borenstein et al., 2017).
Based on the previous research, meta-analysis provided accurate results supported by strong statistical data in making conclusions from some studies. The meta-analysis conducted by Ritchie & Tucker-Drob (2018), for instance, this study succeeded in proving that education with a longer duration (additional duration) has a better effect than education with a shorter duration. Then, the meta-analysis conducted by Adesope et al. (2017) also succeeded in proving that the claims related to exams can hinder learning, in fact, are false. This study presents concrete conclusions based on the strong statistical analysis over the most impressive learning treatment to train students' creative thinking skills in Physics subjects. More specifically, this meta-analysis aims to truly answer the following research questions: What research treatments are most often used to practice creative thinking skills?; Does the difference in learning treatment give different results in creative thinking skills?; What learning treatment is most effective in improving students' creative thinking skills?

METHOD
This research is a quantitative research with a Meta-Analysis approach (Adesope et al., 2017;Mathiesen et al., 2021). Meta-analysis is used because this research aims to find and synthesize various relevant study results related to effective learning treatments to train students' creative thinking skills (Suparman et al., 2021).

Article Sample Selection Details
When searching on the scopus.com page, there are more than 13,000 articles related to creative thinking. However, this research is merely limited to the articles published in 2016-2021, and limited to the journal articles. In addition, the sample of articles were also obtained through manual searches in the Web of Science journals. In this study, two analytical processes were carried out. The first is bibliographic analysis to see some learning treatments which are frequently associated with the keyword 'thinking creatively'. The second, after getting the data, the articles were selected and then processed by utilizing CMA software. Details of the article selection process are described in Figure 2.

Bibliographical Analysis on Keywords
A bibliographic analysis is a process of listing a collection of bibliographic articles to describe a relationship from broad sections (Jiménez et al., 2021). In this study, keywords mapping (bibliography) was used to see the most popular and most frequently associated variables with creative thinking skills based on keywords from 1,663 scopus indexed journal articles and the Web of Science. Through this mapping, learning treatments will be selected in the form of learning models, learning approaches, and learning media that are most often studied along with creative thinking skills. This selection intends to make meta-analysis more focused and easier. The analysis was conducted with the help of vosviewer software.

Meta-Analysis with CMA
After the most popular learning treatment data were obtained, manual analysis was carried out to select the appropriate articles to be tested for metaanalysis with CMA V3. This software facilitates the application of complex concepts with the real data, helping researchers to see the power of the metaanalysis process. Comprehensive Meta-Analysis is easy to learn and use with a clear and friendly interface. The criteria for articles which are selected manually to be processed by using CMA software are: 1) The experimental research; 2) Science and Mathematics Learning; 3) With PBL, Design thinking, STEM/STEAM, and technology-based media variables; 4) There is a sample, mean and standard deviation, or t-value, or p-value. The list of articles processed by meta-analysis can be seen in Table 1. To find out what learning treatment is the most effective to be applied as an effort to develop creative thinking skills, an interpretation of the Effect Size value is used. The Effect Size interpretation in this study uses the interpretation of Thalheimer & Cook (Suparman et al., 2021;Tamur et al., 2020;Thalheimer & Cook, 2002). The interpretation is shown in Table 2.

RESULT AND DISCUSSION Variables that are Often Related to the Keyword 'Creative Thinking' based on Bibliographic Mapping
After the selection was made by limiting the year of publication (2016-2021), articles (proceedings and books excluded), and final articles (not in press) on 13,593 articles from Scopus and Web of Science, bibliographic data were obtained from 1,663 articles from research done in various countries. The distribution of the article locations can be seen in Figure 3. The United States is still the country that publishes the most articles related to creative thinking followed by China and Indonesia. The next step is to do a biblio-graphic analysis to list keywords which are often related to creative thinking. Based on data from Vosviewer, there are 105 keywords appearing, at least, 10 times along with creative thinking. Table  3 presents data for the top 10 keywords that often appear along with data occurrences and total link strength, and the illustration of the relation between keywords from 1,663 creative thinking articles. It can be seen in Figure 4.  Based on the results of bibliographic analysis, from the top 10 keywords, the variables which were taken comprise of learning models, learning approaches, and learning media (technology in learning). The selection process is presented in Figure 5. Based on Figure 5, it can be seen the research related to learning models for creative thinking that are most popular are PBL and Design thinking, while the research on popular learning approaches for creative thinking is STEM and STEAM, and then for learning media on creative thinking, the popular research is technology-based learning media. The purpose of this grouping is to make data search more focused. Therefore, manual analysis was carried out to find the right research articles to be processed by metaanalysis. There were 21 articles which were eligible to enter the meta-analysis stage.

Meta-Analysis of the Effects of Learning Models on Creative Thinking Skills
Meta-analysis was carried out with the help of Comprehensive Meta-Analysis software version V3 by entering data in the form of mean, standard deviation, t value, and p-value. The data are then processed to see the effect size results. The effect size of the learning model on creative thinking skills is shown in Tables 4 and 5. The data in Table 4 shows the effect size values of each study, and it is known that the largest effect size in the excellent category is indicated by the application of the PBL model with 3.765. In this part of the learning model, it is known that there are two models tested, namely design thinking and PBL. Each learning model was tested in different places, with different samples, and the results which were also very different. As in the article with the design thinking model, in the article with code A02, the effect size shows the ignored criteria, which means it cannot be used as a basis, but in the article with code A03, the effect size value is larger (0.430). This points out that the use of the design thinking model is very dependent on the state of the sample and the implementation strategy. Meanwhile, in the case of the PBL learning model, the samples were all from Indonesia albeit from different cities. However, it appears that, compared to design thinking, PBL has more influence on creative thinking skills. If the overall average is calculated, the average effect size of the design thinking model on creative thinking skills is 0.254 in the Moderate category, while the average effect size of the PBL model on creative thinking skills is 0.950 in the High category.
Based on the data in Table 5, it is known that based on the fixed effects estimation model, the overall effect size value (average) is 0.675 in the moderate category. After measuring the level of homogeneity of the article sample, the Q value is 154.464, which means it is higher than the χ 2 table value which is 3.94 (df = 10; α = 0.95). This means that the effect size distribution is heterogeneous. It is not surprising because the sample articles were taken from various countries with different number of research samples. Because the effect size is heterogeneous, we use the effect size from the random effects estimation model, which is 1.075 and is in the high category. The Z value in the random effects estimation model is known to be 3.269 and is declared significant due to the p value < 0.05. The I-squared value of 94.173 indicates that 94.173 % of the observed variance is due to differences in effect size, while about 5 % is caused by random error.
As previously noted that this study took samples from various countries so that the possibility of publication bias is unavoided. The funnel plot in Figure 6 shows the distribution of effect sizes. Based on the funnel plot in Figure  6, it can be seen that the distribution of effect sizes is uneven. Therefore, it is possible that these results are publication bias. To determine the resistance of the data to publication bias, it is calculated by using the Rosenthal's fail-safe N (FSN) statistic. Based on the FSN value calculated by CMA software, it is known that the Rosenthal safe N value is 244.00. Based on formula Mullen et al. (2001), the resistance of the data to research bias can be calculated by: 244.00/(5*10+10)= 4.1; where 4.1 > 1, the data used is resistant to publication bias, so there is no need for data to be discarded or recalculated.

Meta-Analysis of the Effects Learning Approaches on Creative Thinking Skills
A total of 4 articles with the application of the STEM/AM learning approach were involved in this statistical analysis. The effect size of the learning approach on creative thinking skills is shown in Table 6 and 7.  Table 6 shows the effect size of each research result, and it is known that the largest effect size in the excellent category is shown by the application of the STEM Approach with 1.773. This learning approach is applied in different places, with different samples, and produces different effects. As in the article with code A11, it produces an effect size with low criteria, but in articles with code A14, the effect size value is much larger, reaching 1.773. This depicts that the effectiveness of the STEM approach will vary relying on the circumstances of the sample, the learning environment, and the implementation strategy.
Based on the data in Table 7, it is known that based on the fixed effects estimation model, the overall effect size value (average) is 1.250 in the very high category. After measuring the level of homogeneity of the article sample, the Q value is 47.531, which means it is higher than χ 2 table which is 0.7107 (df = 4; α = 0.95). This means that the effect size distribution is heterogeneous. Thus, we use the effect size from the random effects estimation model, which is 1.627 in the excellent category. The Z value in the random effects estimation model is known to be 2.793 and is declared significant because the p value < 0.05. The I-squared value of 93.686 indicates that 93.686 % of the observed variance is due to differences in effect size, while about 6 % is caused by random error.
As previously noted, this study took samples from various countries so that the possibility of publication bias cannot be avoided. The funnel plot in Figure 7 shows the distribution of the effect size. Based on the funnel plot in Figure  7, it can be seen that the distribution of effect sizes is uneven. Therefore, it is possible that these results are publication bias. To determine the resistance of the data to publication bias, it is calculated using the Rosenthal's fail-safe N (FSN) statistic. Based on the FSN value calculated by CMA software, it is known that the Rosenthal safe N value is 99.00 (Mullen et al. 2001). Then: 99.00/(5*4+10)= 3.3; where 3.3 > 1 then the data used is resistant to publication bias, so there is no need for data to be discarded or recalculated.

Meta-Analysis of the Effects Learning Media on Creative Thinking Skills
Different effect size results are shown by the application of learning media. The effect size of the learning media on creative thinking skills is shown in Tables 8 and 9.  Table 8 shows the effect size of each research result, and it is known that the largest effect size in the excellent category is indicated by articles with code A15 with 3.590. Based on the data in Table 9, it is known that based on the fixed effects estimation model, the overall effect size value (average) is 1.242 with the very high category. After measuring the level of homogeneity of the article sample, the Q value is 90.691, which means it is higher than χ 2 table value which is 2.1673 (df = 7; α = 0.95). This means that the effect size distribution is heterogeneous. Because the effect size is heterogeneous, we use the effect size from the random effects estimation model, which is 1.575 in the excellent category. The Z value in the random effects estimation model is known to be 3.698 and is declared significant because the p value < 0.05. The I-squared value of 93.384 indicates that 93.384 % of the observed variance is due to differences in effect size, while about 6 % is caused by random error. Because this study takes samples from various countries so that the possibility of publication bias cannot be avoided, the funnel plot in Figure 8 shows the distribution of effect sizes. Based on the funnel plot in Figure  8, it can be seen that the distribution of effect sizes is uneven. Therefore, it is possible that these results are publication bias. To determine the resistance of the data to publication bias, it is calculated by using the Rosenthal's fail-safe N (FSN) statistic. Based on the FSN value calculated by CMA software, it is known that the Rosenthal safe N value is 298.00. Then: 298.00/(5*7+10)= 6.6; where 6.6 > 1 then the data used is resistant to publication bias (Mullen et al. 2001). Thus, there is no need for data to be discarded or recalculated.

Which Treatment is More Effective?
Based on the results of statistical tests of the three types of treatment, we can compare what treatment is the most effective for improving students' creative thinking skills. This comparison can be seen in Figure 9. Based on Figure 9, it can be seen that the learning approach has a greater effectiveness value than the learning media and learning models. Furthermore, based on Table 4, Table 6, and Table 8, we can clearly see which variables have the greatest effectiveness on creative thinking skills as shown in Figure 10. Although overall the learning approach has a larger average effect size, if we look at each variable, PBL shows the most impressive effect size results, reaching 3.765 (see Table 4) in the excellent category. Several previous studies have revealed that PBL is effective in improving several learning skills, such as problem solving skills (Juandi & Tamur, 2021) and enhance higher order thinking skills (Suparman et al., 2021). The second and third variables with the highest effect size values were the application of technology-based media, followed by the application of the STEAM approach and lastly, PBL once again recorded an excellent effect size value (2.264), strengthening the fact that PBL does have a strong effect in training creative thinking skills.
Based on previous research, metaanalysis proved to be able to provide accurate results supported by strong statistical data in making conclusions from several studies. As in the metaanalysis conducted by Ritchie & Tucker-Drob (2018), this study succeeded in proving that education with a longer duration (additional duration) has a better effect than education with a shorter duration (such as accelerated programs); And the meta-analysis conducted by Adesope et al. (2017) succeeded in proving that claims regarding exams which can hinder learning are false. Furthermore, this study provides a detailed description of information related to the effectiveness of learning treatments which are suitable for practicing creative thinking skills.Thus, for further research, the researcher suggests using one of the research variables mentioned in this study to be applied in learning to train students' creative thinking skills.

CONCLUSION
Research has been done to determine which learning treatment is the most effective to train students' creative thinking skills. Through bibliographic analysis, it is known that the most popular learning treatments studied along with creative thinking skills are design thinking learning models, problem based learning, STEM/AM approaches, and technology-based learning media. Based on the results of the meta-analysis, it is known that the learning approach has the greatest effectiveness value, which is 1.627, followed by learning media at 1.242, and the learning model to 0.675. However, if the value of the greatest effectiveness based on research is sorted, the PBL model occupies the first place, which is 3.765. The results also show that differences in learning treatment have a significant impact on creative thinking skills. Based on the results of the average effectiveness, the learning model is categorized as having a moderate effect in training creative thinking skills. Learning media is categorized as having a very high effect on the ability to think creatively. Then the learning approach has an excellent effect when it is used to train creative thinking skills. This study also reveals that even using the same learning treatment, the results will be different if applied to different places and samples. Especially for science and mathematics teachers, we suggest to consider the using of PBL (learning model), STEM/AM (learning approach), and technologybased learning media to train creative thinking skills.