Pre-Service Teachers' Online Teaching Readiness: Validation of Paliwal and Singh's Questionnaire

Rasch Abstract The present study aimed to investigate the psychometric characteristic of Paliwal and Singh's questionnaire used to evaluate pre-service teachers' readiness to teach online. Non-probability sampling was used to collect the required data from the participants. Using a survey design, the study distributed Paliwal and Singh's (Paliwal & Singh, 2021) online questionnaire that was already translated into Indonesian. A total of 116 preservice teachers took part in this study, but only eighty-eight responses from the participants were analyzed due to data appropriateness. Rasch analysis was carried out to examine the unidimensionality aspect, person-item reliability, item statistics, person-item measure, and finally the item-person Wright map. The findings showed that nine items of OTR developed by Paliwal and Singh questionnaire were valid to collect the data related to online teaching readiness. The Rasch analysis of Online Teaching Readiness revealed that the use of the Rasch model to analyze this scale was suitable and produced a high level of reliability both for person and item. The findings of the study also suggest some implications and recommendations for future use of the (SEM).


INTRODUCTION
Online teaching has been seen to be more complexed than the conventional teaching. To achieve the success of online teaching, teachers is not only required to have sufficient skills in operating electronic and digital devices for their online teaching, but they also must have appropriate competence in developing online teaching and learning design, and making creative and innovative online teaching materials with the use of technology. Alston, et al. (2017) suggest that teachers should have sufficient online teaching strategies to meet students' learning needs. To this end, according to Alston et al., teachers need to investigate some of the existing models in conventional learning to be adapted into the virtual learning environment. For example, teachers may envisage the use of the flipped classroom learning model that is widely implemented in Indonesia during the Covid-19 pandemic time. Davis et al. (2019) also suggest that teachers must have appropriate knowledge on how to provide feedback to their students timely, to provide sufficient information to guide the students, and more importantly, has developed sufficient approach to address students' learning difficulties.
To address the complexity of online teaching, teacher education institution must prepare their students with sufficient skills to achieve the effective online teaching. Faculty administrators must take part to create well-developed materials, assessments, interactions, and courses in dealing with online teaching (Martin et al., 2019). What important to note is that the faculty administrators could not be able to stand alone themselves in preparing certain online teaching activities, but they need consider teacher's need individually . Several studies have evidenced that good deliberation between teachers and their institutions will produce good readiness in online teaching. For examples, Hung (2015) has found that institutional support is the most influences online teaching readiness for teachers. A study by Scherer et al., (2021) also revealed that online teaching readiness could be successful depending on the institutional, cultural, and innovation supports. Findings by Howards et al., (2021) showed that sufficient engagement between both teachers and their faculty can produce a better understanding of teachers' experiences, and the support from institutions towards online teaching.
The current study was a part of a bigger study evaluating the relation between Indonesian pre-service teachers' online teaching readiness and their online teaching during teaching practicum. Particularly, the current study was aimed to examine if the Indonesian translated version of Paliwal and Singh's (2021) questionnaire was reliable to use to evaluate pre-service teachers' readiness to teach online. Two research questions were addressed as below: 1. Is the Indonesian version of Paliwal and Singh's questionnaire was reliable to use to evaluate pre-service teachers' readiness to teach online? 2. What is the psychometric analysis of the Indonesian version of Paliwal and Singh's questionnaire?

LITERATURE REVIEW
Online teaching readiness has been viewed as of key success for online teaching. A body of literature has suggested the role of faculty administrators in preparing pre-service teachers for online teaching, particularly, to the extent they are required to equip the pre-service teachers with sufficient online teaching skills and to provide institutional supports Hung, 2015;Scherer et al., 2021). These two prominent roles are believed to promote certain level of readiness for the pre-service teachers so that they could achieve the online learning success.
Several scales have been developed to measure teachers' readiness for online teaching. In particular, the Faculty Readiness to Teach Online or known as FRTO Scale were offered by Martin et al. (2019). FRTO was developed after the evaluation of the previous literature (e.g. Downing & Dyment, 2013;Gay, 2016;Lichoro, 2016 Paliwal and Singh (2021) reconceptualize Martin et al.'s (2019) FRTO scale by analyzing the most influential competencies and requirements to prepare for online teaching during COVID 19 pandemic. In their study, Paliwal and Singh used confirmatory factor analysis (CFA) and structural equation modelling (SEM). It was found from the analysis that the reliability of the instrument revealed high reliability of the OTR scale (Cronbach alpha = .91) and each of the constructs showed sufficient level of reliability (r >0.70). Moreover, the findings revealed that the proposed instrument fitted with SEM since it met all the validity issues. Unfortunately, the study was limited to the Higher Education Institution (HEI) context situated in India, and further research on teacher competencies in online teaching at the level of primary and secondary school, particularly in the context of pre-service teachers needs to carry out. By doing this, the offered framework for handling online education thus can spread evenly. The current study was aimed to examine Paliwal and Singh's (2021) online teaching readiness questionnaire using Rasch modelling. Rasch analysis was conducted through WINSTEP 4.4.1 to explore the Rasch prominent assumptions such as unidimensionality, person and item reliability, wright map, and person and item measure. According to Wright (1977) as cited in S. W. Chan et al., (2014), there are several advantages using Rasch model for measuring item scale. First of all, Rasch model can classify whether the item is match or biased. Secondly, the sample's ability is free and not influenced its item calibration. Thirdly, the Rasch calibration can be used to test the accuracy of the item. Fourthly, the item difficulties into a common scale from many samples can be estimated. Fifthly, the Rasch model can compare two people's abilities by converting them into a common scale. Sixthly, Chi-square can be used to detect the measurement quality. At last, the instrument assessment can be more easily set up and well-constructed by using Rasch model (S. W. Chan et al., 2014, p. 134). Additionally, according to Rasch model, item inquiry also can perform as an introductory examination that show the difficulty of item and bias, that is attended by distractor analysis (M. Chan & Subramaniam, 2020).
To the best of our knowledge, a study adopting Rasch analysis to validate Online Teaching Readiness questionnaire had not been found anywhere. It is thus, the present study was conducted to promote empirical evidence to the Psychometric assessment of Paliwal and Singh's (2021) online teaching readiness questionnaire using Rasch analysis. More importantly, this study will contribute to the literature about the readiness to teach online within pre-service teachers in the Indonesian context.

Participants
The participants for the current study were selected using non-probability sampling method. A total of 116 pre-service teachers who were currently in an internship at schools agreed to participate and filled out the questionnaire. However, after the misfit analysis was conducted on the data, 28 samples were shown to be outliers, so the remaining 88 samples were further analysed in the second round. The pre-service teacher participants were between 20 and 25 years old, many of them were female respondents (86%, N=76) and 12 males (14%). The participants had shown to have varying levels of technology skills.

Data collecting instrument and collecting method
The quantitative data required for the current study was collected using a survey design. A survey questionnaire adapted the one proposed by Paliwal and Singh (2021), and was distributed to the target participants. The questionnaire was developed using a 5-point Likert scale reflecting on how much the respondents agree or disagree with a specific statement: Strongly Disagree (1); Disagree (2); Neutral (3); Agree (4); Strongly Agree (5) for all items (N = 9 items). The original questionnaire revealed high reliability of the OTR scale (Cronbach alpha = .91). The questionnaire was translated into the Indonesian to help the respondents comprehend the information. The translation was done by the first author and was crosscheck by two independent translators. The second author of the paper then read and reread the translated questionnaire to ensure that the questionnaire did not contain redundant wording and ideas.
To ease the distribution, the questionnaire was made online using Google form. The link access to the questionnaire was posted in several social media platform such as Instagram, Whatsapp, and Twitter. Prior to filling out the questionnaire, participants were asked to provide with several demographic information or personal data questions (age, gender, school level, technology skills, and internship status). It is important to acknowledge that participants' consent was obtained before the data collection and all the data presented in this paper had been anonymised.

Data Analysis
The collected quantitative data were analysed using Rasch analysis with the use of WINSTEP application version 4.4.1. Several data analyses were carried, such as the evaluation of an individual item and person fit was done by evaluating the mean square (Mnsq) and through Outfit before Infit (Ling Lee et al., 2020). In the current study, the threshold of Mnsq were < 0.5 or > 2.0. In addition, as suggested by M. Chan & Subramaniam (2020) and Huang et al., (2019), the current study examine the questionnaire dimensionality, person and item reliability, scoring scale, response map, and item bias. Dimensionality assessment was used to find out the items in the instrument only measure one construct. In the Rasch analysis, the dimensional assessment is carried out by assessing Rasch Principal Component Analysis (PCA), and bias items were assessed through Item Differential function (DIF) with DIF criteria: DIF contrast > 0.5 and the probability value < 0.05.

FINDINGS AND DISCUSSION
The result of the Rasch analysis focuses on two primary objectives: First, it was carried out to examine the extent of the assessment fits with Rasch model. The analysis of item fit can be shown at infit and outfit mean square (MnSq) that demonstrated assemble validity to separate pre-service teachers with varying readiness levels in teaching online. The infit and outfit analysis of overall items were shown <2.0 which implied no mismatch, except Q3. The Rasch analysis also informed the point measure correlation, which was used to measure the correlation between respondents towards a single item. Moreover, the positive values (logits) can affect the internal coherence of the item. In this study, pt. measure values were all positive and not close to zero (0.00). It indicated that all items were internally coherent in the test and understood the whole item well.
Second, the analysis concerned with the evaluation of the questionnaire and its item difficulty measure. The Rasch analysis findings is summarized in Table 1 below, more detail will be presented in Table 2.  Table 2 showed the statistics of all items individually, containing the item measure, standard error of measurement, fit statistics, and point-measure correlation. Globally, the item difficulty levels were appropriate with the levels of pre-service teachers' ability. The statistics as in Table 2 below showed that 9 items of Online Teaching Readiness questionnaire had a good psychometric equity and fit the Rasch model.  Table 2 also summarises the personitem measure. The table presents the item difficulty level range between -.88 logits and .96 logits. The lower the logit is, the easier the item is. Otherwise, and the higher the logit is, the more difficult the item level will be. The hardest item was Q6 with .96 logits whereas the easiest item was Q2 having the value of -.88 logits. Meanwhile, the standard error of measurement mostly was observed at .17 for all items except Q2 (.18 logit). The statement of Q6 "I believe teaching online will offer me more job satisfaction) was viewed as the most difficult item to choose by the respondents. There was a total of 48 respondents (54%) who disagreed with the item. The other items considered difficult item including Q3, Q7, and Q8. On the contrary, the result viewed that pre-service teacher executed best in Q2 suggesting the item as the easiest item to respond.

Unidimensionality of the Items
The analysis of unidimentionality aspect of the questionnaire is used to find out if the items in the instrument only estimate one construct. In Rasch analysis, unidimensionality was carried out by assessing the Rasch Principal Component Analysis (PCA) for the global scale and all subscales. The criteria of PCA eigenvalue for first contrast is > 2 logits if it indicates the existence another dimension, and <2 if it supports  Paliwal and Singh's (2021) questionnaire fitted to the Rasch model. Unfortunately, the results depict a potential existence of other dimensions in the questionnaire.

Person and item Reliabilities
The reliability analysis as shown in table 1 revealed that the Online Teaching Readiness questionnaire possessed a high level of reliability (Cronbach alpha = .91). In addition, the reliability of the person on the global scale revealed a high level too, with value of .90. The findings suggest that the respondents' answers were quite good while filling out the questionnaire. Furthermore, the index of item separation was observed at 3.21 logit with the person separation index showed at 3.03 logit. These two separation values were shown to fit the Rasch separation criteria suggesting that the items on the instrument could be classified into three levels of item difficulty, and the respondents who filled the questionnaire also could be categorised into 3 levels.

Wright Map
Wright Maps analysis is used to determine the distribution of people's levels and item adversity levels on the same gauge (Boone & Noltemeyer, 2017;M. Chan & Subramaniam, 2020;Mulyono et al., 2020;Ningsih et al., 2021). The analysis of Wright Maps is presented in Figure 1, drawing the distribution of measured level of the person on the left side of the map and the distribution of item adversity levels on the right side of the map. The division of participants' responses is shown in the vertical line. The items that are the most challenging for respondents to agree with are at the highest level (upper part), while the items that are the easiest for respondents to agree are at the lowest level (lower part). As shown in Figure 1, this instrument is easy to use to measure online teaching readiness of the pre-service teachers. The Wright map analysis showed that the easiest and most difficult items range from -1.08 to 1.62 logits.

Figure 1. Wright Maps
Figure 1 also shows that the items Q6 (.96 logit), Q3 (.53 logit), and Q7 (.51 logit) respectively were classified as the the least chosen item to agree by respondents. For example, item Q6 (I believe teaching online will offer me more work pleasure) was shown to be the most difficult item to choose by the respondents. It indicated that most respondents (N=48, mostly female) felt that they did not get job satisfaction while doing online teaching, The other Q3 (I believe teaching online takes less time than teaching face to face) reflected that most respondents did not agree that online teaching takes a short time, because in reality they spent more time compared to prepare and manage online teaching. Meanwhile, item Q2 (-.88 logit), Q1 (-.54 logit), and Q9 (-.69 logit) were displayed as the easiest items or the most chosen to agree by respondents. For example, the item Q2 "I am open to learning more ways in using technology in teaching online" indicated that most respondents did not mind or willing to learn different ways of using technology to implement when teaching online. In other words, the item logit value has showed that most respondents were confident and had a willingness to learn the use of technology. In brief, the current study aimed to evaluate the reliability and psychometric characteristic of the Indonesian translated version of Paliwal and Singh's (2021) questionnaire. was reliable to use to evaluate preservice teachers' readiness to teach online. Findings from the Rasch model analysis has showed that the Indonesian translated version of Paliwal and Singh's questionnaire possessed a high level of reliability. The questionnaire was capable in differentiating the difficulty levels among the participants. For instance, item Q6 was observed to be the most difficult item to answer providing prospective teachers with actionable information to identify things they can improve to gain job satisfaction in online teaching. Several factors have been recognized to influence teacher work pleasure; such as leadership (Hui et al., 2013;Menon, 2014); teacher independence (Skaalvik & Skaalvik, 2014); and students' attitude performed (Perrachione et al., 2008). Further, teachers' self-confident is also significantly related to their work pleasure. Teachers who had high self-confident toward their skill feel can involve students with more interactive and they can be more enjoy in teaching and increase their job satisfaction (Liu et al., 2018). A study finding by Chaprara et al. (2003) also revealed that teachers of high-school students in Italia who were confident that they could complete their teaching assignments and overcome difficulties in the classroom had a high level of job satisfaction.
In addition, the findings of the current study have emphasised that pre-service teachers still disagree on several items when doing online teaching because of their unpreparedness and misconception about online teaching among pre-service teachers. The government and, particularly, the teacher education institutions therefore should develop sufficient learning curriculum that prepares the pre-service teachers for online teaching. Such an online teaching readiness may enable them to address online teaching challenges and achieve effective teaching.

CONCLUSION
The current study used the Rasch model to examine a nine-item of the Indonesian translated version of Paliwal and Singh's (2021) online teaching readiness questionnaire. Using the data from eighty-eight preservice teachers the psychometric assessment revealed that the nine-items of OTR developed by Paliwal and Singh questionnaire were valid to collect the data related to online teaching readiness. The Rasch analysis of Online Teaching Readiness revealed that the use of Rasch model to analyze this scale was suitable and produced a high level of reliability both for person and item. Findings of the study also suggest some implications and recommendations for future use of the questionnaire. The findings of the study also revealed that pre-service teachers were viewed to be open and had no hesitation to learn and improve their online teaching skills. Although, the findings also indicated pre-service pessimistic feeling for online teaching due to their negative perception about the work pleasure and online teaching duration. The current study acknowledges its limitations, particularly concerning to the number of its participants. Further study in similar field, therefore, should include more participants across Indonesia. Moreover, the factor contributing online teaching readiness should be further evaluated.