Blog

Week 9: Workshop Developing and Validating Your Scale The first thing you need to do is download your data from Qualtrics and tidy/clean the dataset so that it is ready for analysis. There are two videos in the week 9 folder to help you with these stepsDownload your dataClean your dataset and re-label the variables so that they have appropriate namesCheck that the scoring for each variable is correctFor the other scales that you have included for the purpose of construct/criterion validity testing, you will need to calculate total scores. To do this you may need to reverse score some items and then calculate the total. There is a video in the week 9 folder to help you with this.Once your dataset is formatted appropriately and you have calculated the total scores for your other scales you can begin the item analysis of your item pool. PLEASE NOTE: I am happy for you to complete stage 1 (above) as a group exercise but everything from stage 2 onwards must be completed individually Now you have the data you need to complete your item analysis and reject those items that perform poorly against the item-analytic criteria. You probably want to leave about 15-20 items for the factor analysis, but see how you go.  Many more than that and your factor analysis can become very messy; too few and you are in danger of not leaving yourself sufficient room for manoeuvre. You can always try entering different numbers of items into the factor analysis and see what happens. Once you have reduced your item pool you need to conduct an exploratory factor analysis

(or principle components analysis…I’ll leave that up to you) in order to explore which items can be combined to form a univariate scale. Use a Varimax rotation to interpret your factors and highlight items that load on factors that are independent to the main construct. You should now select your final set of items and interpret the nature of the factor/scale. What do you think it is measuring? Make sure you have reversed the scoring of items as necessary (your factor loadings should tell you if any are scored differently eg you may have positive factor loadings and one item with a negative loading). Check internal consistency and see if Cronbach’s alpha can be improved by the deletion of any further items. Now you need to compute a total score for your new scale. Use the ‘transform’ – ‘compute’ command to add together the individual scores of the remaining items. You will also need to calculate total scores for the other measures you have used if you have not already done so in step 1 (above). Once you have calculated your total score for your new scale, you can explore the correlations between your scale total and the other scales in order to explore the construct & criterion validity of your new scale. You can also want to look at the correlation between age and our new scale. Are we expecting to find an association? Look at the literature. We also want to look at sex/gender differences in scores on the new measure. Do your findings concur with what the literature says about the relative levels of your target construct in men and women?