Participants

Participants were recruited via social networks of University of Sussex students on the Quantitative and Qualitative Methods (QQM) module. 675 initial responses were made, with 642 remaining after exclusions of QQM students (n = 31), under 18 year-old respondents (n = 1) and responses with missing data (n = 1).

Tables 1, 2 and 3 outline sample descriptive statistics.

Table 1

Participant descriptive statistics
N Mean Age SD Age Min Age 1st Q Median Age 3rd Q Max Age
642 29.53 15.09 18 20 21 38 91
Note. Data presented are after response exclusions. Age is measured in years.

Table 2

Participant descriptive statistics by gender
Gender n % of total Mean Age SD Age Min Age 1st Q Median 3rd Q Max Age
Female 422 65.73 29.66 15.46 18 20 21 40 91
Male 207 32.24 29.78 14.65 18 20 22 36 82
Other 12 1.87 20.92 2.94 19 19 19 22 28
Prefer not to say 1 0.16 27.00 NA 27 27 27 27 27
Note. Data presented are after response exclusions. Age is measured in years.

Table 3

Participant employment status by gender
Gender Current studenta Employed Unemployed

Part-time

employment

Searching

for employment 		Retired
Female 220 124 12 41 6 19
Male 103 88 3 5 3 5
Other 7 1 3 0 1 0
Prefer not to say 1 0 0 0 0 0
Note. Data presented are after response exclusions.
a ‘Current student’ refers to participants’ status as a university student in general, rather than specifically a student on the ‘Quantitative and Qualitative Methods’ module at the University of Sussex as these individuals were already excluded from the analysis.

Materials

The questionnaire consisted of demographic items (Tables 1, 2 and 3). Following this were 25 items assessing participants’ self-esteem on specific domains (DSSE), concluding with 1 item measuring global self-esteem (GSE) and 1 measuring openness to experience (OTE).

DSSE items aimed to capture self-evaluations on a number of domains, such as academic competence (e.g. item 17: “When I succeed on an assessment, I feel a sense of self-pride.”) and physical health (e.g. item 25 “Eating unhealthy foods makes me feel bad about myself”). Despite drawing influence from them, the present DSSE scale also aimed to expand on similar pre-existing scales (e.g. Contingencies of Self-worth Scale, Crocker, Luhtanen, Cooper & Bouvrette, 2003; Body-Self Relations Questionnaire, Brown, Cash & Mikulka, 1990) by including items related to social media (e.g. item 5: “Whenever my follower count goes down, my self-esteem suffers”), increasingly relevant to students and non-students alike in contemporary society (Midgley, Thai, Lockwood, Kovacheff & Page-Gould, 2021; Statista, 2022), potentially enhancing ecological validity of results by asking questions with more relevance to the sample (e.g. Szczygieł, 2021). DSSE items were rated on a 5-point Likert scale assessing participant agreement, where 1 = “Strongly disagree” and 5 = “Strongly agree.” To reduce acquiescent responding (Winkler, Kanouse & Ware, 1982), item phrasing was balanced such that 11 items were phrased positively (e.g. item 17), 11 negatively phrased (e.g. item 5), and 3 phrased neutrally (e.g. item 2: “My self-esteem is influenced by how well I get along with others”).

GSE and OTE items were derived from Robins, Hendin and Trzesniewski (2001) and Gosling, Rentfrow and Swann (2003), respectively. These similarly used a 5-point Likert scale, however participants instead rated to what extent they felt statements described them (1 = “Does not describe me” and 5 = “Describes me extremely well”). GSE and OTE items were positively phrased. Furthermore, whilst the Likert scale for DSSE items was balanced around a neutral midpoint (3 = “Neither agree nor disagree”), GSE and OTE scales represented a gradient of positive identification with the statement, with no neutral midpoint (3 = “Describes me moderately well”). Despite this difference, the use of midpoints in each scale also contributes to reducing acquiescent responding (Tourangeau et al., 2000, cited in Podsakoff, MacKenzie, Lee & Podsakoff, 2003).

Procedure

Participants were provided a link via social media platforms to the questionnaire on Qualtrics software (Qualtrics, Provo, UT, 2022). Pre-completion, participants were informed they were taking part in a short 10-minute study aiming to develop a questionnaire that captures subjective self-evaluations of different life-aspects. Participants consented, having their right to anonymity, withdrawal, confidentiality affirmed and were told their data would be analysed and potentially published. Post-completion, participants were provided an opportunity to withdraw and debriefed on research aims.

Results

To improve internal validity (Nichols & Maner, 2008), participants on the QQM module (n = 31) were excluded for being aware of research aims. Individuals under 18 years old constitute a ‘vulnerable population’ for whom additional briefing and approval from guardians is necessary for informed consent (British Psychological Society, 2021) - since neither were assured in this study, under-18s (n = 1)were excluded from analysis. Responses with missing data (n = 1) were also excluded.

Exclusion in all cases involved listwise deletion (LD) of responses. Whilst this can reduce statistical power in analysis (Myers, 2011), it was deemed appropriate as the number of excluded cases (n = 33) was relatively small compared to the remaining sample size (n = 642), making loss of statistical power minimal whilst also improving validity of results (Kang, 2013). Alternative methods such as multiple imputation could have been used with missing data (Rubin, 2004). This was deemed unnecessary as methods like multiple imputation do not impact statistical power significantly differently from LD when dealing with item-level missing data (Parent, 2013), as was the case in this sample where the excluded case was missing data on just one item (QQM).

Inter-item correlations within a scale measuring a construct should at least exceed an “exemplary” value of |.3| (Robinson, Shaver & Wrightsman, 1991, p. 13). Furthermore, inter-item correlations above |.85| indicate that the two items do not measure substantially different aspects of the construct and thus largely equivalent (Paulsen & BrckaLorenz, 2017). Therefore, all items that either: have no inter-item correlations above |.3| or any inter-item correlations above |.85| were excluded from the analysis. These criteria excluded items 3, 14 and 23.

Bartlett’s test of sphericity was significant, (\(x^{2}\)(231) = 5052.349 p < .05) demonstrating that the correlation matrix of this data is significantly different from an identity matrix. Furthermore, a Kaiser-Meyer-Olkin test was used to produce a measure of sampling adequacy (MSA). Overall MSA was .84, with a range of .62-.92, indicating an “meritorious” overall MSA, and ranging from “mediocre” to “marvellous” (Kaiser & Rice, 1974, p. 112). A minimum threshold of .5 for KMO values (Guttman, 1953, cited in Kaiser & Rice, 1974) was exceeded in all cases, suggesting all included variables are suitable for Exploratory factor analysis (EFA).

EFA requires predetermination of factor size (Fabrigar, Wegener, MacCallum & Strahan, 1999). To achieve this, a scree plot (Cattell, 1966) can be used. However, determining factor size from a scree plot alone is inherently subjective (Courtney & Gordon, 2013). Pairing it (Figure 1) with a parallel analysis (PA) (Horn, 1965) reduces subjectivity. PA compares eigenvalues of each factor - “the amount of variance of the variables accounted for by that factor” (Norris & Lecavalier, 2010, p. 9) - with random eigenvalues. Identifying factor eigenvalues greater than 1 (Kaiser, 1960) could be the criteria used to determine factor extraction, however this can (over)underestimate the number of factors (Zwick & Velicer, 1986). In PA, factor and random eigenvalues are ordered and compared. The number of factor eigenvalues greater than random eigenvalues determines the suggested factor size. As seen in Figure 1, 7 factors were extracted.

EFA used a minimum residuals (minres) method (Comrey, 1962), as with PA. Factor rotation method used was oblique. Oblique rotations do not force factors to correlate as orthogonal rotations would, making the former better suited for dealing with interrelated behaviours, like DSSE self-assessments, which are already expected to correlate (Osborne, 2015).

EFA can be iterative, involving reanalysis after exclusions of items and reduction of factor size. Furthermore, factor clustering must be thematically coherent. Such reanalysis and thematic considerations were necessary. Exclusion criteria are:

  1. Significance threshold. Whilst there is no decisive view of what the minimum threshold for factor loading should be (Peterson, 2000), social science favours a |.3| threshold for sample sizes greater than 350 (Hair, 2019), where coefficients near |.5| are considered “practically significant.” This EFA uses this |.3| threshold to determine significance. For items which do not load significantly on any factor, it is suggested to remove items individually, assessing the factor structure after each until a stable factor structure is found (where all items have at least 1 significant factor loading) (Samuels, 2017). This excluded items 7, 15, 16 and 23.

  2. Insignificant factors. Factors with less than 3 significant item loadings are likely weak (Samuels, 2017). As such, it is recommended (Watkins, 2018) to sequentially reduce factor size until a stable factor structure is found (all factors have at least 3 significant factor loadings). This reduced the factor size from 7 to 5.

  3. Thematic consistency Whilst the emergent factor structure may be sound, items can cluster into a factor without actually fitting the theme of that factor. Furthermore, items may significantly cross-load onto multiple items. Determining thematic consistency within factors is inherently subjective, however as the goal with this analysis is to discover coherent DSSE scale factors, some adjustments were made. Items 10, 13 and 25 cross-loaded and thus their primary loading was used to assign them to a factor. Item 11 also cross-loaded but coefficients were very similar (.37 & .39) nor practically significant, thus it was excluded entirely.

Figure 2

Factor loadings after reanalysis exclusions
Variable MR2 MR1 MR3 MR5 MR4 Complexity Uniqueness
DSSE.19 0.81 – – – – 1.03 0.28
DSSE.21 0.78 – – – – 1.13 0.38
DSSE.20 0.74 – – – – 1.01 0.45
DSSE.17 0.54 – – – – 1.86 0.53
DSSE.9 – 0.70 – – – 1.09 0.52
DSSE.12 – 0.67 – – – 1.05 0.49
DSSE.13 – 0.52 – 0.33 – 1.82 0.47
DSSE.10 – 0.49 – 0.33 – 2.02 0.42
DSSE.18 – 0.43 – – – 1.63 0.65
DSSE.8 – – 0.83 – – 1.01 0.29
DSSE.4 – – 0.76 – – 1.05 0.36
DSSE.5 – – 0.65 – – 1.34 0.54
DSSE.6 – – – 0.73 – 1.04 0.38
DSSE.1 – – – 0.70 – 1.03 0.47
DSSE.2 – – – 0.59 – 1.13 0.62
DSSE.24 – – – – 0.71 1.02 0.50
DSSE.22 – – – – 0.67 1.01 0.53
DSSE.25 – 0.35 – – 0.48 1.94 0.69

EFA returned 5 factors accounting for 52% of overall variance. Oblique rotation was appropriate given all factor correlations were non-zero.

Factors were as follows: MR1 = Appearance self esteem - including 5 items (e.g. item 9 “I try to avoid seeing my body in the mirror”); MR2 = Academic self-esteem - including 4 items (e.g. Item 17 “When I succeed on an assessment, I feel a sense of self-pride.”); MR3 = Social media self-esteem - including 3 items (e.g. Item 4 “My sense of self-worth increases when I get positive engagement on my social media”); MR4 = Health-based self-esteem - including 3 items (e.g. Item 22 “My self-esteem goes up when I am healthy”); MR5 = Social acceptance self-esteem - including 3 items (e.g. Item 1 “Knowing someone liked me would make me feel good about myself”).

Hu and Bentler (1999) suggest model fit cut-off values of either a TLI > .96 & SRMR < .06 or RMSEA < .05 and SRMR < .09, these cut-offs are not absolute. Likelihood Chi square was highly significant \(x^{2}\) = 209.07, p < .001, likely due to the large sample size (n = 642). Tucker Lewis index was .929, SRMR was .02, and RMSEA was .054, 90% CI [.045, .063]. Evidently, this indicates poor model fit. Such a model likely has enhanced type I and II error rates (Hu & Bentler, 1999).

Reliability analysis using McDonald’s \(\omega\) (McDonald, 1999) reported three items (9, 22 & 24) with low general factor loadings, indicating a common factor model is inappropriate (Field, 2022). An \(\omega\) hierarchical of 0.58 was found, where items explained 35% of the variance in the general common factor, considered to be fairly low. An overall \(\omega\) total of 0.9 was observed, indicating the overall scale is highly reliable. Model fit indexes (\(x^{2}\) = 2130.86, p < 0; RMSEA = .152, 10% CI [.146-.158]) indicated that the 5-factor model is a better fit than a single-factor model. \(\omega\) totals for each sub-scale were: MR1 = .84, MR2 = .79, MR3 = .74, MR4 = .8, MR5 = .67. All sub-scales are reliable, with MR5 being the least reliable.

Convergent validity was tested by computing composite scores for each factor and correlating them with the GSE item scores. Divergent validity was similarly tested, but instead used OTE scores.

Figure 4

Correlation matrix for factor composites
Parameter OPEN SISE MR5_comp MR4_comp MR3_comp MR2_comp
MR1_comp -0.55 -0.55 0.42 0.18 0.34 0.35
MR2_comp -0.16 -0.16 0.36 0.15 0.20 –
MR3_comp -0.15 -0.15 0.36 0.10 – –
MR4_comp 0.03 0.03 0.17 – – –
MR5_comp -0.21 -0.21 – – – –
SISE 1.00 – – – – –

SISE and OPEN both negatively correlate with MR1, MR2, MR3 and MR5, whilst not correlating with MR4. This indicates poor convergent validity for this scale as factors composites are expected to correlate positively with GSE as it is a related construct. Poor discriminant validity was also observed for all but one factor (MR4), where factor composites are expected to not correlate with OTE as it is an unrelated construct.

Discussion

Overall, inter-item correlations revealed 3 items (3, 14 & 23) did not significantly correlate with any others, whilst all others were satisfactory. EFA revealed 5 coherent factors, accounting for 52% of overall variance. However, model fit indexes were slightly below recommended thresholds, suggesting type I and II error rates are enhanced in this model. Despite reliable omega values for the scale and sub-scales, some items had no significant general factor loadings and thus a common factor model is inappropriate here. Convergent and divergent validity was poor for all factors, except factor MR4 which had good discriminant validity.

These issues with the analysis suggest the items included and the scale itself poorly reflect the target construct. Future research could benefit from exploring alternative DSSE items in conjunction with highly performing items from this analysis (e.g. item 19) in order to produce a scale which more accurately reflects underlying DSSE constructs, providing greater practical utility than this scale.

References

British Psychological Society. (2021). Code of ethics and conduct. https://www.bps.org.uk/news-and-policy/bps-code-ethics-and-conduct
Brown, T. A., Cash, T. F., & Mikulka, P. J. (1990). Attitudinal body-image assessment: Factor analysis of the body-self relations questionnaire. Journal of Personality Assessment, 55(1), 135–144. https://doi.org/10.1080/00223891.1990.9674053
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276. https://doi.org/10.1207/s15327906mbr0102_10
Comrey, A. L. (1962). The minimum residual method of factor analysis. Psychological Reports, 11(1), 15–18. https://doi.org/10.2466/pr0.1962.11.1.15
Courtney, M., & Gordon, R. (2013). Determining the number of factors to retain in EFA: Using the SPSS r-menu v2 0 to make more judicious estimations. Practical Assessment, Research & Evaluation, 18, 8. https://www.proquest.com/docview/2366799655/abstract/B50A6AA27AFC4CC0PQ/1
Crocker, J., Luhtanen, R. K., Cooper, M. L., & Bouvrette, A. (2003). Contingencies of self-worth in college students: Theory and measurement. Journal of Personality and Social Psychology, 85(5), 894–908. https://doi.org/10.1037/0022-3514.85.5.894
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
Field, A. P. (2022, March 10). Discovering statistics using r and RStudio. SAGE publications inc. https://us.sagepub.com/en-us/nam/discovering-statistics-using-r-and-rstudio/book261351
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37(6), 504–528. https://doi.org/10.1016/S0092-6566(03)00046-1
Hair, J. F. (2019). Multivariate data analysis. (Eighth edition / Joseph F. Hair, Jr., William C. Black, Barry J. Babin, Rolph E. Anderson.). Cengage.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151. https://doi.org/10.1177/001316446002000116
Kaiser, H. F., & Rice, J. (1974). Little jiffy, mark iv. Educational and Psychological Measurement, 34(1), 111–117. https://doi.org/10.1177/001316447403400115
Kang, H. (2013). The prevention and handling of the missing data. Korean Journal of Anesthesiology, 64(5), 402–406. https://doi.org/10.4097/kjae.2013.64.5.402
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates Publishers.
Midgley, C., Thai, S., Lockwood, P., Kovacheff, C., & Page-Gould, E. (2021). When every day is a high school reunion: Social media comparisons and self-esteem. Journal of Personality and Social Psychology, 121(2), 285–307. https://doi.org/10.1037/pspi0000336
Myers, T. A. (2011). Goodbye, listwise deletion: Presenting hot deck imputation as an easy and effective tool for handling missing data. Communication Methods and Measures, 5(4), 297–310. https://doi.org/10.1080/19312458.2011.624490
Nichols, A. L., & Maner, J. K. (2008). The good-subject effect: Investigating participant demand characteristics. The Journal of General Psychology, 135(2), 151–166. https://doi.org/10.3200/GENP.135.2.151-166
Norris, M., & Lecavalier, L. (2010). Evaluating the use of exploratory factor analysis in developmental disability psychological research. Journal of Autism and Developmental Disorders, 40(1), 8–20. https://doi.org/10.1007/s10803-009-0816-2
Osborne, J. (2015). What is rotating in exploratory factor analysis? Assessment, 20, 1–8.
Parent, M. C. (2013). Handling item-level missing data: Simpler is just as good. The Counseling Psychologist, 41(4), 568–600. https://doi.org/10.1177/0011000012445176
Paulsen, J., & BrckaLorenz, A. (2017). Internal consistency [Technical Report]. Faculty Survey of Student Engagement. https://scholarworks.iu.edu/dspace/handle/2022/24498
Peterson, R. A. (2000). A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis. Marketing Letters, 11(3), 261–275. https://doi.org/10.1023/A:1008191211004
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
Qualtrics, Provo, UT. (2022, March). https://www.qualtrics.com/uk/
Robins, R. W., Hendin, H. M., & Trzesniewski, K. H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the rosenberg self-esteem scale. Personality and Social Psychology Bulletin, 27(2), 151–161. https://doi.org/10.1177/0146167201272002
Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Measures of personality and social psychological attitudes (Vol. 1). Academic Press.
Rubin, D. B. (2004). Multiple imputation for nonresponse in surveys. John Wiley & Sons.
Samuels, P. (2017, June 9). Advice on exploratory factor analysis [Monograph]. https://www.researchgate.net/publication/319165677_Advice_on_Exploratory_Factor_Analysis
Statista. (2022). Social media - statistics & facts. Statista. https://www.statista.com/topics/1164/social-networks/
Szczygieł, M. (2021). Not only reliability! The importance of the ecological validity of the math anxiety questionnaire for adults. European Journal of Psychological Assessment, 38(2), 78–90. https://doi.org/10.1027/1015-5759/a000646
Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807
Winkler, J. D., Kanouse, D. E., & Ware, J. E. (1982). Controlling for acquiescence response set in scale development. Journal of Applied Psychology, 67(5), 555–561. https://doi.org/10.1037/0021-9010.67.5.555
Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99(3), 432–442. https://doi.org/10.1037/0033-2909.99.3.432

Appendix

############### YAML header ###############

# title: 
# author:
# date: 
# output: 
#   html_document:
#     css: styles.css
# bibliography: QQM.bib
# csl: apa.csl

############### Set-up ###############
  
# used {r setup, include=FALSE} as chunk header

# knitr::opts_chunk$set(echo = TRUE)
# library(tidyverse)
# library(kableExtra)
# library(psych)
# library(GPArotation)

############### load data ###############

# used {r load data, include = FALSE} as chunk header 

# cand_no_alt <- 218340 # enter your candidate number
# source("https://raw.githubusercontent.com/vlad-costin/qqm/main/rename_n_sample_alt.R")
# report_data <- data_alt

############### data inspection/cleaning ###############

# used {r data inspection/cleaning, include = FALSE} as chunk header

# check data classes of variables in dataset (all should be numeric)
sapply(report_data, class)
##        AGE     GENDER OCCUPATION        QQM     DSSE.1     DSSE.2     DSSE.3 
##  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric" 
##     DSSE.4     DSSE.5     DSSE.6     DSSE.7     DSSE.8     DSSE.9    DSSE.10 
##  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric" 
##    DSSE.11    DSSE.12    DSSE.13    DSSE.14    DSSE.15    DSSE.16    DSSE.17 
##  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric" 
##    DSSE.18    DSSE.19    DSSE.20    DSSE.21    DSSE.22    DSSE.23    DSSE.24 
##  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric"  "numeric" 
##    DSSE.25       SISE       OPEN 
##  "numeric"  "numeric"  "numeric"
##### all variables are numeric

# initial n
nrow(report_data)
## [1] 642
##### initial n = 675

# exclude NAs    
report_data <- na.omit(report_data) 
##### excluded 1 response

# exclude inappropriate age responses
report_data <- report_data %>% 
  dplyr::filter(
    ., 
    AGE >= 18 & AGE <= 120
  ) 
##### excluded 1 response

# exclude QQM students
report_data <- report_data %>% 
  dplyr::filter(
    ., 
    QQM == 2
  )
##### excluded 31 responses

# total n excluded after data cleaning
675 - nrow(report_data)
## [1] 33
##### total n excluded = 33

############### Pre-analysis ###############

# used {r pre-analysis, include = FALSE} as chunk header

# Polychoric correlation
report_poly_before <- report_data %>% 
  dplyr::select(., DSSE.1:DSSE.25) %>% 
  psych::polychoric(.,)

# DSSE correlation matrix
psych::cor.plot(
  report_poly_before$rho,
  cex = .75,
  upper = FALSE
)

##### DSSE 3, 14 & 23 should be removed
report_poly_after <- report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.14, DSSE.23, SISE, OPEN)) %>% 
  psych::polychoric(.,)

# correlation matrix after exclusions
psych::cor.plot(
  report_poly_after$rho,
  cex = .75,
  upper = FALSE
)

##### all correlations are fine

# Bartlett’s test of sphericity 
psych::cortest.bartlett(
  report_poly_after$rho, 
  n = nrow(report_data)
)
## $chisq
## [1] 5052.349
## 
## $p.value
## [1] 0
## 
## $df
## [1] 231
##### highly significant: x^2(231) = 5052.349 p < .05

# Kaiser-Meyer-Olkin test
psych::KMO(report_poly_after$rho)
## Kaiser-Meyer-Olkin factor adequacy
## Call: psych::KMO(r = report_poly_after$rho)
## Overall MSA =  0.84
## MSA for each item = 
##  DSSE.1  DSSE.2  DSSE.4  DSSE.5  DSSE.6  DSSE.7  DSSE.8  DSSE.9 DSSE.10 DSSE.11 
##    0.88    0.88    0.81    0.80    0.89    0.83    0.81    0.80    0.92    0.91 
## DSSE.12 DSSE.13 DSSE.15 DSSE.16 DSSE.17 DSSE.18 DSSE.19 DSSE.20 DSSE.21 DSSE.22 
##    0.86    0.90    0.62    0.65    0.87    0.91    0.84    0.86    0.83    0.72 
## DSSE.24 DSSE.25 
##    0.65    0.76
##### Overall MSA = 0.84
##### MSA range = .62-.92

############### Parallel analysis ###############

# used {r parallel analysis, include = FALSE} as chunk header

# parallel analysis scree plot
report_para <- psych::fa.parallel(
  report_poly_after$rho, 
  n.obs = nrow(report_data), 
  fa = "fa"
)

## Parallel analysis suggests that the number of factors =  7  and the number of components =  NA
##### parallel analysis suggests the number of factors is: 7

############### Factor analysis ###############

# used {r factor analysis, include = FALSE} as chunk header

# initial factor analysis
report_fa <- psych::fa(
  report_poly_after$rho, 
  n.obs = nrow(report_data), 
  nfactor = report_para$nfact, 
  scores = "tenBerge"
)

parameters::model_parameters(
  report_fa,
  sort = TRUE,
  threshold = .3
)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR2  | MR1  | MR3  | MR6  | MR4  | MR5  | MR7  | Complexity | Uniqueness
## -----------------------------------------------------------------------------------
## DSSE.21  | 0.84 |      |      |      |      |      |      |    1.11    |    0.29   
## DSSE.19  | 0.78 |      |      |      |      |      |      |    1.08    |    0.30   
## DSSE.20  | 0.70 |      |      |      |      |      |      |    1.05    |    0.47   
## DSSE.17  | 0.49 |      |      |      |      |      |      |    2.39    |    0.47   
## DSSE.12  |      | 0.75 |      |      |      |      |      |    1.04    |    0.45   
## DSSE.9   |      | 0.66 |      |      |      |      |      |    1.30    |    0.54   
## DSSE.13  |      | 0.62 |      |      |      |      |      |    1.42    |    0.40   
## DSSE.10  |      | 0.50 |      |      |      |      |      |    1.87    |    0.42   
## DSSE.11  |      | 0.43 |      | 0.33 |      |      |      |    1.99    |    0.54   
## DSSE.18  |      | 0.40 |      |      |      |      |      |    1.71    |    0.67   
## DSSE.8   |      |      | 0.83 |      |      |      |      |    1.03    |    0.29   
## DSSE.4   |      |      | 0.78 |      |      |      |      |    1.12    |    0.32   
## DSSE.5   |      |      | 0.64 |      |      |      |      |    1.43    |    0.50   
## DSSE.1   |      |      |      | 0.67 |      |      |      |    1.05    |    0.47   
## DSSE.6   |      |      |      | 0.65 |      |      |      |    1.11    |    0.42   
## DSSE.2   |      |      |      | 0.61 |      |      |      |    1.28    |    0.53   
## DSSE.24  |      |      |      |      | 0.70 |      |      |    1.06    |    0.50   
## DSSE.22  |      |      |      |      | 0.66 |      |      |    1.06    |    0.52   
## DSSE.25  |      |      |      |      | 0.52 |      |      |    1.92    |    0.62   
## DSSE.16  |      |      |      |      |      | 0.93 |      |    1.00    |    0.13   
## DSSE.15  |      |      |      |      |      | 0.46 |      |    1.51    |    0.74   
## DSSE.7   |      |      |      |      |      |      | 0.51 |    1.80    |    0.59   
## 
## The 7 latent factors (oblimin rotation) accounted for 53.70% of the total variance of the original data (MR2 = 10.55%, MR1 = 10.75%, MR3 = 8.99%, MR6 = 9.04%, MR4 = 5.90%, MR5 = 5.32%, MR7 = 3.14%).
##### 2 factors do not load significantly
##### Reduce factor size

# first reduce factor size to 6 per recommendation
psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.14, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 6, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR2  | MR6  | MR1  | MR3  | MR4  | MR5  | Complexity | Uniqueness
## ----------------------------------------------------------------------------
## DSSE.19  | 0.82 |      |      |      |      |      |    1.03    |    0.28   
## DSSE.21  | 0.76 |      |      |      |      |      |    1.16    |    0.40   
## DSSE.20  | 0.74 |      |      |      |      |      |    1.01    |    0.45   
## DSSE.17  | 0.56 |      |      |      |      |      |    1.86    |    0.50   
## DSSE.9   |      | 0.69 |      |      |      |      |    1.13    |    0.53   
## DSSE.12  |      | 0.63 |      |      |      |      |    1.09    |    0.51   
## DSSE.13  |      | 0.52 | 0.35 |      |      |      |    1.90    |    0.43   
## DSSE.10  |      | 0.46 | 0.33 |      |      |      |    2.27    |    0.42   
## DSSE.18  |      | 0.41 |      |      |      |      |    1.73    |    0.67   
## DSSE.7   |      |      |      |      |      |      |    3.76    |    0.78   
## DSSE.1   |      |      | 0.69 |      |      |      |    1.06    |    0.47   
## DSSE.6   |      |      | 0.68 |      |      |      |    1.08    |    0.41   
## DSSE.2   |      |      | 0.62 |      |      |      |    1.22    |    0.55   
## DSSE.11  |      | 0.35 | 0.39 |      |      |      |    2.17    |    0.55   
## DSSE.8   |      |      |      | 0.85 |      |      |    1.01    |    0.27   
## DSSE.4   |      |      |      | 0.75 |      |      |    1.06    |    0.37   
## DSSE.5   |      |      |      | 0.65 |      |      |    1.42    |    0.52   
## DSSE.24  |      |      |      |      | 0.74 |      |    1.01    |    0.47   
## DSSE.22  |      |      |      |      | 0.65 |      |    1.03    |    0.54   
## DSSE.25  |      | 0.38 |      |      | 0.45 |      |    2.09    |    0.69   
## DSSE.16  |      |      |      |      |      | 1.00 |    1.00    |  3.44e-03 
## DSSE.15  |      |      |      |      |      | 0.39 |    1.63    |    0.79   
## 
## The 6 latent factors (oblimin rotation) accounted for 51.72% of the total variance of the original data (MR2 = 10.78%, MR6 = 10.28%, MR1 = 10.01%, MR3 = 9.10%, MR4 = 5.87%, MR5 = 5.69%).
##### 1 factor remains with less than 3 loadings
##### item 7 is insignificant, remove this first 

# exclude item 7
psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.14, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 6, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR3  | MR4  | MR6  | MR1  | MR2  | MR5  | Complexity | Uniqueness
## ----------------------------------------------------------------------------
## DSSE.19  | 0.82 |      |      |      |      |      |    1.03    |    0.28   
## DSSE.21  | 0.77 |      |      |      |      |      |    1.14    |    0.39   
## DSSE.20  | 0.74 |      |      |      |      |      |    1.01    |    0.46   
## DSSE.17  | 0.55 |      |      |      |      |      |    1.86    |    0.52   
## DSSE.9   |      | 0.70 |      |      |      |      |    1.13    |    0.52   
## DSSE.12  |      | 0.65 |      |      |      |      |    1.08    |    0.50   
## DSSE.13  |      | 0.53 |      | 0.36 |      |      |    1.87    |    0.42   
## DSSE.10  |      | 0.47 |      | 0.33 |      |      |    2.22    |    0.42   
## DSSE.18  |      | 0.41 |      |      |      |      |    1.70    |    0.67   
## DSSE.8   |      |      | 0.85 |      |      |      |    1.01    |    0.27   
## DSSE.4   |      |      | 0.76 |      |      |      |    1.05    |    0.36   
## DSSE.5   |      |      | 0.63 |      |      |      |    1.41    |    0.55   
## DSSE.1   |      |      |      | 0.68 |      |      |    1.06    |    0.48   
## DSSE.6   |      |      |      | 0.68 |      |      |    1.09    |    0.41   
## DSSE.2   |      |      |      | 0.60 |      |      |    1.23    |    0.57   
## DSSE.11  |      | 0.37 |      | 0.40 |      |      |    2.13    |    0.54   
## DSSE.24  |      |      |      |      | 0.73 |      |    1.02    |    0.49   
## DSSE.22  |      |      |      |      | 0.65 |      |    1.03    |    0.54   
## DSSE.25  |      | 0.35 |      |      | 0.47 |      |    1.99    |    0.69   
## DSSE.16  |      |      |      |      |      | 1.00 |    1.00    |  3.68e-03 
## DSSE.15  |      |      |      |      |      | 0.40 |    1.56    |    0.78   
## 
## The 6 latent factors (oblimin rotation) accounted for 53.12% of the total variance of the original data (MR3 = 11.29%, MR4 = 10.50%, MR6 = 9.33%, MR1 = 10.00%, MR2 = 6.10%, MR5 = 5.91%).
##### 1 factor remains with less than 3 loadings

# reduce factor size to 5
psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.14, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 5, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR2  | MR1  | MR5  | MR3  | MR4  | Complexity | Uniqueness
## ---------------------------------------------------------------------
## DSSE.19  | 0.82 |      |      |      |      |    1.03    |    0.28   
## DSSE.21  | 0.77 |      |      |      |      |    1.13    |    0.39   
## DSSE.20  | 0.74 |      |      |      |      |    1.01    |    0.46   
## DSSE.17  | 0.54 |      |      |      |      |    1.96    |    0.51   
## DSSE.12  |      | 0.66 |      |      |      |    1.06    |    0.49   
## DSSE.9   |      | 0.66 |      |      |      |    1.12    |    0.57   
## DSSE.13  |      | 0.53 | 0.35 |      |      |    1.83    |    0.42   
## DSSE.10  |      | 0.49 | 0.31 |      |      |    2.02    |    0.42   
## DSSE.18  |      | 0.41 |      |      |      |    1.66    |    0.67   
## DSSE.1   |      |      | 0.69 |      |      |    1.05    |    0.48   
## DSSE.6   |      |      | 0.68 |      |      |    1.07    |    0.42   
## DSSE.2   |      |      | 0.63 |      |      |    1.11    |    0.57   
## DSSE.11  |      | 0.38 | 0.39 |      |      |    2.13    |    0.54   
## DSSE.8   |      |      |      | 0.82 |      |    1.01    |    0.31   
## DSSE.4   |      |      |      | 0.76 |      |    1.05    |    0.36   
## DSSE.5   |      |      |      | 0.65 |      |    1.36    |    0.54   
## DSSE.15  |      |      |      |      |      |    3.12    |    0.87   
## DSSE.22  |      |      |      |      | 0.68 |    1.01    |    0.52   
## DSSE.24  |      |      |      |      | 0.65 |    1.03    |    0.57   
## DSSE.25  |      | 0.35 |      |      | 0.48 |    1.94    |    0.68   
## DSSE.16  |      |      |      |      |      |    2.59    |    0.84   
## 
## The 5 latent factors (oblimin rotation) accounted for 48.01% of the total variance of the original data (MR2 = 11.25%, MR1 = 10.60%, MR5 = 10.18%, MR3 = 9.43%, MR4 = 6.55%).
##### DSSE 15 and 16 now have no significant loadings

# Exclude DSSE 15
psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.14, DSSE.15, DSSE.23, SISE, OPEN)),
  n.obs = nrow(report_data), 
  nfactor = 5, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR2  | MR1  | MR5  | MR3  | MR4  | Complexity | Uniqueness
## ---------------------------------------------------------------------
## DSSE.19  | 0.82 |      |      |      |      |    1.03    |    0.28   
## DSSE.21  | 0.77 |      |      |      |      |    1.13    |    0.39   
## DSSE.20  | 0.74 |      |      |      |      |    1.01    |    0.46   
## DSSE.17  | 0.54 |      |      |      |      |    1.89    |    0.52   
## DSSE.9   |      | 0.68 |      |      |      |    1.10    |    0.54   
## DSSE.12  |      | 0.67 |      |      |      |    1.05    |    0.49   
## DSSE.13  |      | 0.53 | 0.35 |      |      |    1.83    |    0.42   
## DSSE.10  |      | 0.48 | 0.31 |      |      |    2.04    |    0.42   
## DSSE.18  |      | 0.42 |      |      |      |    1.64    |    0.67   
## DSSE.1   |      |      | 0.69 |      |      |    1.06    |    0.48   
## DSSE.6   |      |      | 0.68 |      |      |    1.07    |    0.42   
## DSSE.2   |      |      | 0.63 |      |      |    1.10    |    0.57   
## DSSE.11  |      | 0.37 | 0.39 |      |      |    2.13    |    0.54   
## DSSE.8   |      |      |      | 0.83 |      |    1.01    |    0.30   
## DSSE.4   |      |      |      | 0.77 |      |    1.06    |    0.35   
## DSSE.5   |      |      |      | 0.66 |      |    1.34    |    0.54   
## DSSE.22  |      |      |      |      | 0.69 |    1.01    |    0.51   
## DSSE.24  |      |      |      |      | 0.68 |    1.03    |    0.54   
## DSSE.25  |      | 0.35 |      |      | 0.48 |    1.92    |    0.68   
## DSSE.16  |      |      |      |      |      |    2.40    |    0.89   
## 
## The 5 latent factors (oblimin rotation) accounted for 49.95% of the total variance of the original data (MR2 = 11.79%, MR1 = 11.09%, MR5 = 10.69%, MR3 = 9.68%, MR4 = 6.71%).
##### DSSE 16 still has no significant loadings

# Exclude DSSE 16
psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.14, DSSE.15, DSSE.16, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 5, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
## # Rotated loadings from Factor Analysis (oblimin-rotation)
## 
## Variable | MR2  | MR1  | MR5  | MR3  | MR4  | Complexity | Uniqueness
## ---------------------------------------------------------------------
## DSSE.19  | 0.82 |      |      |      |      |    1.03    |    0.28   
## DSSE.21  | 0.78 |      |      |      |      |    1.13    |    0.38   
## DSSE.20  | 0.74 |      |      |      |      |    1.01    |    0.45   
## DSSE.17  | 0.54 |      |      |      |      |    1.88    |    0.53   
## DSSE.9   |      | 0.69 |      |      |      |    1.11    |    0.53   
## DSSE.12  |      | 0.66 |      |      |      |    1.05    |    0.50   
## DSSE.13  |      | 0.54 | 0.36 |      |      |    1.84    |    0.42   
## DSSE.10  |      | 0.48 | 0.32 |      |      |    2.07    |    0.42   
## DSSE.18  |      | 0.41 |      |      |      |    1.65    |    0.67   
## DSSE.1   |      |      | 0.69 |      |      |    1.05    |    0.48   
## DSSE.6   |      |      | 0.69 |      |      |    1.07    |    0.41   
## DSSE.2   |      |      | 0.61 |      |      |    1.10    |    0.60   
## DSSE.11  |      | 0.37 | 0.39 |      |      |    2.13    |    0.54   
## DSSE.8   |      |      |      | 0.84 |      |    1.01    |    0.28   
## DSSE.4   |      |      |      | 0.76 |      |    1.06    |    0.36   
## DSSE.5   |      |      |      | 0.65 |      |    1.35    |    0.54   
## DSSE.24  |      |      |      |      | 0.71 |    1.02    |    0.50   
## DSSE.22  |      |      |      |      | 0.67 |    1.01    |    0.53   
## DSSE.25  |      | 0.35 |      |      | 0.47 |    1.95    |    0.69   
## 
## The 5 latent factors (oblimin rotation) accounted for 52.06% of the total variance of the original data (MR2 = 12.40%, MR1 = 11.66%, MR5 = 11.11%, MR3 = 10.12%, MR4 = 6.77%).
##### All factors have at least 3 significant item loadings. 
##### 4 Items cross-load. Items will be assigned according to their primary loadings and subjective evaluation of thematic coherence.
##### Item 11 has cross-loadings which are very similar and below practical significance, this item will be removed.

##### Exclude item 11
report_fa_final <- psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.11, DSSE.14, DSSE.15, DSSE.16, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 5, 
  scores = "tenBerge",
  cor = "poly"
) %>% 
  parameters::model_parameters(
    .,
    sort = TRUE,
    threshold = .3)
##### All factors have at least 3 significant item loadings. 
##### 3 items cross-load. Items will be assigned according to their primary loadings and subjective evaluation of thematic coherence.
##### check model fit indexes

psych::fa(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.11, DSSE.14, DSSE.15, DSSE.16, DSSE.23, SISE, OPEN)), 
  n.obs = nrow(report_data), 
  nfactor = 5, 
  scores = "tenBerge",
  cor = "poly"
)
## Factor Analysis using method =  minres
## Call: psych::fa(r = report_data %>% dplyr::select(., -c(AGE, GENDER, 
##     OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.11, DSSE.14, DSSE.15, 
##     DSSE.16, DSSE.23, SISE, OPEN)), nfactors = 5, n.obs = nrow(report_data), 
##     scores = "tenBerge", cor = "poly")
## Standardized loadings (pattern matrix) based upon correlation matrix
##           MR2   MR1   MR3   MR5   MR4   h2   u2 com
## DSSE.1   0.08 -0.03  0.00  0.70 -0.03 0.53 0.47 1.0
## DSSE.2  -0.06  0.13  0.02  0.59  0.03 0.38 0.62 1.1
## DSSE.4   0.00 -0.06  0.76  0.09  0.07 0.64 0.36 1.1
## DSSE.5  -0.09  0.20  0.65 -0.12 -0.09 0.46 0.54 1.3
## DSSE.6   0.01  0.01  0.10  0.73  0.01 0.62 0.38 1.0
## DSSE.8   0.05 -0.04  0.83  0.02 -0.01 0.71 0.29 1.0
## DSSE.9   0.05  0.70  0.03 -0.13 -0.01 0.48 0.52 1.1
## DSSE.10  0.12  0.49  0.10  0.33  0.03 0.58 0.42 2.0
## DSSE.12  0.06  0.67  0.03  0.07 -0.05 0.51 0.49 1.0
## DSSE.13  0.05  0.52  0.04  0.33  0.10 0.53 0.47 1.8
## DSSE.17  0.54 -0.23  0.12  0.09  0.21 0.47 0.53 1.9
## DSSE.18  0.14  0.43  0.03  0.19 -0.03 0.35 0.65 1.6
## DSSE.19  0.81  0.02 -0.06  0.08  0.00 0.72 0.28 1.0
## DSSE.20  0.74 -0.03  0.02  0.01 -0.03 0.55 0.45 1.0
## DSSE.21  0.78  0.15  0.03 -0.12 -0.01 0.62 0.38 1.1
## DSSE.22  0.03 -0.01  0.05  0.00  0.67 0.47 0.53 1.0
## DSSE.24  0.01 -0.05 -0.05 -0.01  0.71 0.50 0.50 1.0
## DSSE.25 -0.09  0.35  0.01 -0.05  0.48 0.31 0.69 1.9
## 
##                        MR2  MR1  MR3  MR5  MR4
## SS loadings           2.31 2.04 1.87 1.93 1.28
## Proportion Var        0.13 0.11 0.10 0.11 0.07
## Cumulative Var        0.13 0.24 0.35 0.45 0.52
## Proportion Explained  0.25 0.22 0.20 0.20 0.14
## Cumulative Proportion 0.25 0.46 0.66 0.86 1.00
## 
##  With factor correlations of 
##      MR2  MR1  MR3  MR5  MR4
## MR2 1.00 0.24 0.23 0.45 0.19
## MR1 0.24 1.00 0.27 0.24 0.00
## MR3 0.23 0.27 1.00 0.44 0.12
## MR5 0.45 0.24 0.44 1.00 0.24
## MR4 0.19 0.00 0.12 0.24 1.00
## 
## Mean item complexity =  1.3
## Test of the hypothesis that 5 factors are sufficient.
## 
## The degrees of freedom for the null model are  153  and the objective function was  6.62 with Chi Square of  4197.3
## The degrees of freedom for the model are 73  and the objective function was  0.33 
## 
## The root mean square of the residuals (RMSR) is  0.02 
## The df corrected root mean square of the residuals is  0.03 
## 
## The harmonic number of observations is  642 with the empirical chi square  97.96  with prob <  0.027 
## The total number of observations was  642  with Likelihood Chi Square =  209.07  with prob <  4.7e-15 
## 
## Tucker Lewis Index of factoring reliability =  0.929
## RMSEA index =  0.054  and the 90 % confidence intervals are  0.045 0.063
## BIC =  -262.84
## Fit based upon off diagonal values = 0.99
## Measures of factor score adequacy             
##                                                    MR2  MR1  MR3  MR5  MR4
## Correlation of (regression) scores with factors   0.93 0.89 0.92 0.90 0.84
## Multiple R square of scores with factors          0.86 0.79 0.84 0.81 0.71
## Minimum correlation of possible factor scores     0.72 0.58 0.68 0.61 0.41
##### SRMR = .02
##### Likelihood Chi Square = 209.07, p < .001
##### TLI = .929
##### RMSEA = .054
##### model fit criteria marginally not met, increases risk of type 1 & 2 error

############### Reliability analysis ###############

psych::omega(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.11, DSSE.14, DSSE.15, DSSE.16, DSSE.23, SISE, OPEN)),
  nfactors = 5,
  fm = "minres",
  rotate = "oblimin",
  digits = 5,
  poly = TRUE
) 

## Omega 
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip, 
##     digits = digits, title = title, sl = sl, labels = labels, 
##     plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option, 
##     covar = covar)
## Alpha:                 0.84 
## G.6:                   0.88 
## Omega Hierarchical:    0.58 
## Omega H asymptotic:    0.65 
## Omega Total            0.9 
## 
## Schmid Leiman Factor loadings greater than  0.2 
##             g   F1*   F2*   F3*   F4*   F5*   h2   u2   p2
## DSSE.1   0.59              0.42             0.53 0.47 0.66
## DSSE.2   0.50              0.35             0.38 0.62 0.66
## DSSE.4   0.47                    0.65       0.64 0.36 0.34
## DSSE.5   0.25                    0.55       0.46 0.54 0.14
## DSSE.6   0.65              0.43             0.62 0.38 0.68
## DSSE.8   0.47                    0.71       0.71 0.29 0.31
## DSSE.9               0.65                   0.48 0.52 0.07
## DSSE.10  0.57        0.45  0.20             0.58 0.42 0.56
## DSSE.12  0.33        0.62                   0.51 0.49 0.21
## DSSE.13  0.52        0.48                   0.53 0.47 0.51
## DSSE.17  0.41  0.45 -0.21              0.20 0.47 0.53 0.36
## DSSE.18  0.39        0.41                   0.35 0.65 0.45
## DSSE.19  0.50  0.68                         0.72 0.28 0.34
## DSSE.20  0.41  0.62                         0.55 0.45 0.31
## DSSE.21  0.40  0.65                         0.62 0.38 0.27
## DSSE.22  0.22                          0.65 0.47 0.53 0.10
## DSSE.24                                0.68 0.50 0.50 0.04
## DSSE.25              0.32              0.46 0.31 0.69 0.09
## 
## With eigenvalues of:
##   g F1* F2* F3* F4* F5* 
## 3.3 1.5 1.6 0.6 1.3 1.2 
## 
## general/max  1.99   max/min =   2.74
## mean percent general =  0.34    with sd =  0.21 and cv of  0.62 
## Explained Common Variance of the general factor =  0.35 
## 
## The degrees of freedom are 73  and the fit is  0.33 
## The number of observations was  642  with Chi Square =  209.07  with prob <  4.7e-15
## The root mean square of the residuals is  0.02 
## The df corrected root mean square of the residuals is  0.03
## RMSEA index =  0.054  and the 10 % confidence intervals are  0.045 0.063
## BIC =  -262.84
## 
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 135  and the fit is  3.36 
## The number of observations was  642  with Chi Square =  2130.86  with prob <  0
## The root mean square of the residuals is  0.14 
## The df corrected root mean square of the residuals is  0.15 
## 
## RMSEA index =  0.152  and the 10 % confidence intervals are  0.146 0.158
## BIC =  1258.14 
## 
## Measures of factor score adequacy             
##                                                  g  F1*  F2*   F3*  F4*  F5*
## Correlation of scores with factors            0.81 0.84 0.86  0.59 0.84 0.82
## Multiple R square of scores with factors      0.66 0.70 0.73  0.35 0.70 0.68
## Minimum correlation of factor score estimates 0.33 0.40 0.46 -0.30 0.40 0.35
## 
##  Total, General and Subset omega for each subset
##                                                  g  F1*  F2*  F3*  F4*  F5*
## Omega total for total scores and subscales    0.90 0.84 0.79 0.74 0.80 0.67
## Omega general for total scores and subscales  0.58 0.29 0.29 0.51 0.22 0.05
## Omega group for total scores and subscales    0.24 0.55 0.50 0.24 0.58 0.62
##### 3 items did not have general factor loadings, common factor model inappropriate
##### omega h was .58, accounting for 35% variance
##### omega t overall was .9
##### MR1 omega 1 = .84, MR2 = .79, MR3 = .74, MR4 = .8, MR5 = .67

############### Table 1 ###############

# used {r table 1 overall descriptive statistics, echo = FALSE} as chunk header

report_data %>% 
  dplyr::summarise(
    dplyr::n(),
    mean(AGE),
    sd(AGE),
    min(AGE),
    quantile(AGE, .25),
    median(AGE),
    quantile(AGE, .75),
    max(AGE)
  ) %>% 
  knitr::kable(
    caption = "Table 1
    \n *Participant descriptive statistics*",
    col.names = c(
      "***N***", 
      "Mean Age", 
      "SD Age",
      "Min Age",
      "1st Q",
      "Median Age",
      "3rd Q",
      "Max Age"
      ),
    align = "cccccccc",
    digits = 2
  ) %>% 
  kable_classic(
    html_font = "Times New Roman"
  ) %>% 
  footnote(
    general = "Data presented are after response exclusions. Age is measured in years.",
    general_title = "*Note.*",
    footnote_as_chunk = TRUE
  )

Table 1

Participant descriptive statistics
N Mean Age SD Age Min Age 1st Q Median Age 3rd Q Max Age
642 29.53 15.09 18 20 21 38 91
Note. Data presented are after response exclusions. Age is measured in years. 
############### Table 2 ###############

# used {r table 2 descriptive statistics by gender, echo = FALSE} as chunk header

report_data %>%
  dplyr::mutate(
    .,
    GENDER = dplyr::recode(
      GENDER,
      '1' = "Male",
      '2' = "Female",
      '3' = "Other",
      '4' = "Prefer not to say"
    )
  ) %>%
  dplyr::group_by(GENDER) %>% 
  dplyr::summarise(
    dplyr::n(),
    dplyr::n()/nrow(report_data)*100,
    mean(AGE),
    sd(AGE),
    min(AGE),
    quantile(AGE, .25),
    median(AGE),
    round(quantile(AGE, .75), 0),
    max(AGE),
  ) %>% 
  knitr::kable(
    .,
    caption = "Table 2
    \n *Participant descriptive statistics by gender*",
    col.names = c(
      "Gender", 
      "n", 
      "% of total", 
      "Mean Age", 
      "SD Age", 
      "Min Age",
      "1st Q",
      "Median",
      "3rd Q",
      "Max Age"
      ),
    align = "lccccccccc",
    digits = 2
  ) %>% 
  kable_classic(
    html_font = "Times New Roman",
  ) %>% 
  footnote(
    general = "Data presented are after response exclusions. Age is measured in years.",
    general_title = "*Note.*",
    footnote_as_chunk = TRUE
  )

Table 2

Participant descriptive statistics by gender
Gender n % of total Mean Age SD Age Min Age 1st Q Median 3rd Q Max Age
Female 422 65.73 29.66 15.46 18 20 21 40 91
Male 207 32.24 29.78 14.65 18 20 22 36 82
Other 12 1.87 20.92 2.94 19 19 19 22 28
Prefer not to say 1 0.16 27.00 – 27 27 27 27 27
Note. Data presented are after response exclusions. Age is measured in years. 
############### Table 3 ###############

# used {r table 3 employment descriptive statistics by gender, echo = FALSE} as chunk header

report_data %>%
  dplyr::mutate(
    .,
    GENDER = dplyr::recode(
      GENDER,
      '1' = "Male",
      '2' = "Female",
      '3' = "Other",
      '4' = "Prefer not to say"
    )
  ) %>%
  dplyr::group_by(GENDER) %>% 
  dplyr::summarise(
    sum(OCCUPATION == 1),
    sum(OCCUPATION == 2),
    sum(OCCUPATION == 3),
    sum(OCCUPATION == 4),
    sum(OCCUPATION == 5),
    sum(OCCUPATION == 6)
  ) %>% 
  knitr::kable(
    .,
    caption = "Table 3
    \n *Participant employment status by gender*",
    col.names = c(
      "Gender", 
      "Current student^a^", 
      "Employed", 
      "Unemployed", 
      "Part-time 
      \n employment", 
      "Searching 
      \n for employment", 
      "Retired"
      ),
    align = "lcccccc"
    ) %>% 
  kable_classic(
    html_font = "Times New Roman"
  ) %>%
  footnote(
    general = "Data presented are after response exclusions.", 
    general_title = "*Note.*",
    alphabet = "*'Current student'* refers to participants' status as a university student in general, rather than specifically a student on the 'Quantitative and Qualitative Methods' module at the University of Sussex as these individuals were already excluded from the analysis.", 
    footnote_as_chunk = TRUE
  )

Table 3

Participant employment status by gender
Gender Current studenta Employed Unemployed

Part-time

employment

Searching

for employment 		Retired
Female 220 124 12 41 6 19
Male 103 88 3 5 3 5
Other 7 1 3 0 1 0
Prefer not to say 1 0 0 0 0 0
Note. Data presented are after response exclusions.
a ‘Current student’ refers to participants’ status as a university student in general, rather than specifically a student on the ‘Quantitative and Qualitative Methods’ module at the University of Sussex as these individuals were already excluded from the analysis. 
############### Figure 1 ###############

# used {r factor extraction, echo = FALSE, results = FALSE} as chunk header

psych::fa.parallel(
  report_poly_after$rho, 
  n.obs = nrow(report_data), 
  fa = "fa",
  main = "Figure 1: Parallel analysis scree plot"
)

## Parallel analysis suggests that the number of factors =  7  and the number of components =  NA
############### Figure 2 ###############

# used {r factor loading table, echo = FALSE} as chunk header

options(knitr.kable.NA = '--')
knitr::kable(
  report_fa_final,
  caption = "Figure 2
  \n *Factor loadings after reanalysis exclusions*",
  align = "lccccccccc",
  digits = 2
) %>% 
kable_classic(
  html_font = "Times New Roman"
)

Figure 2

Factor loadings after reanalysis exclusions
Variable MR2 MR1 MR3 MR5 MR4 Complexity Uniqueness
DSSE.19 0.81 – – – – 1.03 0.28
DSSE.21 0.78 – – – – 1.13 0.38
DSSE.20 0.74 – – – – 1.01 0.45
DSSE.17 0.54 – – – – 1.86 0.53
DSSE.9 – 0.70 – – – 1.09 0.52
DSSE.12 – 0.67 – – – 1.05 0.49
DSSE.13 – 0.52 – 0.33 – 1.82 0.47
DSSE.10 – 0.49 – 0.33 – 2.02 0.42
DSSE.18 – 0.43 – – – 1.63 0.65
DSSE.8 – – 0.83 – – 1.01 0.29
DSSE.4 – – 0.76 – – 1.05 0.36
DSSE.5 – – 0.65 – – 1.34 0.54
DSSE.6 – – – 0.73 – 1.04 0.38
DSSE.1 – – – 0.70 – 1.03 0.47
DSSE.2 – – – 0.59 – 1.13 0.62
DSSE.24 – – – – 0.71 1.02 0.50
DSSE.22 – – – – 0.67 1.01 0.53
DSSE.25 – 0.35 – – 0.48 1.94 0.69 
############### Figure 3 ###############

# {r relability analysis plot, echo = FALSE, results = FALSE} as chunk header
# produces Figure 3 in HTML output, but same code also outputs omega values/g values in console

psych::omega(
  report_data %>% 
  dplyr::select(., -c(AGE, GENDER, OCCUPATION, QQM, DSSE.3, DSSE.7, DSSE.11, DSSE.14, DSSE.15, DSSE.16, DSSE.23, SISE, OPEN)),
  nfactors = 5,
  fm = "minres",
  rotate = "oblimin",
  digits = 5,
  poly = TRUE,
  title = "Figure 3: Omega coefficients"
) 

## Figure 3: Omega coefficients 
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip, 
##     digits = digits, title = title, sl = sl, labels = labels, 
##     plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option, 
##     covar = covar)
## Alpha:                 0.84 
## G.6:                   0.88 
## Omega Hierarchical:    0.58 
## Omega H asymptotic:    0.65 
## Omega Total            0.9 
## 
## Schmid Leiman Factor loadings greater than  0.2 
##             g   F1*   F2*   F3*   F4*   F5*   h2   u2   p2
## DSSE.1   0.59              0.42             0.53 0.47 0.66
## DSSE.2   0.50              0.35             0.38 0.62 0.66
## DSSE.4   0.47                    0.65       0.64 0.36 0.34
## DSSE.5   0.25                    0.55       0.46 0.54 0.14
## DSSE.6   0.65              0.43             0.62 0.38 0.68
## DSSE.8   0.47                    0.71       0.71 0.29 0.31
## DSSE.9               0.65                   0.48 0.52 0.07
## DSSE.10  0.57        0.45  0.20             0.58 0.42 0.56
## DSSE.12  0.33        0.62                   0.51 0.49 0.21
## DSSE.13  0.52        0.48                   0.53 0.47 0.51
## DSSE.17  0.41  0.45 -0.21              0.20 0.47 0.53 0.36
## DSSE.18  0.39        0.41                   0.35 0.65 0.45
## DSSE.19  0.50  0.68                         0.72 0.28 0.34
## DSSE.20  0.41  0.62                         0.55 0.45 0.31
## DSSE.21  0.40  0.65                         0.62 0.38 0.27
## DSSE.22  0.22                          0.65 0.47 0.53 0.10
## DSSE.24                                0.68 0.50 0.50 0.04
## DSSE.25              0.32              0.46 0.31 0.69 0.09
## 
## With eigenvalues of:
##   g F1* F2* F3* F4* F5* 
## 3.3 1.5 1.6 0.6 1.3 1.2 
## 
## general/max  1.99   max/min =   2.74
## mean percent general =  0.34    with sd =  0.21 and cv of  0.62 
## Explained Common Variance of the general factor =  0.35 
## 
## The degrees of freedom are 73  and the fit is  0.33 
## The number of observations was  642  with Chi Square =  209.07  with prob <  4.7e-15
## The root mean square of the residuals is  0.02 
## The df corrected root mean square of the residuals is  0.03
## RMSEA index =  0.054  and the 10 % confidence intervals are  0.045 0.063
## BIC =  -262.84
## 
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 135  and the fit is  3.36 
## The number of observations was  642  with Chi Square =  2130.86  with prob <  0
## The root mean square of the residuals is  0.14 
## The df corrected root mean square of the residuals is  0.15 
## 
## RMSEA index =  0.152  and the 10 % confidence intervals are  0.146 0.158
## BIC =  1258.14 
## 
## Measures of factor score adequacy             
##                                                  g  F1*  F2*   F3*  F4*  F5*
## Correlation of scores with factors            0.81 0.84 0.86  0.59 0.84 0.82
## Multiple R square of scores with factors      0.66 0.70 0.73  0.35 0.70 0.68
## Minimum correlation of factor score estimates 0.33 0.40 0.46 -0.30 0.40 0.35
## 
##  Total, General and Subset omega for each subset
##                                                  g  F1*  F2*  F3*  F4*  F5*
## Omega total for total scores and subscales    0.90 0.84 0.79 0.74 0.80 0.67
## Omega general for total scores and subscales  0.58 0.29 0.29 0.51 0.22 0.05
## Omega group for total scores and subscales    0.24 0.55 0.50 0.24 0.58 0.62
############### Figure 4 ###############

# used {r convergent and divergent validities, echo = FALSE} as chunk header

# composite scores
report_comp <- report_data %>% 
  dplyr::rowwise() %>% 
  dplyr::summarise(
    MR1_comp = mean(c(DSSE.9, DSSE.10, DSSE.12, DSSE.13, DSSE.18)),
    MR2_comp = mean(c(DSSE.17, DSSE.19, DSSE.20, DSSE.21)),
    MR3_comp = mean(c(DSSE.4, DSSE.5, DSSE.8)),
    MR4_comp = mean(c(DSSE.22, DSSE.24, DSSE.25)),
    MR5_comp = mean(c(DSSE.1, DSSE.2, DSSE.6)),
    SISE = mean(SISE),
    OPEN = mean(OPEN)
  )

report_comp %>% 
  dplyr::select(., MR1_comp, MR2_comp, MR3_comp, MR4_comp, MR5_comp, SISE, OPEN) %>% 
  correlation::correlation() %>% 
  summary() %>% 
  knitr::kable(
    caption = "Figure 4
    \n *Correlation matrix for factor composites*",
    digits = 2
  ) %>% 
  kable_classic(
    html_font = "Times New Roman"
  )

Figure 4

Correlation matrix for factor composites
Parameter OPEN SISE MR5_comp MR4_comp MR3_comp MR2_comp
MR1_comp -0.55 -0.55 0.42 0.18 0.34 0.35
MR2_comp -0.16 -0.16 0.36 0.15 0.20 –
MR3_comp -0.15 -0.15 0.36 0.10 – –
MR4_comp 0.03 0.03 0.17 – – –
MR5_comp -0.21 -0.21 – – – –
SISE 1.00 – – – – – 
##### convergent validity poor for all factors
##### divergent validity only good for factor 4