Confirmatory Factor Analysis

What is Confirmatory Factor Analysis (CFA)?

Questionnaires and surveys often have items that reflect multiple sub-scales or multiple (possibly related) factors (AKA components or sub-scales), in a single measure. In Confirmatory Factor Analysis you already have allocated items to specific factors (e.g. based on literature), but need to confirm whether these are valid allocations.

For example, the empathy quotient [@Lawrence2004] has multiple types of empathy within it:

  • cognitive empathy

  • emotional reactivity

  • social skills

Let’s use one publicly available data set to check if we can confirm that the allocations of items to each of these subscales is consistent with the current data:

https://github.com/bhismalab/EyeTracking_PlosOne_2017/blob/master/EQ_Data.csv

eq <- read.csv("EQ_Data.csv")
# focusing on q1.raw to q40.raw
eq_processed <- eq[,c(paste("Q",1:40,".processed",sep=""))]
rmarkdown::paged_table(eq_processed)
colnames(eq_processed) <- paste("eq",1:40, sep="")
rmarkdown::paged_table(data.frame(cor(eq_processed)))

Confirmatory factor analysis

We already have three subscales for the empathy quotient, so let’s confirm whether our current data is consistent with them. The subscales are:

Some of the items below are reverse scored

The items that would go against the component, e.g. “I find it hard to know what to do in a social situation” if the participant agreed are reverse scored, meaning that strongly disagreeing increases their score for the relevant subscale (in this case “social skills”).

Cognitive:

  • 14: I am good at predicting how someone will feel.

  • 15: I am quick to spot when someone in a group is feeling awkward or uncomfortable.

  • 29: I can sense if I am intruding, even if the other person doesn’t tell me.

  • 34: I can tune into how someone else feels rapidly and intuitively.

  • 35: I can easily work out what another person might want to talk about.

Social Skills:

  • 2: I find it difficult to explain to others things that I understand easily, when they don’t understand it first time.

  • 4: I find it hard to know what to do in a social situation.

  • 7: Friendships and relationships are just too difficult, so I tend not to bother with them.

  • 8: I often find it difficult to judge if something is rude or polite.

  • 21: I don’t tend to find social situations confusing.

Emotion:

  • 3: I really enjoy caring for other people.

  • 16: If I say something that someone else is offended by, I think that that’s their problem, not mine.

  • 19: Seeing people cry doesn’t really upset me.

  • 33: I usually stay emotionally detached when watching a film.

  • 39: I tend to get emotionally involved with a friend’s problems.

Let’s start by looking at correlation matrices to see if the items tend to correlate with each other in the current data set:

Cognitive empathy

cor(eq_processed[,c("eq14","eq15","eq29","eq34","eq35")])
          eq14       eq15      eq29      eq34       eq35
eq14 1.0000000 0.34501695 0.2905797 0.4416185 0.27504384
eq15 0.3450170 1.00000000 0.3721252 0.2532783 0.07192026
eq29 0.2905797 0.37212520 1.0000000 0.2468024 0.28854264
eq34 0.4416185 0.25327834 0.2468024 1.0000000 0.23509011
eq35 0.2750438 0.07192026 0.2885426 0.2350901 1.00000000

So far looking good, as everything positively correlates - mostly with r-values greater than .25.

Social Skills

cor(eq_processed[,c("eq2","eq4","eq7","eq8","eq21")])
             eq2       eq4         eq7         eq8        eq21
eq2   1.00000000 0.2067016 -0.06714029  0.06086131  0.30817434
eq4   0.20670163 1.0000000  0.16596510  0.16674209  0.25085525
eq7  -0.06714029 0.1659651  1.00000000 -0.02137924  0.21971081
eq8   0.06086131 0.1667421 -0.02137924  1.00000000 -0.04748266
eq21  0.30817434 0.2508553  0.21971081 -0.04748266  1.00000000

This is a bit less convincing, as the 8th item doesn’t consistently correlate with other items in this subscale.

Emotional empathy

cor(eq_processed[,c("eq3","eq16","eq19","eq33","eq39")])
           eq3       eq16       eq19      eq33      eq39
eq3  1.0000000 0.19886409 0.23743638 0.2444680 0.3494996
eq16 0.1988641 1.00000000 0.09384386 0.2801024 0.2080836
eq19 0.2374364 0.09384386 1.00000000 0.1884394 0.3160680
eq33 0.2444680 0.28010241 0.18843936 1.0000000 0.2923009
eq39 0.3494996 0.20808361 0.31606800 0.2923009 1.0000000

Emotional empathy seems similarly consistent to cognitive empathy.

Let’s now see how well each item loads onto the total of these scores:

library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.3     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(kableExtra)

Attaching package: 'kableExtra'

The following object is masked from 'package:dplyr':

    group_rows
eq_subscales_r <- data.frame(
  cor(
    eq_processed[,c(
      # Cog
      "eq14","eq15","eq29","eq34","eq35",
      
      # Social Skills
      "eq2","eq4","eq7","eq8","eq21",
      
      # Emotion
      "eq3","eq16","eq19","eq33","eq39"
      )],
    matrix(data = c(
      rowSums(eq_processed[,c("eq14","eq15","eq29","eq34","eq35")]),
      rowSums(eq_processed[,c("eq2","eq4", "eq7", "eq8", "eq21")]),
      rowSums(eq_processed[,c("eq3","eq16","eq19","eq33","eq39")])
      ), ncol = 3
    )
  )
)

trying to create a proper table…

psych::fa(eq_processed, nfactors = 3, rotate="oblimin")
Loading required namespace: GPArotation
Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
am sorry, to do these rotations requires the GPArotation package to be
installed
Factor Analysis using method =  minres
Call: psych::fa(r = eq_processed, nfactors = 3, rotate = "oblimin")
Standardized loadings (pattern matrix) based upon correlation matrix
       MR1   MR2   MR3    h2   u2 com
eq1   0.53 -0.11  0.03 0.297 0.70 1.1
eq2   0.13  0.24  0.24 0.130 0.87 2.5
eq3   0.49  0.11 -0.20 0.287 0.71 1.4
eq4   0.08 -0.03  0.56 0.325 0.68 1.0
eq5   0.22  0.59  0.01 0.397 0.60 1.3
eq6   0.25  0.05 -0.13 0.085 0.92 1.6
eq7   0.15  0.26  0.24 0.145 0.85 2.6
eq8   0.31  0.13  0.16 0.141 0.86 1.9
eq9   0.28  0.38  0.24 0.282 0.72 2.6
eq10  0.26  0.05 -0.27 0.142 0.86 2.1
eq11  0.21 -0.42  0.11 0.237 0.76 1.6
eq12  0.38  0.23  0.45 0.398 0.60 2.5
eq13  0.57 -0.05 -0.06 0.330 0.67 1.0
eq14  0.61 -0.23 -0.05 0.430 0.57 1.3
eq15  0.43 -0.22  0.13 0.246 0.75 1.7
eq16  0.42  0.42 -0.03 0.347 0.65 2.0
eq17  0.20  0.29 -0.22 0.170 0.83 2.7
eq18  0.46  0.22  0.36 0.394 0.61 2.4
eq19  0.36 -0.12  0.04 0.150 0.85 1.3
eq20  0.26  0.55  0.15 0.390 0.61 1.6
eq21  0.12 -0.08  0.48 0.249 0.75 1.2
eq22  0.62  0.03 -0.27 0.454 0.55 1.4
eq23  0.10  0.11 -0.18 0.055 0.95 2.3
eq24  0.20 -0.18 -0.27 0.146 0.85 2.6
eq25  0.19  0.28 -0.42 0.286 0.71 2.2
eq26  0.42 -0.42  0.12 0.370 0.63 2.2
eq27  0.24 -0.06 -0.01 0.060 0.94 1.1
eq28  0.53  0.18 -0.31 0.413 0.59 1.9
eq29  0.54 -0.19 -0.08 0.337 0.66 1.3
eq30  0.16  0.40 -0.04 0.184 0.82 1.3
eq31  0.35  0.60  0.01 0.483 0.52 1.6
eq32  0.29  0.19  0.20 0.162 0.84 2.6
eq33  0.33  0.30  0.08 0.209 0.79 2.1
eq34  0.59 -0.22 -0.12 0.414 0.59 1.4
eq35  0.41 -0.25  0.23 0.283 0.72 2.3
eq36  0.52 -0.49  0.09 0.516 0.48 2.1
eq37 -0.06 -0.04  0.08 0.013 0.99 2.3
eq38  0.30 -0.54  0.08 0.386 0.61 1.6
eq39  0.47  0.00 -0.31 0.319 0.68 1.7
eq40  0.58 -0.28  0.00 0.419 0.58 1.4

                       MR1  MR2  MR3
SS loadings           5.70 3.36 2.02
Proportion Var        0.14 0.08 0.05
Cumulative Var        0.14 0.23 0.28
Proportion Explained  0.51 0.30 0.18
Cumulative Proportion 0.51 0.82 1.00

Mean item complexity =  1.8
Test of the hypothesis that 3 factors are sufficient.

df null model =  780  with the objective function =  22.46 with Chi Square =  1209.17
df of  the model are 663  and the objective function was  14.34 

The root mean square of the residuals (RMSR) is  0.09 
The df corrected root mean square of the residuals is  0.1 

The harmonic n.obs is  69 with the empirical chi square  833.94  with prob <  6.5e-06 
The total n.obs was  69  with Likelihood Chi Square =  743.36  with prob <  0.016 

Tucker Lewis Index of factoring reliability =  0.754
RMSEA index =  0.039  and the 90 % confidence intervals are  0.02 0.058
BIC =  -2063.86
Fit based upon off diagonal values = 0.79
Measures of factor score adequacy             
                                                   MR1  MR2  MR3
Correlation of (regression) scores with factors   0.95 0.92 0.87
Multiple R square of scores with factors          0.90 0.85 0.75
Minimum correlation of possible factor scores     0.80 0.70 0.50
# from https://www.anthonyschmidt.co/post/2020-09-27-efa-tables-in-r/
flex <- function(data, title=NULL) {
  # this grabs the data and converts it to a flextbale
  flextable(data) %>%
  # this makes the table fill the page width
  set_table_properties(layout = "autofit", width = 1) %>%
  # font size
  fontsize(size=10, part="all") %>%
    #this adds a ttitlecreates an automatic table number
      set_caption(title, 
                  autonum = officer::run_autonum(seq_id = "tab", 
                                                 pre_label = "Table ", 
                                                 post_label = "\n", 
                                                 bkm = "anytable")) %>%
  # font type
  font(fontname="Times New Roman", part="all")
}

fa_table <- function(x, cut) {
  #get sorted loadings
  loadings <- psych::fa.sort(x)$loadings %>% round(3)
  #supress loadings
  loadings[loadings < cut] <- ""
  #get additional info
  add_info <- cbind(x$communalities, 
                    x$uniquenesses,
                    x$complexity) %>%
    # make it a data frame
    as.data.frame() %>%
    # column names
    rename("Communality" = V1,
           "Uniqueness" = V2,
           "Complexity" = V3) %>%
    #get the item names from the vector
    rownames_to_column("item")
  #build table
  loadings %>%
    unclass() %>%
    as.data.frame() %>%
    rownames_to_column("item") %>%
    left_join(add_info) %>%
    mutate(across(where(is.numeric), round, 3))
}

fa_table(
  psych::fa(eq_processed, nfactors = 3, rotate="oblimin"), 
  .5
)
Loading required namespace: GPArotation
Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
am sorry, to do these rotations requires the GPArotation package to be
installed
Joining with `by = join_by(item)`
Warning: There was 1 warning in `mutate()`.
ℹ In argument: `across(where(is.numeric), round, 3)`.
Caused by warning:
! The `...` argument of `across()` is deprecated as of dplyr 1.1.0.
Supply arguments directly to `.fns` through an anonymous function instead.

  # Previously
  across(a:b, mean, na.rm = TRUE)

  # Now
  across(a:b, \(x) mean(x, na.rm = TRUE))
   item   MR1   MR2   MR3 Communality Uniqueness Complexity
1  eq22 0.617                   0.454      0.546      1.373
2  eq14 0.613                   0.430      0.570      1.287
3  eq34 0.591                   0.414      0.586      1.371
4  eq40 0.584                   0.419      0.581      1.434
5  eq13 0.569                   0.330      0.670      1.036
6  eq29 0.544                   0.337      0.663      1.280
7  eq28 0.535                   0.413      0.587      1.857
8   eq1 0.532                   0.297      0.703      1.099
9  eq36 0.515                   0.516      0.484      2.063
10  eq3                         0.287      0.713      1.437
11 eq39                         0.319      0.681      1.748
12 eq18                         0.394      0.606      2.360
13 eq15                         0.246      0.754      1.679
14 eq26                         0.370      0.630      2.167
15 eq16                         0.347      0.653      2.009
16 eq35                         0.283      0.717      2.294
17 eq19                         0.150      0.850      1.261
18 eq33                         0.209      0.791      2.105
19  eq8                         0.141      0.859      1.885
20 eq32                         0.162      0.838      2.589
21  eq6                         0.085      0.915      1.623
22 eq27                         0.060      0.940      1.117
23 eq31       0.602             0.483      0.517      1.598
24  eq5       0.589             0.397      0.603      1.285
25 eq20       0.548             0.390      0.610      1.602
26 eq38                         0.386      0.614      1.612
27 eq11                         0.237      0.763      1.637
28 eq30                         0.184      0.816      1.338
29  eq9                         0.282      0.718      2.558
30 eq17                         0.170      0.830      2.708
31  eq7                         0.145      0.855      2.578
32  eq4             0.564       0.325      0.675      1.041
33 eq21                         0.249      0.751      1.176
34 eq12                         0.398      0.602      2.493
35 eq25                         0.286      0.714      2.195
36 eq24                         0.146      0.854      2.639
37 eq10                         0.142      0.858      2.085
38  eq2                         0.130      0.870      2.510
39 eq23                         0.055      0.945      2.305
40 eq37                         0.013      0.987      2.339
cog_eq_sum = (
  eq_processed$eq14 +
  eq_processed$eq15 +
  eq_processed$eq29 +
  eq_processed$eq34 +
  eq_processed$eq35
)

cog_eq_mean = (
  eq_processed$eq14 +
  eq_processed$eq15 +
  eq_processed$eq29 +
  eq_processed$eq34 +
  eq_processed$eq35
)/5


lm(eq14 ~ cog_eq_sum, eq_processed)

Call:
lm(formula = eq14 ~ cog_eq_sum, data = eq_processed)

Coefficients:
(Intercept)   cog_eq_sum  
    -0.3124       0.2341  
lm(eq14 ~ cog_eq_mean, eq_processed)

Call:
lm(formula = eq14 ~ cog_eq_mean, data = eq_processed)

Coefficients:
(Intercept)  cog_eq_mean  
    -0.3124       1.1705  
bop<-lm(cog_eq_mean ~ eq14 + eq15 + eq29 + eq34 + eq35, eq_processed)

bop$coefficients 
(Intercept)        eq14        eq15        eq29        eq34        eq35 
-1.1501e-15  2.0000e-01  2.0000e-01  2.0000e-01  2.0000e-01  2.0000e-01 
colnames(eq_subscales_r) <- c( "Cognitive", "Social", "Emotion")

eq_subscales_r %>% mutate_all(~cell_spec(
        .x, 
        color = ifelse(.x < .4, "black", "white"),
        background = ifelse(.x > .4, "red"," white"))) %>%
    kable(escape = F) %>%
    kable_styling()
Cognitive Social Emotion
eq14 0.730729714723979 0.110723735489975 0.31320589104921
eq15 0.60185382395657 0.198874708000659 0.137720182774352
eq29 0.659975490377257 0.0278299656284012 0.364451955577018
eq34 0.679619374819311 0.0226190168052008 0.395730784538071
eq35 0.587492289787915 0.122610871425514 0.242014658835711
eq2 -0.00813200484463844 0.576298173261772 0.133855862092061
eq4 0.0479274896411267 0.662778025977272 0.123591773099612
eq7 -0.0312109379222552 0.438752142983534 0.234889555117118
eq8 0.226816177181639 0.427950397740909 0.134334810841783
eq21 0.161873895295539 0.627680352407105 -0.0206828796782751
eq3 0.29722368237889 -0.00177049925158063 0.62739169422544
eq16 0.268239998089082 0.168294312535106 0.592732439474545
eq19 0.35758009314024 0.183400506448371 0.552243103025108
eq33 0.159850007854409 0.302368129555882 0.665487951247779
eq39 0.349069094803534 0.0110077011019322 0.690638221783324

This is quite dense, but in broad terms, we want to cluster items that correlate with each other into one component (AKA factor AKA subscale).

If we use a package in R we can start identifying the top 3 components and check if the questions map on to what we would expect for each of the three subscales:

eq_pca <- prcomp(eq_processed)
sort(abs(eq_pca$rotation[,1]),decreasing = T)[1:5]
     eq16      eq20      eq22      eq18      eq13 
0.2752500 0.2436235 0.2428536 0.2329153 0.2315572 
sort(eq_pca$rotation[,1],decreasing = T)[1:5]
       eq37        eq11        eq21         eq4        eq38 
 0.03183132 -0.02479062 -0.03616440 -0.04118648 -0.04219994

Component 1 involves questions 20, 33, 31, 16 and 5. These items are: - 20: I am very blunt, which some people take to be rudeness, even though this is unintentional. - 33: I usually stay emotionally detached when watching a film. - 31: Other people often say that I am insensitive, though I don’t always see why. - 16: If I say something that someone else is offended by, I think that that’s their problem, not mine. - 5: People often tell me that I went too far in driving my point home in a discussion.

I think the below are just notes or wrong

So how do we start doing this? We can make a regression to determine the loading of all the items onto the total:

eq_total <- rowSums(eq_processed)
first_component <- lm(eq_total ~ ., eq_processed)
first_component$coefficients
 (Intercept)          eq1          eq2          eq3          eq4          eq5 
1.542403e-14 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
         eq6          eq7          eq8          eq9         eq10         eq11 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
        eq12         eq13         eq14         eq15         eq16         eq17 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
        eq18         eq19         eq20         eq21         eq22         eq23 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
        eq24         eq25         eq26         eq27         eq28         eq29 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
        eq30         eq31         eq32         eq33         eq34         eq35 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
        eq36         eq37         eq38         eq39         eq40 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00

explained in smaller steps

using r package

eigen(cov(eq_processed))
eigen() decomposition
$values
 [1] 2.93449333 1.87628459 1.31590409 1.12748372 0.94180395 0.88649785
 [7] 0.79441522 0.71321627 0.65949310 0.62037952 0.57051522 0.51529423
[13] 0.47943423 0.44518243 0.42671811 0.40955342 0.38885355 0.36458886
[19] 0.34093142 0.32034798 0.28981434 0.26346324 0.25212709 0.21377123
[25] 0.19943788 0.16612460 0.15125182 0.13004091 0.12441560 0.11618794
[31] 0.11047414 0.10347115 0.08948276 0.08151707 0.06526776 0.05568862
[37] 0.05048182 0.04280785 0.03199620 0.01706141

$vectors
             [,1]        [,2]        [,3]         [,4]         [,5]
 [1,] -0.14885205  0.10112567 -0.02509082 -0.071014782  0.070127637
 [2,] -0.10091649 -0.12487961 -0.21344804  0.126134563 -0.363701783
 [3,] -0.21319619  0.06215300  0.14461470  0.111668352  0.254921340
 [4,] -0.04118648 -0.01513016 -0.42425919  0.110096470  0.018531412
 [5,] -0.17453482 -0.27933600  0.05769899 -0.012773827 -0.094372919
 [6,] -0.12317922  0.04277050  0.13093569 -0.152834097 -0.075972782
 [7,] -0.09429054 -0.12047107 -0.11068727  0.095173646  0.321122230
 [8,] -0.15532360 -0.04014554 -0.11777900 -0.162564085 -0.112894310
 [9,] -0.15175665 -0.13375762 -0.09957822  0.015337922  0.099157025
[10,] -0.11476213  0.07181521  0.20893937  0.185060033  0.179800156
[11,] -0.02479062  0.20731727 -0.11512269 -0.129484221 -0.124767585
[12,] -0.20001559 -0.08797362 -0.28406761 -0.149859215  0.179441277
[13,] -0.23155718  0.14482426  0.02484372 -0.102876589  0.133912627
[14,] -0.18108913  0.18299383 -0.01014458 -0.166832869 -0.090845908
[15,] -0.10440156  0.11709966 -0.07772977 -0.105781123  0.105817944
[16,] -0.27525004 -0.17463360  0.05807836 -0.074587632 -0.109099073
[17,] -0.12515385 -0.11189375  0.22004369 -0.070312126  0.144586699
[18,] -0.23291526 -0.05944152 -0.24972629 -0.118728554 -0.070205907
[19,] -0.13481111  0.15230707 -0.07563699  0.265291859 -0.141997509
[20,] -0.24362346 -0.36238558 -0.07453948 -0.156177866  0.025176784
[21,] -0.03616440  0.03942237 -0.34412315  0.148213368 -0.048368899
[22,] -0.24285356  0.12463983  0.13604868 -0.051930456 -0.246722473
[23,] -0.05203000 -0.01501208  0.16511453 -0.002450736  0.270528451
[24,] -0.05906693  0.19421309  0.15273056  0.358550925 -0.177778979
[25,] -0.09644040 -0.08117186  0.26316869 -0.140791173 -0.035738385
[26,] -0.07345369  0.18152254 -0.07792673 -0.124229730  0.065045677
[27,] -0.10514313  0.11214456 -0.02311665  0.353333232 -0.154066342
[28,] -0.21837854  0.02627596  0.18307314 -0.127036457 -0.207679119
[29,] -0.14871777  0.15334195  0.01875033  0.017018645 -0.094104526
[30,] -0.13421971 -0.19925686  0.06446130  0.102339786 -0.341431242
[31,] -0.22781047 -0.24536195  0.04637429  0.050756852  0.010756140
[32,] -0.16086212 -0.03560941 -0.12984607  0.332415006  0.208100395
[33,] -0.22804044 -0.11299201 -0.01468625  0.293444598  0.219353354
[34,] -0.17436789  0.18970876  0.03337218 -0.049325165 -0.007173955
[35,] -0.11389235  0.14957934 -0.15942072 -0.172264522  0.002624950
[36,] -0.10887641  0.25415327 -0.08877280 -0.068476179  0.041093920
[37,]  0.03183132  0.01079756 -0.09170238  0.184201107 -0.090322073
[38,] -0.04219994  0.29754522 -0.10121144 -0.115424152  0.104174103
[39,] -0.21909596  0.14852516  0.22629102  0.171059890  0.019044720
[40,] -0.16930983  0.22832068 -0.04107838  0.005916164  0.051211687
              [,6]        [,7]         [,8]          [,9]        [,10]
 [1,]  0.006119948  0.02324630  0.040691595 -0.1294096393 -0.148942216
 [2,]  0.016933565 -0.17665873  0.223096197 -0.0405096824  0.352530196
 [3,] -0.042796961  0.09290645  0.026050627  0.0395456079  0.054063729
 [4,] -0.074503190  0.16521157 -0.202173247 -0.2966167135  0.124578200
 [5,]  0.012713419 -0.06357712 -0.086066025 -0.0491824725 -0.051250400
 [6,]  0.435228223  0.06098821 -0.161284969  0.2577071219  0.117801427
 [7,] -0.271066653 -0.01194754 -0.057270194  0.0979508265 -0.044225282
 [8,]  0.134081558  0.12799009 -0.071293952 -0.2198553070 -0.451131985
 [9,]  0.207132844 -0.14029164  0.093306497  0.0656231074  0.184870075
[10,]  0.142926959 -0.18753662 -0.116615738  0.0570337168  0.013059541
[11,] -0.032643620 -0.04351784 -0.222009296  0.1315299255 -0.013179787
[12,]  0.199806055  0.13708449  0.060524975  0.1992480218 -0.132030286
[13,] -0.153104814  0.04908259  0.342429803  0.1077049680  0.149118464
[14,]  0.003958447 -0.03875740 -0.098081800 -0.0317053321  0.008479807
[15,] -0.127331352 -0.02569759 -0.006488851  0.1161201330 -0.099095263
[16,]  0.198074734  0.32323386 -0.157413533 -0.0004700701  0.258594312
[17,] -0.068202126 -0.31281778  0.085339084 -0.0076273430 -0.058200957
[18,]  0.137998888 -0.16938085  0.201376101  0.1345761916  0.016416638
[19,] -0.117115123  0.05501882 -0.138403111  0.1168033759  0.184423548
[20,] -0.309746658  0.32653654  0.092161901 -0.1094815145 -0.073249089
[21,] -0.030030064 -0.36464316 -0.109756057  0.2834541486 -0.077322145
[22,] -0.073825472 -0.14454128 -0.071059067 -0.0699442597 -0.189200852
[23,]  0.418352031  0.06055086  0.002244558  0.0325248469  0.041387139
[24,]  0.050443616  0.12239003  0.191329884  0.0599348238 -0.209461246
[25,] -0.144190583  0.06700051  0.081587883 -0.0543381124  0.206056218
[26,]  0.070323095  0.01312386  0.048277226 -0.0780515538  0.003983627
[27,]  0.084677432  0.28827864  0.182056470  0.1048929065 -0.270329635
[28,]  0.022764004 -0.15490076 -0.040686689 -0.1208785449 -0.154640753
[29,] -0.028725942  0.03954373 -0.009367995  0.1293606197 -0.006759526
[30,] -0.112625495  0.04595560 -0.210180789  0.2547197346  0.043968622
[31,] -0.060563016 -0.31190190  0.213870711 -0.0204652801 -0.004114626
[32,]  0.061696200  0.12617541  0.076618157  0.1160058088  0.013861306
[33,]  0.048341035 -0.20001719 -0.427379992 -0.2543358862 -0.115205194
[34,] -0.047060618 -0.11709317  0.050869067 -0.1546589529  0.062455555
[35,]  0.101873334  0.04138749 -0.131456373 -0.0606821789  0.037080094
[36,] -0.093811702 -0.09952521 -0.049077877 -0.1227509486 -0.019950696
[37,]  0.309862048 -0.06975852  0.245299005 -0.5309007228  0.118851107
[38,] -0.072224835  0.01754501 -0.065361115 -0.0487848696  0.308351032
[39,] -0.168248949  0.12052077 -0.146240053 -0.1386520615  0.229948767
[40,] -0.055521030  0.03278789  0.280474287 -0.0010935095 -0.095309650
             [,11]        [,12]        [,13]        [,14]       [,15]
 [1,] -0.253868134 -0.026378923  0.139382716  0.044761895  0.08948919
 [2,] -0.022810667  0.020994584 -0.077046746  0.346488286 -0.08691651
 [3,]  0.120914913 -0.072314901  0.187086205  0.301066309  0.32564864
 [4,] -0.088050052 -0.243189034 -0.267756742  0.072911971  0.03630678
 [5,] -0.072822847 -0.288424864 -0.078123589 -0.273594921  0.01349082
 [6,] -0.203816671 -0.275828938 -0.203716874  0.094047851 -0.11443579
 [7,]  0.005326963  0.006993484  0.014542079 -0.149687164 -0.15293618
 [8,]  0.045712400  0.076175741 -0.265109868  0.202428397 -0.02931851
 [9,]  0.138159936 -0.024521607 -0.160244586 -0.348243157  0.23199313
[10,]  0.264929142  0.043684988  0.050768359  0.145715076 -0.18998061
[11,]  0.110870762  0.403955313 -0.073179826  0.010223102 -0.14049182
[12,]  0.070954746  0.093634203 -0.128963760  0.049677031  0.30365893
[13,] -0.114143158 -0.269418949 -0.071534229 -0.078927689 -0.15602113
[14,] -0.173753963 -0.040394180  0.164228968  0.052666105 -0.04110498
[15,] -0.144614985  0.053256450 -0.053968648  0.066967560 -0.28890074
[16,] -0.116896928  0.230041437  0.075767587 -0.041653811 -0.25641550
[17,] -0.429776496  0.243035922 -0.261530553  0.093768297  0.17357371
[18,]  0.316419980  0.178596105  0.051927948  0.007019879 -0.21310623
[19,]  0.136542837  0.018220133  0.087625887  0.084760908  0.21910180
[20,]  0.058038535  0.002380763  0.297734072  0.115009764 -0.12710823
[21,] -0.185384319 -0.138466933  0.134471641  0.071194368 -0.07846002
[22,]  0.168649625 -0.179815303  0.017623692  0.114531283  0.11851323
[23,]  0.011044629 -0.023558273  0.317489916  0.100585874 -0.01445311
[24,] -0.097156222  0.155450721 -0.175588489 -0.079492243 -0.08651245
[25,] -0.066388538  0.306500289 -0.188557761 -0.005850511  0.05072269
[26,] -0.216927702 -0.043321064  0.077391489 -0.210244505 -0.00799991
[27,] -0.149843296  0.131568176  0.016582460 -0.022052581  0.11290888
[28,]  0.072210654 -0.274295719  0.023933918  0.071539745 -0.01341991
[29,]  0.070641695  0.027451376  0.070896954 -0.336881264 -0.19055657
[30,] -0.219397023  0.021368869  0.280512041 -0.193161328  0.25095271
[31,]  0.027619178  0.121965742 -0.060345272 -0.044247910  0.10288207
[32,] -0.110944046 -0.033972122 -0.114297377  0.183704630 -0.03051556
[33,] -0.069234258  0.128769313  0.022270597 -0.116804666 -0.16698083
[34,]  0.017632909  0.061659269  0.160647608  0.162520062  0.04542429
[35,]  0.176403011  0.121207893 -0.055449911 -0.202345939  0.33783961
[36,] -0.065575303  0.141128952  0.145660585 -0.095775914  0.10261327
[37,] -0.091821121  0.066421466  0.215964244 -0.150725673 -0.04461417
[38,] -0.227886499  0.068254196  0.004405031  0.075515277  0.06253901
[39,]  0.210024433 -0.096319225 -0.318256877 -0.058321039 -0.04186827
[40,]  0.098775253 -0.135888673 -0.007911576 -0.228873442 -0.07877680
             [,16]        [,17]       [,18]        [,19]        [,20]
 [1,] -0.094923537 -0.192238711 -0.09534877 -0.159657019 -0.263154099
 [2,] -0.121152744 -0.268133556 -0.02037691 -0.073369010 -0.361937208
 [3,]  0.136792271 -0.170694040 -0.21878266 -0.140875936 -0.102684603
 [4,]  0.149548107  0.007767771 -0.20848356 -0.067509154  0.185568114
 [5,]  0.481794881 -0.188839707  0.11141921  0.038518180 -0.038908091
 [6,] -0.036236361  0.342278526 -0.18683115 -0.093753777 -0.030993709
 [7,] -0.028349776 -0.033031614 -0.06621637  0.343806212 -0.194590135
 [8,] -0.033311715  0.133706153  0.08596254 -0.020568295 -0.217074430
 [9,]  0.055467903  0.043113458 -0.01553761  0.204987535 -0.198908488
[10,]  0.343420406  0.069697224 -0.28911609 -0.273611058  0.069972436
[11,]  0.214466094  0.108083332 -0.13501143  0.101843466 -0.033521223
[12,] -0.056667299 -0.215770902 -0.19729269  0.124579893  0.175846777
[13,] -0.208146922  0.061091112 -0.08383813 -0.164370003  0.230492665
[14,] -0.027594081 -0.007459796 -0.02565145  0.335406003  0.283029151
[15,]  0.079017018 -0.125565686 -0.06598592 -0.042522877 -0.040319330
[16,] -0.003111449 -0.284897471  0.08718054  0.005253693  0.204073473
[17,]  0.013151740  0.083604440  0.23870805 -0.133560752  0.093053343
[18,] -0.010005348  0.192489223  0.20382396 -0.036038300 -0.031016716
[19,] -0.007722025  0.089276259  0.25784135  0.005659660  0.280583546
[20,]  0.027470624  0.239427325 -0.05417795 -0.058508189 -0.056329942
[21,] -0.157064450 -0.136480572 -0.06166973 -0.012169958  0.045251810
[22,]  0.009507169 -0.186098407 -0.10823197  0.227906483  0.032039812
[23,] -0.164121480 -0.078152996  0.13108411  0.243008214 -0.176144583
[24,]  0.086174771 -0.241328036 -0.05969505  0.140081716 -0.027124119
[25,]  0.011038704 -0.172615827 -0.11747661  0.068862712  0.111035856
[26,]  0.128928264 -0.053620643  0.05692409 -0.245700805 -0.216982410
[27,] -0.102461867  0.233335740 -0.15346714  0.017944021 -0.003563527
[28,]  0.013283549  0.023721311  0.14159740  0.176211916  0.042775237
[29,]  0.062276826 -0.144201758 -0.08689177 -0.202104892 -0.100007284
[30,] -0.010394611  0.202151342 -0.02704892 -0.090903607 -0.145816766
[31,] -0.026404981  0.214858055 -0.31276877  0.044056190  0.114886719
[32,]  0.320797860  0.083951261  0.43460631  0.046631181 -0.005060487
[33,] -0.353547954 -0.025710715  0.02007452 -0.076692996  0.079746128
[34,]  0.085732769  0.034358582  0.21586470 -0.122799869  0.069852372
[35,] -0.147264008 -0.100313953  0.12993224 -0.301260599  0.037329184
[36,]  0.185613773  0.122754928 -0.02132291  0.016540313 -0.159091609
[37,]  0.079922147  0.126452053 -0.16708879  0.071711947  0.129862952
[38,]  0.104901845  0.187108097 -0.06353429  0.331486099 -0.184810504
[39,] -0.271893227  0.130433763  0.03299805  0.035128750 -0.284171868
[40,] -0.065532585  0.003331394  0.14907648  0.019462781  0.111738238
             [,21]        [,22]        [,23]         [,24]        [,25]
 [1,]  0.057672069 -0.010551627  0.014968636  0.1097851744 -0.127171088
 [2,]  0.052742405 -0.075167540  0.097918946  0.1288066938 -0.003167578
 [3,] -0.091351983  0.071981822  0.098595627  0.1932563305  0.074172130
 [4,] -0.089750265  0.012887413 -0.069575343  0.0201398071 -0.074999800
 [5,] -0.105054835  0.099999891  0.215121871  0.0320525083  0.276750869
 [6,] -0.018051076 -0.107230686  0.215391029  0.0278297235 -0.261925807
 [7,] -0.110984652 -0.004501101 -0.172779210  0.0437490646 -0.474516306
 [8,]  0.317941149  0.270735936 -0.265285451 -0.0677401142 -0.035585094
 [9,]  0.009873304  0.268701811 -0.057027592  0.1086886812 -0.128163197
[10,]  0.076670473  0.006345776 -0.350082840  0.1281912774 -0.007574883
[11,] -0.065025747 -0.194878502  0.136143733  0.1878198469 -0.053093452
[12,] -0.112253421 -0.256008332  0.062484961 -0.2442605282  0.139343090
[13,]  0.128292244  0.316523701 -0.125319360  0.0784518976  0.091945693
[14,]  0.060555856  0.004547840  0.076081147  0.1079321765 -0.007279708
[15,]  0.225037036 -0.002187989  0.160618807 -0.3477567168  0.200652679
[16,] -0.176525618  0.107213379 -0.255439051 -0.0272481522 -0.115792942
[17,] -0.257692097 -0.186722861 -0.201635226  0.1297006215  0.086662286
[18,] -0.209706251  0.201864780  0.047428530  0.1039642797  0.172330896
[19,]  0.274574355  0.094625818  0.139058644  0.0006627626 -0.099762103
[20,] -0.010208942 -0.239235823  0.091686272  0.1777883597 -0.001568161
[21,] -0.069699526 -0.091453598 -0.290076956 -0.0343322188  0.019308268
[22,] -0.073354275  0.111719789 -0.149595935 -0.1353328673 -0.004963187
[23,]  0.142208245 -0.052849616 -0.015582670 -0.0992049892  0.089362240
[24,]  0.077014285 -0.040263842  0.165036300  0.1958822410 -0.189610287
[25,]  0.231526169  0.098279972 -0.061075475 -0.0125485845 -0.019225859
[26,] -0.004669175  0.030269719  0.151333121 -0.0211666407 -0.203903433
[27,] -0.349408797  0.187095515  0.008207817  0.0295395527  0.173370681
[28,] -0.016886784 -0.246723640 -0.036673036  0.1842840233 -0.010075954
[29,]  0.029013724 -0.167060014 -0.029893424 -0.1945045934  0.167493234
[30,]  0.166349960 -0.010025768 -0.209196032 -0.1056451108  0.009238251
[31,]  0.245671242 -0.080741853  0.230342462 -0.2352927921 -0.138194625
[32,]  0.234658334 -0.172874740 -0.057908496 -0.1222790914 -0.006920263
[33,]  0.056578419  0.135243648  0.327772042  0.1664786261  0.149730837
[34,] -0.341029989  0.155614708  0.176302491 -0.3893493814 -0.362084771
[35,]  0.138372105 -0.191425450 -0.073764745  0.1813234310 -0.060065814
[36,]  0.017882284  0.088301359 -0.056626891 -0.1075930915  0.020749553
[37,]  0.038662979 -0.214313995 -0.172380866 -0.1201924920  0.039706488
[38,]  0.048990490  0.067009239 -0.034395153  0.1525092351  0.325790082
[39,] -0.193488224 -0.243956538 -0.083020835 -0.2744093864  0.146703443
[40,]  0.074083456 -0.267947912 -0.056702045  0.2101688044 -0.049783535
              [,26]         [,27]        [,28]        [,29]        [,30]
 [1,]  0.1261976960  0.4845188015 -0.057770949 -0.140507882 -0.025772829
 [2,]  0.0206144478 -0.0230884426  0.046307863  0.072373564  0.116750147
 [3,] -0.1823751123 -0.3561312586  0.054947467  0.045154503 -0.087922747
 [4,]  0.0628123080  0.2343519072  0.265168309 -0.030656805  0.115630452
 [5,]  0.1310736588 -0.0070743408 -0.236793636  0.081898799  0.211634066
 [6,] -0.0385937111 -0.0711048529 -0.076164795 -0.214972619 -0.145522262
 [7,]  0.1315442218 -0.0630451939 -0.215327781 -0.048833432  0.096518845
 [8,]  0.1082669211 -0.2519390034 -0.070704782  0.135653858  0.045384229
 [9,]  0.0007944122  0.0104994599  0.354746023  0.149582306 -0.404848848
[10,]  0.1693863971  0.2743603613 -0.081317473  0.102996515  0.010999736
[11,] -0.0540792995 -0.1594418821  0.174381159  0.030223976  0.344251588
[12,] -0.0202520296  0.1014750013 -0.181411806 -0.024019058  0.049655459
[13,] -0.1328828723 -0.1352276350 -0.107243577  0.020129689  0.224284343
[14,]  0.1904037295  0.0278573378  0.148270272  0.091071121 -0.130051812
[15,] -0.0500101508  0.1702298879  0.036838424  0.230090443 -0.341186432
[16,] -0.1878720247 -0.0648672691 -0.097741463  0.219430770  0.001730235
[17,]  0.2170308246 -0.0699135511  0.021538774  0.008099121 -0.086690936
[18,]  0.0231632243  0.2378415260 -0.133672035 -0.262831366 -0.045238335
[19,]  0.4182285733  0.0008798547 -0.145151997  0.134670728 -0.074284223
[20,]  0.1228706469 -0.0773181768  0.036874934 -0.028801053 -0.263699822
[21,]  0.0121871913 -0.1863299304  0.034246740  0.091247102 -0.080131451
[22,]  0.0344283810 -0.1150367069  0.007291642 -0.484909202  0.002420392
[23,]  0.1977979813  0.0840537503  0.172241134  0.078706373  0.311691231
[24,] -0.0948390259  0.0524647864 -0.279020221  0.009544148 -0.206553405
[25,]  0.0594849896  0.1131241478  0.287880918 -0.279527786  0.060239844
[26,]  0.2191918037 -0.1605031941  0.056036613  0.055933637  0.198444513
[27,]  0.1295629163  0.0536864258  0.209225746  0.149040005  0.122131930
[28,] -0.1925996747  0.1089079117  0.143821576  0.307285752 -0.007699667
[29,]  0.2814053783 -0.2656112376  0.242592735 -0.161992159 -0.120451289
[30,] -0.1902429386  0.1809669527 -0.043648890 -0.098491666  0.114197959
[31,] -0.0155084650 -0.0417931104 -0.065406659  0.193334011  0.228112371
[32,] -0.2192327733 -0.0642663334  0.158876673 -0.273290637  0.034993759
[33,] -0.1525868353 -0.0520388087  0.046093380 -0.168706570  0.011836742
[34,] -0.0269458077  0.0352368012  0.008205491  0.010828145 -0.000423724
[35,] -0.0329725008 -0.0413906875 -0.186302842  0.027645914 -0.039720160
[36,] -0.4193815101  0.0725158797  0.032464427  0.093732182 -0.024994670
[37,]  0.0150865842 -0.1766856815 -0.162448843 -0.112129715 -0.128435264
[38,]  0.0649681396 -0.0300664309 -0.297040601 -0.058548849 -0.019733263
[39,]  0.0403027971  0.0616921497 -0.089968170  0.124814120 -0.005028767
[40,] -0.1082104252  0.1119012875  0.129789422 -0.024299983  0.172421644
             [,31]        [,32]        [,33]        [,34]        [,35]
 [1,] -0.005285036 -0.095671366  0.362669996 -0.075026530  0.256949901
 [2,]  0.047241314  0.129880583 -0.171559066  0.062743492 -0.011831095
 [3,] -0.179963877  0.182123169  0.229944920  0.052705836 -0.243545580
 [4,]  0.065330152  0.006442492  0.024953194  0.104563270 -0.225150004
 [5,] -0.099246907  0.165657889 -0.017576778 -0.094494620  0.125111701
 [6,]  0.059648892  0.241617969  0.028130687 -0.106557684  0.044271074
 [7,]  0.043603714  0.164488559  0.036121656 -0.145801394 -0.268208583
 [8,] -0.145619084  0.104948317  0.116804770  0.070658222 -0.032969525
 [9,]  0.072534068 -0.160581542  0.006577853  0.183568634  0.134790458
[10,] -0.043770994 -0.074002509 -0.199079595  0.036967109  0.071912736
[11,]  0.127870572 -0.138243771  0.103167731  0.176322129  0.044751754
[12,]  0.105832554 -0.040878065 -0.086816445  0.038220317 -0.077038975
[13,]  0.179294997 -0.207385697 -0.103749520  0.033863939 -0.084784671
[14,] -0.529819581  0.070162827 -0.307188494  0.057524336 -0.082446819
[15,]  0.125905285  0.085982086  0.013164536  0.159670837 -0.184808072
[16,]  0.053060644  0.005975088  0.155502672  0.046662792  0.203062673
[17,]  0.176991929  0.113835961 -0.032571784  0.195103156 -0.097601918
[18,] -0.106374816  0.067451324  0.062083040 -0.100758259 -0.290523662
[19,]  0.338499323  0.197941205  0.143060977 -0.046817105  0.022158101
[20,]  0.121979299 -0.097650425 -0.228833000  0.008650852  0.204777437
[21,] -0.139447134 -0.013564983  0.013160832 -0.242048930  0.215330608
[22,]  0.267660218 -0.072902335 -0.171193028  0.177660811  0.183336672
[23,]  0.159061372  0.036625689 -0.129476338  0.066902922 -0.077137870
[24,] -0.001760595 -0.149350084 -0.165395951  0.079268775 -0.127497158
[25,] -0.101506547  0.132957416 -0.003244448 -0.409846775 -0.077441806
[26,]  0.041238846 -0.167653465 -0.287651030 -0.013824433 -0.131389342
[27,]  0.012913498  0.018952004 -0.049094191 -0.230797744  0.078360916
[28,]  0.215459561 -0.259797084  0.206417389 -0.330109002 -0.306608622
[29,]  0.021474757 -0.034663509  0.251998016 -0.097911377 -0.102434203
[30,] -0.111966020 -0.046733618 -0.006348328  0.296835870 -0.306842570
[31,] -0.125057428 -0.100691359  0.134749324 -0.049949966  0.187708449
[32,] -0.120308756 -0.218260472 -0.043708172 -0.133946110  0.102506222
[33,]  0.049378303 -0.035365911 -0.024571688  0.049369481 -0.004062730
[34,] -0.160559696 -0.155298421  0.083712835  0.087568979  0.016760536
[35,] -0.121709018 -0.108066329 -0.202750214 -0.197156269 -0.004401274
[36,]  0.266985321  0.424111202 -0.278967339 -0.246098196  0.127739333
[37,]  0.057620012  0.068837143  0.085662409 -0.001810040 -0.163490334
[38,] -0.035917404 -0.231057296  0.202006601  0.008825165  0.113581983
[39,] -0.175329611  0.005886550 -0.108221959 -0.006970761  0.045771796
[40,] -0.135862455  0.405241707  0.181725669  0.347622397  0.204562583
             [,36]         [,37]         [,38]       [,39]        [,40]
 [1,]  0.244326165  0.0501747541 -0.2315007970  0.17547431  0.078770995
 [2,] -0.085172793 -0.0759354260 -0.1858912569 -0.11116683 -0.215113387
 [3,]  0.068358919  0.0398635451  0.0260460921  0.02390101  0.178979276
 [4,]  0.195618764 -0.0008855799  0.3197706794 -0.09261320 -0.006997310
 [5,]  0.001502148  0.2576485944 -0.0212546037  0.05249582 -0.090096107
 [6,] -0.038159226  0.0670506960  0.0140246217 -0.03845332 -0.045665900
 [7,] -0.029133901  0.0497009518 -0.1920633907 -0.18738462 -0.088841123
 [8,] -0.053461951 -0.0218726127  0.0461044923  0.12622211 -0.106671788
 [9,] -0.043885140  0.0862392074 -0.0614831164  0.09778282 -0.020327619
[10,] -0.186209621 -0.0888611492 -0.0706582484 -0.01518650 -0.114230174
[11,]  0.120340393  0.3278214335 -0.1641696384  0.21115901  0.057664465
[12,] -0.229308135 -0.1979234421 -0.1168577444  0.27029655 -0.151943116
[13,]  0.144553818  0.1675228333 -0.1400509886  0.17850566 -0.154876675
[14,]  0.218005076 -0.1311067276 -0.2524749274  0.04521873 -0.044447817
[15,] -0.115393273  0.3053230377 -0.0969374632 -0.29192731  0.195652441
[16,]  0.063659508 -0.1636733999  0.0206192800 -0.12419974  0.175717931
[17,]  0.141229582 -0.0208500695  0.0345781766 -0.03327614 -0.025750113
[18,]  0.150466496 -0.0492700753  0.1441325754  0.02837943  0.241912548
[19,] -0.017377953 -0.0230954095 -0.0519550921  0.14972177  0.086935243
[20,] -0.051367020  0.0979172830  0.1714410958  0.08356675 -0.061367095
[21,] -0.091586177  0.2201593051  0.3064906796  0.21811964  0.131133911
[22,]  0.023719095  0.0177512129 -0.0228983328 -0.24364490  0.208023114
[23,]  0.183655595  0.2118349124  0.3173764676 -0.04431162 -0.087332055
[24,]  0.137963463  0.0264323982  0.3851340417  0.13456836 -0.077712060
[25,] -0.301414941  0.1852321285  0.1478868193  0.14162335  0.039431992
[26,] -0.272319726 -0.2741940562  0.0486628335  0.08115931  0.468360897
[27,] -0.129320753  0.0979887680 -0.1528028907 -0.27741106  0.003484819
[28,] -0.147749684 -0.1038570541 -0.0292333278 -0.01503290 -0.016058463
[29,]  0.197127324 -0.2683584658  0.0487320315 -0.06206444 -0.367092546
[30,] -0.089745164  0.0348900477  0.0139841398  0.04933249 -0.034584420
[31,]  0.217162094 -0.1668069152  0.1585038194 -0.23097806  0.138162324
[32,]  0.080284021 -0.0379183151 -0.1966031382 -0.05139139  0.048958139
[33,] -0.187418155 -0.0762541603 -0.0460621960  0.02493730 -0.099627545
[34,] -0.196632023  0.1957248038  0.1228955621  0.02848968 -0.368780317
[35,]  0.089749909  0.3050936576  0.0004155838 -0.42447525 -0.070759808
[36,]  0.203597809 -0.1878196348  0.0624564958  0.11330999 -0.133260770
[37,] -0.051601091  0.2322390049 -0.1856901054  0.11559402  0.090593822
[38,] -0.229945498 -0.1338359623  0.2083725429 -0.19506462 -0.111049767
[39,]  0.092251226  0.0630984759 -0.0212128281  0.23065965  0.195879030
[40,] -0.311167064 -0.0321359755  0.1218812603 -0.08324526  0.010208762

learning from https://uc-r.github.io/pca

library(tidyverse)  # data manipulation and visualization
library(gridExtra)  # plot arrangement

Attaching package: 'gridExtra'
The following object is masked from 'package:dplyr':

    combine
data("USArrests")
head(USArrests, 10)
            Murder Assault UrbanPop Rape
Alabama       13.2     236       58 21.2
Alaska        10.0     263       48 44.5
Arizona        8.1     294       80 31.0
Arkansas       8.8     190       50 19.5
California     9.0     276       91 40.6
Colorado       7.9     204       78 38.7
Connecticut    3.3     110       77 11.1
Delaware       5.9     238       72 15.8
Florida       15.4     335       80 31.9
Georgia       17.4     211       60 25.8
scaled_df <- apply(USArrests, 2, scale)

arrests.cov <- cov(scaled_df)
arrests.eigen <- eigen(arrests.cov)

arrests.cov
             Murder   Assault   UrbanPop      Rape
Murder   1.00000000 0.8018733 0.06957262 0.5635788
Assault  0.80187331 1.0000000 0.25887170 0.6652412
UrbanPop 0.06957262 0.2588717 1.00000000 0.4113412
Rape     0.56357883 0.6652412 0.41134124 1.0000000
arrests.eigen
eigen() decomposition
$values
[1] 2.4802416 0.9897652 0.3565632 0.1734301

$vectors
          [,1]       [,2]       [,3]        [,4]
[1,] 0.5358995  0.4181809 -0.3412327  0.64922780
[2,] 0.5831836  0.1879856 -0.2681484 -0.74340748
[3,] 0.2781909 -0.8728062 -0.3780158  0.13387773
[4,] 0.5434321 -0.1673186  0.8177779  0.08902432
temp_1 <- lm(scaled_df[,1] ~ scaled_df[,2])
cov(temp_1$residuals, scaled_df[,1])
[1] 0.3569992
cov(temp_1$residuals, scaled_df[,2])
[1] 5.328079e-17
str(arrests.eigen)
List of 2
 $ values : num [1:4] 2.48 0.99 0.357 0.173
 $ vectors: num [1:4, 1:4] 0.536 0.583 0.278 0.543 0.418 ...
 - attr(*, "class")= chr "eigen"
arrests.eigen
eigen() decomposition
$values
[1] 2.4802416 0.9897652 0.3565632 0.1734301

$vectors
          [,1]       [,2]       [,3]        [,4]
[1,] 0.5358995  0.4181809 -0.3412327  0.64922780
[2,] 0.5831836  0.1879856 -0.2681484 -0.74340748
[3,] 0.2781909 -0.8728062 -0.3780158  0.13387773
[4,] 0.5434321 -0.1673186  0.8177779  0.08902432
arrests.eigen$values
[1] 2.4802416 0.9897652 0.3565632 0.1734301
arrests.eigen$vectors
          [,1]       [,2]       [,3]        [,4]
[1,] 0.5358995  0.4181809 -0.3412327  0.64922780
[2,] 0.5831836  0.1879856 -0.2681484 -0.74340748
[3,] 0.2781909 -0.8728062 -0.3780158  0.13387773
[4,] 0.5434321 -0.1673186  0.8177779  0.08902432
(beep <- arrests.eigen$vectors[,1:2])
          [,1]       [,2]
[1,] 0.5358995  0.4181809
[2,] 0.5831836  0.1879856
[3,] 0.2781909 -0.8728062
[4,] 0.5434321 -0.1673186
beep <- prcomp(scaled_df)
beep$rotation <- beep$rotation * -1
beep$rotation <- beep$x * -1

biplot(beep, scale = 0)

To start