Admixtools 2

[Арсен]

I guess this is a bug in Admixtools2.  Doesn't matter what dataset we use. We should get the same or near the same results if we specify the same criteria. 
But as we see: running the same script - same options - the results are disaster.
This is only for groups statistics.
When I run F4 on individuals I get perfect and stable results.
The bug is definately in Admixtools.
Thanks for confirming that you already have seen that.

dude, teach me how to use this program, question, do I have to download heavy bam ENA files from data library sites for this? and do you need a serious, powerful processor to calculate mixtures?

There is already such topic in Eupedia. Start from there.

https://www.eupedia.com/forum/threads/ad...ows.42684/

Regarding converting converting  bam to plink format - this is another topic. It is a different process. For now try to make your software working with the already available data. (in plink bed,bim fam  format).

Thank you very much for your answer, I think I’ll figure it out in the future

[Kale]

do I have to download heavy bam ENA files from data library sites for this? and do you need a serious, powerful processor to calculate mixtures?

No and no. My computer is 19 years old.

[TanTin]

For the Admixtools2 bug again. Here is another example. Several runs for the same data, same test.

[AimSmall]

What version of AT2 are you using? Platform?, though it shouldn't matter.

sessionInfo()

[1] admixtools_2.0.0

[TanTin]

What version of AT2 are you using?  Platform?, though it shouldn't matter.

sessionInfo()

[1] admixtools_2.0.0

> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                    LC_TIME=English_Canada.1252   
system code page: 65001

attached base packages:
[1] stats    graphics  grDevices utils    datasets  methods  base   

other attached packages:
[1] Hmisc_5.1-0      forcats_0.5.1    stringr_1.4.0    dplyr_1.0.7      purrr_0.3.4      readr_2.1.1      tidyr_1.1.4      tibble_3.1.6    ggplot2_3.3.5    tidyverse_1.3.1  admixtools_2.0.0

loaded via a namespace (and not attached):
[1] Rcpp_1.0.8        lubridate_1.8.0  digest_0.6.29    assertthat_0.2.1  foreach_1.5.1    utf8_1.2.2        R6_2.5.1          cellranger_1.1.0  backports_1.4.1  reprex_2.0.1      evaluate_0.14   
[12] httr_1.4.2        pillar_1.6.5      rlang_1.0.0      readxl_1.3.1      data.table_1.14.2 rstudioapi_0.13  rpart_4.1-15      checkmate_2.1.0  rmarkdown_2.11    foreign_0.8-81    htmlwidgets_1.5.4
[23] igraph_1.2.11    munsell_0.5.0    broom_0.7.11      xfun_0.29        compiler_4.1.2    modelr_0.1.8      pkgconfig_2.0.3  base64enc_0.1-3  htmltools_0.5.2  nnet_7.3-16      tidyselect_1.1.1
[34] gridExtra_2.3    htmlTable_2.4.1  codetools_0.2-18  fansi_1.0.2      crayon_1.4.2      tzdb_0.2.0        dbplyr_2.1.1      withr_2.4.3      grid_4.1.2        jsonlite_1.7.3    gtable_0.3.0   
[45] lifecycle_1.0.1  DBI_1.1.2        magrittr_2.0.2    scales_1.1.1      cli_3.1.1        stringi_1.7.6    fs_1.5.2          xml2_1.3.3        ellipsis_0.3.2    generics_0.1.1    vctrs_0.3.8     
[56] Formula_1.2-5    iterators_1.0.13  tools_4.1.2      glue_1.6.1        hms_1.1.1        fastmap_1.1.0    abind_1.4-5      colorspace_2.0-2  cluster_2.1.2    rvest_1.0.2      knitr_1.37     
[67] haven_2.4.3

[AimSmall]

So, apparently version 2.01 was released on March 3rd, 2024.

https://github.com/uqrmaie1/admixtools/r...tag/v2.0.1

You can install the packages normally, or you might consider compiling from source.

Running devtools::install_github("uqrmaie1/admixtools") will compile C++ from source code (this isn’t necessary when installing packages from CRAN with install.packages()).

https://github.com/uqrmaie1/admixtools/

Not sure if your bug is corrected here, you might be able to check the changelog.

[TanTin]

So, apparently version 2.01 was released on March 3rd, 2024.

https://github.com/uqrmaie1/admixtools/r...tag/v2.0.1

You can install the packages normally, or you might consider compiling from source.

Running devtools::install_github("uqrmaie1/admixtools") will compile C++ from source code (this isn’t necessary when installing packages from CRAN with install.packages()).

https://github.com/uqrmaie1/admixtools/

Not sure if your bug is corrected here, you might be able to check the changelog.

Thanks for the help !

Except these bugs that I see at random  my Admixtools is running pretty good. I have it also installed on the second computer, so I may do another test there. And I can try to run another version in VM under Linux.
For the moment I don't want to uninstall /reinstall it here, because in general it is running fine.  Will see if the same error will reproduce under different installation.

[TanTin]

I was able to find what is causing the bug...

As you know, Admixtools2 may work by using both plink and geno file format as  source data...
So I was running my Admixtools with plink format files (.bed .bim .fam files.)
And it seems this is what's  causing such bug.  When I did the rerun on the second computer I used other file format (.geno, .ind, .snp files)  The results are stable ,  same results on each rerun.

As soon as I changed the prefix to use  plink(.bed .bim .fam files.)  I got the same bug on the second computer as well.

Here are the final results for my last F4 tests (stable).

Show Content

Spoiler
> R17DM[1:20,]
# A tibble: 20 x 9
  pop1        pop2                  pop3                pop4                          est      se      z          p      n
  <chr>      <chr>                  <chr>              <chr>                        <dbl>    <dbl>  <dbl>      <dbl>  <dbl>
1 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Altai_Neanderthal.DG    0.0479    0.000545  87.9    0        1060509
2 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Russia_Ust_Ishim.DG      0        0        NaN    NaN        1060690
3 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Russia_MA1_HG.SG        -0.000426  0.000329  -1.29    1.96e- 1  746081
4 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Luxembourg_Loschbour.DG -0.0000887 0.000306  -0.290  7.72e- 1 1060579
5 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Bulgaria_N              -0.000351  0.000294  -1.19    2.34e- 1  868705
6 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Iran_GanjDareh_N        -0.000601  0.000287  -2.09    3.64e- 2  980813
7 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Iran_HajjiFiruz_N      -0.000140  0.000287  -0.487  6.27e- 1  934961
8 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Georgia_Kotias.SG      -0.000253  0.000323  -0.782  4.34e- 1 1057249
9 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Georgia_Satsurblia.SG  -0.000531  0.000344  -1.54    1.23e- 1  629462
10 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Israel_Natufian        -0.000727  0.000456  -1.59    1.11e- 1  209038
11 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Israel_Natufian_contam  -0.000332  0.000369  -0.900  3.68e- 1  434546
12 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG South_Africa_400BP.SG  -0.00185  0.000295  -6.28    3.32e-10 1060671
13 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG South_Africa_1900BP.SG  -0.00157  0.000325  -4.82    1.46e- 6 1060489
14 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Saharawi.DG            -0.000559  0.000277  -2.02    4.37e- 2 1045888
15 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Lebanon_MBA.SG          -0.000412  0.000278  -1.48    1.38e- 1 1026497
16 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Israel_MLBA            -0.000388  0.000261  -1.49    1.36e- 1  957438
17 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Israel_PPNB            -0.000564  0.000355  -1.59    1.12e- 1  689573
18 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Morocco_Iberomaurusian  -0.000946  0.000294  -3.21    1.31e- 3  998410
19 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Ethiopia_4500BP.SG      -0.00182  0.000313  -5.82    5.95e- 9 1060616
20 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG JPT.SG                  -0.000195  0.000252  -0.774  4.39e- 1 1060689
> R17DM[21:40,]
# A tibble: 20 x 9
  pop1        pop2                  pop3                pop4                                      est      se        z        p      n
  <chr>      <chr>                  <chr>              <chr>                                  <dbl>    <dbl>    <dbl>    <dbl>  <dbl>
1 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Malawi_Hora_LSA_15500BP            -0.00175  0.000526  -3.32  8.99e-  4  189881
2 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Malawi_Hora_LSA_8500BP            -0.00209  0.000898  -2.33  1.99e-  2  64131
3 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Malawi_Fingira_LSA_6000BP          -0.00171  0.000452  -3.79  1.48e-  4  326047
4 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Malawi_Fingira_LSA_2500BP          -0.00204  0.000331  -6.16  7.22e- 10  926878
5 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Malawi_Chencherere_LSA_5200BP      -0.00122  0.000504  -2.41  1.60e-  2  222739
6 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Denisova.DG                        -0.0609    0.000542 -112.    0        1060690
7 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Vindija_Neanderthal.DG              0.0577    0.000565  102.    0        1060690
8 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG DenisovaNeanderthalMix.SG          -0.00333  0.000403  -8.27  1.29e- 16  804333
9 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Ignore_Yoruba(discovery).DG        -0.00151  0.000310  -4.88  1.09e-  6 1060439
10 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Bulgaria_BachoKiro_LatePleistocene  0.000686  0.000308    2.23  2.61e-  2 1002219
11 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Turkey_N                          -0.000324  0.000259  -1.25  2.12e-  1 1049500
12 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Micronesia_Pohnpei_400BP          -0.000667  0.000283  -2.36  1.83e-  2  931991
13 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Ignore_Mbuti(discovery).DG        -0.00211  0.000311  -6.77  1.30e- 11 1060337
14 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Romania_Oase_UP                    0.00189  0.000589    3.21  1.34e-  3  184370
15 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Romania_Oase_UP_enhanced            0.00170  0.000451    3.78  1.59e-  4  361736
16 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Ignore_Ju_hoan_North(discovery).DG  0.0000156 0.000327    0.0476 9.62e-  1 1060245
17 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Chimp.REF                          -0.0120    0.000375  -32.1    4.60e-226 1020079
18 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Iranian.DG                        -0.000343  0.000275  -1.25  2.13e-  1 1045889
19 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Mozabite.DG                        -0.000318  0.000284  -1.12  2.62e-  1 1045797
20 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Ju_hoan_North.DG                  -0.00151  0.000296  -5.12  3.01e-  7 1045911
> R17DM[41:50,]
# A tibble: 10 x 9
  pop1        pop2                  pop3                pop4                                est      se      z        p      n
  <chr>      <chr>                  <chr>              <chr>                              <dbl>    <dbl>  <dbl>    <dbl>  <dbl>
1 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Egypt_ThirdIntermediatePeriod -0.000272  0.000354 -0.768 4.42e- 1  517355
2 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Morocco_EN.SG                -0.000822  0.000361 -2.28  2.27e- 2  521366
3 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Morocco_LN.SG                -0.0000868 0.000407 -0.213 8.31e- 1  276886
4 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Morocco_unknown.SG            -0.00143  0.000919 -1.56  1.20e- 1  46552
5 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Palestinian.DG                -0.000526  0.000263 -2.00  4.58e- 2 1045912
6 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Turkey_N.SG                  -0.000371  0.000277 -1.34  1.80e- 1 1060617
7 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Mbuti.DG                      -0.00196  0.000284 -6.90  5.17e-12 1045919
8 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Mende.DG                      -0.00167  0.000287 -5.84  5.32e- 9 1045824
9 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Yoruba.DG                    -0.00173  0.000282 -6.16  7.23e-10 1045917
10 Denisova.DG Vindija_Neanderthal.DG Russia_Ust_Ishim.DG Mezmaiskaya2_Neanderthal.SG    0.0499    0.000559 89.3  0        766512

[AimSmall]

That's interesting....

[TanTin]

Here are the same results, visualizing Z.

[Jerome]

do I have to download heavy bam ENA files from data library sites for this? and do you need a serious, powerful processor to calculate mixtures?

No and no. My computer is 19 years old.

Kale,can admixtools1 be run smoothly on a device with 4gb ram and i3 processor?
That's like the minimum,right?
I am not talking about admixtools2 but about 1.

[Kale]

Kale,can admixtools1 be run smoothly on a device with 4gb ram and i3 processor?
That's like the minimum,right?
I am not talking about admixtools2 but about 1.

1 crucial difference between Admixtools1 and 2 is that 1 needs to load the entire dataset in order to run any analysis, whereas admixtools2 doesn't.
You can certainly get away with using Admixtools1 with low resources, but you will need to make smaller datasets with just a batch of samples you are interested in (200 samples runs quick enough, 5000 samples is a nightmare, I haven't tried anything in between)

[Aramu]

I can confirm what TanTin says.

When I tried to use the packedped (bed/bim/fam) version of AADR 1240k in Admixtools 2 ( running in Rstudio/Windiws11) I got very erratic results. In each run of qpadm I was getting a different result. But when the same file was converted back to eigenstrat then it worked perfectly.

[TanTin]

There is a newer version of Admixtools 2 where a small fix has been applied so now this issue should be fixed.
To update your Admixtools2 try: install_github() or devtools::install_github("uqrmaie1/admixtools") depending how it is installed on your system.
With this update Admixtools 2 should be running smootly with plink files without any issues.
The new version that I have after the update is version 2.0.5.

[AimSmall]

There is a newer version of Admixtools 2  where a small fix has been applied so now this issue should be fixed.
To update your Admixtools2 try:  install_github()  or devtools::install_github("uqrmaie1/admixtools")  depending how it is installed on your system.
With this update Admixtools 2 should be running smootly with plink files  without any issues.
The new version that I have after the update is version 2.0.5.

changed permutation p-value correction in compare_fits
https://github.com/uqrmaie1/admixtools/c...c7d250d9dc

placed rightfix before right in qpadm_rotate
https://github.com/uqrmaie1/admixtools/c...a8c516124b