TY - JOUR

T1 - Comparing the frequentist and Bayesian periodic signal detection: rates of statistical mistakes and sensitivity to priors

AU - Baluev, Roman V.

PY - 2022/6/1

Y1 - 2022/6/1

N2 - We perform extensive Monte Carlo simulations to systematically compare the frequentist and Bayesian treatments of the Lomb–Scargle periodogram. The goal is to investigate whether the Bayesian period search is advantageous over the frequentist one in terms of the detection efficiency, how much if yes, and how sensitive it is regarding the choice of the priors, in particular in case of a misspecified prior (whenever the adopted prior does not match the actual distribution of physical objects). We find that the Bayesian and frequentist analyses always offer nearly identical detection efficiency in terms of their trade-off between type-I and type-II mistakes. Bayesian detection may reveal a formal advantage if the frequency prior is non-uniform, but this results in only ∼1 per cent extra detected signals. In case if the prior was misspecified (adopting non-uniform one over the actual uniform) this may turn into an opposite advantage of the frequentist analysis. Finally, we revealed that Bayes factor of this task appears rather overconservative if used without a calibration against type-I mistakes (false positives), thereby necessitating such a calibration in practice.

AB - We perform extensive Monte Carlo simulations to systematically compare the frequentist and Bayesian treatments of the Lomb–Scargle periodogram. The goal is to investigate whether the Bayesian period search is advantageous over the frequentist one in terms of the detection efficiency, how much if yes, and how sensitive it is regarding the choice of the priors, in particular in case of a misspecified prior (whenever the adopted prior does not match the actual distribution of physical objects). We find that the Bayesian and frequentist analyses always offer nearly identical detection efficiency in terms of their trade-off between type-I and type-II mistakes. Bayesian detection may reveal a formal advantage if the frequency prior is non-uniform, but this results in only ∼1 per cent extra detected signals. In case if the prior was misspecified (adopting non-uniform one over the actual uniform) this may turn into an opposite advantage of the frequentist analysis. Finally, we revealed that Bayes factor of this task appears rather overconservative if used without a calibration against type-I mistakes (false positives), thereby necessitating such a calibration in practice.

KW - methods: data analysis

KW - methods: statistical

KW - surveys

UR - https://www.mendeley.com/catalogue/99885a89-449b-33dc-9945-1df8f68ae08a/

UR - http://www.scopus.com/inward/record.url?scp=85130062707&partnerID=8YFLogxK

U2 - 10.1093/mnras/stac762

DO - 10.1093/mnras/stac762

M3 - Article

VL - 512

SP - 5520

EP - 5534

JO - Monthly Notices of the Royal Astronomical Society

JF - Monthly Notices of the Royal Astronomical Society

SN - 0035-8711

IS - 4

ER -