Pusa sibirica, the Baikal seal, is the only extant, exclusively freshwater, pinniped species. The pending issue is, how and when they reached their current habitat—the rift lake Baikal, more than three thousand kilometers away from the Arctic Ocean. To explore the demographic history and genetic diversity of this species, we generated a de novo chromosome-length assembly, and compared it with three closely related marine pinniped species. Multiple whole genome alignment of the four species compared with their karyotypes showed high conservation of chromosomal features, except for three large inversions on chromosome VI. We found the mean heterozygosity of the studied Baikal seal individuals was relatively low (0.61 SNPs/kbp), but comparable to other analyzed pinniped samples. Demographic reconstruction of seals revealed differing trajectories, yet remarkable variations in Ne occurred during approximately the same time periods. The Baikal seal showed a significantly more severe decline relative to other species. This could be due to the difference in environmental conditions encountered by the earlier populations of Baikal seals, as ice sheets changed during glacial–interglacial cycles. We connect this period to the time of migration to Lake Baikal, which occurred ~3–0.3 Mya, after which the population stabilized, indicating balanced habitat conditions.