The Carboniferous is characterized by drastic climatic and environmental fluctuations, which include multiple phases of glaciation resulting in an icehouse climate. Additionally, dynamic continental reconfigurations forced the contraction of the Rheic Ocean resulting in the closure of the Rheic–Tethyan Gateway, which precluded further faunal exchanges between the North American and Eurasian marine realms. Interestingly, cartilaginous fishes seem to be relatively immune to these drastic climatic and environmental changes. The Eurasian fossil record of Paleozoic sharks is strongly biased towards intensively sampled localities from England, Ireland, Scotland, and the Russian Platform. Here we present rare dental material from the Serpukhovian (early Carboniferous) of Austria, adding new information to the paleogeographic distribution of ctenacanthiform sharks. The new material revealed the first record of the genus Saivodus in Central Europe and allowed us to recognize a new species, Cladodus gailensis sp. nov., and a remnant of fossilized cartilage. In an attempt to identify possible linkages between climatic or environmental fluctuations on shark diversity throughout the Carboniferous, we provide a synopsis of the distribution and diversity of elasmobranchs based on primary literature. This preliminary assessment at genus level indicates two pronounced events of extinction, with the first one occurring during the latest Mississippian and the second one towards the end of the Pennsylvanian. The first extinction event distinctly correlates with the known diversity decline of other marine inhabitants and the second occurred during an unstable period of multiple phases of glaciation.