In the declining phase of the solar cycle (SC), when the new-polarity fields of the solar poles are strengthened by the transport of same-signed magnetic flux from lower latitudes, the polar coronal holes expand and form non-axisymmetric extensions toward the solar equator. These extensions enhance the occurrence of high-speed solar wind (SW) streams (HSS) and related co-rotating interaction regions in the low-latitude heliosphere, and cause moderate, recurrent geomagnetic activity (GA) in the near-Earth space. Here, using a novel definition of GA at high (polar cap) latitudes and the longest record of magnetic observations at a polar cap station, we calculate the annually averaged SW speeds as proxies for the effective annual occurrence of HSS over the whole Grand Modern Maximum (GMM) from 1920s onward. We find that a period of high annual speeds (frequent occurrence of HSS) occurs in the declining phase of each of SCs 16-23. For most cycles the HSS activity clearly reaches a maximum in one year, suggesting that typically only one strong activation leading to a coronal hole extension is responsible for the HSS maximum. We find that the most persistent HSS activity occurred in the declining phase of SC 18. This suggests that cycle 19, which marks the sunspot maximum period of the GMM, was preceded by exceptionally strong polar fields during the previous sunspot minimum. This gives interesting support for the validity of solar dynamo theory during this dramatic period of solar magnetism.