Nanostructures of three binary Al-Mg alloys and a commercial AA5182 alloy subjected to high pressure torsion at room temperature were comparatively investigated using transmission electron microscopy, high-resolution transmission electron microscopy, and X-ray line profile analysis. Grain size distributions, dislocation densities, and densities of planar defects including stacking faults and microtwins were quantified. The average subgrain size decreased considerably from 120 to 55 nm as the Mg content increased from 0.5 to 4.1 wt%. The average dislocation density in the alloys first increased to a maximum and then decreased as the Mg content increased and the average subgrain size decreased. The role of Mg solute on these features and the refinement mechanisms associated with the typical nanostructures and faults were interpreted.