The optical conductivity of a d-CDW conductor is calculated for electrons on a square lattice and a nearest-neighbor charge-charge interaction using the lowest-order conserving approximation. The spectral properties of the Drude-like peak at low frequencies and the broad hump due to transitions across the gap at large frequencies are discussed, also as a function of temperature and of the second-nearest neighbor hopping term t′. We find that vertex corrections enhance the dc conductivity, make the Drude peak narrower and provide a smooth transition from a renormalized regime at low to the bare theory at high frequencies. It is also shown that vertex corrections enhance the temperature dependence of the restricted optical sum leading to a non-negligible violation of the sum rule in the d-CDW state.