Two-particle angular correlations between trigger particles in the forward pseudorapidity range (2.5 <vertical bar eta vertical bar <4.0) and associated particles in the central range(vertical bar eta vertical bar <1.0) are measured with the ALICE detector in p-Pb collisions at a nucleon-nucleon centre-of-mass energy of 5.02 TeV. The trigger particles are reconstructed using the muon spectrometer, and the associated particles by the central barrel tracking detectors. In high-multiplicity events, the double-ridge structure, previously discovered in two-particle angular correlations at midrapidity, is found to persist to the pseudorapidity ranges studied in this Letter. The second-order Fourier coefficients for muons in high-multiplicity events are extracted after jet-like correlations from low-multiplicity events have been subtracted. The coefficients are found to have a similar transverse momentum(p(T)) dependence in p-going (p-Pb) and Pb-going (Pb-p) configurations, with the Pb-going coefficients larger b