Nickel orthoborate Ni3(BO3)2 having a complex orthorhombic structure Pnnm (No. 58, Z=2) of the kotoite type is known for quite a long time as an antiferromagnetic material below TN=46K, but up to now its physical properties including the lattice dynamics have not been explored. Six [NiO6] units of 2a and 4f types are linked via rigid [BO3] groups and these structural particularities impose restrictions on the lattice dynamics and spin-phonon interactions. We performed the symmetry analysis of the phonon modes at the center of the Brillouin zone. The structural parameters and phonon modes were calculated using the dmol3 program. We report and analyze results of infrared and Raman studies of phonon spectra measured in all required polarizations. Most of the even and odd phonons predicted on the basis of the symmetry analysis and theoretical calculations were reliably identified in the measured spectra. Clear evidence of the spin-phonon interaction was found for some particular phonons below TN. An unexpected emergence of several very narrow and weak phonon lines was observed in the infrared absorption spectra exactly at the magnetic ordering temperature TN. Moreover, anomalous behavior was found for some Raman phonons. The emergence of new phonon modes in the infrared and Raman spectra exactly at TN proves the existence of a magnetostructural phase transition of a new type in Ni3(BO3)2. A possible nature of this transition is discussed.