We consider quantum mechanics with non-Hermitian quasi-diagonalizable Hamiltonians, i.e. the Hamiltonians having a number of Jordan cells, in particular, biorthogonal bases. The 'self-orthogonality' phenomenon is clarified in terms of a correct spectral decomposition and it is shown that 'self-orthogonal' states never jeopardize a resolution of identity and thereby quantum averages of observables. The example of a complex potential leading to one Jordan cell in the Hamiltonian is constructed and its origin from level coalescence is elucidated. Some puzzles with zero-binorm bound states in a continuous spectrum are unravelled with the help of a correct resolution of identity.