This note is devoted to parameter estimation in linear regression and filtering, where the observation noise does not possess any "nice" probabilistic properties. In particular, the noise might have an "Unknown-but-bounded" deterministic nature. The basic assumption is that the model regressors (inputs) are random. Optimal rates of convergence for the modified stochastic approximation and least-squares algorithms are established under some weak assumptions. Typical behavior of the algorithms in the presence of such deterministic noise is illustrated by numerical examples.