We study an ultracold gas of neutral atoms subject to the periodic optical potential generated by a high-Q cavity mode. In the limit of very low temperatures, cavity field and atomic dynamics require a quantum description. Starting from a cavity QED single atom Hamiltonian we use different routes to derive approximative multiparticle Hamiltonians in Bose-Hubbard form with rescaled or even dynamical parameters. In the limit of large enough cavity damping the different models agree. Compared to free space optical lattices, quantum uncertainties of the potential and the possibility of atom-field entanglement lead to modified phase transition characteristics, the appearance of new phases or even quantum superpositions of different phases. Using a corresponding effective master equation, which can be numerically solved for few particles, we can study time evolution including dissipation. As an example we exhibit the microscopic processes behind the transition dynamics from a Mott insulator like state to a self-ordered superradiant state of the atoms, which appears as steady state for transverse atomic pumping.