Rocket science, is the study of Rockets and rocketry. The physics of rockets is discussed in the spacecraft propulsion article.
Rocket science is a slang term for anything complicated. For example, in financial circles, quantitative analysts are sometimes called rocket scientists.