This paper reports a theoretical study of axial compressor surge. A nonlinear model is developed to predict the transient response of a compression system subsequent to a perturbation from steady operating conditions. It is found that for the system investigated there is an important nondimensional parameter on which this response depends. Whether this parameter is above or below a critical value determines which mode of compressor instability, rotating stall or surge, will be encountered at the stall line. For values above the critical, the system will exhibit the large amplitude oscillatory behavior characteristic of surge; while for values below the critical it will move toward operation in rotating stall, at a substantially reduced flow rate and pressure ratio. Numerical results are presented to show the motion of the compression system operating point during these two basic modes of instability, and a physical explanation is given for the mechanism associated with the generation of surge cycle oscillations.