Many rotor assemblies of industrial turbomachines are supported by oil-lubricated bearings. It is well known that the operation safety of these machines is highly dependent on rotors whose stability is closely related to the whirling motion of lubricant oil. In this paper, the problem of transverse motion of rotor systems considering bearing nonlinearity is revisited. A symmetric, rigid Jeffcott rotor is modeled considering unbalanced mass and short bearing forces. A semi-analytical, seminumerical approach is presented based on the generalized harmonic balance method (GHBM) and the Newton–Raphson iteration scheme. The external load of the system is decomposed into a Fourier series with multiple harmonic loads. The amplitude and phase with respect to each harmonic load are solved iteratively. The stability of the motion response is analyzed through identification of eigenvalues at the fixed point mapped from the linearized system using harmonic amplitudes. The solutions of the present approach are compared to those from time-domain numerical integrations using the Runge–Kutta method, and they are found to be in good agreement for stable periodic motions. It is revealed through bifurcation analysis that evolution of the motion in the nonlinear rotor-bearing system is complicated. The Hopf bifurcation (HB) of synchronous vibration initiates oil whirl with varying mass eccentricity. The onset of oil whip is identified when the saddle-node bifurcation of subsynchronous vibration takes place at the critical value of parameter.