In this paper we present a new algorithm for fast discriminative training based on Minimum Classification Error (MCE). This algorithm is intended for continuous speech recognition using HMMs, where the gradient of the objective function is almost cancelled in each step. We also introduce a new objective function to solve some problems identified in the traditional sigmoid function used in MCE. We derived the closed-form formulas of the algorithm, to estimate all the parameters of the HMMs. This new algorithm is compared against resilient backpropagation in a continuous speech recognition task, using HMMs to model monophones, triphones and words. After several experiments we concluded that the proposed algorithm completely outperforms the gradient descent method.