The challenge of this paper is to extend the concept of discriminative training to a hybrid ANN/HMM phoneme recognition system. The main goal is to improve phoneme accuracy in the aligned output string, instead of in the Multi Layer Perceptron (MLP) output, as usually done. The method uses the difference between the reference and the best acoustic likelihood of the observation sequences to update the MLP weights. Results are presented for the TIMIT phoneme recognition task and show that this method leads to significant improvements, comparing to the baseline.