Neural networks are employeed as part of an adaptive critic controller. The neural networks give the adaptive critic the ability to adapt to unmodeled dynamics. The neural networks are designed using knowledge of the nonlinear system. The training method for the neural networks incorporates that same knowledge during adaptation.