PPMAC jogs in wrong direction, throws spurious following errors

[mshaver]

Problem 1: I've had several instances now where either the PPMAC or the IDE gets in a mode whereby pressing either the Jog+ or the Jog- button on the jog ribbon causes the axis to jog negative. Regardless of which Jog button I pressed the axis moved in the negative direction but the indicated position counted upward in the positive direction. Eventually the count reached the positive soft limit and the positive jog button ceased to function as would be expected. However, the negative jog button continued to function causing the axis to move in the negative direction, however the displayed position continued to increment in the positive direction. Eventually I hit the negative limit switch and could now not move the axis in either direction. I am using stepper motors on linear stages with linear encoders driven by a Delta Tau 24E2A axis card. While the system was in this condition, I put a differential scope on the direction signal leads from the axis card to the stepper drive. The axis card was putting out the same direction polarity regardless of which Jog button I pressed. The system had been working fine for months and continuously for several hours today when it got in this mode.

 

I eventually killed the DC power to just the stepper drives (but not the PPMAC) and manually turned the motor to get the stage to the physical home position, then issued a HomeZ command. For whatever reason, the axis began to work properly in all ways, ie; I observed the direction output polarity switching properly according to which jog button was pressed and everything worked as expected.

 

Being concerned about what triggered this, I tried to reproduce this by injecting a fixed polarity signal into the direction input of the drive so that the motor will move the same direction regardless of which jog button is pressed. This does not produce the same behavior as above. The system jogs properly with one jog button and immediately shows a fatal following error in the other direction as one would expect.

 

I've been unable to reproduce this behavior by deliberately messing with the step and direction signals or any of my hard wired safety logic. The fact that the 24E2A was sending step pulses but not changing the direction polarity regardless of which jog button was pressed and the fact that the displayed position moved upward despite the actual direction being negative when either jog button was pressed leads me to beleive this is a Delta Tau hardware/firmware problem. Anything I do to deliberately make the system run negative when the +jog button is pressed is immediately recognized by the Delta Tau and it throws a following error as it should.

 

Problem 2: I've had other intermittent situations where one particular axis will get in a mode where it will throw a series of apparently spurious Fatal Following Errors until power is cycled which then magically makes the problem go away. Once this problem starts to occur, about one time out of two moves the axis starts to move in the correct direction and then stops with a fatal following error. The system will go for days without this problem occuring, then once the problem crops up the system fails on following error about 50% of the moves until power is cycled.

 

It's as though the axis suddenly is experiencing much higher mechanical loading or the speed and acceleration has been turned way up, or the .FatalFeLimit for this axis has been turned way down.

 

I do not hear any bumping, grinding, or the characteristic chirping of a stepper motor missing pulses because the accel and/or speed is too high for the mechanical intertia. I've been unable to find any reason for the Fatal Following Errors. I have the following error limit set at 20. I'm using 1 micron linear encoders. When I use the tuning screen to move the axis duplicating the move parameters that throw fatal following errors, I'm not seeing any following errors approaching 20. Maximum following error during a move is typically under 4. I've recorded the encoder pulses on a recording scope and I don't see any problems with the encoder such as dead spots, or areas that might indicate a damaged lead screw, etc. I've turned the lead screw by hand and cannot feel any rough or sticking spots. Nothing is jammed, hitting, or dragging. I've tried deliberately adding mechanical load to the axis and using the tuning screen to move it and I conclude that I am only using a fraction of the available torque, ie; I am unable to increase the following error significantly by adding any reasonable mechanical loading.

 

The only way I can deliberately come close to duplicating this behavior is to turn up the speed and acceleration rediculously high. Then on the tuning screen I can see the following error approaching 18 to 20 with an occaisonal fatal following error. However, in this scenario, I can also hear the chirping of the motor losing step pulses because of the high acceleration. I do not hear this when I get the real problem.

 

Because I can't duplicate this mechanically and I don't hear chirping or feel any rough spots when turning the stage by hand, and because I don't see any anomolies in the encoder pulses, logic leads me to suspect that there is a glitch in the PPMAC itself or I have a bug in my code that is somehow setting the .FatalFeLimit to a low value, for example, until I cycle power which then resets to my default values. So far, I've not been able to find any way in my code that the .FatalFeLimit is getting changed and when I poll this value from the terminal window it is still set at 20.

 

Any idea what's going on with these two issues?

[steve.milici]

This is something I have not seen. The next time it occurs contact me for some techsupport: 818-717-5658

[Richard Naddaf]

Problem 1: You may have been outside of the limit (hard or soft).

 

Problem 2: Could be anywhere from tuning, commanded move(s), to encoder count error.

Is your hardware sampling clock turned up?

 

I suggest contacting us for a web or phone call with these issues.

[mshaver]

Problem 1: You may have been outside of the limit (hard or soft).

 

Problem 2: Could be anywhere from tuning, commanded move(s), to encoder count error.

Is your hardware sampling clock turned up?

 

I suggest contacting us for a web or phone call with these issues.

 

Response To Richard From M. Shaver:

Problem 1: At some point I was definitely beyond the positive software limit, according to the PPMAC, but in reality, the axis was moving negative. I think this is a symptom, not a cause. When the problem first started, I was well within all limits.

 

Problem 2: Please tell me more about the hardware sampling clock. I have not adjusted anything in the way of clocks. Whatever the system defaulted to is the way it is. Please give me some specific variable names or structure elements to check and I'll report back on them.

 

For both issues, I wrote up a detailed troubleshooting procedure to try to capture more information when this happens again. How should I get this information to Delta Tau if I think it's something you might be able to make sense of?

 

Thanks for your help.

 

My email address is mshaver@pekoprecision.com.

 

Thanks;

[steve.milici]

Send all collected information to support@deltatau.com ATTN: Steve Milici.

[steve.milici]

Keep in mind if you somehow get past the software limit a motor command will move you back to the limit.