mshaver

Thanks Brad: One more question. You didn't mention anything about firmware so it sounds like there are not any known firmware bugs that would explain this and I don't need to upgrade firmware or anything like that? Thanks; mshaver

A Power PMAC appears to be freezing up or going to sleep. PMAC was purchased in spring of 2016. Symptoms: Symptom 1: Loss of Ethernet communications. Restarting the computer or the Windows app that is communicating with the PMAC does not reestablish communications. Cycling power to the PMAC does reestablish communications. Symptom 2: We've had two instances whereby the PMAC appears to have stopped running and has left its outputs in their last state. IE; in one case, the PMAC kept a digital output on despite the fact that the condition that caused the output to turn on having been reset. In another case, when the same input should have caused the PMAC to turn on an output, it did not do so. As soon as we cycled power to the PMAC the unit operated as expected. When we try to recreate this ourselves, of course, we cannot. Everything works as it should. Any advice on how to troubleshoot this? I've searched this forum for similar systems but the communications failures discussed herein don't seem relevant to what I'm seeing. Have there been any similar problems reported in the last few years? Are there any issues with firmware such that I need to update firmware? When we bought the unit in the spring of 2016 we used it "as is" without updating firmware. Thanks; MShaver

I am using a Axis Expansion Board ACC-24E2A (with a Power PMAC) to drive stepper motors using step and direction outputs. The step pulses from the ACC-24E2A are low for only about 4 microseconds. One of the amplifiers I am using needs considerably longer "off" times of about 80 or so microseconds in order for it to recognize this as a step pulse. I can determine exactly what it needs for off time using a signal generator if needed. The other 3 amplifiers are a different type and model and deal just fine with the 4 microsecond off times. Is there any way to lengthen the "off" time of the step pulses going just to the 4th axis outputs of the ACC-24E2A? Thanks

Response To Richard From M. Shaver: Problem 1: At some point I was definitely beyond the positive software limit, according to the PPMAC, but in reality, the axis was moving negative. I think this is a symptom, not a cause. When the problem first started, I was well within all limits. Problem 2: Please tell me more about the hardware sampling clock. I have not adjusted anything in the way of clocks. Whatever the system defaulted to is the way it is. Please give me some specific variable names or structure elements to check and I'll report back on them. For both issues, I wrote up a detailed troubleshooting procedure to try to capture more information when this happens again. How should I get this information to Delta Tau if I think it's something you might be able to make sense of? Thanks for your help. My email address is mshaver@pekoprecision.com. Thanks;

Problem 1: I've had several instances now where either the PPMAC or the IDE gets in a mode whereby pressing either the Jog+ or the Jog- button on the jog ribbon causes the axis to jog negative. Regardless of which Jog button I pressed the axis moved in the negative direction but the indicated position counted upward in the positive direction. Eventually the count reached the positive soft limit and the positive jog button ceased to function as would be expected. However, the negative jog button continued to function causing the axis to move in the negative direction, however the displayed position continued to increment in the positive direction. Eventually I hit the negative limit switch and could now not move the axis in either direction. I am using stepper motors on linear stages with linear encoders driven by a Delta Tau 24E2A axis card. While the system was in this condition, I put a differential scope on the direction signal leads from the axis card to the stepper drive. The axis card was putting out the same direction polarity regardless of which Jog button I pressed. The system had been working fine for months and continuously for several hours today when it got in this mode. I eventually killed the DC power to just the stepper drives (but not the PPMAC) and manually turned the motor to get the stage to the physical home position, then issued a HomeZ command. For whatever reason, the axis began to work properly in all ways, ie; I observed the direction output polarity switching properly according to which jog button was pressed and everything worked as expected. Being concerned about what triggered this, I tried to reproduce this by injecting a fixed polarity signal into the direction input of the drive so that the motor will move the same direction regardless of which jog button is pressed. This does not produce the same behavior as above. The system jogs properly with one jog button and immediately shows a fatal following error in the other direction as one would expect. I've been unable to reproduce this behavior by deliberately messing with the step and direction signals or any of my hard wired safety logic. The fact that the 24E2A was sending step pulses but not changing the direction polarity regardless of which jog button was pressed and the fact that the displayed position moved upward despite the actual direction being negative when either jog button was pressed leads me to beleive this is a Delta Tau hardware/firmware problem. Anything I do to deliberately make the system run negative when the +jog button is pressed is immediately recognized by the Delta Tau and it throws a following error as it should. Problem 2: I've had other intermittent situations where one particular axis will get in a mode where it will throw a series of apparently spurious Fatal Following Errors until power is cycled which then magically makes the problem go away. Once this problem starts to occur, about one time out of two moves the axis starts to move in the correct direction and then stops with a fatal following error. The system will go for days without this problem occuring, then once the problem crops up the system fails on following error about 50% of the moves until power is cycled. It's as though the axis suddenly is experiencing much higher mechanical loading or the speed and acceleration has been turned way up, or the .FatalFeLimit for this axis has been turned way down. I do not hear any bumping, grinding, or the characteristic chirping of a stepper motor missing pulses because the accel and/or speed is too high for the mechanical intertia. I've been unable to find any reason for the Fatal Following Errors. I have the following error limit set at 20. I'm using 1 micron linear encoders. When I use the tuning screen to move the axis duplicating the move parameters that throw fatal following errors, I'm not seeing any following errors approaching 20. Maximum following error during a move is typically under 4. I've recorded the encoder pulses on a recording scope and I don't see any problems with the encoder such as dead spots, or areas that might indicate a damaged lead screw, etc. I've turned the lead screw by hand and cannot feel any rough or sticking spots. Nothing is jammed, hitting, or dragging. I've tried deliberately adding mechanical load to the axis and using the tuning screen to move it and I conclude that I am only using a fraction of the available torque, ie; I am unable to increase the following error significantly by adding any reasonable mechanical loading. The only way I can deliberately come close to duplicating this behavior is to turn up the speed and acceleration rediculously high. Then on the tuning screen I can see the following error approaching 18 to 20 with an occaisonal fatal following error. However, in this scenario, I can also hear the chirping of the motor losing step pulses because of the high acceleration. I do not hear this when I get the real problem. Because I can't duplicate this mechanically and I don't hear chirping or feel any rough spots when turning the stage by hand, and because I don't see any anomolies in the encoder pulses, logic leads me to suspect that there is a glitch in the PPMAC itself or I have a bug in my code that is somehow setting the .FatalFeLimit to a low value, for example, until I cycle power which then resets to my default values. So far, I've not been able to find any way in my code that the .FatalFeLimit is getting changed and when I poll this value from the terminal window it is still set at 20. Any idea what's going on with these two issues?

I need help with basic setup and addressing of an ACC-65E. I have a PPMAC with a ACC-24E2A in the first (non-CPU) slot and an ACC-65E in the second slot. From reading the ACC-65E User Manual, here's what I think I am supposed to do. >> Since this is the first ACC-65E in the rack, I think I am supposed to set ALL SW1 switches to ON. Is this correct? Please recalibrate me if not. >> To access the intputs and outputs, I declare pointers such as; PTR Input1->ACC65E[0].DataReg[0].0.1; PTR Input2->ACC65E[0].DataReg[0].1.1; etc. and PTR Output1->ACC65E[0].DataReg[3].0.1; PTR Output2->ACC65E[0].DataReg[3].1.1; Must I do anything else or is this it? Thanks; MShaver

Customer who wrote the Windows app is on site today and confirms that he was, in fact, using ppmacserver all along until very recently and that the version of his app that I was using until yesterday was based on ppmacserver. He recently switched to SSH and I now have a copy using this communication. We hope this cures the problem. Do we have a reason to be optimistic? Knowing what we now know, it seems pretty clear that an application that uses ppmacserver is going to have trouble when ppmacserver shuts down. Now that we've confirmed that the problem, did, in fact, involve the use of ppmacserver and that we've switched to SSH, do we have a reason to be optimistic that this will solve the problem? Were/are there known problems with ppmacserver shutting down that would explain this?

"curtwilson" asked; "Did you change Motor[x].Ctrl from the default of Sys.ServoCtrl to Sys.PidCtrl? If you selected this very basic PID algorithm, the deadband terms are not used. In the standard algorithm selected by Sys.ServoCtrl, the suggested deadband terms stop dithering at rest from fractional count errors even with a non-zero integral gain value." And MShaver is replying: No I did not change the setting. The setting is Sys.ServoCtrl and you either have bad information or different software or firmware. The deadband terms discussed above do not stop the dithering unless the integral gain term is zero. I'm sitting here with all the deadband terms set as described above and any non-zero integral value will eventually integrate out of the deadband and cause dithering. The only way to stop the dithering is to set the integral term to zero. So either your understanding is wrong, the documentation is wrong, or someone has introduced or fixed an error in the firmware. Do I need to update firmware possibly?

Attention Delta Tau Insiders: This is the same issue being described by MShaver in http://forums.deltatau.com/showthread.php?tid=1640 IE; vtmtnbiker's application he is describing problems with here is the application I'm referring to in my posts. I've determined that "ppmacserver" on the PPMAC is shutting down and this is the reason that vtmtnbiker's application can no longer communicate with the PPMAC. I don't know whether ppmacserver shutting down is the root cause or a symptom of something else. We need Delta Tau's help understanding why ppmacserver is shutting down and whether it is the root cause of why vtmtnbiker's app cannot communicate or whether vtmtnbiker's app is doing something to cause ppmacserver to shut down. M. Shaver

I started the thread below which Delta Tau has not responded to. http://forums.deltatau.com/showthread.php?tid=1626 ppmacserver is shutting down. When it does, my customer's windows app written using the Delta Tau Dot-Net communications library, is unable to communicate with the PPMAC until the PPMAC has power cycled which restarts ppmacserver. Obviously the IDE does not use ppmacserver because it continues to communicate fine. I don't know whether ppmacserver shutting down is the root cause of my customer's windows app losing communications or if it is a symptom of something the customer's app is doing causing ppmacserver to shut down. I need the delta tau guys to tell me what ppmacserver is, what can cause it to shut down, could the external Windows app be doing anything to cause it to shut down, etc. and help me get to the bottom of this. What is ppmacserver? Is it a communications server put there for the Dot-Net communications library to access? Apparently the Dot-Net communications library is not using gpascii. In any case, I need some help asap please. Thanks; M Shaver

Here is some very fresh information that I hope the Delta Tau experts can make some sense of: 1. The customer's windows application locked up several times today while using it to jog and move axis in the PPMAC for testing purposes. In each case, the following occurred. 2. We were able to shut down the windows app and restart it but it would not communicate with the PPMAC. 3. The IDE was running at the time and it continued to communicate with the PPMAC without any problems. 4. We shut down and restarted the Windows computer. 5. The customer's Windows app still would not communicate with the PPMAC but the IDE would. 6. We checked the PPMAC OS Processes from the IDE. > gppmac was running. > ppmacserver WAS NOT RUNNING. 7. We then cycled power to the PPMAC. 8. The IDE connected without incident. 9. The customer's Windows app connected and communicated normally. 10. We checked the PPMAC OS Processes from the IDE. > BOTH gppmac AND ppmacserver ARE NOW RUNNING. > The customer's Windows app is running and communicating normally. So tell me about ppmacserver. What is it. What does it take to make it crash or shut down? Is the fact that it is disappearing a symptom or a cause of the problem with the customer's app? Below is some information being logged by the customer's app. I don't know if the innermost exception is being thrown by a function in the Delta Tau Dot-Net component classes or in the customer's code. ================================ First Excerpt From Application Log ================================ //This is an excerpt from one type of log that the customer's app creates. First part is normal logging of polling activities, then the exception gets thrown. 6/2/2014 12:46:14.186, Debug, DeltaTauPowerPmac, 0, < M4004=425999 6/2/2014 12:46:14.186, Info, DeltaTauPowerPmac, 0, CheckAxisStatus complete 6/2/2014 12:46:14.186, Info, DeltaTauPowerPmac, 0, CheckControllerStatus complete 6/2/2014 12:46:14.451, Debug, DeltaTauPowerPmac, 0, > M4000 // application polling PMAC for data in M4000 6/2/2014 12:46:14.451, Debug, DeltaTauPowerPmac, 0, < M4000=425995 // PMAC responding 6/2/2014 12:46:14.498, Debug, DeltaTauPowerPmac, 0, > M4001 6/2/2014 12:46:14.498, Debug, DeltaTauPowerPmac, 0, < M4001=425999 6/2/2014 12:46:14.498, Info, DeltaTauPowerPmac, 0, CheckAxisStatus complete 6/2/2014 12:46:18.881, Error, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4002 // Attepting to poll M4002 // ONCE THE ERROR ABOVE OCCURS, THE WINDOWS APPLICATION WILL BE UNABLE TO COMMUNICATE WITH THE PPMAC UNTIL POWER IS CYCLED TO THE PPMAC // THE IDE WILL CONTINUE TO COMMUNICATE WITHOUT ANY PROBLEMS. 6/2/2014 12:46:18.881, Info, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4002 6/2/2014 12:46:18.881, Debug, DeltaTauPowerPmac, 0, > M4002 6/2/2014 12:46:18.881, Exception, DeltaTauPowerPmac, 0, Exception in SendGetResponse: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index 6/2/2014 12:46:18.881, Error, DeltaTauPowerPmac, 0, CheckAxisStatus error 6/2/2014 12:46:22.953, Error, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4003 6/2/2014 12:46:22.953, Info, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4003 6/2/2014 12:46:22.953, Debug, DeltaTauPowerPmac, 0, > M4003 6/2/2014 12:46:22.953, Exception, DeltaTauPowerPmac, 0, Exception in SendGetResponse: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index 6/2/2014 12:46:22.953, Error, DeltaTauPowerPmac, 0, CheckAxisStatus error 6/2/2014 12:46:26.993, Error, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4004 6/2/2014 12:46:26.993, Info, DeltaTauPowerPmac, 0, Status = Failed Comm Error with Command: M4004 6/2/2014 12:46:26.993, Debug, DeltaTauPowerPmac, 0, > M4004 6/2/2014 12:46:26.993, Exception, DeltaTauPowerPmac, 0, Exception in SendGetResponse: Index was out of range. Must be non-negative and less than the size of the collection. //and on and on //once the error above occurs, the Windows app is no longer able to coumminicate with the PPMAC until power to the PPMAC is cycled. ============================== Second Excerpt: ============================== // This is an excerpt from a different type of log that the customer's app creates. // Note two types of exceptions being logged. // Once we see these types of exceptions in the customer's application log, the application will no longer communicate with the PPMAC // Until power is cycled to the PPMAC. 6/2/2014 12:28:17.851, Exception, DeltaTauPowerPmac, 0, Exception in GetPosition: Conversion from string "-inf" to type 'Double' is not valid.: System.InvalidCastException: Conversion from string "-inf" to type 'Double' is not valid. ---> System.FormatException: Input string was not in a correct format. at Microsoft.VisualBasic.CompilerServices.Conversions.ParseDouble(String Value, NumberFormatInfo NumberFormat) at Microsoft.VisualBasic.CompilerServices.Conversions.ToDouble(String Value, NumberFormatInfo NumberFormat) --- End of inner exception stack trace --- at Microsoft.VisualBasic.CompilerServices.Conversions.ToDouble(String Value, NumberFormatInfo NumberFormat) at Microsoft.VisualBasic.CompilerServices.Conversions.ToDouble(String Value) at DeltaTauPowerPmacController.classDeltaTauPowerPmacController.GetAxisPosition(Int32 AxisNumber, Double& Position) at DeltaTauPowerPmacController.classDeltaTauPowerPmacController.GetPosition(String AxisName, Double& CurrentPosition) 6/2/2014 12:46:18.881, Exception, DeltaTauPowerPmac, 0, Exception in SendGetResponse: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index: System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index at System.ThrowHelper.ThrowArgumentOutOfRangeException() at System.Collections.Generic.List`1.get_Item(Int32 index) at DeltaTauPowerPmacController.classDeltaTauPowerPmacController.SendGetResponse(String CommandString, String& ResponseString)

My IT administrator logged on, and was able to run the IDE as administrator without issue. He has attempted to set things up to give me selected admin permissions to be able to run the IDE but without giving me full admin permissions on the entire computer. So far no luck. Any suggestions?

I was finally able to duplicate your results (no dithering). The key difference is your use of zero integral gain, Motor[x].Servo.Ki = 0, which I had overlooked. By setting the integral gain to zero, the deadband terms Motor[x].Servo.OutDbOn, Motor[x].Servo.OutDbOff, Motor[x].Servo.OutDbSeed, and Motor[x].Servo.BreakPosErr, Motor[x].Servo.Kbreak are able to create a true deadband preventing the dithering. The integral component is not suppressed by the above deadband parameters and any non-zero value of Ki will eventually integrate out of the deadband and cause dithering. I would ordinarily use Ki to decrease the following error during a move. Ordinarily setting it to zero allows a larger steady state position error during accelerations and constant velocities. Your use of Motor[1].Servo.Kvff = 15 largely compensates for a lack of integral term by causing the velocity to increase immediately upon a change in commanded position rather than having to wait for a following error to develop. Thanks for your help. M. Shaver

1. I'll try to get details of the exception being thrown by the Windows app. 2. How do I verify that the Delta Tau processes are still running (gppmac, ppmacserver)? 3. The windows application updates quite a bit of information from the Delta Tau very frequently. IE; multiple requests for information per second and sometimes it locks up within a few seconds of starting up so there have been no operator actions yet. Are there any upper limits on these background information request rates that would cause something to lock up? 4. In all cases, the IDE continues to work fine. The windows app has locked up with and without the IDE running so I don't think the IDE has anything to do with it. However, if the IDE was running, it continues to communicate fine. If it was not running, I am always able to start it up and communicate without incident. 5. I have no C programs running in the PPMAC.

I haven't heard back on this issue. I am currently unable to do the work I need to do. Needless to say, getting very frustrated. Should the IDE work on Windows 7 professional 64 bit?