Bartat Posted July 17, 2015 Posted July 17, 2015 Hello, since about one year we have a strange issue with PowerPMac which occurs very very rarely. We are not able to find out what could cause this problem. It occurs in random intervals between 2 - 10 days and not in a special situation. We cannot intentionally reproduce it. In that situation our system stops running with a sys.WdtFault=1, a so-called software watchdog fault. But this is only the consequence of the fact that the gppmac application has been terminated or has been crashed, possibly caused by an internal fault. The gppmac application executes our background script plcs and (of course) if gppmac stops working the sys.wdtimer is countdown by RTI and the system stops with Sys.WdtFault=1. What we see in that situation is that all background script plcs has finished their execution regularly. We are using two extra C background applications, which are accessing the user shared memory of the PPMac. One is an own written Modbus client, which communicates with three modbus slaves over the second ethernet port (eth1). But these applications are still running without failures when gppmac stops working. So it is unlikely, that the problem is caused by these applications. It happens on differnet CPUs and with different firmware versions. We have tested several firmware versions from 1.6.1.1 up to 2.0.0.14. Maybe somebody else has some similar experiences? Any ideas?
Omron Forums Support Posted July 17, 2015 Posted July 17, 2015 What are your clock speeds? Are you running any motion programs or foreground PLCs (e.g. RTICPLC or Script PLC within the range of Sys.MaxRtPlc)? If you're running motion programs, do you have very closely-spaced programmed points that might be overloading the CPU? Are you sure your C programs are still running when you get WdtFault=1? How are you checking this? Can you list your C code? Many WdtFault=1 conditions I have encountered were caused by C programs.
Bartat Posted July 19, 2015 Author Posted July 19, 2015 What are your clock speeds? 20KHz Phase and 10 KHz Servo clock Are you running any motion programs or foreground PLCs (e.g. RTICPLC or Script PLC within the range of Sys.MaxRtPlc)? If you're running motion programs, do you have very closely-spaced programmed points that might be overloading the CPU? The problem occurs also, when no motion program or RTICPLC is running Are you sure your C programs are still running when you get WdtFault=1? How are you checking this? Can you list your C code? Many WdtFault=1 conditions I have encountered were caused by C programs. Yes, they are still running. I checked this with the Linux console "top" command. But the gppmac processes have been terminated. They are not listed anymore, when i type at the Linux console the "more stat" command at folder /proc/xenomai. Four month ago, i tested a firmware from DTCA with debug infos. The gppmac exits with an error message. The C-programs are doing some TCPIP socket communications with other controllers for reading/writing some IO data over ethernet. They are still communicating with the other controllers after the fault has occurred. And they still increment global P variables, which I can see at watch window of the PowerPMac IDE. One of the c-programs can definitely excluded, because I removed it from the project and the problem still occurs. Only the modbus tcp client must running. I attach the modbus source and a part of the background plc, which copies the data from and to the modbus buffers.modbus.zip
Recommended Posts