Jump to content
OMRON Forums

Writing data to file using the gather buffer


maartenvervelde

Recommended Posts

Hi,

 

I'm trying to make a gather function that keeps on running and adding the data to a txt file. The code I have in the background c app is this:

 

fpGather = fopen("/var/ftp/gather/gather.txt", "w");

while (!done)
{
	pGatherBuffer = (char *)(pshm->Gather.Buffer);
	samples = pshm->Gather.Index;

	if(pshm->P[xsaveValues] == 1)
	{
		for (i = 0; i < samples; i++)
		{
			memcpy(&time, (unsigned char*)pGatherBuffer, sizeof(int));
			pGatherBuffer += sizeof(int);
			fprintf(fpGather, "%d \r\n", time);
		}
		pshm->P[xsaveValues] = 0; 
	}
}

 

The value pshm->P[xsaveValues] is changed in my servo loop so it is only updated when the loop is used.

 

the gather values are the following:

gather.addr[0] = sys.servocount.a
gather.items = 1
gather.period = 1
gather.maxlines = 1048576
gather.maxsamples = 1048575
gather.enable = 2

 

The problem I have is that the servo loop runs at 5kHz, so it will update 5000 times a second. I timed the saving of values and in 10 second I got a .txt file of 53 MB with about 5.6 million times the servocount printed, where I excpected 50.000 lines.

 

The first value I get is 2.453.738 which is normal, and it goes up by one until it reaches 3.373.594 on line 919857. After that, on the next line it starts again at 2.453.738 until it reaches 3381841. This pattern continues until the last line where it ends at the highest value of 3.415.154.

 

I don't know where it goes wrong, to me it looks correct. Maybe someone has seen this before and knows how to solve it.

Link to comment
Share on other sites

  • Replies 21
  • Created
  • Last Reply

Top Posters In This Topic

I tried the thing Charles said, but I didn't see a file anywhere on the file system. When I looked further I saw this on the forum:

 

Currently the only customer supported usage is:

gather_csv [-u] [filename]

 

The “-u” will stream the contents of the most recent data gather to the standard output as in the current SSH, Telnet or serial port connection. The “-u” with a file name will stream it to that file. I would suggest using the “/var/ftp/usrflash/” directory as this is already writable and will allow ftp uploads without any permission changes. For example:

gather_csv -u /var/ftp/usrflash/steve

 

This puts the last gathered data in a text file named “steve” in the “/var/ftp/usrflash/” directory where you now have ftp access.

 

Keep in mind the gather files can be very large.

 

This is the only thing I could find about how to use the gathered data. This was a while ago so I dont know if there are different way to do this now.

 

Than I found this tutorial you made to gather data using a capp.

 

The Gather Buffer is just a series of bytes so to access the Gather Buffer Data from a C App you must know what type of variables you gathered. Then you can access the buffer directly and offset your pointer the correct number of bytes for each element that was gathered.

 

 

pGatherBuffer = (char *)(pSharedMem->Gather.Buffer);

samples = pSharedMem->Gather.Index;

*timeConstant = 0.0;

 

fpGather = fopen ("/var/ftp/gather/autotune_gather.txt", "w");

 

 

//--------------------------------------------------------------------------

// Read in gathered data and compute the following error and min max

// following error

//--------------------------------------------------------------------------

for (i = 0; i < samples; i++)

{

memcpy(&t, (unsigned char*)pGatherBuffer, sizeof(unsigned));

pGatherBuffer += sizeof(unsigned);

memcpy(&cmdpos,(unsigned char*) pGatherBuffer, sizeof(double));

pGatherBuffer += sizeof(double);

memcpy(&actpos, (unsigned char*)pGatherBuffer, sizeof(double));

pGatherBuffer += sizeof(double);

memcpy(&dac, (unsigned char*)pGatherBuffer, sizeof(float));

pGatherBuffer += sizeof(float);

fprintf(fpGather, "%u %lf %lf %lf\r\n", time, cmdpos, actpos, dac);

}

fclose(fpGather);

 

 

That was why I tried it in a capp.

If there is a way without it I would be thrilled, but the way Charles explained it, didn't work for me.

Link to comment
Share on other sites

I wondered about what was written. The command is not gather_csv it is just gather. So you should do gather -u /var/ftp/gather/ggg.txt to send the already gathered data from the structures inside the ppmac to the file ggg.txt.

 

I just tested this and it is working. My test was simple. I open the plot program and gathered some data and plotted it to be sure the gather worked. Then I ran the command and saw my file was created. Then I listed the file contents to verify it had the data inside.

Link to comment
Share on other sites

I am not sure I understand what you are saying. What I understand is that you plan to store the data on the PPmac, therefore you can not be planning to gather more data than you have disk space on the PPmac. PPmac gives you 1Mb of internal buffer space for gathering. If you need more than this you must use streaming either to a file on the PPmac disk or to another host on the stdio. See Gather.Enable, Gather.MaxLines, Gather.MaxSamples for a start at looking at how this works. This of course means the streaming must be active while the gather is running.

 

If you are planning on gathering more than can be stored in the internal PPmac buffer then you must stream the data to a host or a file on disk and this is also done with the gather program but you use Gather.Enable=3 then you start the gather {filename} and you must make sure not to overflow the disk. Note in this streaming mode the -u option is not used.

Link to comment
Share on other sites

That worked, thank you for your help.

 

As a side note, I don't know if I could't find it or if it's not available, but I could find not much documentation on the Gather function. I could find that it was available, but not how to use it. This guide(http://forums.deltatau.com/showthread.php?tid=99) did help, but that was about the most information I could find.

Link to comment
Share on other sites

These type of "appware" that we have in the controller are not well documented. Most only have a limited embedded help with the --h runtime switch. Some do not have that. I have already mentioned this to the documentation dept. based on your recent post so a future manual will hopefully have this documented.
Link to comment
Share on other sites

  • 1 year later...

If you are planning on gathering more than can be stored in the internal PPmac buffer then you must stream the data to a host or a file on disk and this is also done with the gather program but you use Gather.Enable=3 then you start the gather {filename} and you must make sure not to overflow the disk. Note in this streaming mode the -u option is not used.

 

Hi all

this gathering mode is really interesting and useful.

However, I'm trying to automate some programs and plcs, and I cannot make the PPMAC start the streaming gathering from a prog.

Inside of a program I have:

Gather.Enable = 3;		// start rotary gathering
dwell(2000);
system "gather /var/ftp/usrflash/stepXXX.txt" // 

but the gather save only an empty file. I think that's because the gather software must run without interruptions in order to work in background.

I tried all the following commands without success:

system "nohup gather /var/ftp/usrflash/stepXXX.txt"
system "gather /var/ftp/usrflash/stepXXX.txt & " 
system "nohup gather /var/ftp/usrflash/stepXXX.txt & " 
system "nohup gather /var/ftp/usrflash/stepXXX.txt \046 "

am I missing some information regarding the system command?

 

thanks

gigi

Link to comment
Share on other sites

gigi,

 

I just tested and it works as described.

1. I first went to the plot program and configured my items to gather.

2. I then test by pressing Enable Gather, wait a bit, Stop, upload and plot. I see I get the data correctly.

3. In the terminal I enter Gather.Enable=3

4. Then from a puTTy SSH session and enter gather /var/ftp/Temp/g1.txt

5. I wait a bit and enter gather.enable=0 from the IDE terminal

6. I look at the file on the ftp and see it has data.

 

This is not a perfect test but shows the function works. You should make sure this works for you before we go further. It might give some hint.

Link to comment
Share on other sites

Hi Brad

gigi,

 

I just tested and it works as described.

1. I first went to the plot program and configured my items to gather.

2. I then test by pressing Enable Gather, wait a bit, Stop, upload and plot. I see I get the data correctly.

3. In the terminal I enter Gather.Enable=3

4. Then from a puTTy SSH session and enter gather /var/ftp/Temp/g1.txt

5. I wait a bit and enter gather.enable=0 from the IDE terminal

6. I look at the file on the ftp and see it has data.

 

sorry, I think I explained myself wrong. The procedure you are describing is the one that I am using right now, and it is working.

However, I'd like to start the gathering from a PROG or a PLC, not using SSH. And here comes my problem with the SYSTEM function: it seems that stop the execution of 'gather', even if I use the linux parameters to run in background.

 

Thanks

gigi

Link to comment
Share on other sites

looks to me like the system command in firmware 2.3.1.0 and perhaps earlier is broken in a program. Or I just do not know how to use it and neither does the SRM. Anyways I am testing on a motion machine. Other ppmac types might behave differently. Here is the problem I see. I do as the manual explains

 

open plc 1

Gather.enable=3

p11=Sys.Time+2

while(p11>Sys.Time){}

system "/opt/ppmac/gather_csv/gather_csv /var/ftp/Temp/g5.txt"

p11=Sys.Time+5

while(p11>Sys.Time){}

Gather.enable=0

disable plc 1

close

 

But the file g5.txt never gets created. So I do a workaround since from the IDE terminal the file would get created. I do this.

 

open plc 2

Gather.enable=3

p11=Sys.Time+2

while(p11>Sys.Time){}

cmd"system /opt/ppmac/gather_csv/gather_csv /var/ftp/Temp/g5.txt"

p11=Sys.Time+5

while(p11>Sys.Time){}

Gather.enable=0

disable plc 1

close

 

Now the file g5.txt is created.

 

But either from the terminal or from PLC 2 the file g5.txt is always 0 bytes.

 

I can not go further with this. Someone at ODT factory will need to test and look at this and tell use how to make this work from a script PLC.

Link to comment
Share on other sites

Corrections / additions:

 

don't forget the "-u" switch:

cx p1=sprintf(256, "/opt/ppmac/gather_csv/gather_csv -u /var/ftp/usrflash/Temp/g5.txt")

cx system "%s", 256

 

And when using the "Plot" application to set it up make sure to set "Legacy Mode" to create the old style gather parameters setup file.

 

Also don't rely on Windows "IE" or "Explorer" to view or copy the files as it LIES. Use a good ftp or sftp client like FileZilla to see the data in the generated file. I just tried this on my UMAC 465, 2.3.1.0 and it worked well.

Link to comment
Share on other sites

just tested. This building the string solution does not work on motion machine. What did you test with? What firmware?

 

I also cannot use the CX command since it is only a online command, I need to put that on a Prog.

 

Thanks

ciao

gigi

Link to comment
Share on other sites

Here's my latest findings: if I use the following code in a prog, the behavior is the one that I want, but the gather function is using 150% of CPU, making the IDE pretty unstable

 

system "nohup gather /var/ftp/usrflash/stepscript.txt > /dev/null  <&- &"

 

This is equivalent in commanding on the shell:

nohup gather /var/ftp/usrflash/stepscript.txt > /dev/null  <&- &

and that's using only 70% of CPU time. Any idea on why the prog is doing that?

 

thanks

ciao

gigi

Link to comment
Share on other sites

  • 2 months later...
Guest
This topic is now closed to further replies.

×
×
  • Create New...