Author Topic: source, output attributes  (Read 340 times)

Kevin

  • Newbie
  • *
  • Posts: 32
source, output attributes
« on: February 10, 2018, 06:45:37 AM »
Dear Dr.Stukowski,

I observed a weird behavior in both OVITO 2.9.0 and 3.0.0, so I report it here.

The following code reads in two files(dump.200.0000 and dump.200.0001).
Each timestep is incremented by 2500.
So, node.compute(3) should result in timestep 7500.

Since node.compute() should cache the most recent compute to the node.output field,
expected outputs for the last 3 lines of the codes are as follows:
0        => node.source.attributes['Timestep']   
7500  => node.output.attributes['Timestep']
7500  => theOutput.attributes['Timestep']

However, the outputs from 2.9.0 version are as follows:
7500  => (source should remain at the timestep 0)
7500
7500


*I was going to say 3.0.0 version also behaves weirdly as follwos,
*but I guess 3.0.0 version doesn't have output field anymore.
Also, the outputs from 3.0.0 version are as follows:
0
0    =>(I don't think the most recent compute is cached into output field)
7500


Code: [Select]
from ovito.io import *
from ovito import dataset
from ovito.modifiers import *
import numpy

node = import_file("dump.200.*",
                   multiple_frames = True)
# Modifiers                                                                     
# Modifier 1: CNA                                                               
node.modifiers.append(CommonNeighborAnalysisModifier
                      (mode=CommonNeighborAnalysisModifier.Mode.FixedCutoff,
                       cutoff=3.5))
# Modifier 2: Selection                                                         
node.modifiers.append(SelectExpressionModifier
                      (expression = 'StructureType==3'))

theOutput=node.compute(3)
print(node.source.attributes['Timestep'])
print(node.output.attributes['Timestep'])
print(theOutput.attributes['Timestep'])

Alexander Stukowski

  • Administrator
  • Sr. Member
  • *****
  • Posts: 476
Re: source, output attributes
« Reply #1 on: February 10, 2018, 08:51:21 AM »
Dear Kevin,

Let's first speak about OVITO 2.9.0:

To me, the output seems reasonable. Both fields, node.source and node.output, represent data caches for the input and output data of the pipeline, respectively. If you evaluate the pipeline at timestep 7500, then it's expected that after evaluation is complete both caches contain data from timestep 7500. Why would you expect to find data from a completely different timestep (0) in the pipeline input cache?

The input cache (node.source) works as follows: Whenever a pipeline evaluation is requested for a certain animation time (timestep 7500 in your case), OVITO checks if the data present in node.source is for that timestep. If yes, it is directly used as input for the modifier pipeline. If not, the data for the requested timestep is loaded from the external input file and stored in the cache before passing it on to the modifier pipeline.

Now about OVITO 3.0.0:

The caching mechanisms have become more complex in this version, because the pipeline tends to be evaluated at different timesteps simultaneously in this version. From the Python perspective you only see one of these caches. node.source is the cache for the current animation time, which is controlled by the global ovito.dataset.anim.current_frame setting (basically the time slider in the GUI). This setting is not affected when you call node.compute(3). Thus, you'll still find data for the current animation time (frame 0) in that input cache.

(The second input cache, which contains the data from the most recent evaluation (frame 3), is not accessible from Python. I'm not sure if that poses a limitation from the user's perspective or not.)

Furthermore, as you said, in OVITO 3 the node.output cache for the output data is gone (should no longer be used). Instead, the Python script should take care of the "caching" itself by storing the output DataCollection returned by compute() in a local Python variable (e.g. "theOutput" in your case).

In OVITO 3.0, accessing the node.output field is equivalent to invoking node.compute() (without frame parameter). Thus, it will always yield the pipeline output for the current animation time. It's basically a poor emulation of the old node.output field from OVITO 2.9.0, just to provide some limited backward compatibility.
« Last Edit: February 10, 2018, 09:16:47 AM by Alexander Stukowski »

Kevin

  • Newbie
  • *
  • Posts: 32
Re: source, output attributes
« Reply #2 on: February 11, 2018, 02:26:33 AM »
Thank you for the reply.
I came up with one additional question as follows:

Code: [Select]
# Note                                                                         
# :This works with ovito 3.0.0                                                 

from ovito.io import *
from ovito import dataset
from ovito.modifiers import *
import numpy

node = import_file("dump.200.*",
                   multiple_frames = True)
# Modifiers                                                                     
# Modifier 1: CNA                                                               
node.modifiers.append(CommonNeighborAnalysisModifier
                      (mode=CommonNeighborAnalysisModifier.Mode.FixedCutoff,
                       cutoff=3.5))
# Modifier 2: Selection                                                         
node.modifiers.append(SelectExpressionModifier
                      (expression = 'StructureType==3'))


print(node.source.attributes['Timestep'])
print(node.compute(3).attributes['Timestep'])
print(dataset.anim.current_frame)
dataset.anim.current_frame=3   
node.compute()
print(dataset.anim.current_frame)
print(node.source.attributes['Timestep'])

The results from the last 6 lines of the code are as follows:
print(node.source.attributes['Timestep'])           => 0   [This is OK]
print(node.compute(3).attributes['Timestep'])    => 7500  [This is OK]
print(dataset.anim.current_frame)                    => 0  [This is OK]
dataset.anim.current_frame=3
node.compute()
print(dataset.anim.current_frame)                     => 3  [This is OK]
print(node.source.attributes['Timestep'])            => 0  [This should be 7500]

Quote
node.source is the cache for the current animation time, which is controlled by the global ovito.dataset.anim.current_frame setting (basically the time slider in the GUI). This setting is not affected when you call node.compute(3). Thus, you'll still find data for the current animation time (frame 0) in that input cache.

According to the quote above, I expect the last line of the code to print 7500 instead of 0. Could you explain why this prints 0?

UPDATE
1. this happens only when previous node.compute(3) is there.
If 'frame1' in node.compute(frame1) (6th line from the bottom) and
'frame2' in dataset.anim.current_frame = frame2  (4th line from the bottom) is different, this behavior is not observed.

2. node.source seems to update only after both 'current_frame' has changed and node.compute() has been used.
« Last Edit: February 11, 2018, 05:52:50 AM by Kevin »

Alexander Stukowski

  • Administrator
  • Sr. Member
  • *****
  • Posts: 476
Re: source, output attributes
« Reply #3 on: February 12, 2018, 09:51:34 PM »
I guess this behaviour is due to the two parallel caches the FileSource maintains. There is one cache (let's call it 'A') that contains the data for the most recently requested frame, and a second cache (let's call it 'B') for the data at the current animation time. When a new request for a frame is processed, the FileSource first checks if the data for the requested frame is already available in either of these two caches. Only if that is not the case, the data is retrieved from the external file(s).

So in your case, the following happens:

Code: [Select]
import_file(...) # This fills cache B with data of frame 0
node.compute(3)  # This fills cache A with data of frame 3
dataset.anim.current_frame=3
node.compute() # Cache A already contains the data for the requested frame. Returns immediately.

Cache B is not updated again in this case. But cache B is exactly the one that the FileSource object exposes to the Python script. This explains the output you see, I think.

Based on your input, I extended the Python interface of the FileSource class a bit. In the newest development build of OVITO, the class provides a new method compute(), allowing you to request arbitrary frames from the FileSource. Now it no longer matters which data is currently in the visible cache of the FileSource. Please see the updated documentation for details:

http://ovito.org/manual_testing/python/modules/ovito_pipeline.html#ovito.pipeline.FileSource