VDB loading times

[Frischvergiftung]

Hi!

We have this rather huge VDB sequence that we load into cinema using the Redshift Volume Loader. One VDB frame has up to 2.5GB. The sequence is stored on a PCIe 4.0 Nvme which ( we checked) is able to deliver 7GB/Sec.

When moving around in the timeline it takes around ten seconds for the viewer to update. We set the preview to dots, switching to box doesn’t help either. Cinema takes very long to load these files. Is that expected behavior? What could we do to speed up things?

We have a midrez VDB as a stand in, but it’s not much faster and not if help when actually trying to shade the VDB to our liking.

Any ideas?

Thx, Max

[Dr. Sassi]

Hi Max,

I'm sure you are aware of the Play all frames setting.

The drive might have a max of 7GB/sec, but if your scene is 30 FPS times 2.5 GB =75GB/sec, you would need ten times the speed to load the sequence. (Perhaps it is 24 or 25 fps, but you get the idea where I'm heading with that.) Ten times, sounds familiar?

This is just the load time. There is processing time and the need to shuffle the RAM in and out. While the amount of RAM needed plays a critical role here.

I have no idea about your setup and how it sustains the speed over time, but if you have a four-second clip, that would be 300GB to load.

If I'm not mistaken, that goes into the GPU, and if this can't sustain 75GB/s incoming data and the translation into results while adding other requests to keep all needed calculations running, things will slow down. As far as I know, cards are between 20GB/s and 1.5TB/s; you will know better what each part of the pipeline does in your setup.

However, that is not my area to discuss here, as it belongs to the hardware section. I have only a limited amount of hardware here and close to half of your speed, perhaps only a fraction of your RAM. I'm guessing here! It's not my place, as the Q&A forum is not a tech support alternative. I'm not supposed to do that.

Even if you take only 500MB per frame, that would already exceed the 7GB/s.

I tested it with an M1Max, 64GB of RAM, and an SSD with 3GB/s that has nothing else to do but support the Cache. The app is on an SSD, and the OS is on another. All nicely separated, even with 500MB, my FPS is 1.3.

Perhaps tech support can put more insight into this discussion.
https://www.maxon.net/en/support-center

All the best

[Frischvergiftung]

Hi Sassi, you‘re a legend, thanks for the detailed reply! Of course I don’t expect this to play smoothly. It’s just when I am, for example, looking at frame 40 and then click on frame 80, the scene takes 10 seconds to update. This just seems very, very long doesn’t it?

[Dr. Sassi]

Thank you very much, Frischvergiftung!

I would open a ticket and see what will happen.
I feel that the request for speed will be received with great appreciation. Seriously.

Yes, today, I think that 10 seconds can often feel long. I am not very patient with hardware, but 30 years ago, the SCSI connector allowed for 5MB/second from a drive, so I'm perhaps equipped with a particular multifaceted perspective.
Anyway, I firmly believe that one of the most significant parts of an intuitive workflow is to be as close (or better) to real-time response.

This allows us to see connections, and the learning and adjusting are better and faster.
So, yes, I undoubtedly see the value in speed. It is not a luxury; it is a necessity.

By the way, reading about your 7GB/sec hardware gave me some wishes.

Thanks again, and enjoy your weekend

[Frischvergiftung]

Ok thanks we will talk to tech support!

It‘s Christmas soon, time to write your wish list

[Dr. Sassi]

Hi Frischvergiftung,

Yes, that list needs to come in on time!

Fingers crossed it works.

Have a great Sunday!