Hi MFVis,
The single object that needs to be moved in your scene via the Driver Field is a Linear Field (or plural). I would assume so (being faster), as the geometry is not generated (blend) between two.
You wanted the option to animate the parameter, which is not the idea of the Field Driver. Hence, it deletes the Keyframes it covers by default.
Outside of this, and as a general idea, with all the problems general opinions have, the typical suggestion is that many single objects are slower than otherwise merged into one object.
However, if the single object has the need to run through a generator and needs only a small part changed, perhaps two objects might be faster. In short, general rules never go the full way; even the name might insist they do. This is a reason why I ask normally for project files, to answer specifically. However, speed is a thing that relies on your setup in a dramatic way, and it is not only the GPU/CPU combination. Hence, my exploration might not match what you get out of your setup. Optimizing a scene is something I haven't found any timeless general tips over the past 30+ years I have done 3D, besides the obvious, like don't use slow drives, etc. In the end, the rendering has options to optimize, and then each frame provides a precise number. (Varying by background activities - of course)
What typically works is the test of the scene with the Main Menu> Windows> Scene Profiler. Both versions will typically provide different results. I suggest doing it a few times, as the computer does things in the background, and even having Chrome open with a lot of tabs might change the outcome.
All the best