Hi MFVis,
I hope I get your target correctly.
What you like are two independent systems that work together anyway. Correct.
Here is a suggestion
https://stcineversityprod02.blob.core.windows.net/$web/Cineversity_Forum_Support/2023_PROJECTS_DRS/20230609_CV4_2023_drs_23_MGdy_01.zip
I baked the Voronoi Fracture setup into an Alembic and de-activated the initial Voronoi Fracture setup.
Voronoi Fracture Object selected: Object Manager> Object> Bake Alembic.
Since the motion of the sphere is short, I just traced it. While moving and matching the position, I keyframed the start and end of it, then one in the middle, and perhaps some more.
In the F-Curve Editor, you will see a specific Curve now. That should work already just fine.
In the example, I took an F-Curve. Snapshot, and deleted all keyframes except the first and the last, then called up the Snapshot and used only two keyframes to shape the F-Curve. It is smoother that way.
My best wishes for your project