Mixamo Rig Retargeting Inquiry
-
Hi there,
I'm currently testing out retargeting rokoko mocap animation via the character definition & solver tag onto a mixamo control rig. I'm getting issues with the fingers and arm joints being rotated in different ways than the mocap rig (pictured). I was wondering if this was some error in the mixamo rig I made (regarding the axis/orientation of the joints) or if this outcome is normal for mocap retargeting (and simply needs to be hand cleaned-up)? Any info would be appreciated
wetransfer project link: https://we.tl/t-GbQo2XBmtQ
-
Hi rent-crash,
I hope (no promise) to get some time today to check everything with the latest releases (Rokoko/Suit/Coil/Studio and Cinema 4D). I haven't used it for a while, … working on another project. Since I like to answer with practical experience from current setups and not from memory, I handed it over to Sam:
To give you a reply more quickly, please explore this link:
https://youtu.be/dGBw_KKRfLA?feature=shared&t=512
Sam explains two workflows!My impression of your file, you have seen this video, correct? But your character is in A pose and the Mocap is in T pose, correct?
I have to explore this more, it might take some time.
My experience over the years with Rokoko is simple, if the initial alignment is not done properly, it is much more work down the road.
These steps need to be good.The Character Definition Tag maps between source and target, which means it is relative! Any derivation from an idea state will show up, and yes, that happens predominately with the fingers for me.
Enjoy your MoCap session!
Cheers
-
Hi Dr. Sassi,
Thanks for the quick reply. And I have seen that video, but I'll review it once more to see if there's anything I can get a refresher on. I think I initially rigged the character in an A pose but oriented it to a t pose (and updated the bind pose) before applying the character definition tag and solver.
One detail that could be important is that the mixamo rig I set up for the character is larger than the rig of the recorded mocap... which may be causing the discrepancies in finger joint rotation. I tried to scale it to about the same size before applying the character solver, but that didn't seem to remedy the issue. If possible, I'd like to see if I can get the finger animation to map properly by adjusting the rig (or bind settings) as opposed to recreating the rig entirely and weight painting again. Thank you for the good wishes and I'll take a look at the video you posted again in the mean time.
-
quick update- I think the issue is with the scaling differences between the two rigs. The problem areas (arms and fingers) are occurring at the areas of my character rig that have slightly larger scale than the rokoko mocap mixamo rig. Specifically the clavical, biceps, forearms, and hands on my character rig are longer than the mixamo mocap rig which I think is the cause of the issue. Mentioning this just to provide more context to my previous post
-
Hi rent-crash,
Please focus on the part of Rokoko/Sam's video after 20:30.
Here is what is different:
The Character Definition Tag is placed in the file (from above) to the Character Rig, which indicates a setup strategy that is new to me, which is why I wrote that I need more time with it.I needed more time to explore it, as I hadn't seen it in all these years. I always question my own knowledge to stay open-minded.
My current conclusion is, that is not a way I would even remotely suggest.
Why? Take a look at the definition found, what it has connected, and if that fits one's overall understanding of the process.
The Character Rig has several Joints rigs, not just one. Each has a specific purpose. What a Character Definition tag does, when applied to this Character object, is it finds all and connects all. Here is one problem, not the only one.To my understanding, Sam connects the MoCap rig (Snaps to it…). This now influences the Character Object's Rig (the Retarget part of it, not anything else.) This is important to understand as this external Joint rig needs to be Defined and Solved with any incoming new MoCap, NOT the Character Object's Rig.
In short, we have then three separate rigs: Incoming and exchangeable, the rig "snapped" to the Character Rig, and the Character's Rig. I use the term "snapped' to differentiate the idea of the definition tag.
If you have any reasons to stay with the setup I saw in the file, I have no idea how to solve that; quite honestly, I can't see how that could work.
As a side note, all three must be in a T-Pose during the initial process, and any misalignment will cause problems.
All the best.
-
Hi Dr. Sassi,
I was wondering why the character definition tag was finding so many bones, thank you for the detailed analysis. I'll go ahead and go through the latter part of Sam's video and watch his process. I don't have any reason to stay with the character definition tag on the character object, so I'll change that. I assume that if the elbow and hand joints of the current rig are misaligned to the incoming mocap rig, then I'll need to adjust the position of those joints. Would I just do that in the adjust tab of the character object? I guess I'm at the stage of figuring out how to adjust certain scales of my existing rig that the mesh is already painted to, before mapping the mocap over with the character definition & solver tags.
Thank you for your time
-
Hi rent-crash,
Please remember that the rotation is the key, not the position.
The following approach benefits those who learn best by doing and reflects a hands-on, trial-and-error methodology often seen in creative fields like 3D modeling, animation, and game design. Since I know very little about your background, I might be dead wrong here.
My suggestion is to set up a few joints in a new file. Freely. Then again, try to match them roughly; the key is to be different for this exploration. I even suggest rotating some of them. Copy them for a backup, just in case.
Save the scene. Save iterations.Set up a cylinder with no caps that fits closely to the joints and has enough density (not too high) in the mesh. Bind it. Name all joints similar to the Mixamo rig that you like to improve.
Then, use the Definition workflow to get a good motion transfer from the source mesh to the target mash( the one with the cylinder).
Save this. Take the definition tag of the source, rotate a few joints, and then define again. Explore how that affects the target rig.
Why do I suggest this? The typical problem I have seen over the past two decades is starting with an important (your robot) model and missing perhaps the playful and destructive exploration of meaning setup. Something that you can trash without any regret. But here is the time investment that lets you focus on more complex things.
If you save this, returning after a while and messing around is easy to refresh your memory.
Sure, that sounds like a lot of time when something bigger and better is waiting for you. But you will gain a solid command of this area faster when you explore it without consequences.
This is just a suggestion. You can toss it, but I don't mind. Perhaps it doesn't fit your learning style, so ignore it. But can 10 minutes do a lot of damage?
Enjoy
-
Hi Dr. Sassi,
Thank you for the detailed response. I did end up figuring out a work around to my issue. First I reset the bind pose of my character into a T pose as you advised and from there I retargeted the mocap rig in the master controller settings of the character object. This gave me much more accurate results of movement and joint rotation, and I no longer have my characters' fingers crossing over.
I also went through working on testing different retargeting approaches in new scene files as not to contaminate the original project. It is interesting how certain movements can render model designs basically useless as you detailed above. I guess that's part of the process of figuring things out, trial and error like always!
Anyways, I'll let you back to your work. Thank you for the help and I'll take it from here
-
Thanks for exploring this, rent-crash.
I'm sure the investment has already paid out for you this time. Thanks for your feedback.
Cheers
