Exporting rigging to Apple SceneKit

[bhartsb]

Anyone on this forum have experience Exporting mGear rigging to Apple SceneKit?

I am an experienced iOS developer (Objective-C, C++, and Swift) but have little experience with 3D. I am hiring a rigging expert to rig a model I have in Maya using mGear. The plan is for him to create rigging for game engines. Normally this would be things like Unity, Unreal Engine etc., but I want to use SceneKit. What I’m doing is NOT intended for a game and exporting baked animations will not suffice. I want to programmatically articulate the rigging.

SceneKit historically supported Collada (DAE), Alembic and USDZ formats.

Alembic is not suitable for bringing rigging to SceneKit as per their own documentation “Alembic will not attempt to store a representation of the network of computations (rigs, basically) which are required to produce the final, animated vertex positions and animated transforms.”

USDZ doesn’t support rigging either. Pixar supports their own proprietary Presto rigging in
a USDZ file but not other rigging formats: https://graphics.pixar.com/usd/workflows.html

Collada (DAE) MAYBE supports some rigging but it isn’t clear how to bring it into SceneKit.

FBX: I believe supports rigging and can perhaps be brought in to SceneKit using:

dmsurti/AssimpKit

Apple doesn’t support FBX for SceneKit but this seems the most promising so far though I have yet to try it.

These are the Apple Classes that I’m assuming would be used to fully support rigging:
https://developer.apple.com/documentation/scenekit/scnskinner
https://developer.apple.com/documentation/scenekit/scnmorpher
https://developer.apple.com/documentation/scenekit/scnshadable
https://developer.apple.com/documentation/scenekit/scnprogram

I’m further assuming that to support mGear’s rigging I would have to build mGear as a library and then integrate the FBX rigging imported into SceneKit with the library in a similar fashion to how Maya currently integrates with mGear to interactively change the rigging and morph the polygons using mGear’s solvers. I’m assuming but don’t know for sure that solvers are essentially the same as shaders (but the terminology is all new to me).

Anyone have experience here or thoughts?
If anyone also has knowledge to share of how Maya interacts and integrates with mGear API to API that would be very helpful.
mGear uses Python and C++ so what areas each language is used for with regard to integration would also be helpful.

Thanks in advance.

[Jerome]

Hello @bhartsb

What you are tying to do (as far as I know) wont work. What I mean by this is that mGear heavily relies on Maya nodes and nodes hierarchies. Porting mGear solvers to another engine wouldn’t suffice to actually run an mGear rig as it is represented in Maya.

That been said I haven’t done so many exports for games on my side. I do know that Maya can export DAE files that I think are compatible with Apple’s SceneKit right? The DAE export are part of Maya’s FBX plugin so in order to be able to export them you need to load that plugin inside Maya.

That been said exporting to that format won’t allow you to have the Maya rig inside the SceneKit… My guess is that you can have your asset skeleton and skinning in maya and you can export that (with or without baked animation) and you will probably need to build on your own rig iks/solvers in SceneKit… I sincerely don’t know what there is inside the SceneKit in terms of rigging related things so I can’t really help on that matter.
From what I am reading in the doc I think you can’t really interact with rigs in the SceneKit because simply there is no rig in it. So my guess is that you need to have all your animations done in a 3D app and have those animations baked out into the character joints so that you can really trigger them inside the SceneKit. This is just a quick interpretation when seen the doc.

[bhartsb]

Hi @Jerome

SceneKit has a concept of nodes and node hierarchies. Wouldn’t we have to see how they differ before saying it won’t work? Even if they differ maybe there is enough similarity to make some transform that allows it to work, or do you disagree?

Here is the SceneKit node: https://developer.apple.com/documentation/scenekit/scnnode

Regarding “build on your own rig iks/solvers in SceneKit…” I don’t think there is any rigs in SceneKit so that is why I am proposing to use mGear as a library. It seems on the surface that translating or mapping Maya node hierarchies and node properties to SceneKit node hierarchies and node properties, and using mGear’s solvers would be easier and more exact than recreating solvers. Especially if my goal (which it is) is to get the same rigging for my model within SceneKit that exists in Maya. I want to be able to programmatically articulate the rigging in SceneKit.

I’m thinking that after mapping the node hierarchies and properties that the mGear solvers could be wrapped in some way by https://developer.apple.com/documentation/scenekit/scnshadable

Pre baked animations is really very non optimal for my non game application.
However, pre-baked animations might be useful with regard to getting the SceneKit implemented rigging to match the rigging and controls in Maya.

I welcome your further input.

Thanks,
Brian

[Jerome]

Hello @bhartsb

So I just wanted to say first that I didn’t meant that it isn’t going to work in a non constructive way. I was just imagining the amount of work that needs to be done for this to work and I am not well aware of the end result, but sure with enough work and efforts everything is possible and we (@mgear_dev) are here to help.

Lets start simple. Here is a Maya File and and DAE export of that Maya file. It is a simple rig with only FK hiearchy joints and a simple skinning applied to a combination of cubes combined together.

The files
https://we.tl/t-l3N3zUaxpe

Now, if you are able to already port this to SceneKit and be able to interact with the joints (that can work as offset nodes) then it is a starting point of course. Then you need to start trying out node by node. I don’t know how much work will be intended but my guess would be that doing the simples rig as possible inside Maya would be the best. If the above example works then you can already ask your rigger to build the joints/bones skeleton and bind it to your model correctly and send that to you as is to start with.

What comes next isn’t worth talking to much if the above does not succeed but I can already say that doing a realtime rig for you to interact with in SceneKit will take a lot of work if SceneKit doesn’t have anything for rigging.

Let me know if this helps

Cheers

[bhartsb]

Hi @Jerome

I really appreciate your willingness to help! And know that I’m very serious about getting this to work, knowing well it might take months.

I will start with what you’ve provided and update you.

Thanks again!

[Jerome]

Keep me posted then because it is better to tackle this in an incremental way.

[bhartsb]

Can you provide the maya project as an FBX? Loading the DAE is only bringing in the cubes.

Also just to note I found this SceneKit import tool (in addition to the other one in my OP, both rely on a library called assimp): https://github.com/eugenebokhan/AssetImportKit

Thanks,
Brian

[Jerome]

Here is an FBX export
https://drive.google.com/file/d/1dyC__LVlZmJ8go9WcEXSuwvn5gu5YFIp/view?usp=sharing

[bhartsb]

Thank you!
I will let you know my results.