[Epic] Rejuvenate M3D model loading

[tellypresence]

Goal: restore the ability for users to opt-in for M3D model support, and be able to load as many available sample models as possible.

• Simplify re-enabling M3D build support #5835
  Enable users to easily opt-in via CMake options rather than having to patch assimp CMake scripts to override hardcoded #defines
  With this change users can load basic M3D models, excluding those that are

  • textured
  • rigged/animated
  • voxel-based

• Fix issues 5847 and 5836 (M3D was accessing stb_image objects not in public API) #5848
  Support loading textured M3D models

• Fix issue 5834 (voxel-based M3D models fail to load) #5841
  Support loading voxel-based M3D models

• Bug: M3D model import failures due to bone problems #5837
  Support loading rigged/animated M3D models
  Rigged/animated models "sort of" worked as of

  • 12 Feb 2020 assimp commit 58b9dcb "Merge pull request Additional checks on invalid input #2973 from bztsrc/master"

  Orginal author will not be fixing this (comments below), so not worth the effort on assimp side to restore to what was at best the partially working state of 12 Feb 2020 commit 58b9dcb

• Bug: M3D "elephant" model import failure due to texture problem #5843
  The "elephant" sample M3D model fails to load.
  The download page says the model is untextured, but even with the patch in place to allow loading textured models, the model still fails to load for apparent texture-related reasons.

  "Elephant" model was not working as of the final M3D PR:

  • 12 Feb 2020 assimp commit 58b9dcb "Merge pull request Additional checks on invalid input #2973 from bztsrc/master"

  Orginal author will not be fixing this (comments below); this model never worked and so nothing to be done on assimp side.

[tellypresence]

@bztsrc I think I've taken this effort as far as I can. At this stage, if a couple of PRs get merged, users will be able to load all of your sample models except

• the rigged/animated ones
• the elephant

If you're able to spend a few hours some weekend to try to fix the problems with those models, can get to a state where users can load 100% of your sample models via assimp. Not sure whether you're still actively maintaining the M3D format, but if so please consider addressing these issues to expand your user base -- thank you

[bztsrc]

Thank you so much for your efforts!

On Sun, 3 Nov 2024 at 02:05, Steve M ***@***.***> wrote: @bztsrc <https://github.com/bztsrc> I think I've taken this effort as far as I can. At this stage, if a couple of PRs get merged, users will be able to load all of your sample models except - the rigged/animated ones - the elephant If you're able to spend a few hours some weekend to try to fix the problems with those models,

Frankly I'm not really interested. There are no problems with the M3D models, they are the same models as before, which were working and they are still working with all the other M3D applications. can get to a state where users can load 100% of your sample models via

assimp. Not sure whether you're still actively maintaining the M3D format,

Well, I'm actively maintaining the M3D code base, but the M3D format and the in-memory structure that the SDK uses hasn't changed from the very beginning, it is as stable as it can be. Nothing has changed on the M3D side as long as Assimp concerned.

but if so please consider addressing these issues to expand your user base -- thank you

I'm sorry, but I'm the M3D developer and not an Assimp developer. As I've said, the M3D SDK's API and in-memory format hasn't changed, and it used to work in Assimp but if not any more then the problem is on the Assimp side for sure, and I'm not willing to spend time with that because the Assimp devs were total j**ks (except kimkulling, he was always nice and professional). I wrote a detailed description <https://gitlab.com/bztsrc/model3d/-/tree/master/assimp#model-with-skeletons> on how the M3D in-memory structure should be mapped to Assimp in-memory nodes, also here is my Assimp wrapper <https://gitlab.com/bztsrc/model3d/-/blob/master/assimp/code/M3D/M3DImporter.cpp> code (with lots of comments and explanations) that used to work with rigged models and with the elephant too. I'm afraid that's all the help I can provide for Assimp integration. Cheers, bzt

…

Message ID: ***@***.***>

[tellypresence]

@bztsrc Thank you for your response.

I wanted to check whether the code did in fact work at one point so went back in time to the commit where you introduced M3D to assimp:

• 4 Nov 2019 assimp commit 308db73 "Merge pull request Added M3D format support #2736 from bztsrc/master"

Then built assimp and retried loading the sample models

Summary

• most of the models cause catastrophic crash on load, including the "elephant" model
• two (non-textured) rigged/animated models worked at that point

Details

Most sample models cause catastrophic crash on load, including (but not limited to) anything with textures. 2 rigged/animated models were able to load and appear to animate OK, though the "swat" model seems to have insane bone count, and the bone check indicates an incorrect number of meshes, so possibly there was a problem with converting from M3D to assimp skeleton in the code at that commit.

The "elephant" model is among the ones that crash, so based on this test it appears that the elephant never worked.

Screenshots

Models that load OK

4 Nov 2019 assimp commit 308db73
Model	Screenshot	Notes
teapot.m3d
madmax.m3d
r2d2.m3d
swat.m3d		Model has 4 meshes Model has 24 animations Model indicates it has 47,120 bones in 760 meshes (despite only having 4 meshes) (Because of massive bone count, implementation can't keep up and model appears fragmented)
aliveai_character.m3d		Model has 1 mesh Model has 1 animation Model indicates it has 6 bones in 1 mesh (seems reasonable)

Models that crash on load

All these models caused catastrophic crash on import attempt:

4 Nov 2019 assimp commit 308db73
Models with catastrophic crash-on-load	Notes
bat.m3d	Rigged/animated model
lara.m3d	Rigged/animated model
seagull.m3d	Rigged/animated model
dragon_vrip_res3.m3d
Elephant
atat.m3d
batmobile.m3d
cessna.m3d
fishboat.m3d
castle.m3d
medieval.m3d
bender.m3d
gandalf.m3d
wolfzoid.m3d
yoda.m3d
awing.m3d
birdofprey.m3d
viper.m3d
enterprise.m3d
mfalcon.m3d
alkesh.m3d
tie.m3d
vaderstie.m3d
xwing.m3d

Do you recall if you had made fixes to address the crashing and weird bone problems? If so please let me know the specific git commit, and I can try again -- would be great to establish if there was a commit where things were working so we can figure out where things broke. Thank you

[tellypresence]

Never mind, I may have found it -- looks like there were indeed fixes made; AFAICT the final PR for M3D was

• 12 Feb 2020 assimp commit 58b9dcb "Merge pull request Additional checks on invalid input #2973 from bztsrc/master"

Built assimp at this commit and retried loading the sample models

Summary

• most of the models load OK at this commit
• "elephant" and "CesiumMan" models failed to load; seems clear these models never worked in assimp at all
• Most of the rigged/animated models load and animate, but bat/seagull have animation artifacts and "swat" has insane bones

Details

Textured models were mostly working at this commit. Rigged/animated models were able to load, but while "Lara" appeared to animate correctly and have reasonable bone count, "bat" and "seagull" had severe animation artifacts, and "swat" still has insane bone count.

The "elephant" and "CesiumMan" models fail to load; it is clear these models never worked in assimp at all.

Screenshots

Models that load OK

With the exceptions of "elephant" and "CesiumMan" most models load OK (so not all are shown here for brevity)

12 Feb 2020 assimp commit 58b9dcb
Model	Screenshot	Notes
aliveai_character.m3d		1 mesh 6 bones 1 animation Looks correct
bat.m3d		Obvious animation artifacts
lara.m3d		Model has 1 mesh Model has 313 animations Model indicates it has 15 bones in 1 mesh (seems correct)
mobs_dwarves_character.m3d		1 mesh 6 bones 1 animation Looks correct
seagull.m3d		Obvious animation artifacts
swat.m3d		Model has 4 meshes Model has 24 animations Model indicates it has 47,120 bones in 760 meshes (despite only having 4 meshes) (Because of massive bone count, implementation can't keep up and model appears fragmented)
batmobile.m3d
tie.m3d

Models that fail to load

Forced to conclude these models never worked at all with assimp

12 Feb 2020 assimp commit 58b9dcb
Models failing to load	Notes
Elephant	Validation failed: aiTexture::pcData is NULL
CesiumMan.m3d	Validation failed: aiTexture::pcData is NULL

@bztsrc So you were correct that most of the models loaded at one point, except that "elephant" and "CesiumMan" models clearly never worked in assimp, so there's no working state to be restored.
Also, as of the final M3D PR, there were obvious problems with skeletal animations; doesn't seem worth the effort to restore broken animations so will leave things here (unless I missed a commit fixing the situation, in which case please let me know)

[tellypresence]

Feedback from the original author below, in case there's useful information for future reference in the event someone else wants to investigate

Thank you so much for your efforts!

assimp. Not sure whether you're still actively maintaining the M3D format,

Well, I'm actively maintaining the M3D code base, but the M3D format and the in-memory structure that the SDK uses hasn't changed from the very beginning, it is as stable as it can be. Nothing has changed on the M3D side as long as Assimp concerned.

but if so please consider addressing these issues to expand your user base -- thank you

I'm sorry, but I'm the M3D developer and not an Assimp developer. As I've said, the M3D SDK's API and in-memory format hasn't changed, and it used to work in Assimp but if not any more then the problem is on the Assimp side for sure, and I'm not willing to spend time with that because the Assimp devs were total j**ks (except kimkulling, he was always nice and professional).

I wrote a detailed description https://gitlab.com/bztsrc/model3d/-/tree/master/assimp#model-with-skeletons on how the M3D in-memory structure should be mapped to Assimp in-memory nodes, also here is my Assimp wrapper https://gitlab.com/bztsrc/model3d/-/blob/master/assimp/code/M3D/M3DImporter.cpp code (with lots of comments and explanations) that used to work with rigged models and with the elephant too.

I'm afraid that's all the help I can provide for Assimp integration.

Cheers, bzt