Import instruments from ADL format

[Wohlstand]

I have got a question about Dune II songs and they're sounding on OPL chip: WohlSoft/Moondust-Project#294. I thought they are using AIL's mechanism with AD/OPL banks like other games do, however, they are using AIL for SFX, GeneralMIDI, and MT32. For OPL they are using own odd crap.

Music from the game include ADL files: http://www.vgmpf.com/Wiki/index.php/Dune_II:_The_Building_of_a_Dynasty_(DOS)

The ADL format itself: http://www.shikadi.net/moddingwiki/ADL_Format

I think the format is similar to the IMF, but that possibly something different. I would like to try to play it on libADLMIDI also if that possible. First off I need the proper specification of format to work on it.

[Wohlstand]

Получаетя, что нету дампов музыки в формате OPL в свободном доступе?
я тут на vogons пообщался с хозяином. Он просвятил в xmi формате, что это для MT-32 синтезаторов.Поэтому я и не мог из этих xmi получить , ТО что слышу в DOSBOX.

Хотел бы еще спросить, в dune0.xmi там идут всякие треки после самого длинного трека.
Сейчас прослушал три штуки, оказывается эти все "после" какой то набор звуков одинаковый в каждом XMI.
Получается, там можно хранить разные звуки переключаясь между ними по номер трека?

Мне тут одна личность, сделала DRO =DosBox Raw OPL дамп.
у меня DRO файл. Приложил. Я его проиграл в winamp+in_adlib плагин, проигралось, но не так прям как в dosbox. Но это самое близкое.
dune2_000.dro.zip

CREATIVE LABS WAVEBLASTER OWNERS:

  As of 09/22/93 this patch has been updated to support Sound Blaster 16     
  and Wave Blaster owners.  To receive mixed sound support (digitized  
  sound from the SB16, and music from the Wave Blaster) do the following:

  1.  Follow steps 1 - 3 as shown above.
  
  2.  In the SETUP program assign sound (music) option to 
  Sound Canvas (General MIDI), sound effect option to SBPRO, and 
  digitized option to SBPRO.
  
  3.  After exiting SETUP.EXE, copy the files WAVESET.BAT and WAVESET.DAT
  to your LANDS of LORE directory.

  4.  Type WAVESET anytime you wish to play DUNE II in
  Wave Blaster/SB16 mode.         

-Westwood Studios Audio Department-

Поможет ли это ? что за WAVESET

Оставлю это тут:
soundChunk[Sound_CreditsTick] = loadMixFromADL("DUNE1.ADL", 52, 4*MIX_MAX_VOLUME); soundChunk[Sound_Tick] = loadMixFromADL("DUNE1.ADL", 38);

`
sdl2::mix_chunk_ptr SFXManager::loadMixFromADL(const std::string& adlFile, int index, int volume) const {

auto rwop = pFileManager->openFile(adlFile);
auto pSoundAdlibPC = std::make_unique<SoundAdlibPC>(rwop.get(), AUDIO_FREQUENCY);
pSoundAdlibPC->setVolume(volume);
sdl2::mix_chunk_ptr chunk{ pSoundAdlibPC->getSubsong(index) };

return chunk;

}

Mix_Chunk* SoundAdlibPC::getSubsong(int Num) {
Uint8* buf = nullptr;
int bufSize = 0;
bool bSilent = true;

playTrack(Num);

do {
    bufSize += 1024;
    if((buf = static_cast<Uint8*>(SDL_realloc(buf, bufSize))) == nullptr) {
        THROW(std::runtime_error, "Cannot allocate memory!");
    }

    memset(buf + bufSize - 1024, 0, 1024);

    SoundAdlibPC::callback(this, buf + bufSize - 1024, 1024);

    bSilent = true;
    for(Uint8* p = buf + bufSize - 1024; p < buf + bufSize; p++) {
        if(*p != 0) {
            bSilent = false;
            break;
        }
    }

    if(bufSize > 1024*1024*16) {
        SDL_Log("SoundAdlibPC::getSubsong(): Decoding aborted after 16MB have been decoded.");
        break;
    }

} while(isPlaying() || !bSilent);

Mix_Chunk* myChunk;
if((myChunk = static_cast<Mix_Chunk*>(SDL_calloc(sizeof(Mix_Chunk), 1))) == nullptr) {
    return nullptr;
}

myChunk->volume = 128;
myChunk->allocated = 1;
myChunk->abuf = (Uint8*) buf;
myChunk->alen = bufSize;
return myChunk;

}
`

Смотри, я нашел в проекте dunelegacy , код, который читает музыку из ADL файла.

Приложил сюда исходник этих двух файлов.
adlib.zip

@alexeik dude gave a useful information for work, and he have attached a code sample that is able to work with Dune's ADL files: adlib.zip

[Wohlstand]

Мне тут одна личность, сделала DRO =DosBox Raw OPL дамп.
у меня DRO файл. Приложил. Я его проиграл в winamp+in_adlib плагин, проигралось, но не так прям как в dosbox. Но это самое близкое.
dune2_000.dro.zip

in_adlib использует очень неточный эмулятор чипа OPL3, и очень сильно глючит, пробовал я через него играть разную музыку в своё время, в итоге из всего, что было, самый лучший вариант, это конечно же самопал на базе одного из имеющихся на публике решений.

[alexeik]

Перешел в эту ветку.

да in_adlib играет тяп ляп через xmplay или winamp

у меня есть полный код этого проекта из которого взял adlib.zip, он из dune legacy проекта.

Смотрел уже целые сутки на этот C++ код.
Чтото магии много стало в С++.
Трюк с этим:

    memset(buf + bufSize - 1024, 0, 1024);
    SoundAdlibPC::callback(this, buf + bufSize - 1024, 1024);
    bSilent = true;

Не понятен. Этот callback уходит в такие дали, что даже код не понятно как читать.

+почему код, так плохо форматируется на гитхабе?
+какую книжку посмотреть, чтобы взять в толк этот callback вызов ?

[alexeik]

Что, скажешь?

[Wohlstand]

Пока ещё не посмотрел детально, но как посмотрю, отпишусь обязательно 🦊

[alexeik]

https://github.com/OmniBlade/libeastwood

еще вот, проект, который может читать из ADL.
оказывается, этот проект, того же автора , что и dunelegacy:)

[Wohlstand]

Надо будет глянуть...

[alexeik]

может я тебе задоначу 1к , чтобы ты посмотрел? :)
в дискорде есть? я могу показать, готовность дюны2.
может тебе дать git путь для dunelegacy проекта
там есть sln для студии под c++
но я не осилил настроить окружение под c++
А если настроить, то там будет легко через дебаг все разобрать.
git://dunelegacy.git.sourceforge.net/gitroot/dunelegacy/dunelegacy

вот тут лежит sln:
dunelegacy\IDE\VC\DuneLegacy.sln
собственно, тот класс, которые я в зипе прикладывал, ты там и найдешь.

[Wohlstand]

В Дискорде есть, конечно же 🦊, я там Wohlstand#4881.
Умеешь уговорить, а так, спасибо ещё раз за напоминание, надо бы проверить всё это... Всё ещё зависит от свободного времени, от нахождения меня дома, и от срочности имеющихся задач (сегодня я исправил некоторые критические баги в одном проекте, на которые жаловались)

[Wohlstand]

Можешь даже подключить моего друга @jpcima, он тоже смыслит в этих делах нехило, и мы часто работаем вместе. Он говорит по-французски, но английский хорошо знает, если что.

[jpcima]

I've no idea what's said about me here.. let's translate this 😃

[jpcima]

It seems there doesn't exist a clean specification unfortunately.
It's reverse-engineered code which has numerous unknowns.

There exists some players of it. If it's esentially extracting instruments which is desired, it's no doubt feasible to dump the tracks into vgm files.

[alexeik]

@Wohlstand окей)
@jpcima hi
i asked @Wohlstand about midi format that dune2(westwood studios game) has.

1. post dune2_000.dro.zip zip file where is the music placed.

2. git://dunelegacy.git.sourceforge.net/gitroot/dunelegacy/dunelegacy
   we have this repo where is a code that can play ADL as FM OPL3 ( as dosbox does)

3. dunelegacy\IDE\VC\DuneLegacy.sln it has visual studio 2010 solution. but i cant setup env for c++.

i dont understand about midi playing.
so im searching for someone who can help to make a player for this ADL files or just dump them to mp3.

because of that @Wohlstand suggest your person :)

and u MUST DOWNLOAD DUNE2 ORIGINAL for PAK files to run DuneLegacy
Dune Legacy.zip
or use that.

[Wohlstand]

вот тут лежит sln: dunelegacy\IDE\VC\DuneLegacy.sln

Мы для разработки используем Qt Creator, CLion или Emacs, поэтому, нам хватит CMakeLists.txt для сборки на CMake, и да, мы все линуксоиды, на винду залазим только для отладки виндоспецифичных фич.

Я только что стянул репу с Dune Legacy, и вижу там сборку на AutoTools, так что "./confugure; make -j 9; make install" всегда рулит тут 😁

[alexeik]

@Wohlstand you are lucky))
я не думал, что вы линуксоиды, кстати.

[jpcima]

The adlib.zip, it's the same code I've seen in a variety of places.
It's also included in the tool adplay which is able to play back these files.

[alexeik]

@jpcima but they doesnt play adl like a dosbox i suppose.
i used AdPlug in winamp and its has poor quality of playing

[jpcima]

A tool has been created based on parts of dunelegacy engine. (found at jpcima/dunelegacy repository)

It plays back the set of tracks, and also it has permitted to extract a set of VGM dumps, that can be replayed on more accurate emulators.

It's not a full track set, for now it's only non-looping ones.
vgm.zip

[alexeik]

nice. tested.
plays well.

[Wohlstand]

Dune Music build for Windows
DuneMusic.zip
(To make it work, put all .PAK files from the DUNE II game into the "data" folder you see in the archive)

[alexeik]

@jpcima
hi again
can i ask u to help me , make dunemusic as a library ?

i imagine only one function for this library:
bytes() getPCMfromADL( "dune0.adl",tracknumber")

[jpcima]

@alexeik ok I don't mind to look at the OpenRA project, if non-Windows are able to run it.
Keep in mind, I've no experience in C# development whatsoever.

[alexeik]

@jpcima to look openRA?
why openRA?
You misunderstand me:)
C-library from dunemusic.exe :)
and compile it with /clr flag.
its enough to consume C-library in .net

and if u want to loop openRA u r welcome:)

[jpcima]

@alexeik ok I understand ! well yeah preferably I like to have it interact with the engine.
I can surely make a library of it.

[alexeik]

maybe this can be suitable for .net interop
https://github.com/mono/CppSharp

seems to be /clr does not what i want. it tries to compile C++ to MSIL :(

[jpcima]

I've posted the music player as a library. See dunemusic.h inside repository.
https://github.com/jpcima/libdunemusic

A problem is : Nuked OPL works as emulator, Woody OPL also, but not DOSbox.
The music plays very noisy in that emulator I can't find why, so I leave it like this for a moment.

[alexeik]

thank you
strange. strange. wait for @Wohlstand

[alexeik]

@jpcima
how to use that?:)

[jpcima]

It contains play.cpp as a usage example.

• The function DuneMusic_Init is called, setting a folder where PAK files are found.

• DuneMusic_GetSamples obtains 16bit PCM samples. count is number of frames (same as number of samples / number of channels).

• DuneMusic_ChangeMusic(Ex) to control the current track playing.

[alexeik]

@jpcima
can u make another function that accept stream of bytes instead folder where PAK files placed?

[Wohlstand]

@alexeik , I don't think it's a good idea, you better to just pass a path to PAK files. Pointer to raw bytes of PAK files is required for only cases where these packs are packed inside of another archive.

[alexeik]

@Wohlstand its only one idea.
openRA has own filesystem that can give a stream of bytes for given files.

[jpcima]

The access into files is wrapped by SDL's RWops, so it can't be too difficult to make it to wrap streams of anything.

@alexeik are byte streams representing PAK files or ADL files?

[alexeik]

adl files

[alexeik]

@jpcima can u to consider
soundChunk[Sound_CreditsTick] = loadMixFromADL("DUNE1.ADL", 52, 4*MIX_MAX_VOLUME); soundChunk[Sound_Tick] = loadMixFromADL("DUNE1.ADL", 38);
this interface to access this TWO sounds placed in ADL ?

[jpcima]

I've added the memory API DuneMusic_InsertMemoryFile at jpcima/libdunemusic@e31f280

I hope it's working, I have't tried myself.
Filename is supposed to be such as "DUNE0.adl"

this interface to access this TWO sounds placed in ADL ?

there exists a sound effects management that I'm aware of but it's not yet in the library.
It would have to simulate the operation of SDL-mixer's Mix_Chunk and also handle the mixing volumes between music and sfx.

[alexeik]

@jpcima thank u
DUNE1.ADL 52 and 38 means not effects
i mean to allow read this elements from ADL file as PCM.

[Wohlstand]

i mean to allow read this elements from ADL file as PCM.

The only way is only play them. They are should be generated in real time: you getting a context of specific song or SFX, and then, you streaming the audio with DuneMusic_GetSamples() call

[jpcima]

@Wohlstand I refer to this when I mentioned Adlib-based sfx.
You can see Dunelegacy is invoking the Adlib callback and caching the result into a sample buffer.

https://github.com/jpcima/dunelegacy/blob/64600047e5b1270449b2667475fc40d40c6ad488/src/FileClasses/adl/sound_adlib.cpp#L2533

[Wohlstand]

I see, which is used for SFX, then yeah.

[jpcima]

I've added a function to synthesize a sample for audio effects.
jpcima/libdunemusic@1e08d36

I also have to modify the generating function.
It checks for a sequence of zeroes to determine silence, but silent nukedOPL can produce ±1 values.

DuneMusic_SynthesizeAudio returns the data pointer, gets the number of stereo frames in numFramesReturned.

[alexeik]

nice work
how to use?

[jpcima]

It's how to produce and save a chunk to a file, for example.
The file can be loaded by raw import into Audacity as 16bit signed.
(also I have passed -1 as the volume, which has identical meaning to passing the default volume 64.)

    DuneMusic_Init(kSampleRate, argv[1], oplEmu);

    size_t length;
    int16_t *audio = DuneMusic_SynthesizeAudio("DUNE1.ADL", 38, -1, &length);
    if (!audio) return;

    fprintf(stderr, "Got Audio %p Length %zu\n", audio, length);

    FILE *fh = fopen("sfx.dat", "wb");
    if (fh) {
        fwrite(audio, sizeof(int16_t), 2 * length, fh);
        fclose(fh);
    }

    DuneMusic_FreeAudio(audio);

[jpcima]

In the latest, Dosbox emulator was fixed.
As it turned out, it would fill only half buffer when it's computing in OPL2 mode, compared to OPL3.

[alexeik]

@jpcima can u make some binary for windows?
or tell me how i can make it

[jpcima]

@alexeik I added some automatic builds. The build process is in appveyor.yml.
It needs SDL2.dll at the dll's side, and probably visual studio runtimes installed.

https://github.com/jpcima/libdunemusic/releases

[alexeik]

seems to i must to compilte it myself because i need to attach this library to .net

[jpcima]

@alexeik, check this https://github.com/flibitijibibo/SDL2-CS/blob/master/src/SDL2.cs

@Wohlstand has shared me this method which is used by a SDL binding to C#.

It's able to use the native library direct from C# language, by the DllImport functionality.
The dunemusic library can be used the same. What is needed, it's to translate the interface of dunemusic.h into C# equivalent.

[jpcima]

@alexeik I've made the following in Monodevelop a bit ago. It builds, but I don't know if it works.
https://github.com/jpcima/libdunemusic/blob/csharp/binding/DuneMusic-CS/DuneMusic.cs

[alexeik]

cant get, how to use that
so much methods.
if i have Stream and want to get PCM.

its all about filename i must pass to dunemusic.
can i pass stream of bytes?

void FileManager::insertMemoryFile(const std::string &filename, const void *data, size_t length) { memoryFiles[strToUpper(filename)] = {data, length}; }
found

got some

nor x86 , nor x64 can not be loaded from DllImport.
seems to be all code are valid, but .net cant find