MVP for the GRIN Project

[GunpowderGuy]

The GRIN project has been dormant recently, and I’d like to reignite interest by developing an MVP (Minimum Viable Product) that includes essential features for a functional backend capable of handling real-world programs. I believe the Idris2 backend could serve this purpose with a few key enhancements: basic FFI (Foreign Function Interface) support and garbage collection.
Proposed Features:

Foreign Function Interface (FFI):
Currently, only GRIN backends can define FFI functions (RTS functions). To quickly enable user programs to perform FFI, a potential solution is to implement an RTS function that indirectly calls the user-specified external function. While this is a "quick and dirty" approach, it could expedite FFI support for user programs in the MVP phase.

Garbage Collection
I’ve found little concrete documentation about using LLVM’s GC facilities in an actual compiler. While LLVM's GC infrastructure is powerful, it might be too complex for an MVP. Although reference counting is simple to implement, it would likely result in poor performance without optimizations.
A potential compromise could be integrating a plug-and-play garbage collector like Boehm. It offers a balance between ease of integration and reasonable performance, allowing us to move forward quickly without significant performance trade-offs.

Call for Feedback and Partners:

I’m looking for feedback on this approach and seeking partners who would like to work this out alongside me. I will be available for full production starting in mid-December, and I’m eager to collaborate to push GRIN forward with a solid foundation for real-world usage.

[luc-tielen]

I haven't spoken with @csabahruska in a while, but last I heard there was a lot of focus on tooling (like GHC-WPC) to get better insights while working on GRIN.

[csabahruska]

Recently I've learned GHC Cmm language, and it is also suitable to use as a GRIN target language. In that case we also could use all GHC RTS features, like GC, STM, concurrency. With GHC's LLVM backend which translates Cmm to LLVM we can have the benefits of LLVM also. IMO this would be the simplest way to have a GRIN MVP.
What do you think?

[GunpowderGuy]

Grin is still not anywhere near from needing anything other than a basic GC, right? Taking that into account i wanted to reuse the existing code generation for the minimal changes necessary. However the haskell llvm library Grin depends on, has bit rotten. If that isnt easy to fix, we might as well try to build a better backend soon.

Besides cmm, i can think of 3 alternatives.

Mlton/MPL MPLLang/mpl#206

This idris2 llvm backend has a GC the creator and i can be reused by grin : https://sr.ht/~cypheon/rapid/
The llvm datatypes defined there could also be reused for re writing code gen in idris2. Of note is that he is beginning to add concurrency to that RTS and will also experiment with a cranelift backend. If so that would be another option

Wasm GC. I gather wasm has become a good backend for functional languages due to its support of GC, which some engines implement with Generations. This may not be the fastest option but could be a good starting point as that IR has loads of documentation

[luc-tielen]

@GunpowderGuy you could consider using https://GitHub.com/luc-tielen/llvm-codegen. Its not on hackage (I should add it there 😅) and has support for newer llvm versions, with a similar API. But it can only generate IR, not parse it. (Idk if grin needs that capability.)

[GunpowderGuy]

@luc-tielen I dont think Grin needs to parse llvm.

Would using your library be a closer to rewriting the code gen or be a more plug and play change? If the former, i would personally prefer to write a new code gen ( targetting llvm or otherwise ) in idris2. I havent actually written haskell projects yet

[luc-tielen]

@GunpowderGuy the API is very similar, with a few small changes for more type safety (but I think that was mostly internal).

[GunpowderGuy]

Kickass, then i could try to port GRIN to use that library. Btw here is the current code : https://github.com/csl-club/Idris2-Grin

That repo contains both the idris2 backend and a GRIN version modified by Z-snails . No other Grin branch/fork has more recent changes.

@csabahruska Would you still be interested in writing a new code gen in the near term? or should i update the current one when i become free ?

[GunpowderGuy]

@luc-tielen Could you upload your llvm library to stackage?

[luc-tielen]

@GunpowderGuy I added it to hackage, that should be usable from stack too: https://hackage.haskell.org/package/llvm-codegen-0.1.0.0

No docs atm, but the API is very similar (mostly imports differ). Feel free to read through the code (IRBuilder / ModuleBuilder modules), and look at the exported functions / instructions.

Let me know if some are missing and we can add more instructions as needed! For the Eclair compiler I only needed a very minimal set..

[GunpowderGuy]

@csabahruska I am attempting to upgrade Grin's dependencies, but the package "infix functor" doesnt compile with modern stackage versions and hasnt been updated since 2017

[csabahruska]

I've forked and updated the functor-infix to compile with newer GHC and TH. Please use this fork for now.
https://github.com/csabahruska/functor-infix

[GunpowderGuy]

Thanks csaba. I have kept updating the dependencies and found out there is a current llvm hs fork that is trying to get merged .

llvm-hs/llvm-hs#430

We should be able to use it instead of revamping the backend. However a bug in that branch is currently holding me back from completing GRIN's transition to a modern stackage version. I hope it gets solved in the following days

[GunpowderGuy]

@luc-tielen Given you seem to be an expert on llvm, could i ask you some questions privately about it? The point is learn how to best use it, or whether i should focus on another tecnology ( such as cranelift )

[luc-tielen]

Yeah sure, but just keep in mind that I am super busy irl so finding a time to talk may not be easy. 😅