Support AutoAWQ

[daniel-fahey]

No description provided.

[MatthewCroughan]

@daniel-fahey I see in the readme of AutoAWQ that they say:

AutoAWQ has been merged into 🤗 transformers.

Does this not mean we only have to update transformers and AutoAWQ will be present inside of it?

[daniel-fahey]

@MatthewCroughan I think the developer means support for AutoAWQ has been merged in to transformers, but you'd still need the autoawq package available (see upstream: https://github.com/oobabooga/text-generation-webui/blob/f7534b2f4b6ceca5e00d9cf6af2d25b744af3d06/modules/models.py#L299C39-L299C39). I marked this PR as draft because after needing to bump text-generation-webui 's version there are still quite a few changes to get this flake working nicely with it (need to upgrade Gradio with a new Nix expression, and update the patchedSrc in projects/textgen/package.nix, probably more to do). After a relatively long compile I was able to get quantised AWQ models running with fairly impressive performance though.

[daniel-fahey]

@MatthewCroughan further re: AutoAWQ merging into transformers (v4.35.0) not only does text-generation-webui still automatically select the AutoAWQ loader as mentioned above, transformers also still requires the autoawq package even when using transformer's AutoModelForCausalLM class (see transformers upstream).