Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New Feature] Add new route for dllama api for embeding models #96

Open
testing0mon21 opened this issue Jul 1, 2024 · 5 comments
Open

Comments

@testing0mon21
Copy link

std::vector<Route> routes = {
    {
        "/v1/chat/completions",
        HttpMethod::METHOD_POST,
        std::bind(&handleCompletionsRequest, std::placeholders::_1, &api)
    },
    {
        "/v1/models",
        HttpMethod::METHOD_GET,
        std::bind(&handleModelsRequest, std::placeholders::_1)
    }
};

in ddlama api at master branch we have only 2 routes /v1/chat/completions and /v1/models but some model looks like llama3:8b has embedding functionality. Can add you add new route for /api/embeddings ?
@testing0mon21
Copy link
Author

@b4rtaz what do you think about new route?

@testing0mon21
Copy link
Author

i mean this for ddlama api

Generate Embeddings

POST /api/embeddings

Generate embeddings from a model

@testing0mon21
Copy link
Author

i hope that i explain) @b4rtaz

@testing0mon21
Copy link
Author

what do you think about this revision or should I explain in more detail? @b4rtaz

@b4rtaz
Copy link
Owner

b4rtaz commented Jul 10, 2024

Hello @testing0mon21 I'm not too familiar with embedding, but if I see correctly, llama.cpp supports it. This is not a priority for me, but contributions are welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants