Skip to content

Latest commit

 

History

History
88 lines (54 loc) · 4.33 KB

README.md

File metadata and controls

88 lines (54 loc) · 4.33 KB
Raw

Pico MLX Server

Pico MLX Server is the easiest way to get started with Apple's MLX AI framework

Pico MLX Server provides a GUI for MLX Server. MLX server provides an API for local MLX models conform the OpenAI API. This allows you to use most existing OpenAI chat clients with Pico MLX Server.

Menu extra screenshot

Highlights

  • Start and stop servers quickly via menu bar extra
  • Download MLX models from the MLX community on HuggingFace
  • Install and setup Python environment, MLX, and MLX Server from within Pico MLX Server.
  • Run multiple servers on different ports
  • View logs of the servers in separte windows
  • Custom link to open your favorite chat client (defaults to Pico AI Assistant)

Supported MLX models

See MLX Community on HuggingFace

Getting Started

To install Pico MLX Server, build the source using Xcode, or download the notarized executable directly from GitHub.

To set up Pico MLX Server, open the app and

  • Install and set up Python, pip, MLX and MLX Server and optionally Conda manually
  • Use Pico MLX Server's automated Setup (MLX -> Setup...)
  • Pico MLX Server uses Conda by default to create a virtual environment for the servers called pico. This (hopefully) will avoid any Python version issues. Conda can be disabled in Settings

Setup window screenshot

Requirements

  • MacOS 14.0 (Sonoma) or later

Create a New Server

Menu extra screenshot

  • Select MLX -> Servers -> New Server...
  • Press Create to create the default server mlx-community/Nous-Hermes-2-Mistral-7B-DPO-4bit-MLX on port 8080
  • To use a different model, click on the v button or type in a model manually from the MLX Community on HuggingFace (make sure to use the mlx-community/ prefix)
  • Press the View Logs buttons to open a window with the server's real-time logs

Menu extra screenshot

Use Pico MLX Server with an AI client

  • Point any OpenAI API compatible AI assistant to http://127.0.0.1:8080 (or any other port you used in Pico MLX Server). (Instructions for Pico AI Assistant coming soon)

  • Curl:

    curl -X GET 'http://127.0.0.1:8080/generate?prompt=write%20me%20a%20poem%20about%the%20ocean&stream=true'

API Endpoints

Known Issues

  • Pico MLX Server doesn't detect if a port is already in use (use lsof -i:8080 in the terminal to find the PID of the running server)
  • There is a SwiftUI issue in New Server window and the Servers menu where the state of servers isn't updated

Roadmap

  • Switch from Python to MLX Swift
  • Swift-based HTTP server

Related projects

Pico MLX Server is part of a bundle of open source Swift tools for AI engineers. Looking for a server-side Swift OpenAI proxy to protect your OpenAI keys? Check out Swift OpenAI Proxy.

Authors and Acknowledgements

Pico MLX Server, Swift OpenAI Proxy, and Pico AI Assistant were created by Ronald Mannak with help from Ray Fernando

MLX Server was created by Mustafa Aljadery & Siddharth Sharma

Code used from MLX Swift Chat and Swift Chat