Open-Source LLM API

Our OpenAI-compatible chat API is trained to be a helpful assistant. We are supporting 3 state of the art open-source AI models (Mixtral AI, Llama 3.1 8B and Llama 3.1 70B). They can be utilized for a wide range of applications and excel in tasks such as commonsense reasoning, world knowledge, reading comprehension, code-related tasks, and more. Test it for yourself in our AI API playground.

API Usage

Our OpenAI-compatible API takes a list of messages as input and provides an AI-generated (assistant) message as output. If you haven't already, you will need to create an API key to authenticate your requests.

// npm install --save openai or yarn add openai
import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.lemonfox.ai/v1",
});

const completion = await openai.chat.completions.create({
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "How many days are in a year?" },
  ],
  model: "llama-8b-chat",
});

console.log(completion.choices[0].message.content);

API Response

{
  "id": "cmpl-8a9ba025b8a744e881636351a26e4642",
  "object": "chat.completion",
  "created": 1697721484,
  "model": "llama-8b-chat",
  "choices": [
      {
          "index": 0,
          "message": {
              "role": "assistant",
              "content": "There are typically 365 days in a year. However, in a leap year, which occurs every four years, there are 366 days. Leap years are used to account for the extra fraction of a day that is not included in a regular year."
          },
          "finish_reason": "stop"
      }
  ],
  "usage": {
      "prompt_tokens": 30,
      "total_tokens": 90,
      "completion_tokens": 60
  }
}

You can obtain only the generated assistant message with the following code:

response['choices'][0]['message']['content']

API Parameters

The API POST https://api.lemonfox.ai/v1/chat/completions takes the following parameters:

Tips

• Conversations often begin with a system message, followed by alternating user and assistant messages.
• The system message shapes the assistant's behavior and can be customized, but it's not mandatory.
• User messages make requests or comments, while assistant messages store prior responses or demonstrate desired behavior.
• It can be helpful to include conversation history when user instructions refer to past messages, as models have no memory of previous requests.