Skip to main content

ChatKonko

Konko API is a fully managed Web API designed to help application developers:

Konko API is a fully managed API designed to help application developers:

  1. Select the right LLM(s) for their application
  2. Prototype with various open-source and proprietary LLMs
  3. Access Fine Tuning for open-source LLMs to get industry-leading performance at a fraction of the cost
  4. Setup low-cost production APIs according to security, privacy, throughput, latency SLAs without infrastructure set-up or administration using Konko AI’s SOC 2 compliant, multi-cloud infrastructure

Steps to Access Models​

  1. Explore Available Models: Start by browsing through the available models on Konko. Each model caters to different use cases and capabilities.

  2. Identify Suitable Endpoints: Determine which endpoint (ChatCompletion or Completion) supports your selected model.

  3. Selecting a Model: Choose a model based on its metadata and how well it fits your use case.

  4. Prompting Guidelines: Once a model is selected, refer to the prompting guidelines to effectively communicate with it.

  5. Using the API: Finally, use the appropriate Konko API endpoint to call the model and receive responses.

To run this notebook, you’ll need Konko API key. You can create one by signing up on Konko.

This example goes over how to use LangChain to interact with Konko ChatCompletion models

To run this notebook, you’ll need Konko API key. You can create one by signing up on Konko.

from langchain.schema import HumanMessage, SystemMessage
from langchain_community.chat_models import ChatKonko

2. Set API Keys​


Option 1: Set Environment Variables​

  1. You can set environment variables for
    1. KONKO_API_KEY (Required)
    2. OPENAI_API_KEY (Optional)
  2. In your current shell session, use the export command:
export KONKO_API_KEY={your_KONKO_API_KEY_here}
export OPENAI_API_KEY={your_OPENAI_API_KEY_here} #Optional

Alternatively, you can add the above lines directly to your shell startup script (such as .bashrc or .bash_profile for Bash shell and .zshrc for Zsh shell) to have them set automatically every time a new shell session starts.

Option 2: Set API Keys Programmatically​

If you prefer to set your API keys directly within your Python script or Jupyter notebook, you can use the following commands:

konko.set_api_key('your_KONKO_API_KEY_here')  
konko.set_openai_api_key('your_OPENAI_API_KEY_here') # Optional

Calling a model​

Find a model on the Konko overview page

Another way to find the list of models running on the Konko instance is through this endpoint.

From here, we can initialize our model:

chat = ChatKonko(max_tokens=400, model="meta-llama/llama-2-13b-chat")
messages = [
    SystemMessage(content="You are a helpful assistant."),
    HumanMessage(content="Explain Big Bang Theory briefly"),
]
chat(messages)
AIMessage(content="  Sure thing! The Big Bang Theory is a scientific theory that explains the origins of the universe. In short, it suggests that the universe began as an infinitely hot and dense point around 13.8 billion years ago and expanded rapidly. This expansion continues to this day, and it's what makes the universe look the way it does.\n\nHere's a brief overview of the key points:\n\n1. The universe started as a singularity, a point of infinite density and temperature.\n2. The singularity expanded rapidly, causing the universe to cool and expand.\n3. As the universe expanded, particles began to form, including protons, neutrons, and electrons.\n4. These particles eventually came together to form atoms, and later, stars and galaxies.\n5. The universe is still expanding today, and the rate of this expansion is accelerating.\n\nThat's the Big Bang Theory in a nutshell! It's a pretty mind-blowing idea when you think about it, and it's supported by a lot of scientific evidence. Do you have any other questions about it?")