Supported Models

Comprehensive guide to AI models and parameters supported by Julep

Overview

Julep leverages LiteLLM to seamlessly connect you to a wide array of Language Models (LLMs). This integration offers incredible flexibility, allowing you to tap into models from various providers with a straightforward, unified interface.

With our unified API, switching between different providers is a breeze, ensuring you maintain consistent functionality across the board.

Available Models

While we provide API keys for quick testing and development, you’ll need to use your own API keys when deploying to production. This ensures you have full control over your usage and billing.

Looking for top-notch quality? Our curated selection of models delivers excellent outputs for all your use cases.

Anthropic

Here are the Anthropic models supported by Julep:

Model Name	Context Window	Best For
claude-3-opus	200K tokens	Complex reasoning, analysis
claude-3-sonnet	200K tokens	General purpose tasks
claude-3-haiku	200K tokens	Quick responses
claude-3.5-sonnet	200K tokens	Improved reasoning
claude-3.5-sonnet-20240620	200K tokens	Enhanced reasoning capabilities
claude-3.5-sonnet-20241022	200K tokens	Latest improvements

Google

Here are the Google models supported by Julep:

Model Name	Context Window	Best For
gemini-1.5-pro	1M tokens	Complex tasks
gemini-1.5-pro-latest	1M tokens	Cutting-edge performance

OpenAI

Here are the OpenAI models supported by Julep:

Model Name	Context Window	Best For
gpt-4-turbo	128K tokens	Advanced reasoning
gpt-4o	128K tokens	Balanced performance
o1-mini¹	128K tokens	Quick tasks
o1-preview¹	128K tokens	Testing features
o1¹	128K tokens	General tasks

¹ Heads up: The O1 series models are temporarily unavailable due to some OpenAI service issues. We’re working to get them back online soon!

Groq

Here are the Groq models supported by Julep:

Model Name	Context Window	Best For
llama-3.1-70b	8K tokens	Long-form content
llama-3.1-8b	8K tokens	Quick processing

OpenRouter

Here are the OpenRouter models supported by Julep:

Model Name	Context Window	Best For
mistral-large-2411	128K tokens	High performance
qwen-2.5-72b-instruct	131K tokens	Complex instructions
eva-llama-3.33-70b	128K tokens	Creative tasks
l3.1-euryale-70b	128K tokens	Creative tasks
l3.3-euryale-70b	8K tokens	Creative tasks
magnum-v4-72b	8K tokens	Creative tasks
eva-qwen-2.5-72b	8K tokens	Creative tasks
hermes-3-llama-3.1-70b	8K tokens	Creative tasks
deepseek-chat	32K tokens	Conversational AI

Embedding

Here are the embedding models supported by Julep:

Model Name	Embedding Dimensions	Best For
text-embedding-3-large	1024	High-quality vectors
voyage-multilingual-2	1024	Cross-language tasks
voyage-3	1024	Advanced embeddings
Alibaba-NLP/gte-large-en-v1.5	1024	Cost-effective solutions
BAAI/bge-m3	1024	Cost-effective solutions
vertex_ai/text-embedding-004	1024	Google Cloud integration

Though the models mention above support different embedding dimensions, Julep uses fixed 1024 dimensions for all embedding models for now. We plan to support different dimensions in the future.

Supported Parameters

These parameters allow fine-tuning of model behavior. Note that not all parameters are supported by every model.

Core Parameters

Parameter	Range	Description
temperature	0.0 - 5.0	Controls randomness in outputs. Higher values (e.g., 0.8) increase randomness, while lower values (e.g., 0.2) make output more focused and deterministic
top_p	0.0 - 1.0	Alternative to temperature for nucleus sampling. Only tokens with cumulative probability < top_p are considered. We recommend adjusting either this or temperature, not both
max_tokens	≥ 1	Maximum number of tokens to generate in the response

Penalty Parameters

Parameter	Range	Description
frequency_penalty	-2.0 - 2.0	Penalizes tokens based on their frequency in the text. Positive values decrease repetition
presence_penalty	-2.0 - 2.0	Penalizes tokens based on their presence in the text. Positive values decrease likelihood of repeating content
repetition_penalty	0.0 - 2.0	Penalizes repetition (1.0 is neutral). Values > 1.0 reduce likelihood of repeating content
length_penalty	0.0 - 2.0	Penalizes based on generation length (1.0 is neutral). Values > 1.0 penalize longer generations

Advanced Controls

Parameter	Range	Description
min_p	0.0 - 1.0	Minimum probability threshold compared to the highest token probability
seed	integer	For deterministic generation. Set a specific seed for reproducible results
stop	list[str]	Up to 4 sequences where generation should stop

Best Practices:

• Start with default values and adjust based on your needs
• Use temperature (0.0 - 1.0) for most cases
• Avoid setting multiple penalty parameters simultaneously
• Test different combinations for optimal results

Setting extreme values for multiple parameters may lead to unexpected behavior or poor quality outputs.

Usage Guidelines

Model Selection Guidelines

When picking a model, think about:

• Your budget and cost constraints
• How fast you need responses
• The quality you’re aiming for
• The context window size you require

Best Practices

• Start with smaller models for development and testing
• Use larger context windows only when necessary
• Keep an eye on token usage to manage costs

For more information, please refer to the LiteLLM documentation.