@0xglitch/llm-sdk

@0xglitch/llm-sdk

Unified LLM Gateway Library with Google Gemini primary support.

Features

• Google Gemini Support (PRIMARY): gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.0-flash, gemini-2.0-pro, gemini-1.5-pro, gemini-1.5-flash
• OpenAI Support (OPTIONAL): gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-mini, gpt-4-turbo, o3-mini
• Anthropic Support (OPTIONAL): claude-sonnet-4.5, claude-opus-4.1, claude-sonnet-4, claude-3.5-sonnet, claude-3.5-haiku
• Intelligent Model Routing: Complexity-based model selection
• Retry Logic: Exponential backoff (3 retries, 2s/4s/8s delays)
• Token Counting: Accurate token usage tracking for all providers
• Streaming Support: Real-time response streaming
• Fallback Mechanism: Automatic provider switching on failure

Installation

npm install @workflow/llm-sdk

Quick Start

import { LLMClient } from '@workflow/llm-sdk';

// Initialize with Gemini only (default)
const client = new LLMClient({
  geminiApiKey: process.env.GOOGLE_API_KEY
});

// Simple completion
const response = await client.complete({
  prompt: 'Explain quantum computing in simple terms',
  complexity: 'simple' // Uses gemini-2.5-flash-lite (2025 default)
});

console.log(response.content);
console.log(`Tokens used: ${response.tokenUsage.total}`);

Configuration

import { LLMClient } from '@workflow/llm-sdk';

const client = new LLMClient({
  // Primary provider (required)
  geminiApiKey: process.env.GOOGLE_API_KEY,

  // Optional fallback providers
  openaiApiKey: process.env.OPENAI_API_KEY,
  anthropicApiKey: process.env.ANTHROPIC_API_KEY,

  // Default provider preference
  defaultProvider: 'gemini', // 'gemini' | 'openai' | 'anthropic'

  // Retry configuration
  retries: 3,
  retryDelays: [2000, 4000, 8000]
});

Model Selection

The library automatically selects the best model based on task complexity:

• Simple tasks → gemini-2.5-flash-lite (2025 default: fast, low-cost, high-performance)
• Complex tasks → gemini-2.5-pro (2025 default: most powerful with adaptive thinking)
• Fallback → OpenAI (gpt-4.1-nano/gpt-4.1) or Anthropic (claude-sonnet-4/claude-sonnet-4.5) if configured

// Simple task (uses gemini-2.5-flash-lite - 2025 default)
const simpleResponse = await client.complete({
  prompt: 'What is 2+2?',
  complexity: 'simple'
});

// Complex task (uses gemini-2.5-pro - 2025 default)
const complexResponse = await client.complete({
  prompt: 'Write a comprehensive analysis of quantum computing applications',
  complexity: 'complex'
});

Streaming

const stream = await client.streamComplete({
  prompt: 'Tell me a story',
  complexity: 'simple'
});

for await (const chunk of stream) {
  process.stdout.write(chunk.content);
}

Token Counting

// Count tokens before making a request
const tokens = await client.countTokens('Your prompt here');
console.log(`Estimated tokens: ${tokens}`);

// Check token usage after completion
const response = await client.complete({
  prompt: 'Your prompt'
});
console.log(`Input tokens: ${response.tokenUsage.input}`);
console.log(`Output tokens: ${response.tokenUsage.output}`);
console.log(`Total tokens: ${response.tokenUsage.total}`);

Error Handling

try {
  const response = await client.complete({
    prompt: 'Your prompt'
  });
} catch (error) {
  if (error.code === 'RATE_LIMIT_EXCEEDED') {
    console.error('Rate limit exceeded, try again later');
  } else if (error.code === 'PROVIDER_UNAVAILABLE') {
    console.error('All providers unavailable');
  } else {
    console.error('Unexpected error:', error.message);
  }
}

Environment Variables

# Required
GOOGLE_API_KEY=your_gemini_api_key

# Optional (for fallback)
OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key

API Reference

See API.md for complete API documentation.

License

MIT