Workflow:Togethercomputer Together python Chat Completion
| Knowledge Sources | |
|---|---|
| Domains | LLMs, Inference, API_Integration |
| Last Updated | 2026-02-15 16:00 GMT |
Overview
End-to-end process for generating chat completions using the Together Python SDK, supporting text, multimodal, streaming, and tool-calling interactions.
Description
This workflow covers the standard procedure for sending conversational messages to Together AI's hosted language models and receiving generated responses. It supports plain text messages, multimodal inputs (images and videos), streaming token delivery, async parallel requests, logprobs retrieval, and function/tool calling. The SDK wraps the Together REST API with Pydantic-typed request and response models, providing both synchronous and asynchronous clients.
Usage
Execute this workflow when you need to interact with a hosted LLM through a conversational interface. This applies to building chatbots, generating text completions from instructions, analyzing images or videos with vision models, calling external tools via function calling, or processing multiple prompts concurrently with the async client.
Execution Steps
Step 1: Client Initialization
Create a Together client instance configured with your API key. The client can be initialized with an explicit API key parameter or by reading the TOGETHER_API_KEY environment variable automatically. For concurrent workloads, use the AsyncTogether client instead.
Key considerations:
- The API key can be passed directly or set via environment variable
- The base URL defaults to the Together API but can be overridden
- Use AsyncTogether for parallel request patterns with asyncio
Step 2: Message Construction
Build the messages array following the OpenAI-compatible chat format. Each message has a role (system, user, assistant) and content. For multimodal inputs, the content field becomes an array of typed content objects (text, image_url, video_url).
Key considerations:
- System messages set the assistant's behavior and persona
- For vision models, content is an array with text and image_url/video_url objects
- Multiple images can be included in a single message for comparison tasks
- Message history should be maintained for multi-turn conversations
Step 3: Request Configuration
Configure generation parameters such as model selection, temperature, max_tokens, top_p, stop sequences, and optional features like streaming, logprobs, response format (JSON mode), and tool definitions.
Key considerations:
- Model must be a valid Together-hosted model identifier
- Temperature controls randomness (0 = deterministic, higher = more creative)
- Set stream=True for token-by-token delivery
- Tools array defines available functions for tool-calling models
- response_format can enforce JSON output structure
Step 4: Response Handling
Process the API response based on the request mode. For standard requests, access the generated text via response.choices[0].message.content. For streaming requests, iterate over chunks and concatenate delta content. For tool calls, inspect the tool_calls array in the response message.
Key considerations:
- Standard responses return a ChatCompletionResponse with choices array
- Streaming returns an iterator of ChatCompletionChunk objects
- Each chunk's delta may contain partial content or tool call fragments
- Logprobs are available on the response when requested
- Token usage statistics are included in the response object
Step 5: Error Handling
Handle SDK-specific exceptions that map to HTTP status codes. The SDK raises typed errors for authentication failures, rate limits, invalid requests, and server errors, each inheriting from a common TogetherException base.
Key considerations:
- AuthenticationError for invalid or missing API keys
- RateLimitError when throughput limits are exceeded
- InvalidRequestError for malformed parameters
- The SDK includes automatic retry with exponential backoff for transient errors