Inkling vs. GPT-4V (Vision)

Inkling is an open-weights multimodal AI model from Thinking Machines built to support customization, agentic workflows, coding, reasoning, vision, audio, and enterprise AI use cases. The model is a Mixture-of-Experts transformer with 975 billion total parameters, 41 billion active parameters, 256 routed experts per MoE layer, and six routed experts active per token. It supports context windows up to 1 million tokens and was pretrained on 45 trillion tokens across text, images, audio, and video. Inkling is designed as a broad foundation model rather than a narrowly optimized benchmark model, giving it balanced capabilities across reasoning, coding, factuality, instruction following, vision, audio, tool use, and safety. Its controllable thinking effort lets developers adjust how much computation and generated reasoning the model uses, helping teams balance quality, latency, and cost for different production needs. The model can run agentic coding tasks, use tools, create web apps, generate polished multi-page artifacts, reason over long contexts, and work through iterative refinement loops. For multimodal tasks, Inkling can process images, answer questions about visual content, transcribe and reason over audio, follow spoken instructions, and combine visual reasoning with code-based tools such as Python. Thinking Machines trained Inkling for calibration, instruction following, factual reliability, refusal behavior, and safety across multiple modalities, including evaluations for dangerous capabilities and human-AI threat vectors. Inkling is available on Tinker for fine-tuning, with 64K and 256K context options, an Inkling Playground for testing, cookbook recipes, and support for multimodal post-training workflows. Its full weights are available on Hugging Face, and deployment support is available through APIs and infrastructure partners such as TogetherAI, Fireworks, Modal, Databricks, Baseten, SGLang, vLLM, llama.cpp, and transformers.

The recent development of GPT-4 with vision (GPT-4V) empowers users to instruct GPT-4 to analyze image inputs they submit, representing a pivotal advancement in enhancing its capabilities. Experts in the domain regard the fusion of different modalities, such as images, with large language models (LLMs) as an essential facet for future advancements in artificial intelligence. By incorporating these multimodal features, LLMs have the potential to improve the efficiency of conventional language systems, leading to the creation of novel interfaces and user experiences while addressing a wider spectrum of tasks. This system card is dedicated to evaluating the safety measures associated with GPT-4V, building on the existing safety protocols established for its predecessor, GPT-4. In this document, we explore in greater detail the assessments, preparations, and methodologies designed to ensure safety in relation to image inputs, thereby underscoring our dedication to the responsible advancement of AI technology. Such initiatives not only protect users but also facilitate the ethical implementation of AI breakthroughs, ensuring that innovations align with societal values and ethical standards. Moreover, the pursuit of safety in AI systems is vital for fostering trust and reliability in their applications.