How to Use OpenAI’s GPT-OSS Models: A Hands-on Tutorial

The world of artificial intelligence was recently met with a significant development: OpenAI, a company long associated with powerful but proprietary models, has released a family of new “open-weight” models called GPT-OSS. This is a game-changer for the open-source community, providing access to powerful models trained with OpenAI’s advanced techniques, specifically designed for robust reasoning and efficient deployment.

This blog post will serve as your comprehensive guide to the GPT-OSS models. We’ll dive into their unique technical specifications, show you how to run them using popular tools like Hugging Face, and discuss the hardware requirements and best practices for getting the most out of these groundbreaking models.

The GPT-OSS Family

The GPT-OSS family currently consists of two key models:

• GPT-OSS-120B: A larger, more powerful model with 117 billion parameters. It’s designed to be on par with models like OpenAI’s o4-mini on reasoning benchmarks. This model requires significant hardware, but its performance on complex tasks is exceptional.
• GPT-OSS-20B: A smaller, more efficient model with 21 billion parameters. It is optimized for use on consumer hardware, offering performance comparable to o3-mini on many tasks. This model is a perfect entry point for those wanting to run a powerful local AI without a massive hardware investment.

Both models are released under a permissive Apache 2.0 license, signifying a major commitment by OpenAI to the open-source ecosystem.

Technical Specifications and Unique Features

Unlike many other models, the GPT-OSS family is specifically optimized for advanced reasoning and “agentic” workflows. Here’s what sets them apart:

• Mixture-of-Experts (MoE) Architecture: Both models utilize an MoE architecture, which allows them to achieve high performance while being more efficient than a traditional dense model of a similar size. This architecture allows the models to activate only a subset of their parameters for any given task, leading to faster inference.
• 4-bit Quantization (MXFP4): To make these large models more accessible, they are heavily optimized with a 4-bit quantization scheme. This significantly reduces their memory footprint and allows them to run on hardware with less VRAM. The GPT-OSS-20B model, for instance, can run on a single GPU with as little as 16 GB of memory.
• Exceptional Reasoning and Tool Use: The models were trained using techniques from OpenAI’s advanced internal models, making them exceptionally good at following instructions, using external tools (like a web browser or calculator), and performing chain-of-thought (CoT) reasoning. This makes them ideal for building sophisticated AI agents.

How to Use GPT-OSS Models

While OpenAI has released the model weights, the most straightforward way to use them is through the robust ecosystem built around the Hugging Face platform.

1. Using Hugging Face Transformers

For developers and those who prefer a command-line interface, the Hugging Face Transformers library is the go-to method.

Installation: First, make sure you have the necessary libraries installed: pip install transformers accelerate bitsandbytes

Loading and Inference: You can load the models directly from the Hugging Face Hub. The bitsandbytes library is crucial here for handling the 4-bit quantized versions.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "openai-gpt-oss/gpt-oss-20b"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    load_in_4bit=True
)

prompt = "What is the capital of France?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    inputs.input_ids,
    max_new_tokens=50
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Using LM Studio and Ollama

For a more user-friendly, no-code experience, tools like LM Studio and Ollama are quickly adding support for the new GPT-OSS models. These applications provide a graphical user interface for downloading, managing, and interacting with local models.

• LM Studio: Simply open LM Studio, navigate to the search bar, and look for “gpt-oss.” You’ll find community-contributed quantized versions of the models. Download the one that fits your hardware, then go to the “Chat” tab to start a conversation.
• Ollama: Ollama’s command-line interface makes it easy to pull and run models. You can use the command ollama run gpt-oss-20b to download and start interacting with the model directly from your terminal.

Hardware Requirements

Running the GPT-OSS models locally requires careful consideration of your hardware.

• GPT-OSS-120B: This model is designed for high-end systems. You will need a GPU with a significant amount of VRAM, with 80 GB being the recommended amount for unquantized inference. You may be able to run quantized versions on GPUs with 48 GB or more of VRAM.
• GPT-OSS-20B: This is the more accessible model. The 4-bit quantized version can run on a single consumer-grade GPU with at least 16 GB of VRAM. This makes it an excellent option for users with GPUs like the NVIDIA RTX 4080/4090 or certain AMD cards. For CPU-based inference, you’ll need at least 32 GB of RAM, but be prepared for significantly slower performance.

Best Practices

• Start with GPT-OSS-20B: If you’re new to local LLMs, the 20B model is the best place to start. Its smaller footprint and efficient architecture make it an ideal choice for experimentation.
• Leverage Agentic Capabilities: These models are not just for basic chat. Test their ability to follow complex instructions, use tools, and perform multi-step reasoning tasks to see their true power.
• Customize and Fine-tune: The Apache 2.0 license allows you to fine-tune these models on your own data. This is perfect for building domain-specific assistants or enhancing their performance on specialized tasks.
• Stay Involved with the Community: The open-source AI community is moving at a rapid pace. Follow forums and the Hugging Face blog to stay updated on new optimizations, fine-tuned versions, and creative use cases for the GPT-OSS models.

The release of the GPT-OSS models is a landmark moment. It blurs the line between proprietary and open-source AI, offering a glimpse into a future where powerful, advanced models are accessible to everyone. By following this guide, you can start exploring the potential of these models and be at the forefront of this new wave of AI innovation.