Skip to content

Pick your style: Denkbaar

Book a conversation

Articles

History

xAI releases Grok-1 as open weights

·

xAI published the weights of Grok-1, a 314-billion-parameter model, under the Apache 2.0 licence. Large, but not trained for conversation.

A worker with a laptop examines a large cabinet of accelerator cards and a purple model block.
Grok-1 is open, but needs a lot of GPU memory. Illustration: Denkbaar.

What happened

On 17 March 2024 xAI published the weights and architecture of Grok-1. Weights are the numbers that hold a model’s knowledge. With the weights, you can run the model yourself. Grok-1 has 314 billion parameters and is a “mixture of experts”: for each token only part of the model works, about 25 percent. xAI released it under Apache 2.0, a licence that also allows business use.

It was the raw base model from October 2023. So it was not trained to hold a conversation or follow instructions. The example code on GitHub itself says that you need a machine with a lot of GPU memory, and that the code is not fast.

A large model cabinet contains many blocks, only a small share of them purple.
For each token, only part of Grok-1 works. Illustration: Denkbaar.

Why it mattered then

In 2024 the number of large open-weight models grew fast. Grok-1 was one of the largest. It showed that companies with a closed chat service also release their older models.

A worker compares a licence booklet, an accelerator card and a laptop with instruction boxes.
Check licence, hardware and instruction training before choosing an open model. Illustration: Denkbaar.

What it means now

Open weights are not the same as usable. A base model of this size, without instruction training, makes no sense for a small company to run itself. With any open model, check three things first: the licence, the size (does it fit on affordable hardware?) and whether it was trained to follow instructions.

Sources

  1. Open Release of Grok-1 (xAI)
  2. xai-org/grok-1 (GitHub)

Topics: open-weightsxaimodels