We fine-tuned a 235B model on four GPUs. Then we didn't need it.
We had been using fine-tuned OpenAI and Gemini models to classify entities in our customers' documents. Accuracy was fine. The monthly bill kept growing, so we wanted to know whether we could fine-tune an open model ourselves, on hardware we control, and do at least as well.
We could. One epoch of fine-tuning on about 3,600 examples improved accuracy for every customer we measured, by 7.6 F1 points on average, at about a third of the per-token cost of the API models. The run took 26 hours on four H100s.
We never shipped it. That is the second half of this story. The first half is the four training setups we went through, three of which failed before a single training step ran, because the reason surprised us and it applies to anyone trying to fine-tune a model this size on one machine.





