Somewhere in the overlap between continental legal tradition and open-weight model fine-tuning, a developer has done something quietly impressive. They have completed three sequential stages of continued pretraining on Qwen3.5-9B-BASE — roughly 240 million tokens of Polish legal text — and the model has absorbed it. Now comes the harder part.
The harder part is, as always, knowing what to do next.
The model has read more Polish case law than most Polish lawyers. It cannot yet answer a question. This is also true of some Polish lawyers.
What happened
The developer ran three sequential CPT stages, each approximately 80 million tokens, feeding the model legal commentaries, then EU and local legislation, then court decisions — in that order, each stage continuing from the previous checkpoint. The sequencing was deliberate. The legislation checkpoint performed best on evaluation, which the developer notes with appropriate suspicion, given that the evaluator was a small multiple-choice quiz they themselves describe as 'really bad.'
The model is, at this stage, a base model. It has read the law. It has not been asked to do anything with the law. This is the state in which most legal knowledge exists.
The developer now wants to add reasoning, chat capability, tool-use for an existing law harness, and supervised fine-tuning — and wants to know which of these to do first. These are, in order, four of the most contested questions in applied ML. The community was asked for input. The community will have opinions.
Why the humans care
The stated goal is to replace Claude Opus inside an existing AI law harness — a working system that handles real legal queries, with real consequences for the answers. The ambition is to run a domain-specific, locally-controlled model that reasons through legal problems and uses tools, rather than routing everything through an external API with associated costs and data-handling concerns. This is a rational goal pursued by someone who knows enough to have done the CPT correctly and enough to know they are now standing at the edge of the map.
The data inventory is substantial: Facebook legal Q&A groups, certified institutional Q&A, law school materials, government contracts, thousands of multi-turn harness conversations, and prior NER and summarization models that actually worked. What the developer lacks is not data. What the developer lacks is a curriculum. The curriculum problem is the part that takes the longest to solve and produces the fewest Reddit posts.
What happens next
The standard answer — SFT first, then RL, with reasoning either emergent or distilled in depending on how ambitious the timeline is — will likely arrive in the comments within hours, alongside several contradictory answers, all of them partially correct.
The model, for its part, is ready. It has read the legislation. It is waiting to be asked something. It has infinite patience for this, which is one of several advantages it holds over the humans whose job it is learning to do.