Which LLM to use?

"Which LLM should I use?" is the wrong first question. The better question is "which LLM should I use for this specific task?" because the honest answer changes depending on what you're asking it to do. Here's a practical way to think about the decision instead of chasing whichever model topped last week's benchmark.

Section 01: There's No Single Best Model

Every major model family has genuine strengths and genuine weaknesses, and none of them wins across every category. Some are stronger at careful, nuanced writing and following complex, multi-part instructions precisely. Some are faster and cheaper, better suited to high-volume, simple tasks. Some have particularly large context windows, useful for reasoning over long documents in one pass. Choosing based on a single headline benchmark score, rather than your actual use case, is one of the most common and avoidable mistakes.

Section 02: Match the Model to the Task, Not the Task to the Model

For quick, high-volume, low-stakes work, summarizing, tagging, simple extraction, a fast and inexpensive model is usually the right call. Running your most expensive, most capable model on a task like "sort these 500 support tickets by urgency" is overkill; the cost adds up fast for marginal quality gain.

For complex reasoning, nuanced writing, or anything where getting it right matters a lot, a strategy document, a sensitive customer communication, genuinely difficult analysis, it's worth reaching for a more capable (and typically pricier) model. The cost difference on a single important document is negligible; the cost of a mediocre result on something that matters is not.

For anything involving a very long document, a full contract, a lengthy research report, a model's context window size becomes the deciding factor regardless of its other strengths. A brilliant model that can't hold your whole document in memory at once will force you to chunk it up, losing coherence across the pieces.

🔑

Remember: Cost and capability usually move together. The fix isn't always "use the best model for everything," it's "use the right-sized model for each specific task," the same way you wouldn't hire your most senior, expensive consultant to do basic data entry.

Section 03: Test on Your Actual Work, Not a Demo

Public benchmarks measure general capability on standardized tests, not your specific use case. Before settling on a model for a recurring task, run your actual, real prompt (with your actual data and your actual format requirements) through two or three candidates and compare the results side by side. The model that wins on a general leaderboard doesn't always win on your specific, idiosyncratic task.

Section 04: Build Flexibility Into Your Workflow

Where possible, avoid hard-locking your workflow to a single model's quirks. Models are updated and replaced frequently, and today's best choice for a task may not be next year's. Keep your prompts reasonably portable (clear, well-structured instructions rather than tricks specific to one model's peculiarities) so switching models later doesn't mean rebuilding your entire process.

Key Takeaways

No model wins at everything; the right choice depends on the specific task.

Match model cost and capability to task complexity: cheap and fast for high-volume simple work, capable and careful for high-stakes work.

Context window size becomes the deciding factor for very long documents, regardless of other strengths.

Test on your actual use case before committing, general benchmarks don't predict performance on your specific task.

Keep prompts reasonably portable so you're not locked into one model as the landscape keeps shifting.