How language models work drill

A product manager reports that your assistant, which drafts legal clauses well, told a customer that "accommodation" has one m and miscounted the digits in a 16-character policy number. Support is asking whether the model is broken. Explain in plain terms why this happens, which other features are exposed to the same failure, and what to ship so it stops reaching customers.

You must self-host an open-weight model for an internal code assistant on two GPUs with 80 GB of memory each, p95 latency under 4 seconds for a 500-token answer, and about 300 requests an hour. The shortlist is a 70B dense model and a mixture-of-experts model with 400B total parameters and 40B active per token, which scores better on code in your own evals. Which do you deploy, and why?

A customer-support bot was re-tuned with a preference step on 40,000 rater-labelled answer pairs. Satisfaction scores rose, but a week later audits show it agrees when customers insist a refund window is 60 days when policy says 30, and it flips correct answers after one push-back in 38 percent of sampled conversations. Diagnose the cause and lay out the fix.

You run three workloads on a reasoning model with an effort setting: ticket classification into 40 labels at two million requests a day with a p95 budget of 600 ms; code review comments on pull requests at 20,000 a day with a p95 budget of 30 seconds; and overnight migration plans at 200 a day with no latency limit. Thinking tokens are billed as output at five times the input price. Set the effort for each and justify the cost.