its so fascinating to me that model performance can fluctuate so strongly on a day-to-day basis. Why is claude actually performing significantly worse right now? does anyone have a good writeup of why exactly this happens? like, more than just 'overcapacity'? marginlab.ai/trackers/cla...
Replies:
4
like, what actually happens? do they have quantized models that they can route all queries through when they are under high demand?
and if thats the case, the implication that you have a random change to get a significantly worse model while paying the same prices for whats labelled as the same model is uhhh, fascinating, and seems like a massive potential for regulatory conflict
@laurenshof.online but the regulators have no teeth
they dont, but at the same time: lots of companies are paying top dollar to get access to opus, and with this performance degradation they could use other models to get better performance at a fraction of the cost
