@david_chisnall
(settling into my comfy IANAL armchair...)
Maybe, but it complicates the heck out of any trial. Proving workplace discrimination is already outrageously difficult. Now you're facing the additional challenges that the defendant has no discoverable documents describing their arguably discriminatory decision or biased decision-making process. Their contracted ATS AI probably has no useful and discoverable logs, either.
So what are you left with? Maybe you could convince a judge to make the model and its training data available through discovery, but we're talking trade secrets and an unreasonable amount of data. And at best you'd end up with some situation where an expert witness says the model is biased in general when examined in a lab but you can't prove that this specific decision negatively impacting the plaintiff was due to bias. And even getting this far would be prohibitively expensive for any real world plaintiff. And the LLMs themselves are often nondeterministic even with the exact same input, which is often *by design* (i.e., temperature) for aesthetic reasons.
Am I missing something here?