@mayintoronto and that's why the study design is basically one round of a normal tech interview
RE: https://hachyderm.io/@jenniferplusplus/115998391921660848
Complete with weird, unnecessary time pressure and a total fixation on production volume despite insisting the goal is to assess something like learning, thinking, or working style
@jenniferplusplus provocative question: was the study design maybe designed in a way to generate a specific outcome 🤔
@sci_photos I don't have any reason to think so. If that's what they were doing, I doubt they'd be happy with the result they got.
This is complete speculation, but I really do think they just don't know how to design learning studies with human subjects. And it didn't even occur to them that there is such a thing as doing it well or poorly, or that they should consult with an expert. So they just grabbed the first format that came to mind, which was a programming interview. Then they added a control group without ever considering what the actual difference was between the two test conditions.
@jenniferplusplus Okay, now in retrospect; during my Physics diploma I did not encounter “human subjects”, and would also not be prepared to design such a study. But, I guess, I would seek for advice then … ?
@jenniferplusplus how can they possibly link those metrics to the outcomes they claim to be measuring? Like, does velocity go up as learning improves, or does it go down because the dev is taking on more challenging problems?
I hate it when tech bros pretend they understand learning without putting in any effort to learn about learning.
