#Tag · bonfire.cafe

Do people learn more from #AI decision assistants?

This experiment found insignificant improvement in learning during and after three forms of AI-assisted decision-making, compared to human-only decision-making.

https://doi.org/10.48550/arXiv.2602.10222

#edu #teaching #cogSci #eduTech #compSci

4 media

"As shown in Figure 13, there is no significant difference across treatments for both the learning during intervention (𝐹 (3, 398) = 1.046, 𝑝 = 0.372) and learning after intervention (𝐹 (3, 398) = 1.193, 𝑝 = 0.312)."

"Figure 13: Comparisons on participants’ learning (a) while receiving the AI assistance intervention (b) after receiving the AI assistance intervention. Values for the Human-only treatment are computed based on the normalized change of decision accuracy between corresponding sessions of tasks (learning during intervention: tasks 6–15 vs. tasks 1–5, learning after intervention: tasks 16–20 vs. tasks 1–5); they provide a baseline for organic learning happened due to repetitive task completion without AI assistance interventions. Error bars represent 95% confidence intervals of the mean values."

arXiv.org

Understanding the Effects of AI-Assisted Critical Thinking on Human-AI Decision Making

Despite the growing prevalence of human-AI decision making, the human-AI team's decision performance often remains suboptimal, partially due to insufficient examination of humans' own reasoning. In this paper, we explore designing AI systems that directly analyze humans' decision rationales and encourage critical reflection of their own decisions. We introduce the AI-Assisted Critical Thinking (AACT) framework, which leverages a domain-specific AI model's counterfactual analysis of human decision to help decision-makers identify potential flaws in their decision argument and support the correction of them. Through a case study on house price prediction, we find that AACT outperforms traditional AI-based decision-support in reducing over-reliance on AI, though also triggering higher cognitive load. Subgroup analysis reveals AACT can be particularly beneficial for some decision-makers such as those very familiar with AI technologies. We conclude by discussing the practical implications of our findings, use cases and design choices of AACT, and considerations for using AI to facilitate critical thinking.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · last week

Why might #AI-assisted #dermatology decisions improve by about 10%?

Across 4,905 unaided and AI-aided decisions, clinicians were more likely to revise when the AI disagreed than when it agreed — even when the AI was wrong!

http://lup.lub.lu.se/student-papers/record/9221420

#medicine #tech #edu #cogSci

3 media

"Accuracy outcomes reflected this same structural pattern. In True Conflict, where physicians were initially incorrect and the AI correct, decision changes produced a clear net accuracy gain of 13.20%, consistent with movement toward the correct answer. In False Conflict, where physicians began with the correct answer but the AI was incorrect, decision changes resulted in a net accuracy loss of 4.24%, indicating that many revisions replaced a correct answer with an incorrect one."

"where physicians were initially incorrect and the AI provided the correct answer"... "The linear-by-linear association [between clinician accuracy and decision correction] was ...significant, χ² = 44.80, p < .01, indicating a clear monotonic trend: physicians with higher diagnostic accuracy were increasingly likely to correct their initial error when confronted with accurate AI disagreement. The effect size was medium, Cramér’s V = 0.362 (p < .01)."

"when physicians initially provided the correct diagnosis and the AI gave an incorrect answer" there was "no significant association between [clinician accuracy] and decision change, χ²(3, N = 146) = 4.53, p = .210. The linear-by-linear association was also non-significant, χ² = 1.16, p = .281, and the overall association was weak and non-significant (Cramér’s V = 0.176, p = .210)."

Why Better Information Does Not Ensure Better Decisions Evidence from AI-Supported Clinical Diagnosis | LUP Student Papers

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 weeks ago

Do neuroscientists' claims about #freeWill impact students' #ethics?

A "deterministic passage ...from Crick" sometimes reduced decisions to proactively intervene in moral thought experiments compared to a "neutral #neuroscience text".

https://doi.org/10.3389/fpsyg.2026.1748028

#xPhi #cogSci #edu

2 media

Figure 1. "Responses showed the expected baseline patterns across dilemmas. The determinism manipulation reduced willingness to intervene in the Spur dilemma (p = 0.0385, fewer participants pulled the switch) and reduced willingness to help in the Singer scenario (p = 0.0261), but had no detectable effect on Footbridge judgments (p = 0.783)."

Free will and trolley dilemmas: evidence for moral inertia in a Venezuelan sample

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 weeks ago

An #ethics Turing test?

Twenty people texted a #chatbot over a month before a 2-hour interview about whether they thought the #AI was able to extract, embody, and explain human values.

Thirteen "left the study convinced..."

https://doi.org/10.48550/arXiv.2601.22440

#philosophy #edu #LLMs #tech

arXiv.org

AI and My Values: User Perceptions of LLMs' Ability to Extract, Embody, and Explain Human Values from Casual Conversations

Does AI understand human values? While this remains an open philosophical question, we take a pragmatic stance by introducing VAPT, the Value-Alignment Perception Toolkit, for studying how LLMs reflect people's values and how people judge those reflections. 20 participants texted a human-like chatbot over a month, then completed a 2-hour interview with our toolkit evaluating AI's ability to extract (pull details regarding), embody (make decisions guided by), and explain (provide proof of) human values. 13 participants left our study convinced that AI can understand human values. Participants found the experience insightful for self-reflection and found themselves getting persuaded by the AI's reasoning. Thus, we warn about "weaponized empathy": a potentially dangerous design pattern that may arise in value-aligned, yet welfare-misaligned AI. VAPT offers concrete artifacts and design implications to evaluate and responsibly build value-aligned conversational agents with transparency, consent, and safeguards as AI grows more capable and human-like into the future.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 weeks ago

How else can automated text messages educate medical patients?

Adding automated text messages to a paper-based decision aid program improved lung #cancer screening knowledge and screening rates.

https://doi.org/10.2196/69044

#medicine #edu #decisionScience #healthScreening #oncology

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 3 weeks ago

Practice 👏 in 👏 exam 👏 conditions

If you’re not allowed to rely on textbooks, slides, notes, peers, #AI, #chatbots, etc. on an exam, then you haven’t finished studying for that exam until you perform well *without* such assistance.

https://www.oecd.org/en/publications/oecd-digital-education-outlook-2026_062a7394-en.html

#edu #cogSci #tech

Figure 1.5. Successfully performing a task with GenAI does not automatically lead to learning

Türkiye: Comparison of high school students’ maths exam and practice results when practicing alone or with GenAI tools

Note: In this randomised controlled trial, access to GPT-4 improved student performance when practising mathematics exercises — especially the tutoring version. However, during closed-book exams, students who had used the base (or general-purpose) version of GPT-4 performed 17% worse than those who never had access.

Source: Bastani, H. O. Bastani, A. Sungu, H. Ge, O. Kabakcı and R. Mariman (2024), “Generative AI Can Harm Learning”, The Wharton School Research Paper, Available at SSRN: http://dx.doi.org/10.2139/ssrn.4895486. — Figure 1.5. Successfully performing a task with GenAI does not automatically lead to learning Türkiye: Comparison of high school students’ maths exam and practice results when practicing alone or with GenAI tools Note: In this randomised controlled trial, access to GPT-4 improved student performance when practising mathematics exercises — especially the tutoring version. However, during closed-book exams, students who had used the base (or general-purpose) version of GPT-4 performed 17% worse than those who never had access. Source: Bastani, H. O. Bastani, A. Sungu, H. Ge, O. Kabakcı and R. Mariman (2024), “Generative AI Can Harm Learning”, The Wharton School Research Paper, Available at SSRN: http://dx.doi.org/10.2139/ssrn.4895486.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 4 weeks ago

If your objection to something is just “it's biased”, then you do not yet have an objection.

I mention #theBiasFallacy in "Strategic Reflectivism":
https://doi.org/10.48550/arXiv.2505.22987

and explain it on my #blog: https://byrdnick.com/archives/11072/the-bias-fallacy

#logic #argumentation #debate #edu #criticalThinking

4 media

"'They’re biased, so they’re wrong!' That’s a fallacy. We can call it the bias fallacy. Here’s why it’s a fallacy: being biased doesn’t entail being wrong. So when someone jumps from the observation that So-and-so is biased to the conclusion that So-and-so is wrong, they commit the bias fallacy. It’s that simple.

In this post, I’ll give some examples of the fallacy, explain the fallacy, and then suggest how we should respond to the bias fallacy."

https://byrdnick.com/archives/11072/the-bias-fallacy — "'They’re biased, so they’re wrong!' That’s a fallacy. We can call it the bias fallacy. Here’s why it’s a fallacy: being biased doesn’t entail being wrong. So when someone jumps from the observation that So-and-so is biased to the conclusion that So-and-so is wrong, they commit the bias fallacy. It’s that simple. In this post, I’ll give some examples of the fallacy, explain the fallacy, and then suggest how we should respond to the bias fallacy." https://byrdnick.com/archives/11072/the-bias-fallacy

Nick Byrd, Ph.D.

The Bias Fallacy

“They’re biased, so they’re wrong!” That’s a fallacy. Why? Because being biased doesn’t entail being wrong. In this post, I’ll explain the bias fallacy, gi…

arXiv.org

Strategic Reflectivism In Intelligent Systems

By late 20th century, the rationality wars had launched debates about the nature and norms of intuitive and reflective thinking. Those debates drew from mid-20th century ideas such as bounded rationality, which challenged more idealized notions of rationality observed since the 19th century. Now that 21st century cognitive scientists are applying the resulting dual pro-cess theories to artificial intelligence, it is time to dust off some lessons from this history. So this paper synthesizes old ideas with recent results from experiments on humans and machines. The result is Strategic Reflec-tivism, the position that one key to intelligent systems (human or artificial) is pragmatic switching between intuitive and reflective inference to opti-mally fulfill competing goals. Strategic Reflectivism builds on American Pragmatism, transcends superficial indicators of reflective thinking such as model size or chains of thought, applies to both individual and collective intelligence systems (including human-AI teams), and becomes increasingly actionable as we learn more about the value of intuition and reflection.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · last month

Can Socratic reflection improve #AI answers to medical questions?

Adding a critic to a #languageModel pipeline improved performance on two measures of medical question-answering.

The improvement didn't depend on the critic's model.

https://doi.org/10.48550/arXiv.2601.04531

#tech #medicine #edu

4 media

"As illustrated in Fig 1, the system follows a loop that mirrors how clinicians gather evidence, generate a provisional explanation, and reassess whether their reasoning is sufficiently supported. At each iteration, the model retrieves context passages, produces an answer and r ationale, then evaluates that rationale through a scoring module. If parts of the rationale are unsupported or contradictory, the system reformulates the query to target missing information and repeats retrieval and generation. This reflection cycle allows Self MedRAG to progressively strengthen factual grounding while ensuring that the final answer and rationale remain clinically coherent, and evidence based."

"Medical question answering (QA) benchmarks evaluate a model’s ability to generate clinically reliable, evidence-grounded responses. Widely used datasets include MedQA for diagnostic reasoning from medical exams [and] PubMedQA for evidence-based biomedical inference over research abstracts...."

"The results presented in Table 1. demonstrate the performance trends across retrieval strategies and critic configurations. For Base RAG methods, hybrid retrieval using the combination of both BM25 and Contriever using Reciprocal Rank Fusion (RRF) achieves substantially stronger performan ce than any single retriever on both PubMedQA and MedQA dataset. While BM25 and Contriever individually reach accuracies of 66.80% and 67.90% on PubMedQA, their fusion through RRF slightly increases their performance accuracy to 69.10 %. The effect is more pronounced on MedQA, where the method introduces a large jump of performance from 41.74% (BM25 alone) and 43.30% (Contriever alone) to 80.00% accuracy. This dramatic improvement proves that the fused retrieval using RRF provides broader coverage of clinically relevant evidence by integrating both high precision lexical signals from BM25 and semantically aligned passages recovered by Contriever."

"...both critics surpass the non -critic, non-itera baseline, demonstrating that the improve in performance is due to the iteration mechanism itself, rather than the specific critic choice."

"Fig 3 details the cumulative impact of the iterative process done on Self Reflective module for both accuracy and F1 scores. We observe a substantial performance leap between the first and second iterations across both datasets, with MedQA accuracy rising from 79.3% to 86.1% and PubMedQA from 69.8% to 83.3%. The upward trend confirms the potential performance gains done by the Self-Reflective module in identifying and correcting unsupported rationales. Extending the process to a third iteration, however, seems to result in a diminishing return, with performance either plateauing for PubMedQA or slightly declining for MedQA." — "...both critics surpass the non -critic, non-itera baseline, demonstrating that the improve in performance is due to the iteration mechanism itself, rather than the specific critic choice." "Fig 3 details the cumulative impact of the iterative process done on Self Reflective module for both accuracy and F1 scores. We observe a substantial performance leap between the first and second iterations across both datasets, with MedQA accuracy rising from 79.3% to 86.1% and PubMedQA from 69.8% to 83.3%. The upward trend confirms the potential performance gains done by the Self-Reflective module in identifying and correcting unsupported rationales. Extending the process to a third iteration, however, seems to result in a diminishing return, with performance either plateauing for PubMedQA or slightly declining for MedQA."

arXiv.org

Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answering

Large Language Models (LLMs) have demonstrated significant potential in medical Question Answering (QA), yet they remain prone to hallucinations and ungrounded reasoning, limiting their reliability in high-stakes clinical scenarios. While Retrieval-Augmented Generation (RAG) mitigates these issues by incorporating external knowledge, conventional single-shot retrieval often fails to resolve complex biomedical queries requiring multi-step inference. To address this, we propose Self-MedRAG, a self-reflective hybrid framework designed to mimic the iterative hypothesis-verification process of clinical reasoning. Self-MedRAG integrates a hybrid retrieval strategy, combining sparse (BM25) and dense (Contriever) retrievers via Reciprocal Rank Fusion (RRF) to maximize evidence coverage. It employs a generator to produce answers with supporting rationales, which are then assessed by a lightweight self-reflection module using Natural Language Inference (NLI) or LLM-based verification. If the rationale lacks sufficient evidentiary support, the system autonomously reformulates the query and iterates to refine the context. We evaluated Self-MedRAG on the MedQA and PubMedQA benchmarks. The results demonstrate that our hybrid retrieval approach significantly outperforms single-retriever baselines. Furthermore, the inclusion of the self-reflective loop yielded substantial gains, increasing accuracy on MedQA from 80.00% to 83.33% and on PubMedQA from 69.10% to 79.82%. These findings confirm that integrating hybrid retrieval with iterative, evidence-based self-reflection effectively reduces unsupported claims and enhances the clinical reliability of LLM-based systems.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · last month

Looks like I’m not the only one to find that between-person differences in #criticalThinking didn’t replicate in within-person changes over time:

Testing #argumentMapping effects: https://www.researchgate.net/publication/390977878

Assessing #higherEd over time: https://doi.org/10.1007/s10212-025-01000-1

#edu #stats

4 media

"Rational thinking skills were assessed using a German test battery consisting of four subtests of heuristics and biases tasks."

"Cognitive Reflection was assessed using three items of the original CRT..."

"Resistance to Belief Bias was assessed using five items created by Markovits and Nantel (1989) and one new item."

"Resistance to Ratio Bias was assessed using four items by Toplak et al. (2014). In each item, the subjects’ task is to choose between a small and a large tray of white and black marbles...."

"Disjunctive Reasoning was assessed using the knights and knaves problem (Toplak & Stanovich, 2002; adapted from Shafir, 1994) and five new items. In all items, the participants’ task was to decide whether the question can be answered with yes, no, or not at all based on the given information." — "Rational thinking skills were assessed using a German test battery consisting of four subtests of heuristics and biases tasks." "Cognitive Reflection was assessed using three items of the original CRT..." "Resistance to Belief Bias was assessed using five items created by Markovits and Nantel (1989) and one new item." "Resistance to Ratio Bias was assessed using four items by Toplak et al. (2014). In each item, the subjects’ task is to choose between a small and a large tray of white and black marbles...." "Disjunctive Reasoning was assessed using the knights and knaves problem (Toplak & Stanovich, 2002; adapted from Shafir, 1994) and five new items. In all items, the participants’ task was to decide whether the question can be answered with yes, no, or not at all based on the given information."

"Prior to analyses, data were cleaned and checked for outliers. Responses to items of rational thinking skills that were unusually fast (3 SD or more below the item-specific log mean; following recommendations of Ratcliff, 1993) were considered unreliable and were deleted. This applied to 0.27% of all trials at t1 and 0.19% of all trials at t2. For continuous variables (fluid intelligence, rational thinking dispositions, rational thinking skills, student satisfaction), scores deviating more than 3 SD from the sample mean were excluded from path and regression analyses."

"our sample size" was "101, excluding missing values and outliers)" — "Prior to analyses, data were cleaned and checked for outliers. Responses to items of rational thinking skills that were unusually fast (3 SD or more below the item-specific log mean; following recommendations of Ratcliff, 1993) were considered unreliable and were deleted. This applied to 0.27% of all trials at t1 and 0.19% of all trials at t2. For continuous variables (fluid intelligence, rational thinking dispositions, rational thinking skills, student satisfaction), scores deviating more than 3 SD from the sample mean were excluded from path and regression analyses." "our sample size" was "101, excluding missing values and outliers)"

"Regarding rational thinking skills, there were no significant differences between t1 and t2 in the means of cognitive reflection (t(119) = 1.73, p =.086, d = 0.16; all means are depicted in Table 1), resistance to belief bias (t(123) = 0.53, p =.598, d = 0.05), resistance to ratio bias (t(123) = −0.64, p =.521, d = −0.06), and disjunctive reasoning (t(119) = −0.31, p =.756, d = −0.03). Regarding rational thinking dispositions, there was no significant difference in consideration of future consequences between t1 and t2 (t(123) = 0.85, p =.395, d = 0.08). Interestingly, the mean scores of students’ need for cognition slightly, but significantly, declined from t1 to t2 (t(123) = −2.07, p =.040, d = −0.19). Overall, these results suggest that the students’ rational thinking skills and dispositions remained very stable over the course of two semesters in terms of both their absolute level and their relative ranking."

Rational thinking in university students: Examining temporal stability and effects on academic performance and student satisfaction

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 months ago

I finally read this book about how W.E.B. #Dubois and colleagues challenged 19th and 20th century ideas and intuitions by visualizing data: https://amzn.to/49916Dz

Cheers to those who advance debate with clear, cogent, empirical arguments.

#stats #history #sociology #philosophy

Battle-Baptiste, W., & Rusert, B. (2018). W. E. B. Du Bois’s Data Portraits: Visualizing Black America. Princeton Architectural Press. ISBN 978-1-61689-706-2 https://amzn.to/49916Dz

https://amzn.to/49916Dz

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 months ago

Bonus: Dubois' data and visualizations from the early 20th century are available for free on #Github so that anyone can reproduce the infographics, run their own analyses, etc.:

The #dataViz: https://github.com/ajstarks/dubois-data-portraits/blob/master/orig-catalog2.pdf

#dataAnalysis #openAccess #archive #DuBoisChallenge2025 #edu

2 media

All 60-something data visualizations from Dubois et al's 1900 exhibit in Paris in PDF format from GitHub.

The 2025 Du Bois Visualization Challenge README.md file on GitHub.

dubois-data-portraits/orig-catalog2.pdf at master · ajstarks/dubois-data-portraits (PDF)

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 months ago

The #Philosophy dept. at @MontclairState will consolidate into a new "School of Human Narrative and Creative Expression" with #English and #Spanish?

And #Psychology will consolidate too?

No more departments (or dept. chairs)? 🤔

https://www.northjersey.com/story/news/education/2025/12/01/montclair-state-university-humanities-rebrand/87459471007/

#higherEd #philSci #edu

NorthJersey.com

Montclair State's rebrand of the humanities has faculty miffed

Faculty and students at Montclair State University are alarmed by a restructuring move that will eliminate humanities departments.

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 months ago

Are more reflective thinkers more or less likely to use #AI to solve complex problems?

In this sample of 500 students, AI barely correlated with reflective thinking, but the correlation was positive (r = 0.1) rather than negative.

https://doi.org/10.1007/s10639-025-13738-8

#edu #econ #psychology

4 media

"Of the 791 participants, 504 participants completed the sample by the end, and 487 participants remained after the treatment of outliers, and only these participants were considered for the analysis."

"The Pearson correlation coefficients showed that the higher scores on Solving economic problems are associated with high levels of Cognitive reflection, Need for cognition and a higher usage of AI (Table 2)."

"Solving economic problems (SEP) measured with a tool developed by one of the authors – Prof. of Financial Management. ...students were asked to use whatever computational and informational tools they wanted, if they did not communicate with their peers. The following tools were allowed: (1) search engine (Google, Bing or similar); (2) ChatGPT or similar; (3) Excel or similar applications; (4) paper and pencil; (5) others (had to specify which). At the end of each [of the four] problem[s], students had to tick which of the tools they used to solve it."

"For calculating the “Using AI” we computed the score using summing only the “ChatGPT or similar” for the problems. The score was between 0 and 3 (mean 0.91, SD 1.15) since Problem 4 was not completely solved by anyone, and we decided not to use it in the research."

Nick Byrd, Ph.D.

@ByrdNick@nerdculture.de · 2 months ago

#AI may exacerbate bad academic habits:

Science summaries from large language models ( #LLMs) were nearly five times more likely than human-authored summaries to contain broad generalizations (95% CI [3.06, 7.70], p < 0.001).

And newer language models over-generalized more; not less!

https://doi.org/10.1098/rsos.241776

#sciComm #philSci #metascience #logic #edu #academia

Generalization bias in large language model summarization of scientific research

Aktionsb. neue soziale Medien boosted

cyber4EDU e.V.

@cyber4EDU@digitalcourage.social · 2 months ago

Moin #39C3,
an #Tag1 findet wieder ein #Meet-up statt, mit dem wir beim #38C3 aufgehört hatten:

Um 15 Uhr laden wir alle, die im #Bildungsbereich aktiv sind, ganz herzlich zu unserem #Edu-Meet-up an unserer #Assembly ein.
Wir wollen uns vernetzten, gemeinsam ins Programm schauen, was es dieses Jahr so alles rund um das Thema #Bildung so gibt - und auch einfach über aktuelle #Bildungsthemen sprechen.
Also kommt vorbei und trefft Gleichgesinnte!
https://events.ccc.de/congress/2025/hub/de/event/detail/edu-meet-up-du-machst-was-im-bildungsbereich-komm-

#cyber4EDU #cyberLZ #FediLZ #DigitaleBildung #Schule #Didaktik #EduStammtisch #EduMeetup #EduFedi #Bildung #education

39c3

[39c3] EDU-Meet-Up: Du machst was im Bildungsbereich? Komm dazu! (#cyberLZ)

cyber4EDU e.V.

@cyber4EDU@digitalcourage.social · 2 months ago

Moin #39C3,
an #Tag1 findet wieder ein #Meet-up statt, mit dem wir beim #38C3 aufgehört hatten:

#cyber4EDU #cyberLZ #FediLZ #DigitaleBildung #Schule #Didaktik #EduStammtisch #EduMeetup #EduFedi #Bildung #education

39c3

[39c3] EDU-Meet-Up: Du machst was im Bildungsbereich? Komm dazu! (#cyberLZ)

Björn Staschen

@bjoernsta@eimsbuettel.social · 3 months ago

@kaffeeringe wobei die Medienanstalten beim Jugendschutz recht effektiv regulieren, auch durch Kivi

EikeZiegler

@erio7@nfdi.social · 3 months ago

@bjoernsta @kaffeeringe

in der Realität schicken sich aber die kids weiterhin die aufgaben per whatsapp und bekommen in der Schule empfohlen sich per youtube zu informieren
.
#edu

da wären die @bpb und des Länder gefragt ?!?! + #verbraucherschutz

Die Fragen nach
#barrierearm und solchem Gedöns .. sollte man nicht... #gafam überlassen ..