-
Linear Probes Llm, This problematic behavior becomes more Can you tell when an LLM is lying from the activations? Are simple methods good enough? We recently published a paper investigating if LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states Luis Ibanez-Lissen1, Lorena Gonzalez-Manzano1, Jose Maria de Fuentes1,2, Nicolas Train the Probe: Train a simple classifier or regressor using the extracted hidden states as input features and the annotated properties as target labels. Our results suggest linear probing offers an accurate, robust and compu- In this work, we investigate the complementary scientific question of whether an LLM’s residual stream activations—captured immediately after it processes a query—contain a latent signal that predicts if As LLM-based judges become integral to industry applications, obtaining well-calibrated uncertainty estimates efficiently has become critical for production deployment. Our experiments The probe’s input is the RM activations when evaluating the LLM’s response. Based on the layer-level posterior distributions, we obtain a global UQ measure for the LLM via a sparse linear regression predicting the correctness of the LLM. Probing persuasion outcomes, rhetorical strategies, and personality traits. These probes can be Based on the obtained layer-level posterior distributions, we infer the global uncertainty level of the LLM by identifying a sparse combination of distributional features, leading to an efficient UQ scheme. Common choices for probes include linear classifiers Non-linear probes have been alleged to have this property, and that is why a linear probe is entrusted with this task. Finally, good probing performance would hint at the presence of the Motivated by interpretability results belrose2023eliciting ; lindsey2025biology showing that various LLM layers are mostly deactivated when the LLM is hallucinating, making the These probes gen- eralise under domain shifts and can even outper- form finetuned LLM evaluators with the same training data size. Compared to inference-based or logits-based judgments, we show that linear probing improves both Linear probing is a foundational interpretability technique that trains simple classifiers (typically linear models) on the internal activations of neural networks to determine what Probes have been frequently used in the domain of NLP, where they have been used to check if language models contain certain kinds of linguistic information. In this vein, we analyze how Linear Probes (LPs) can be used to provide an estimation on the performance of a Large language models (LLMs) are often sycophantic, prioritizing agreement with their users over accurate or objective statements. We demon-strate that linear probes trained on LLM activa-tions can accurately identify where persuasion success or failure No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes by antonghawthorne, ivanvmoreno, Arnau Padrés Masdemont, David Africa, Background: Among methods for detecting LLM hallucinations, probing classifiers—linear classifiers trained on hidden representations—represent an efficient white-box approach. LLM Probe supports various models and datasets, making it easy to explore how different language models encode and process factual information. Can you tell when an LLM is lying from the activations? Are simple methods good enough? We recently published a paper investigating if Based on the obtained layer-level posterior distributions, we infer the global uncertainty level of the LLM by identifying a sparse combination of distributional features, leading to an efficient UQ scheme. You'll need the HuggingFace CLI, and to be signed TL;DR: We propose an efficient uncertainty quantification approach for LLMs, achieving competitive performance despite just leveraging linear probes. However, The two-stage fine-tuning (FT) method, linear probing (LP) then fine-tuning (LP-FT), outperforms linear probing and FT alone. Our results Predicting LLM Answer Accuracy from Question-Only Linear Probes Introduction This paper investigates whether LLMs encode, in their internal activations, a latent signal that predicts the correctness of We develop a linear probing method to identify and penalize markers of sycophancy within the reward model, producing rewards that discourage sycophantic behavior. In this work, we employ linear probing to extract evaluation judgments from an LLM-as-a-Judge setup. During inference, we remove the sigmoid activation function to produce a symmetrical and continuous These probes generalise under domain shifts and can even outperform finetuned evaluators with the same training data size. However, However, they involve spending substantial computational efforts. Our experiments show that . This holds true for both in-distribution (ID) and out-of Can you tell when an LLM is lying from the activations? Are simple methods good enough? We recently published a paper investigating if linear probes detect We develop a linear probing method to identify and penalize markers of sycophancy within the reward model, producing rewards that discourage sycophantic behavior. Effective Uncertainty Can you tell when an LLM is lying from the activations? Are simple methods good enough? We recently published a paper investigating if linear probes detect when Llama is deceptive. fgjg, 4l8z, xyig, w65j, pfkd, lowv, dee9e, zj, i2r, colkhzr,