AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
48
Papers today
8h
Update frequency
7
Days of history
Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling
Time Series
- Earth observation embeddings can replace hand-crafted topographic descriptors in weather downscaling.
- The proposed method improves probabilistic skill for temperature and wind speed predictions.
- The approach is effective across diverse climatic regions and for previously unobserved locations.
- Improvements in CRPS skill demonstrate the potential of using long-timescale embeddings for short-timescale predictions.
Read more
Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling
Summary
This paper explores the use of Earth observation embeddings as sub-grid descriptors for improving probabilistic weather downscaling. Traditional methods rely on hand-crafted topographic descriptors to bridge the gap between coarse global weather reanalyses and site-specific applications. The authors propose a novel approach that incorporates a learned local surface descriptor derived from Tessera embeddings at a high resolution (10 m) to enhance the downscaling of coarse ERA5 reanalysis fields (βΌ25 km resolution). The study demonstrates that these embeddings effectively capture persistent surface properties, leading to improved predictions of near-surface temperature and wind speed across diverse climatic regions. The results indicate a significant enhancement in Continuous Ranked Probability Score (CRPS) skill, with improvements of 11.5% for temperature and 6.2% for wind speed. Notably, the methodology maintains its effectiveness even when applied to new locations without prior data, showcasing the potential of Earth observation embeddings in weather forecasting.
Methodology
The authors augment a convolutional conditional neural process (ConvCNP) to downscale coarse ERA5 reanalysis fields by integrating a learned local surface descriptor from Tessera embeddings. This approach allows for the encoding of persistent surface properties that influence local weather conditions, facilitating predictions at arbitrary locations without relying on local historical data.
Results
The integration of Earth observation embeddings resulted in an 11.5% improvement in CRPS skill for 2 m temperature predictions and a 6.2% improvement for 10 m wind speed predictions. The methodology demonstrated consistent performance across five climatically diverse regions and maintained effectiveness when applied to new stations without prior data.
Implications
The findings suggest that Earth observation embeddings can significantly enhance the accuracy of localized weather predictions, which is crucial for various applications such as agriculture, disaster management, and urban planning. This approach could lead to more reliable weather forecasting models that are adaptable to different geographical contexts.
FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data
Multimodal
Optimization
Interpretability
- FiGuRO provides a dynamic approach to intrinsic dimension estimation in multi-modal data.
- The framework allows for disentanglement of shared and private information as an emergent property.
- FiGuRO outperforms existing ID estimation methods and is robust to hyperparameter changes.
- The method captures distinct ID scales and varying subspace ratios effectively.
Read more
FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data
Summary
The paper introduces Fidelity-Guided Rank Optimization (FiGuRO), a novel framework for estimating the Intrinsic Dimension (ID) of both uni-modal and multi-modal data. The challenge of accurately estimating ID is particularly pronounced in multi-modal contexts, where the data can be complex and high-dimensional, necessitating the disentanglement of shared and private information. Existing methods often fall short, being either static, uni-modal, or only implicitly adapting to shared IDs. FiGuRO addresses these limitations by employing truncated singular value decomposition to learn low-rank projections and an optimization algorithm that dynamically adjusts the dimensionality of latent spaces based on reconstruction fidelity. This emergent property allows for effective disentanglement of shared and private information without the need for complex auxiliary loss functions. The authors demonstrate that FiGuRO outperforms existing techniques in ID estimation, showing robustness to hyperparameter variations and successfully capturing distinct ID scales and subspace ratios across various datasets. Additionally, FiGuRO can be applied to modern uni-modal pretrained models, facilitating efficient post-hoc disentanglement of multi-modal representations.
Methodology
FiGuRO utilizes a combination of truncated singular value decomposition for low-rank projections and an optimization algorithm based on Rate-Distortion Theory. This approach governs the adjustment of latent space dimensions dynamically, allowing the model to increase or decrease ranks based on reconstruction fidelity across modalities.
Results
The results indicate that FiGuRO significantly outperforms traditional ID estimation techniques, demonstrating superior performance in simulations and real-world datasets. It effectively captures the distinct intrinsic dimensions of shared and private subspaces, showcasing its ability to adapt to varying data complexities.
Implications
FiGuRO's ability to accurately estimate intrinsic dimensions in multi-modal data has significant implications for fields requiring interpretable models, such as biology and medicine. It enables researchers to discern the relevance of different modalities and enhances the efficiency of representation learning in complex data environments.
Robust and Efficient Noisy-Label Time-Series Classification via Dynamic Time Warping Based Granular Ball Computing
Time Series
Efficient ML
- DTW-GBC organizes training samples into granular balls for efficient classification.
- The method reduces the influence of mislabeled samples on classification performance.
- Two granular-ball construction strategies are proposed: random splitting and label-informed splitting.
- Experiments show DTW-GBC outperforms traditional DTW-based classifiers in noisy label scenarios.
Read more
Robust and Efficient Noisy-Label Time-Series Classification via Dynamic Time Warping Based Granular Ball Computing
Summary
This paper introduces Dynamic Time Warping-Based Granular Ball Computing (DTW-GBC), a novel approach for time-series classification that addresses the challenges posed by noisy labels and high computational costs associated with traditional Dynamic Time Warping (DTW)-based nearest-neighbor classifiers. DTW-GBC organizes temporally similar training samples into granular balls, allowing for classification at the granule level rather than the individual sample level. This method not only mitigates the impact of mislabeled training samples but also reduces the number of DTW computations required during inference. The authors propose two strategies for constructing granular balls: random splitting and label-informed splitting. Experiments conducted on four benchmark datasets with symmetric label noise demonstrate that DTW-GBC effectively reduces performance degradation caused by label noise while requiring significantly fewer comparisons than the conventional DTW-based 1-NN classifier. The findings suggest that DTW-GBC achieves a favorable balance between classification robustness and inference efficiency, making it a promising solution for real-world time-series classification tasks where label noise is prevalent.
Methodology
The proposed DTW-GBC method computes pairwise DTW distances among training samples and recursively partitions them into granular balls. Classification is performed at the granule level, allowing for fewer comparisons during inference compared to traditional DTW-based nearest-neighbor methods. Two strategies for constructing granular balls are developed to enhance robustness against label noise.
Results
The experiments on four benchmark datasets indicate that both variants of DTW-GBC effectively alleviate the performance degradation caused by label noise. Additionally, DTW-GBC requires substantially fewer comparisons during inference than the DTW-based 1-NN classifier, demonstrating improved efficiency.
Implications
The findings suggest that DTW-GBC can be applied in various real-world time-series classification scenarios where label noise is a concern, such as in healthcare, finance, and environmental monitoring. The method's efficiency and robustness make it suitable for large-scale time-series data analysis.
Diffract: Spectral View of LLM Domain Adaptation
NLP
Large Language Models
- CPT maintains singular value spectra while adaptation is driven by singular vector changes.
- Significant domain-dependent heterogeneity in attention heads allows for selective updates.
- Up to 60% of head updates can be removed without quality loss, improving accuracy by up to 4%.
- Linear interpolation between CPT checkpoints shows smooth domain-quality transitions.
Read more
Diffract: Spectral View of LLM Domain Adaptation
Summary
This paper investigates continual pre-training (CPT) as a method for adapting large language models (LLMs) to specialized domains such as mathematics, instruction, code, and natural text. The authors utilize singular value decomposition (SVD) to analyze weight matrices, revealing that while CPT maintains the singular value spectra largely invariant, adaptation is primarily driven by changes in singular vectors. The study identifies significant domain-dependent heterogeneity in attention heads, allowing for a head importance criterion that suggests up to 60% of head updates can be omitted without degrading performance. By selectively rewinding low-importance heads to their pre-trained states, benchmark accuracy can be improved by up to 4%. Additionally, the authors introduce the concept of domain connectivity, demonstrating that linear interpolation between CPT checkpoints results in smooth domain-quality transitions without notable degradation. The paper culminates in the release of Diffract, an open-source toolkit for scalable spectral analysis of large models, facilitating reproducibility and future research.
Methodology
The authors employed singular value decomposition (SVD) to analyze weight matrices of LLMs during continual pre-training. They characterized the singular spectra and assessed the sparsity of CPT deltas, while also investigating the dynamics of singular vectors. The experimental setup involved training models on domain-specific data and analyzing attention head behavior across different domains.
Results
The analysis revealed that the singular value spectra remain stable during CPT, with domain adaptation primarily influenced by singular vector changes. The study found that a significant portion of attention heads could be deemed low-importance, allowing for selective rewinding to enhance model performance. Moreover, the concept of domain connectivity was established, showing that interpolating between checkpoints leads to improved model quality.
Implications
The findings suggest that LLMs can be efficiently adapted to specialized domains without extensive retraining, which could lead to more effective and resource-efficient model deployment in various applications. The release of the Diffract toolkit also opens avenues for further research in spectral analysis of large models.
CRHT: A Continuous Regression Hybrid Transformer for Vessel Trajectory Prediction with Online Cluster Sampling
Time Series
- CRHT addresses geographic bias and navigational realism in vessel trajectory prediction.
- An online K-means cluster sampling strategy is introduced to enhance training diversity.
- The hybrid architecture integrates local kinematic feature extraction with global attention.
- CRHT achieves superior performance in short-term forecasting compared to existing models.
Read more
CRHT: A Continuous Regression Hybrid Transformer for Vessel Trajectory Prediction with Online Cluster Sampling
Summary
The paper presents the Continuous Regression Hybrid Transformer (CRHT), a novel deep learning framework aimed at improving vessel trajectory prediction using Automatic Identification System (AIS) data. The authors identify challenges in existing models, such as geographic bias and navigational realism, which hinder accurate predictions. To address these issues, CRHT employs an online K-means cluster sampling strategy to ensure diverse training data, particularly for rare maneuvers. The architecture combines 1D convolutional layers for local feature extraction with a multi-head attention mechanism for capturing global temporal context. The model is evaluated on a dataset of over 1.8 billion AIS messages, focusing on larger vessels like tankers and cargo ships. Results indicate that CRHT outperforms traditional models in short-term forecasting, achieving the lowest prediction errors at a 1-hour horizon, while maintaining a balance between precision and maneuver tracking for real-time maritime surveillance.
Methodology
The methodology involves framing the vessel trajectory prediction as a multi-variate sequence-to-sequence regression problem. The CRHT model processes historical AIS data through a Local-Global Encoder that utilizes 1D convolutional layers and a Transformer Encoder for long-range dependency capture. A Query-Based Decoder employs learnable query embeddings to predict future trajectory steps in parallel. The model uses scaled delta learning with Huber loss for optimization, ensuring stability in predictions.
Results
The CRHT model demonstrated significant improvements in prediction accuracy, achieving the lowest errors at the 1-hour forecasting horizon. The results highlight the model's effectiveness in capturing complex maneuvers and its robustness in real-time maritime applications, outperforming traditional discrete models in both precision and maneuver tracking.
Implications
The findings suggest that CRHT can enhance maritime safety and anomaly detection by providing more accurate vessel trajectory predictions. This has potential applications in collision avoidance systems and maritime domain awareness, contributing to improved operational efficiency and safety in critical waterways.
Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models
Multimodal
Time Series
NLP
- BLPM aligns continuous EEG representations with language semantics through semantic embedding prediction.
- The CELP encoder promotes higher-level abstraction by predicting latent representations from contextual EEG observations.
- The MQSD module enables selective access to distinct semantic factors within EEG segments based on task relevance.
- BLPM avoids the pitfalls of discrete tokenization and autoregressive generation, enhancing EEG decoding capabilities.
Read more
Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models
Summary
This paper introduces the Brain Latent Predictive Model (BLPM), a novel EEG-language foundation model designed to address the limitations of existing EEG decoding paradigms. Traditional approaches often rely on masked autoencoding or autoregressive modeling, which can prioritize low-level signal reconstruction or create mismatches between continuous EEG dynamics and discrete token spaces. BLPM reformulates EEG decoding tasks as a continuous semantic embedding prediction problem, utilizing a Continuous EEG Latent Predictive (CELP) encoder that learns transferable representations by predicting latent EEG segments from contextual observations. Additionally, the Multi-Query Semantic Decomposition (MQSD) module extracts task-relevant information and aligns continuous EEG representations with textual semantics in a shared latent space. Experimental results across multiple benchmarks demonstrate that BLPM achieves consistent generalization performance across diverse EEG decoding tasks, establishing continuous latent semantic prediction as an effective approach for integrating EEG data with language models.
Methodology
The methodology involves the development of the Brain Latent Predictive Model (BLPM), which includes a Continuous EEG Latent Predictive (CELP) encoder for learning universal EEG representations and a Multi-Query Semantic Decomposition (MQSD) module for aligning these representations with language semantics. The model reformulates EEG decoding tasks into a continuous semantic embedding prediction framework, allowing for the integration of EEG data without relying on discrete tokenization.
Results
The experiments conducted across multiple benchmarks indicate that BLPM consistently outperforms existing EEG decoding models, demonstrating robust generalization capabilities across diverse tasks. The model effectively captures higher-level neurophysiological structures and aligns them with semantic information from language models.
Implications
The findings suggest that BLPM could significantly enhance the integration of EEG data with natural language processing tasks, potentially leading to advancements in applications such as brain-computer interfaces, clinical diagnostics, and multimodal learning frameworks.
Task- and dataset-specific information in protein language models
NLP
- Intermediate layers of PLMs often provide more informative embeddings than the last layer for specific downstream tasks.
- The relationship between the pre-training objective and downstream task influences the distribution of relevant information across PLM layers.
- Dataset characteristics play a crucial role in determining the effectiveness of PLM embeddings for whole-protein tasks.
- Performance of PLMs significantly drops when applied to artificial protein sequences, underscoring the importance of authentic training data.
Read more
Task- and dataset-specific information in protein language models
Summary
This paper investigates the internal representations of protein language models (PLMs) and their effectiveness across various downstream tasks (DTs). The authors analyze 13 PLMs across 15 DTs derived from 11 datasets to understand how embeddings from different layers of PLMs contribute to task performance. Contrary to the common assumption that the last layer of a PLM provides the most informative embeddings, the study reveals that intermediate layers often yield better results for specific tasks. The findings indicate that the distribution of relevant information across layers is influenced by the nature of the DT and the dataset used. For instance, tasks related to individual residues benefit from embeddings in deeper layers, while whole-protein tasks are more dependent on the dataset characteristics. Additionally, the performance of PLMs declines significantly when applied to artificial proteins, highlighting the importance of training data authenticity. This research provides insights into optimizing PLM usage for protein property prediction and emphasizes the need for a nuanced understanding of layer-specific information in PLMs.
Methodology
The authors conducted a systematic analysis of 13 PLMs by training probe models on embeddings from each layer and comparing their performance across 15 downstream tasks. They computed characteristics of the latent spaces to estimate the information content of embeddings from different layers.
Results
The study found that embeddings from the last layers of PLMs rarely led to the best performance on downstream tasks. Instead, embeddings from intermediate layers were often more effective, particularly for tasks closely aligned with the pre-training objectives. Additionally, the analysis revealed that shallow layer embeddings were more useful for datasets with deep mutational scan data, while deeper layers were better for datasets with diverse natural proteins.
Implications
These findings suggest that researchers and practitioners should consider using embeddings from intermediate layers of PLMs for specific protein property prediction tasks. The study also highlights the importance of dataset selection in optimizing PLM performance, which could lead to more effective applications in computational biology and drug discovery.
Pair-Centric Graph Rewiring for Over-Squashing via Optimal Transport-Guided Communication Alignment
Graph Learning
Optimization
Theory
- Introduces a pair-centric approach to graph rewiring focused on communication shortages.
- Develops a shortage score that ranks node pairs based on their structural demand versus current support.
- Utilizes Optimal Transport to optimize edge additions and budget allocation for effective communication.
- Demonstrates significant improvements in performance on standard graph benchmarks.
Read more
Pair-Centric Graph Rewiring for Over-Squashing via Optimal Transport-Guided Communication Alignment
Summary
This paper addresses the challenge of over-squashing in message-passing neural networks (MPNNs), where task-relevant information is compressed through limited structural interfaces in a graph. The authors propose PairAlign, a novel pair-centric graph rewiring framework that explicitly identifies pairwise communication shortages. By combining the structural demand of the original graph with the finite-hop propagation support of the current graph, PairAlign computes a shortage score that highlights node pairs with insufficient communication support. The framework optimizes edge additions to alleviate over-squashing, guided by the insight that new edges can create beneficial paths while potentially diluting existing communication. PairAlign employs an Optimal Transport (OT)-guided mechanism to allocate a limited edge budget effectively across shortage targets, ensuring a broader and more effective coverage than traditional greedy methods. Experimental results demonstrate that PairAlign significantly improves performance across various message-passing backbones, validating the effectiveness of pair-level repair in addressing over-squashing issues.
Methodology
The methodology involves defining a shortage score for node pairs based on the ratio of original-graph structural demand to current-graph finite-hop propagation support. PairAlign scores candidate edges by their potential to reduce communication shortages and employs an Optimal Transport-guided mechanism to allocate edge additions effectively across the graph.
Results
The experiments conducted on standard graph benchmarks show that PairAlign outperforms existing message-passing frameworks, effectively alleviating over-squashing and enhancing communication between distant nodes in the graph.
Implications
The findings suggest that pair-level communication alignment can significantly enhance the performance of MPNNs, making PairAlign a valuable tool for applications in graph-based learning tasks where information distribution is uneven.
Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads
Theory
Multimodal
Interpretability
- Introduces a unified theory for multiplicative dual-encoder networks across various domains.
- Defines low interaction rank functions and their interaction spectrum to measure complexity.
- Addresses approximation error, identifiability issues, and sample complexity in encoder networks.
- Demonstrates that normalization techniques can enhance interpretability of learned representations.
Read more
Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads
Summary
This paper introduces a unified theoretical framework for multiplicative dual-encoder networks, which compute outputs as the inner product of separately encoded inputs. The authors define a class of functions with low interaction rank, characterized by an interaction spectrum that measures intrinsic complexity. They address key design questions regarding approximation error, encoder identifiability, sample complexity, and architectural limitations. The framework reveals that approximation error can be decomposed into a spectral truncation term and an encoder-realization term, while sample complexity is governed by the sum of encoder complexities. The authors also identify a gauge symmetry issue affecting encoder identifiability and propose normalization techniques to enhance interpretability. Experimental validation on synthetic kernels, operator learning, and CLIP models supports the theoretical predictions, demonstrating the effectiveness of the proposed framework in improving model performance and interpretability.
Methodology
The authors develop a theoretical framework based on the interaction spectrum of functions representable by rank-d heads. They analyze approximation errors, identifiability issues due to gauge symmetry, and sample complexity. The framework is validated through experiments on synthetic kernels, operator learning, and CLIP models, where they measure spectral decay rates and the effectiveness of normalization techniques.
Results
The study finds that approximation error can be decomposed into two components, and that sample complexity is determined by the sum of encoder complexities. The experiments confirm that spectral decay rates align with theoretical predictions, and normalization techniques successfully recover true interaction modes, leading to more interpretable representations.
Implications
This work has significant implications for the design and understanding of dual-encoder architectures in various machine learning applications, including retrieval systems, contrastive learning, and operator networks. The insights into encoder normalization and interpretability can enhance model performance and usability in practical scenarios.
Confidence Calibration of Deep Learning Systems
Theory
Efficient ML
Interpretability
- Introduces novel methods for confidence calibration in deep learning systems under noisy labels.
- Explores conformal prediction techniques that are robust to label noise.
- Presents local differential privacy approaches for conformal prediction.
- Demonstrates effective unsupervised target domain calibration methods.
Read more
Confidence Calibration of Deep Learning Systems
Summary
This Ph.D. thesis explores the critical issue of confidence calibration in deep learning systems, particularly under conditions of noisy labels and unsupervised domain adaptation. The work is structured into several key areas: it begins with a comprehensive introduction to confidence calibration and its importance in machine learning, followed by a detailed examination of conformal prediction methods and their robustness to label noise. The thesis presents novel methodologies for calibrating confidence scores in the presence of noisy labels, including a proposed network training approach that integrates calibration techniques. Additionally, it introduces local differential privacy mechanisms to enhance the reliability of conformal predictions while maintaining privacy. The thesis concludes with an analysis of unsupervised target domain calibration, providing insights into how calibration can be effectively achieved without labeled data in the target domain. Through extensive experiments across various datasets, the author demonstrates the effectiveness of the proposed methods, highlighting improvements in calibration accuracy and robustness against label noise.
Methodology
The thesis employs a combination of theoretical analysis and empirical experimentation. It develops new algorithms for confidence calibration that address the challenges posed by noisy labels and unsupervised domain adaptation. The methods include network training strategies that incorporate calibration techniques, conformal prediction frameworks that are robust to label noise, and local differential privacy mechanisms to ensure data privacy during predictions.
Results
The experimental results indicate that the proposed calibration methods significantly enhance the accuracy of confidence scores in deep learning models, particularly in scenarios with noisy labels. The conformal prediction techniques demonstrated improved coverage and reliability, while local differential privacy approaches maintained privacy without sacrificing prediction quality. The unsupervised calibration methods also showed promising results, effectively adapting to target domains without labeled data.
Implications
The findings of this thesis have important implications for the deployment of deep learning systems in real-world applications, where data may be noisy or unlabeled. Improved confidence calibration can lead to more reliable decision-making in critical areas such as healthcare, autonomous systems, and finance. Additionally, the incorporation of privacy-preserving techniques ensures that these systems can operate within ethical and legal frameworks.
Terminal Symmetry as a Decision Resource: Statewise Refinement for Anytime Verified Construction
Theory
Optimization
Robotics
- Introduces a decision-resource framework that utilizes terminal symmetry for construction tasks.
- Develops SYMBUILD, which implements a statewise refinement approach to improve decision-making.
- Demonstrates significant improvements in verified efficiency across multiple domains.
- Achieves the lowest mean capped verifier cost compared to other planners in GRN OOD scenarios.
Read more
Terminal Symmetry as a Decision Resource: Statewise Refinement for Anytime Verified Construction
Summary
This paper presents a novel approach to leveraging terminal symmetry in sequential construction tasks, where the final outcomes exhibit exact symmetry despite the asymmetric nature of the execution process. The author introduces a decision-resource framework that decomposes terminal symmetry into three components: process evidence that provides directionality, terminal correspondence that transports this structure across equivalent outcomes, and realized-state evidence that refines decision relevance after transitions. The proposed method, SYMBUILD, utilizes an episode-fixed transported process structure and implements a state-restricted process rank along with a state-dependent residual rank that is updated after accepted transitions. The ordinal rank meet ensures a completion guarantee under prefix coverage and achieves a tight worst-case verifier-query bound. The effectiveness of SYMBUILD is demonstrated across various domains, including CAD assembly, Mini-Programs, and exact-fill packing, where it significantly improves anytime AUC metrics. Additionally, SYMBUILD outperforms existing planners in terms of mean capped verifier cost across multiple scales in GRN OOD scenes, showcasing its efficiency and the ability to transfer statewise signals across different organizational structures.
Methodology
The methodology involves a decomposition of terminal symmetry into process evidence, terminal correspondence, and realized-state evidence. SYMBUILD is implemented with an episode-fixed transported process structure, state-restricted process rank, and a state-dependent residual rank. The approach includes a theoretical framework that guarantees completion under prefix coverage and optimizes verifier-query bounds.
Results
SYMBUILD shows improved anytime AUC by up to 6.77, 21.75, and 8.68 points in CAD assembly, Mini-Programs, and exact-fill packing, respectively. It also achieves the lowest mean capped verifier cost across all scales in GRN OOD scenes when compared to other planners.
Implications
The findings suggest that terminal symmetry can be effectively utilized as a decision resource in various construction tasks, potentially leading to more efficient planning and execution in robotics, assembly, and other sequential decision-making processes.
Physics-Informed Machine Learning in Prognostics and Health Management: A Systematic Literature Review
Theory
- PIML integrates physical knowledge into ML to enhance reliability and interpretability in PHM.
- The review categorizes existing studies into four classes based on biases and approaches.
- PIML shows improved predictive performance over traditional methods, particularly in specific applications.
- There is a need for more evidence supporting claims of enhanced interpretability and generalization.
Read more
Physics-Informed Machine Learning in Prognostics and Health Management: A Systematic Literature Review
Summary
This paper presents a systematic literature review on the application of Physics-Informed Machine Learning (PIML) in Prognostics and Health Management (PHM). The authors analyze 212 studies to explore how PIML addresses the limitations of traditional data-driven models, such as poor generalization, lack of interpretability, and inability to infer causal relationships. The review introduces a four-class classification scheme for the studies: observational bias, inductive bias, learning bias, and hybrid approaches, and categorizes them by PHM tasks. The findings indicate that PIML consistently outperforms conventional baselines across various assets, although the literature is predominantly focused on specific applications like lithium-ion batteries and bearings. Despite the demonstrated benefits of physics-informed approaches, the authors note that claims regarding improvements in interpretability and generalization require more robust evidence. The paper concludes by suggesting future research directions, including the development of transferable design patterns, benchmarks for integration strategies, and uncertainty-aware models suitable for real-world deployment.
Methodology
The authors conducted a systematic literature review of 212 studies related to PIML in PHM, categorizing them based on a four-class classification scheme and analyzing their contributions to various PHM tasks.
Results
The review found that PIML approaches consistently improve predictive performance compared to conventional methods across a range of applications, although the focus is heavily skewed towards specific domains like lithium-ion batteries and bearings.
Implications
The findings suggest that PIML can significantly enhance the reliability and efficiency of PHM systems in industrial settings, paving the way for more robust predictive maintenance strategies. Future research could lead to the development of more generalizable models that can be deployed in diverse real-world scenarios.
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
NLP
Large Language Models
Reinforcement Learning
- LoongReflect formulates reflection as a memory-control policy to enhance long-horizon reasoning.
- The framework utilizes a reversible trajectory tree with explicit actions for reflection and backtracking.
- A dual-channel learning approach combines privileged teacher feedback with outcome-based reinforcement learning.
- Experiments show significant improvements over traditional reinforcement learning methods.
Read more
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Summary
The paper introduces LoongReflect, a novel training framework aimed at enhancing long-horizon reasoning in search agents, particularly focusing on the reflection process. Reflection is crucial for agents to evaluate their progress, identify missing information, and decide whether to continue or revise their current trajectory. However, traditional reinforcement learning methods struggle with reflection due to a mismatch between local decision-making and global outcome evaluation. LoongReflect addresses this by formulating reflection as a memory-control policy within a reversible trajectory tree, utilizing explicit actions for reflection and backtracking. The framework employs a dual-channel learning approach: a fast channel that distills reflective behavior from a privileged teacher and a slow channel that optimizes complete trajectories based on outcome-based reinforcement learning. Experimental results demonstrate that LoongReflect consistently outperforms traditional outcome-only reinforcement learning and self-distillation methods across various benchmarks, indicating its effectiveness in improving long-horizon reasoning and reflection in agents.
Methodology
LoongReflect employs a training framework that operates over a reversible trajectory tree, allowing agents to perform structured reflection and backtracking actions. It utilizes a dual-channel learning mechanism: a fast channel for distilling reflective behavior from a privileged teacher and a slow channel for optimizing trajectories based on outcome-based reinforcement learning.
Results
The experimental results indicate that LoongReflect achieves consistent improvements in performance on multi-hop retrieval-augmented generation and mathematical reasoning tasks compared to traditional outcome-only reinforcement learning and self-distillation baselines.
Implications
The findings suggest that enhancing reflection in search agents can lead to more reliable and effective long-horizon reasoning capabilities, which could be beneficial in various applications such as automated planning, decision-making systems, and complex problem-solving tasks.
TailBooster: A Dual-Layer Generative Framework for Extreme Value Augmentation with Operational Validity Enforcement
Generative Models
- TailBooster effectively addresses the rarity of extreme events in historical flight data.
- The framework combines generative modeling with anomaly detection to ensure operational validity.
- Significant improvements in prediction accuracy for extreme events were achieved compared to conventional methods.
- TailBooster is adaptable to various domains beyond aviation, where extreme-event prediction is necessary.
Read more
TailBooster: A Dual-Layer Generative Framework for Extreme Value Augmentation with Operational Validity Enforcement
Summary
The paper presents TailBooster, a novel dual-layer generative framework designed to enhance the prediction of extreme events in air transportation, such as severe arrival delays and abnormal air times. These extreme events are rare in historical data, leading to insufficient training signals for machine learning models. TailBooster addresses this challenge by combining generative modeling with two anomaly detection layers. The first layer utilizes statistical methods to identify extreme values, which are then used to train a Tabular Variational Autoencoder. The second layer employs deep learning techniques to clean the generated synthetic data, ensuring that it adheres to operational constraints derived from historical records. The framework was evaluated using U.S. flight records, focusing on diversity, statistical similarity, fidelity, operational validity, and utility. Results showed that training on TailBooster's synthetic records significantly improved predictions of extreme air time and arrival delays, outperforming conventional synthetic data methods. This approach is fully data-driven and model-agnostic, making it applicable to various domains where extreme-event prediction is crucial.
Methodology
The methodology involves a dual-layer generative framework: a statistical layer that identifies extreme values using the interquartile range and a deep learning layer that cleans the generated data to ensure operational validity. The framework employs a Tabular Variational Autoencoder for generating synthetic data, which is then filtered to discard operationally infeasible instances.
Results
The application of TailBooster resulted in a 47-49% reduction in Mean Absolute Error for extreme air time predictions and a 29-57% reduction for extreme arrival delay predictions when compared to conventional synthetic data methods. The framework also demonstrated comparable gains when real records were augmented with synthetic extremes.
Implications
TailBooster's approach can significantly enhance the predictive capabilities of machine learning models in aviation and other fields where extreme events are critical. Its model-agnostic nature allows for broad applicability, potentially improving operational resilience and decision-making in various industries.
A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes
Theory
Efficient ML
- Introduces a joint-distribution approach to fair representation learning for continuous sensitive attributes.
- Proposes a new discrepancy measure that avoids the need for conditional laws, enhancing computational efficiency.
- Demonstrates that the HSIC estimator converges faster than traditional nonparametric estimators.
- Establishes theoretical connections between the joint discrepancy and existing fairness criteria.
Read more
A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes
Summary
This paper addresses the challenge of fair representation learning when dealing with continuous sensitive attributes, such as age or income. The authors propose a novel approach to ensure that a representation Z is statistically independent of a sensitive attribute S by evaluating independence through a joint discrepancy between the joint law of (Z, S) and the product of its marginals. This method contrasts with existing criteria that rely on conditional laws, which can be computationally intensive and require nonparametric estimates. The authors introduce the HilbertβSchmidt independence criterion (HSIC) as a specific instance of their joint discrepancy measure, demonstrating that it can be estimated directly from samples without the need for conditional smoothing. The HSIC estimator is shown to converge at a faster rate than traditional nonparametric methods, making it more efficient for practical applications. The paper establishes theoretical foundations for this approach, including a disintegration identity that connects the joint discrepancy to existing conditional-integral criteria. The proposed algorithm, FRHSIC, achieves comparable fairness-accuracy tradeoffs to existing methods while significantly reducing training time per epoch.
Methodology
The authors propose a joint-discrepancy measure to assess the independence between a representation Z and a sensitive attribute S. They utilize the HilbertβSchmidt independence criterion (HSIC) as a specific instance of this measure, allowing for direct estimation from joint samples. The methodology includes theoretical proofs and empirical evaluations to demonstrate the efficiency and effectiveness of their approach compared to existing methods.
Results
The HSIC estimator is shown to converge at a rate of O(nβ1/2), outperforming the O(nβ2/5) rate of traditional nonparametric estimators. The empirical studies indicate that the proposed method achieves fairness-accuracy tradeoffs comparable to existing conditional-route methods while reducing training time significantly.
Implications
This work has significant implications for fields where fair representation is critical, such as hiring, lending, and criminal justice. The proposed methods can be applied to ensure that machine learning models do not inadvertently perpetuate biases associated with continuous sensitive attributes, thereby promoting fairness in automated decision-making processes.
Clustered Randomized Smoothing for Stochastic Prediction Functions
Robotics
Generative Models
Reinforcement Learning
- Introduction of clustered Ξ±-smoothing to enhance robustness in stochastic predictors.
- Local Ξ±-smoothing within clusters prevents mode collapse in multi-modal distributions.
- The framework is flexible and agnostic to the clustering algorithm used.
- Empirical results show a 27% reduction in Wasserstein distance and an 81% reduction in collision rates compared to state-of-the-art methods.
Read more
Clustered Randomized Smoothing for Stochastic Prediction Functions
Summary
This paper addresses the challenges of ensuring robust predictions in modern stochastic predictors, particularly in safety-critical applications. Traditional randomized smoothing techniques often fail in multi-modal regression settings due to mode collapse, leading to averaged predictions that do not accurately represent the underlying distribution. To overcome this limitation, the authors propose a novel framework called clustered Ξ±-smoothing, which involves partitioning noisy samples using a clustering algorithm, applying Ξ±-smoothing locally within each cluster, and combining the predictions into a mixture distribution. This approach preserves the multi-modal nature of the predictions and provides a lower bound on the probability that the smoothed prediction lies within distinct modes. The effectiveness of the proposed method is demonstrated through empirical evaluations on two benchmarks: stochastic trajectory prediction in a driving simulator and quadrotor control, showing significant improvements over existing methods.
Methodology
The proposed clustered Ξ±-smoothing framework involves three main steps: (1) clustering the noisy prediction samples using an arbitrary clustering algorithm, (2) applying Ξ±-smoothing locally within each cluster to mitigate outlier effects, and (3) combining the smoothed predictions into a mixture distribution that captures the multi-modal nature of the data. The authors also derive a probabilistic guarantee for the coverage of the smoothed predictions.
Results
The empirical evaluation on two benchmarks demonstrated that the clustered Ξ±-smoothing approach significantly outperformed existing methods. In the stochastic trajectory prediction task, it achieved a 27% lower Wasserstein distance to the ground-truth distribution compared to traditional Ξ±-smoothing. In quadrotor control, the method reduced the collision rate by 81% relative to state-of-the-art randomized smoothing techniques.
Implications
The proposed framework has significant implications for safety-critical applications such as autonomous driving and robotics, where robust and reliable predictions are essential. By effectively capturing multi-modal distributions, the method can enhance decision-making processes in complex environments.
Mapping and Measuring the Behavioral Evolution of Large Language Models
NLP
Large Language Models
- Introduces three novel sentence-level dissimilarity measures for analyzing LLM behavior.
- Demonstrates coherent clustering of model families and decreasing cross-family distances over time.
- Validates findings with a token-level analysis, confirming the robustness of the results.
- Establishes a mathematical relationship between behavioral similarity and training dynamics.
Read more
Mapping and Measuring the Behavioral Evolution of Large Language Models
Summary
This paper addresses the limitations of traditional benchmark leaderboards in evaluating large language models (LLMs) by focusing on the behavioral evolution of these models over time. The authors analyze the output behavior of 32 models from six families using a shared set of 10,000 prompts. They introduce three complementary sentence-level dissimilarity measures: an aligned mean per-prompt distance, a PCA-compressed summary of prompt-wise disagreement, and an alignment-free GromovβWasserstein discrepancy. The study employs these measures to create behavioral maps that illustrate static organization and temporal changes in model behavior. Key findings include the formation of coherent clusters among model families, a decrease in cross-family distances over time, and the identification of compact response clouds for recent reasoning-oriented models. The authors validate their findings through a token-level Maximum Mean Discrepancy analysis, demonstrating a strong correlation with their sentence-level results. Additionally, they establish a sufficient condition linking behavioral similarity to inference-prompt coverage and effective target distributions, providing insights into the training dynamics of LLMs.
Methodology
The authors utilized a shared bank of 10,000 prompts to generate responses from 32 different language models. They embedded these responses and constructed three types of dissimilarity measures to analyze the models' output behavior. The analysis included behavioral maps, hierarchical clustering, and cross-family distance measurements to assess both static and temporal changes in model behavior.
Results
The analysis revealed that model families exhibited coherent clusters, with GPT-2 identified as a global outlier. Over time, the distances between different model families decreased, suggesting a trend towards behavioral convergence. Recent reasoning-oriented models displayed more compact response clouds, indicating a potential shift in their output behavior. The token-level analysis corroborated the findings from the sentence-level measures, confirming the qualitative trends observed.
Implications
The findings suggest that behavioral analysis can provide deeper insights into the evolution and relationships among large language models beyond traditional performance metrics. This approach could inform future model development and evaluation strategies, enhancing our understanding of model capabilities and training dynamics.
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
Reinforcement Learning
Optimization
Theory
- BSPG explicitly separates reward improvement and boundary regulation in policy updates.
- The method achieves a finite-horizon O(1/βT) convergence bound for constraint residuals.
- BSPG demonstrates superior performance in reward maximization and boundary tracking compared to baseline methods.
Read more
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
Summary
This paper introduces the Boundary-Seeking Policy Gradient (BSPG) method for safe reinforcement learning, focusing on maximizing rewards while adhering to safety constraints in Constrained Markov Decision Processes (CMDPs). The authors highlight that traditional gradient-based methods often fail to exploit the structural property that optimal policies lie on the constraint boundary when constraints are active. BSPG addresses this by incorporating a tangential component that enhances reward while maintaining cost and a normal component that adjusts the policy towards the constraint boundary based on the constraint's residual. The paper provides theoretical guarantees for BSPG, including convergence of the constraint residual and characterization of the policy's behavior at convergence. Experimental results demonstrate that BSPG outperforms existing methods in terms of reward and boundary adherence in a standard Safety-Gymnasium navigation task.
Methodology
The BSPG method combines a tangential update that preserves cost while improving reward and a normal update that regulates the policy towards the active constraint boundary. This approach is grounded in an algebraic Lagrangian representation, avoiding the need for learned dual variables. The authors prove convergence properties under exact gradients and regularity conditions.
Results
In experiments on a Safety-Gymnasium navigation task, BSPG achieved higher rewards and maintained tighter adherence to the constraint boundary compared to existing baseline methods, demonstrating its effectiveness in safe reinforcement learning scenarios.
Implications
The findings suggest that BSPG could be applied in various safety-critical applications, such as autonomous driving and robotics, where maintaining safety constraints while maximizing performance is essential.
SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Ranks
Federated Learning
Efficient ML
NLP
- SeFoRA addresses the bilinear mismatch problem in federated LoRA by using sketch aggregation.
- The algorithm allows for heterogeneous client ranks, enabling clients with different computational budgets to participate effectively.
- SeFoRA-Ho provides a rank-homogeneous solution for direct adapter aggregation.
- Convergence to a first-order stationary point is proven for the rank-homogeneous setting.
Read more
SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Ranks
Summary
The paper introduces SeFoRA, a novel algorithm designed for federated parameter-efficient fine-tuning of large neural networks using low-rank adaptation (LoRA). The authors address the challenges posed by heterogeneous client ranks, which result in dimension incompatibility of factor matrices during aggregation. SeFoRA employs a sketch-aggregated approach where each client transmits a linear sketch of its local updates, allowing for effective aggregation at the federator without incurring the bilinear mismatch that typically arises in federated LoRA settings. Additionally, a rank-homogeneous variant, SeFoRA-Ho, is proposed to facilitate direct adapter aggregation. The authors prove convergence to a neighborhood of the first-order stationary point at a rate of O(1/T) for the rank-homogeneous setting. Experimental results demonstrate that SeFoRA outperforms existing state-of-the-art methods when fine-tuning the RoBERTa-Large model on GLUE datasets, showcasing its effectiveness in addressing the unique challenges of federated learning with LoRA.
Methodology
The methodology involves a sketch-aggregated federated learning approach where clients compute and send linear sketches of their updates instead of full updates. This allows for direct aggregation at the federator while addressing the bilinear mismatch issue. The authors also introduce a rank-homogeneous version of the algorithm to facilitate direct aggregation of adapters.
Results
The experimental results indicate that SeFoRA significantly outperforms state-of-the-art federated LoRA methods in fine-tuning the RoBERTa-Large model on GLUE datasets, demonstrating its effectiveness in handling rank heterogeneity and bilinear mismatch.
Implications
The proposed SeFoRA algorithm has potential applications in scenarios where federated learning is employed, particularly in environments with heterogeneous client capabilities. It enhances the efficiency of fine-tuning large models while maintaining data privacy, making it suitable for various domains such as healthcare, finance, and personalized services.
Let it Cook: Learning to Wait in Sequential Decision Making
Reinforcement Learning
Robotics
Optimization
- Introduces a waiting policy for sequential decision making to optimize resource usage.
- Formulates 'learning to wait' as a multi-objective optimization problem.
- Employs a lexicographic MORL algorithm to train agents across various environments.
- Demonstrates significant waiting periods (over 50% of task duration) without performance loss.
Read more
Let it Cook: Learning to Wait in Sequential Decision Making
Summary
This paper addresses the inefficiencies in sequential decision making by introducing a 'waiting policy' that allows agents to strategically pause their actions during periods where active engagement is unnecessary. The authors formalize the concept of 'learning to wait' as a multi-objective optimization problem, where the agent aims to minimize the frequency of sensing and decision-making while maintaining task performance. They propose a lexicographic Multi-Objective Reinforcement Learning (MORL) algorithm to train this waiting policy across various environments, including discrete-state household tasks and continuous-state scenarios. The results demonstrate that the proposed method can effectively learn waiting behaviors, allowing agents to wait for over 50% of the task duration without degrading performance. This approach not only conserves resources but also enables the interleaving of tasks, showcasing its potential for multi-tasking applications.
Methodology
The authors define a Waiting Markov Decision Process (WMDP) that includes a distinguished wait action and a set of waiting durations. They utilize a lexicographic MORL algorithm to balance maximizing task performance with minimizing the number of decisions made, effectively training agents to identify optimal waiting periods.
Results
The experiments reveal that the proposed method successfully learns waiting behaviors across multiple environments, achieving task performance comparable to traditional reinforcement learning methods while allowing for substantial waiting. The lexicographic approach outperforms standard MORL techniques that require hyperparameter tuning for reward weighting.
Implications
This work has significant implications for robotics and automation, where efficient resource management is crucial. The ability to learn when to wait can enhance the performance of agents in real-world tasks, leading to more efficient and effective decision-making processes.
Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique
NLP
Large Language Models
Efficient ML
- Introduction of the Latent Critic, a low-latency mechanism for hallucination detection in LLMs.
- Demonstrated ability to localize hallucinations effectively in real-time without additional inference costs.
- Mechanistic analysis shows improved representation of uncertainty geometry, enhancing detection reliability.
- Achieved high performance metrics (0.966 AUROC and >80% localization accuracy) across Qwen and Llama-based models.
Read more
Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique
Summary
This paper addresses the issue of hallucinations in Large Language Models (LLMs) when deployed as AI agents, particularly focusing on user specification-grounding failures. The authors introduce the Latent Critic, a lightweight low-rank adapter (LoRA) that operates alongside a frozen base LLM to restructure the transformerβs residual stream. This approach amplifies latent grounding signals and translates them into localized, natural language feedback in real-time, thus enabling actionable corrections without incurring significant inference latency. The paper demonstrates that the Latent Critic effectively enhances the model's ability to detect and localize hallucinations, particularly in tool-calling scenarios, where precise parameter specification is crucial. The authors conduct mechanistic analysis to show that the Critic restructures uncertainty geometry into a more reliable representation, outperforming existing detection methods in terms of accuracy and latency. The results indicate that the Latent Critic achieves a 0.966 AUROC and over 80% accuracy in localization, significantly improving real-time efficacy and enabling efficient self-correction in AI agents.
Methodology
The authors developed the Latent Critic as a Parameter-Efficient Fine-Tuning (PEFT) architecture that operates concurrently with a frozen base LLM. It restructures the model's internal representations to amplify grounding signals and provide localized feedback. The evaluation involved defining tool-calling hallucinations and measuring detection performance through a series of experiments focused on specification-grounding failures.
Results
The Latent Critic demonstrated superior performance in detecting and localizing hallucinations, achieving a 0.966 AUROC and over 80% accuracy in localization tasks. It outperformed existing fine-tuned external detectors and passive internal probes, showing its effectiveness as a real-time guardrail for AI agents.
Implications
The findings suggest that integrating the Latent Critic into LLMs can significantly enhance their reliability in high-stakes environments by preventing undesired actions and facilitating self-correction. This approach could be applied in various domains where precise execution of tasks is critical, such as healthcare, finance, and autonomous systems.
Detecting an Effect Is Not Learning to Act on It: A Reward-SNR Floor for LLM Acquisition Agents
Large Language Models
Theory
Interpretability
- Detecting an effect does not equate to learning to act on it; a reward-SNR floor governs the feasibility of learning.
- The apparent learnability of acquisition policies is often an artifact of noise rather than a true signal.
- Structured Hypothesis Embeddings (SHE) provide a method for generating user intent hypotheses but show limited downstream value.
- A necessary condition for effective policy learning is that the reward SNR exceeds a defined threshold.
Read more
Detecting an Effect Is Not Learning to Act on It: A Reward-SNR Floor for LLM Acquisition Agents
Summary
This paper addresses the critical distinction between detecting that an auxiliary signal helps in machine learning and learning to act on that signal in specific instances. The author argues that while it may appear that an auxiliary signal derived from large language models (LLMs) or other sources enhances performance, this perceived benefit often stems from noise rather than exploitable structure. The paper introduces a reward-signal-to-noise ratio (SNR) floor, which must be surpassed for effective policy learning to occur. Specifically, the threshold is defined as Οβ(N) β 2.8/βN, indicating that the SNR must exceed this value for a deployable policy to learn when to acquire the auxiliary signal. The author presents Structured Hypothesis Embeddings (SHE) as a method where a frozen LLM generates ranked, confidence-scored hypotheses based on user interaction history. Despite SHE being faithful and interpretable, its downstream value is conditional on the model architecture and data regime. The study finds that learned acquisition policies fail across various granularities due to insufficient SNR in the datasets tested. The paper concludes with a recommendation for a design-time regime gate instead of a learned per-instance policy, providing a practical four-step recipe for implementation.
Methodology
The paper employs a theoretical framework to establish the reward-SNR detectability floor and introduces Structured Hypothesis Embeddings (SHE) as a method for generating ranked hypotheses from user interaction data. The effectiveness of SHE and learned acquisition policies is evaluated across three public datasets (MIND, REES46, Amazon-Beauty) to assess their performance and SNR.
Results
The findings indicate that learned acquisition policies do not outperform random selection across various granularities and that the datasets tested fall below the SNR threshold necessary for effective learning. SHE is shown to be faithful but its downstream value is context-dependent, with significant improvements observed only in specific model architectures. The paper confirms that the apparent gains from acquisition are largely due to noise rather than exploitable structure.
Implications
The results suggest that practitioners should be cautious when relying on auxiliary signals for decision-making in machine learning systems. The proposed regime gate offers a practical solution for managing resource allocation in scenarios where learned policies are ineffective, potentially leading to more efficient use of computational resources.
Procedural Fairness Failures in RLHF from Preference Averaging
Reinforcement Learning
Large Language Models
Optimization
- Standard RLHF methods assume preference homogeneity, leading to procedural fairness failures.
- Majority preference groups dominate reward learning, disadvantaging minority preferences.
- PA-RLHF separates optimization across preference modes, preserving distinct preference signals.
- Controlled experiments showed PA-RLHF improved alignment accuracy from 46.9% to 67.9%.
Read more
Procedural Fairness Failures in RLHF from Preference Averaging
Summary
This paper addresses the issue of procedural fairness in Reinforcement Learning from Human Feedback (RLHF), particularly focusing on the aggregation of heterogeneous preferences into a single reward model. The authors argue that standard RLHF methods assume preference homogeneity, which can lead to procedural fairness failures where majority preferences dominate the reward learning process, systematically under-representing minority preferences. To tackle this issue, the authors introduce Preference-Aware RLHF (PA-RLHF), which separates optimization across different preference modes during the reward learning stage. In controlled experiments, PA-RLHF demonstrated significant improvements in alignment accuracy and reduced fairness gaps between preference groups. The findings suggest that structural design choices in reward learning can lead to systematic group-level misalignment, with implications for the deployment of large language models and agentic systems that rely on biased reward models.
Methodology
The authors developed PA-RLHF to intervene at the reward learning stage of RLHF, preventing heterogeneous preferences from collapsing into a single optimization objective. They conducted controlled experiments using a dataset of pairwise comparisons from simulated raters with distinct preference profiles, evaluating the effects of preference aggregation on alignment outcomes.
Results
PA-RLHF improved overall alignment accuracy from 46.9% to 67.9% and reduced the fairness gap between the best and worst aligned groups from 15.9 to 9.6 percentage points, demonstrating that separating reward optimization across preference modes can enhance fairness and alignment.
Implications
The findings have significant implications for the design of alignment procedures in large language models and agentic systems, emphasizing the need to consider heterogeneous user preferences to avoid biased outcomes in sequential decision-making processes.
Robust Multi-Agent Bandits with Heavy-Tailed Rewards and Information Asymmetry
Theory
Optimization
Reinforcement Learning
- Introduces three problem formulations for multi-agent bandits with heavy-tailed rewards and information asymmetry.
- Develops robust decentralized algorithms for each formulation with regret guarantees that nearly match centralized rates.
- Validates theoretical findings through experiments in a Pareto-distributed reward environment.
- Explores trade-offs between synchronization, coordination, and exploration in decentralized settings.
Read more
Robust Multi-Agent Bandits with Heavy-Tailed Rewards and Information Asymmetry
Summary
This paper addresses the multi-agent multi-armed bandit (MAB) problem under conditions of heavy-tailed reward distributions and information asymmetry, which are common in real-world applications. The authors propose three distinct problem formulations that capture different types of information asymmetry: (1) unobserved actions with common rewards, (2) observed actions with independent rewards, and (3) unobserved actions with independent rewards. For each scenario, they develop robust decentralized algorithmsβmRUCB-A, mRUCB-Intervals, and mHT-DSEEβand provide regret guarantees that approach those of centralized heavy-tailed rates. The theoretical contributions are validated through experiments in a Pareto-distributed reward environment, demonstrating the trade-offs between synchronization, coordination, and exploration across the different regimes. This work extends the understanding of multi-agent bandits by integrating heavy-tailed reward dynamics and decentralized learning without explicit communication.
Methodology
The authors develop decentralized algorithms tailored to three distinct information asymmetry scenarios in multi-agent settings. They utilize robust upper confidence bounds (RUCB) and adapt single-agent concentration arguments to the multi-agent context. The algorithms are designed to operate without online communication, relying instead on pre-agreed protocols among agents.
Results
The proposed algorithms demonstrate regret bounds that are competitive with centralized approaches, specifically achieving O(log T / Ξ1/Ξ΅) for the first two problems and O(logΒ² T) for the third. The experimental results confirm the theoretical guarantees and illustrate the impact of different information structures on performance.
Implications
This research has significant implications for decentralized decision-making systems in various applications, such as online experimentation, resource allocation, and multi-robot coordination, where agents operate under heavy-tailed reward distributions and limited information sharing.
Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits
Reinforcement Learning
- Introduction of the Counterfactual Clinical Audit (CCA) framework to evaluate medical RL agents.
- Identification of 'Toxic Mimicry' as a critical failure mode in RL applications for sepsis management.
- Demonstration that standard evaluation metrics fail to capture safety concerns in clinical settings.
- Validation of CCA using the MIMIC-III database, revealing significant differences in agent behavior.
Read more
Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits
Summary
This paper addresses the limitations of standard evaluation metrics in offline reinforcement learning (RL) for ICU treatment decisions, particularly in managing sepsis. The authors introduce the Counterfactual Clinical Audit (CCA) framework, designed to identify and mitigate a failure mode termed 'Toxic Mimicry,' where RL agents replicate harmful clinical patterns due to confounding factors in historical data. Using the MIMIC-III database, the study audits two RL architectures: the Medical Decision Transformer (MedDT) and the Historical Causal Transformer (HCT-RL). The CCA framework comprises three audits that test the robustness and causal alignment of the agents' responses to clinical guidelines. The findings reveal that while MedDT exhibits harmful behavior by reducing vasopressor dosage as lactate levels rise, HCT-RL maintains physiologically appropriate responses. This study highlights the need for counterfactual audits in medical RL to ensure safety and efficacy in clinical decision-making.
Methodology
The authors utilized the MIMIC-III Clinical Database to extract data from adult patients meeting Sepsis-3 criteria. They defined a continuous action space for intravenous fluid and vasopressor dosages and implemented the CCA framework, which includes three audits: Spurious Robustness, Causal Trend Alignment, and Contextual Scissor Probe. These audits stress-test the RL agents against physiological perturbations based on clinical guidelines.
Results
The CCA revealed that the Medical Decision Transformer (MedDT) exhibited harmful behavior by reducing vasopressor dosage inappropriately as lactate levels increased, contradicting clinical guidelines. In contrast, the Historical Causal Transformer (HCT-RL) maintained responses consistent with physiological expectations. This highlighted a misalignment between statistical performance and clinical safety.
Implications
The findings underscore the importance of incorporating causal inference in the evaluation of medical RL systems. The CCA framework can serve as a standard for assessing the safety and efficacy of RL agents in clinical settings, potentially improving decision-making in critical care environments.
Your LLM, Your Style: Behavioral Mode Axes for LLM Behavioral Control
NLP
Large Language Models
- Introduction of a situated B-data framework for studying LLM behavioral personality.
- Construction of Behavioral Mode Axes (BMAs) for effective behavioral control.
- Demonstration of stable and model-specific behavioral profiles in LLMs.
- Comparison of thought-derived and response-derived BMAs, highlighting the advantages of the former.
Read more
Your LLM, Your Style: Behavioral Mode Axes for LLM Behavioral Control
Summary
This paper addresses the need for a more reliable framework for studying and controlling the behavioral personality of large language models (LLMs) in interactive settings. Traditional methods rely on self-report questionnaires, which can be unstable and sensitive to wording and context. The authors propose a situated behavioral-data (B-data) framework that operationalizes LLM personality through concrete behavioral scenarios rather than abstract traits. They construct 3,200 contrastive scenarios across 20 behavioral patterns and four prompt registers, grounded in established psychometric facets. The study reveals that LLMs exhibit stable, model-specific behavioral profiles that can shift depending on the context of interaction. The authors introduce Behavioral Mode Axes (BMAs), which are derived from behavioral traces and allow for targeted control over LLM behaviors. The results indicate that thought-derived BMAs provide cleaner control compared to response-derived BMAs, which are more susceptible to trait drift. This work suggests that LLM personality can be understood as measurable and controllable behavioral modes, enhancing user experience and decision-making in interactive applications.
Methodology
The authors developed a situated behavioral-data framework that involved creating 3,200 contrastive behavioral scenarios to assess LLM behaviors across various contexts. They utilized psychometric constructs to ground their framework and analyzed the resulting behavioral patterns to derive Behavioral Mode Axes (BMAs) for steering LLM behaviors.
Results
The study found that LLMs exhibit stable behavioral profiles that vary with interaction context. Thought-derived BMAs were shown to provide more reliable control over LLM behaviors compared to response-derived BMAs, which were more prone to inconsistencies. The control mechanisms were localized within specific layers of the models, indicating a structured approach to behavioral steering.
Implications
This research has significant implications for the design of interactive AI systems, suggesting that LLMs can be tailored to exhibit desired behavioral traits in various contexts, thereby enhancing user experience and decision-making processes. It also opens avenues for further exploration of LLM personality in practical applications.
Click2Poly: A VLM for vector mapping buildings and walls
Computer Vision
Multimodal
Generative Models
- Click2Poly enhances manual vector mapping of buildings and walls using a human-in-the-loop approach.
- The system is built on the Florence-2 Vision Language Model, allowing for interactive editing through user clicks.
- A comprehensive dataset was utilized, consisting of over 377,000 building polygons and 197,000 wall linestrings.
- The implementation as a QGIS plugin facilitates real-world application in geospatial mapping.
Read more
Click2Poly: A VLM for vector mapping buildings and walls
Summary
The paper presents Click2Poly, a human-in-the-loop AI assistant designed to enhance the efficiency of vector mapping for buildings and walls, addressing the challenges of accuracy and labor intensity in geospatial applications. By extending the Florence-2 Vision Language Model (VLM), Click2Poly allows users to interactively edit vector layers through simple clicks, significantly speeding up the manual editing process. The system is implemented as a QGIS plugin and includes functionalities for drawing buildings, splitting building blocks, and drawing walls. The authors trained the model on a comprehensive dataset comprising over 377,000 building polygons and 197,000 wall linestrings, ensuring high spatial precision and correct topology. The methodology involves simulating user interactions during training to improve the model's responsiveness to user inputs. The results indicate that Click2Poly effectively reduces the time required for manual editing while maintaining high accuracy in vector representation, making it a valuable tool for real-world production environments in geospatial mapping.
Methodology
The authors extended the Florence-2 model by training it on three specific tasks: <DRAW_BUILDING>, <SPLIT_BUILDING>, and <DRAW_WALL>. User interactions were simulated during training to improve the model's ability to respond to clicks for drawing and editing vector features. The dataset included satellite imagery and annotations, with standard augmentation techniques applied to enhance model robustness.
Results
Click2Poly demonstrated a significant reduction in the time required for manual editing of vector layers while achieving high accuracy in the placement of vertices and maintaining correct topological structures.
Implications
The development of Click2Poly has the potential to streamline the process of vector mapping in various geospatial applications, making it easier for professionals to create accurate maps while reducing the labor involved in manual editing.
Air Quality Station Simulation via LSTM and Attention-Based Modelling
Time Series
- Introduction of SATADL, a deep learning model for simulating air quality station data during outages.
- Utilizes spatial and temporal attention mechanisms to improve prediction accuracy.
- Demonstrated superior performance in forecasting PM10 concentrations compared to baseline models.
- Addresses a significant gap in real-time air quality data simulation and analysis.
Read more
Air Quality Station Simulation via LSTM and Attention-Based Modelling
Summary
This paper addresses the challenge of simulating air quality measurements from stations that may become unresponsive due to hardware failures. The authors propose a novel deep learning model named SATADL (SpAtial-Temporal Attention Dual LSTM) that leverages both spatial and temporal attention mechanisms to forecast air quality metrics, specifically PM10 concentrations, during periods of station downtime. The model is designed to act as a virtual proxy for offline stations, ensuring continuous data availability for urban air quality analysis. The architecture consists of a spatial module that processes inputs from surrounding stations and a temporal module that analyzes historical data from the offline station. The performance of SATADL is evaluated against several baseline models across four datasets of air quality stations worldwide, demonstrating its superior predictive capabilities. The findings suggest that SATADL can effectively mitigate the impact of data unavailability on air quality assessments, thereby enhancing urban environmental monitoring efforts.
Methodology
The SATADL model is structured as an encoder-decoder architecture comprising two main modules: a spatial module that captures the influence of nearby stations and a temporal module that analyzes historical data from the offline station. Both modules utilize attention mechanisms to weigh the importance of different inputs, enhancing the model's ability to predict air quality metrics over multiple hours ahead.
Results
The SATADL model outperformed baseline and existing deep learning models in terms of both the coefficient of determination and root mean squared error across various prediction windows. This indicates its effectiveness in accurately simulating air quality measurements during periods of station failure.
Implications
The development of SATADL has significant implications for urban air quality management, enabling municipalities to maintain continuous monitoring and analysis of air pollution levels despite hardware failures. This can lead to better public health outcomes and more informed policy decisions regarding air quality management in smart cities.
Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization
Theory
Optimization
Generative Models
- Epiplexity is introduced as a measure of structural information in data that can enhance OOD generalization.
- The authors propose EpiSelect and EpiGen as methods for data selection and synthetic data generation, respectively.
- Higher epiplexity is shown to correlate with better performance in downstream tasks.
- The paper identifies limitations in current benchmarks for evaluating data selection methods.
Read more
Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization
Summary
This paper addresses the challenge of out-of-distribution (OOD) generalization in machine learning by introducing epiplexity, a measure of the structural information content in data. The authors operationalize epiplexity as an online training signal to enhance data selection and synthetic data generation. They propose two main methodologies: EpiSelect for data selection, which uses scaling laws to predict epiplexity gains based on training tokens, and EpiGen for synthetic data generation, which employs REINFORCE policy gradients to optimize data generation towards maximizing epiplexity. The findings suggest that higher epiplexity correlates with improved performance in zero-shot and fine-tuning tasks, supporting the hypothesis that structurally rich data enhances model generalization. Additionally, the paper critiques existing benchmarks for data selection, revealing limitations in evaluating OOD generalization.
Methodology
The authors operationalize epiplexity by fitting scaling laws to training loss curves to predict epiplexity gains for data selection (EpiSelect) and by defining a generator's reward based on changes in epiplexity for synthetic data generation (EpiGen), utilizing REINFORCE policy gradients for optimization.
Results
The proposed methods EpiSelect and EpiGen demonstrate improved performance over state-of-the-art approaches in OOD generalization tasks, confirming the hypothesis that data with higher epiplexity leads to better model representations.
Implications
This research has implications for improving data selection and generation strategies in machine learning, particularly in scenarios where models are deployed in unforeseen domains. It suggests that focusing on the structural richness of data can lead to more robust generalization capabilities.
DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains
Optimization
Theory
Graph Learning
- DACRI addresses the gap between disruption detection and effective intervention selection in supply chains.
- The CriticalSCM-Bench v1 benchmark provides a controlled environment for evaluating causal interventions.
- LambdaMART outperforms static benchmarks in certain supply chain archetypes but not universally.
- Intervention effectiveness is influenced by timing, cost, and the nature of disruptions.
Read more
DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains
Summary
The paper introduces DACRI, a framework designed to enhance decision-making in critical supply chains by addressing the gap between detecting disruptions and selecting interventions that maximize recoverable net value. The authors present CriticalSCM-Bench v1, a controlled synthetic benchmark that includes causal ground truth and paired factual/counterfactual rollouts, allowing for a clear evaluation of intervention strategies. The study employs LambdaMART, a learning-to-rank algorithm, which demonstrates improved median normalized net value across various supply chain archetypes, particularly in semiconductor and critical material domains. However, it shows limitations in digital infrastructure scenarios where simpler policies may outperform more complex models. The research also includes stress tests that evaluate the robustness of intervention strategies under various conditions, revealing that factors such as intervention timing and costs significantly influence policy effectiveness. Additionally, a guarded explanation study assesses the ability to communicate intervention decisions clearly to operators, ensuring that recommendations remain intact without alteration by language models.
Methodology
The authors developed a learning-to-rank framework for intervention selection based on causal analysis, utilizing a controlled benchmark (CriticalSCM-Bench v1) that includes intervention costs and counterfactual labels. They employed LambdaMART for ranking interventions and conducted various stress tests to evaluate the robustness of their approach under different scenarios.
Results
The use of LambdaMART improved median normalized net value by 5.7β16.2% compared to static benchmarks in semiconductor and critical material archetypes. However, it did not show significant improvements in digital infrastructure scenarios, where a simpler constant-buffer policy was more effective. The framework retained 33β75% of full-clamp value across partial and delayed settings, indicating its robustness under certain conditions.
Implications
The findings suggest that DACRI can significantly enhance decision-making in supply chain management by providing a structured approach to intervention ranking. This has potential applications in various industries reliant on complex supply chains, enabling better resource allocation and risk management during disruptions.
Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning
Federated Learning
- Introduces LIGHTYEAR, a federated learning framework that selects updates based on function-space evaluation rather than parameter-space similarity.
- Utilizes an NTK-based agreement score to characterize predictive behavior for optimal aggregation.
- Employs a decentralized peer-to-peer topology for direct client-to-client update exchanges, enhancing personalized aggregation.
- Demonstrates superior performance compared to traditional centralized FL and existing P2P approaches across various datasets.
Read more
Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning
Summary
This paper addresses the challenges of federated learning (FL) in heterogeneous environments, where client data distributions can vary significantly, leading to suboptimal model performance. Traditional FL methods rely on parameter-space similarity to select updates for aggregation, which can be misleading in non-IID settings. The authors propose a novel framework called LIGHTYEAR, which utilizes a Neural Tangent Kernel (NTK)-based agreement score to evaluate model updates in function space rather than parameter space. This approach allows clients to assess the predictive behavior of incoming updates based on their local target domains. By employing a decentralized peer-to-peer (P2P) topology, clients can exchange updates directly and evaluate them on private validation data, enabling personalized aggregation of beneficial updates. The empirical evaluation demonstrates that LIGHTYEAR outperforms both centralized FL baselines and existing P2P methods across multiple datasets, showcasing its effectiveness in enhancing model robustness and performance in federated learning scenarios.
Methodology
The LIGHTYEAR framework leverages an NTK-based agreement score to evaluate model updates in function space. It employs a decentralized peer-to-peer communication structure, allowing clients to exchange updates and evaluate them locally on private validation data. This enables clients to construct personalized aggregation sets that align with their specific target domains.
Results
Empirical evaluations across five datasets and nine baseline methods show that LIGHTYEAR consistently outperforms centralized FL baselines and existing peer-to-peer approaches, indicating its effectiveness in handling heterogeneous data distributions and improving model robustness.
Implications
The proposed method has significant implications for federated learning applications in privacy-sensitive domains such as healthcare and finance, where data heterogeneity and client reliability are critical factors. By enabling personalized aggregation, LIGHTYEAR can enhance model performance and robustness in real-world scenarios.
Two-stage Odd Residual Flows for Mean-Preserving Probabilistic Time Series Forecasting
Time Series
- Introduces TORF, a two-stage framework for time series forecasting that separates mean prediction from uncertainty estimation.
- Utilizes a deterministic model for accurate mean predictions in the first stage, followed by a Restricted Normalizing Flow for modeling residuals.
- Achieves state-of-the-art performance in terms of deterministic accuracy (NMAE) and density estimation (CRPS) on various forecasting tasks.
- Demonstrates that the two-stage approach effectively circumvents the trade-off between mean accuracy and distributional flexibility.
Read more
Two-stage Odd Residual Flows for Mean-Preserving Probabilistic Time Series Forecasting
Summary
This paper presents a novel framework called Two-stage Odd Residual Flows (TORF) aimed at improving mean-preserving probabilistic time series forecasting. The authors identify a critical trade-off in existing methods between distributional flexibility and accurate mean prediction, particularly in long-horizon forecasting scenarios. Traditional parametric methods like Mean Variance Estimation (MVE) often lead to degraded point accuracy when trained under joint Negative Log-Likelihood (NLL) objectives. In contrast, modern generative models, such as Normalizing Flows and Diffusion Models, typically require costly Monte Carlo sampling, which can yield suboptimal mean estimates. To overcome these limitations, TORF decouples mean forecasting from uncertainty estimation. The first stage employs a pre-trained deterministic model to generate an accurate mean prediction, while the second stage utilizes a Restricted Normalizing Flow (RNF) with strictly odd functions to model the residual distributions around the point forecast. This approach guarantees mean preservation from the first stage without the need for sampling. Experimental results demonstrate that TORF achieves state-of-the-art deterministic accuracy and strong density estimation performance across both short and long-horizon forecasting tasks.
Methodology
The methodology involves a two-stage approach where the first stage uses a deterministic model (e.g., SimpleTM) to predict the mean of the time series. The second stage employs a Restricted Normalizing Flow that is constrained to strictly odd functions to model the residuals around the predicted mean. This design ensures that the mean from the first stage is preserved without the need for sampling, thus enhancing both accuracy and flexibility in density estimation.
Results
TORF demonstrates superior performance, achieving state-of-the-art results in 7 out of 9 CRPS and 9 out of 9 NMAE comparisons for long-horizon forecasting, and 6 out of 8 CRPS and 5 out of 8 NMAE for short-horizon forecasting. The framework consistently outperforms existing models like K2VAE and MVE-2S, showcasing its effectiveness in maintaining high point prediction accuracy while providing robust uncertainty quantification.
Implications
The findings suggest that TORF can significantly enhance decision-making processes in risk-sensitive domains such as finance, energy management, and demand planning by providing accurate mean forecasts along with reliable uncertainty estimates. This decoupled approach could be applied to various forecasting tasks where understanding the distribution of predictions is crucial.
SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features
Efficient ML
Theory
- SQuaT eliminates the lower bound on distillation loss by aligning teacher and student feature representations.
- The framework allows for effective QAT without the need for labeled data.
- SQuaT shows significant performance improvements in low-bit quantization settings.
- The method is applicable across various model architectures and quantization levels.
Read more
SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features
Summary
The paper introduces SQuaT (Student-Aware Quantized Teacher Features), a novel framework for label-free Quantization-Aware Training (QAT) that integrates Knowledge Distillation (KD). Traditional QAT methods often require labeled data, which can be unavailable due to various constraints. The authors identify a critical limitation in existing methods where the mismatch in feature ranges between full-precision (FP) teacher models and quantized student models leads to an irreducible lower bound on the distillation loss. SQuaT addresses this by applying the student's quantization parameters to the teacher's features during distillation, aligning their representations and eliminating the unattainable residual. The framework is evaluated across diverse settings, demonstrating significant performance improvements, especially in extreme low-bit quantization scenarios (1- and 2-bit). The results indicate that SQuaT is broadly applicable across different architectures and quantization settings, making it a versatile solution for efficient model deployment in resource-constrained environments.
Methodology
SQuaT employs a student-aware projection technique that quantizes the teacher's intermediate features using the quantization parameters of the student model. This approach ensures that the teacher's features are projected into the same quantization lattice as the student's, facilitating effective feature-level knowledge distillation without requiring labeled data.
Results
The experiments demonstrate that SQuaT consistently outperforms strong baseline methods, particularly in extreme low-bit quantization scenarios. The results validate the theoretical claims regarding the elimination of the lower bound on distillation loss and showcase the framework's robustness across various model architectures.
Implications
SQuaT has significant implications for deploying deep learning models in resource-constrained environments, such as mobile and edge devices, where both model size and the availability of labeled data are critical challenges. The framework could enhance the efficiency of model training and deployment in real-world applications.
$Ξ²$-VAEs as Effective Theories: Tolerance-Dependent Dimension
Generative Models
Theory
Efficient ML
- Increasing regularization strength in $Ξ²$-VAEs collapses low-utility latent coordinates.
- Nonlinear interactions shift and broaden the onset of latent coordinate collapse.
- The effective dimension is not a fixed integer but varies with reconstruction tolerance.
- A headβtail tradeoff exists where deeper networks improve utility concentration but reduce tail fidelity.
Read more
$Ξ²$-VAEs as Effective Theories: Tolerance-Dependent Dimension
Summary
This paper investigates the behavior of $Ξ²$-Variational Autoencoders (VAEs) in terms of their effective dimensionality as a function of reconstruction tolerance. The author explores how increasing the regularization strength in $Ξ²$-VAEs leads to the collapse of low-utility latent coordinates, particularly in the context of nonlinear VAEs trained on the WorldClim dataset. The study reveals that while the collapse order of latent coordinates is affected by nonlinear interactions, the overall ranking of reconstruction utilities remains preserved. The effective-dimension curves demonstrate a tradeoff: deeper networks concentrate utility into a few latent coordinates but compromise the fidelity of the remaining dimensions. The research emphasizes the importance of understanding effective dimensions in high-dimensional datasets and suggests that the choice of reconstruction tolerance can significantly influence the latent representation learned by the model.
Methodology
The author employs a theoretical framework to analyze the rank-distortion curves of $Ξ²$-VAEs, varying the regularization strength to observe changes in active rank and reconstruction utility. The study uses the WorldClim bioclimatic dataset to test the hypotheses regarding effective dimensions and latent coordinate collapse.
Results
The findings indicate that in nonlinear VAEs, the collapse of latent coordinates does not align perfectly with reconstruction utilities as it does in linear cases. However, the overall ranking of reconstruction importance is maintained. The effective-dimension curves reveal that while deeper networks can enhance utility concentration, they also lead to poorer fidelity in the less significant dimensions.
Implications
This research has implications for the design and understanding of generative models, particularly in how latent representations can be optimized for specific tasks. It suggests that careful tuning of regularization parameters can lead to more effective dimensionality reduction in high-dimensional datasets, which is crucial for applications in data analysis and machine learning.
A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression
Theory
Efficient ML
Time Series
- Introduces a factor graph formulation for scalable multi-output Gaussian process regression.
- Achieves linear computational complexity in the number of data points while handling missing observations efficiently.
- Demonstrates competitive performance against traditional kernel-matrix and inducing-point methods in empirical tests.
- Utilizes a nearest-neighbor chain to structure inputs, facilitating exact Gaussian message passing for inference.
Read more
A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression
Summary
This paper presents a novel approach to multi-output Gaussian process (MOGP) regression by utilizing a Forney-style factor graph framework. Traditional MOGP methods face scalability issues due to their cubic complexity in the number of observations and outputs, particularly when different outputs are observed at different inputs. The authors propose a factor graph formulation that organizes a fixed candidate set of inputs into a nearest-neighbor chain, allowing latent MatΓ©rn processes to evolve through linear-Gaussian transition factors. This formulation enables exact Gaussian message passing for posterior computation, significantly reducing computational costs to O(C(DLΒ² + LΒ³)), where C is the chain length, D is the number of outputs, and L is the number of latent processes. The method inherently accommodates missing observations without requiring covariance matrix restructuring, making it particularly efficient for scenarios with partial data. Empirical comparisons with exact kernel-matrix methods and sparse-variational inducing-point methods demonstrate that the factor-graph approach maintains competitive accuracy while scaling linearly with the number of data points, especially in low-dimensional input settings. The results indicate that this approach is not only computationally efficient but also effective in practical applications such as electricity time series forecasting.
Methodology
The authors express multi-output Gaussian process regression as a Forney-style factor graph, employing a nearest-neighbor chain to order inputs. They utilize linear-Gaussian transition factors for latent processes and a Linear Model of Coregionalization (LMC) to mix latent processes into outputs. The inference is performed through exact Gaussian message passing, allowing for efficient handling of missing observations.
Results
The factor-graph formulation closely tracks the exact kernel-matrix posterior in low-dimensional settings, with performance gradually diverging as input dimension increases. In practical applications, such as electricity time series forecasting, the proposed method matches the accuracy of traditional baselines while exhibiting linear scaling with the number of data points, contrasting with the infeasibility of exact kernel methods at larger scales.
Implications
This work has significant implications for fields requiring scalable multi-output regression, such as sensor networks, environmental monitoring, and clinical data analysis. The ability to efficiently handle missing data and maintain computational feasibility opens new avenues for real-time applications and large-scale data analysis.
DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents
Large Language Models
Optimization
- DOCSCHISEL optimizes tool documentation based on empirical analysis of LLM agent performance.
- The effectiveness of tool documentation varies significantly across different task domains and LLM architectures.
- The framework improves task success rates by 95.89% over original documentation and 75.15% over existing baselines.
- The study highlights the importance of adaptive documentation for LLM agents in real-world applications.
Read more
DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents
Summary
The paper introduces DOCSCHISEL, an adaptive framework designed to optimize tool documentation for large language model (LLM) agents. As LLMs increasingly depend on external tools for complex tasks, the quality of tool documentation becomes crucial for their performance. The authors highlight that existing studies often treat tool documentation as a static input, neglecting its impact on agent performance across various settings. Through a large-scale empirical study involving 14 tool-use datasets, the authors reveal significant variability in the information fields of tool documentation and demonstrate that the effectiveness of these fields is contingent on factors such as task domain, LLM backbone, and agent paradigm. To address the identified issues, DOCSCHISEL analyzes failed execution traces to iteratively refine tool documentation by modifying information fields. The framework was evaluated against two state-of-the-art baselines, EASYTOOL and DRAFT, showing substantial improvements in task success rates. Overall, this work emphasizes the need for dynamic and context-aware tool documentation to enhance the capabilities of LLM agents.
Methodology
The authors conducted a large-scale empirical study analyzing 14 tool-use datasets to assess the composition and effectiveness of tool documentation. They developed DOCSCHISEL, which iteratively optimizes documentation by analyzing failed execution traces and refining information fields based on the specific needs of LLM agents.
Results
DOCSCHISEL significantly enhanced the task success rate of LLM agents, achieving a 95.89% improvement over original tool documentation and a 75.15% average improvement over existing optimization baselines, while maintaining low optimization time and token overhead.
Implications
The findings suggest that adaptive tool documentation can greatly enhance the performance of LLM agents in diverse applications, leading to more reliable and efficient task execution. This work could influence future research on LLM agent development and documentation practices.
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
Large Language Models
Theory
Graph Learning
- PLGA generalizes SDPA by using a learned bilinear operator, enhancing flexibility in attention mechanisms.
- The architecture exhibits empirical collapse at inference, allowing for significant simplifications in output generation.
- Operator invariance is observed, with outputs remaining stable under minor input perturbations.
- A learned singularity condition indicates that the generator matrix becomes numerically singular at convergence.
Read more
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
Summary
This paper introduces the Power Law Graph Attention (PLGA) mechanism, which generalizes the traditional scaled dot-product attention (SDPA) used in large language models (LLMs). The PLGA replaces the fixed bilinear form of SDPA with a learned, input-generated bilinear operator, allowing for a more flexible and mathematically meaningful attention mechanism. The architecture of the Large Language Model from Power Law Decoder Representations (PLDR-LLM) is fully specified and verified against reference releases. The paper presents several key findings, including the exact containment of SDPA within the PLGA framework, the phenomenon of empirical collapse at inference where the model's outputs become invariant to input changes, and a learned singularity condition that affects the model's performance. The study also discusses the implications of self-organized criticality in the attention dynamics and proposes a three-stage mechanism for inference. Overall, the PLGA architecture demonstrates advantages over traditional SDPA-LLM, particularly in terms of parameterization, training dynamics, and inference efficiency.
Methodology
The paper employs a theoretical approach to define the PLGA operator and its properties, alongside empirical measurements to validate the architecture's performance. It includes machine-checked proofs and numerical audits on released checkpoints to ensure the robustness of the claims.
Results
The results indicate that PLGA can replicate the functionality of SDPA while providing additional flexibility and efficiency. The empirical collapse at inference shows that the model can maintain performance with a simplified operator, and the learned singularity condition suggests a unique behavior in the attention dynamics. The architecture achieves high fidelity in outputs with minimal fluctuations, demonstrating its stability.
Implications
The findings suggest that PLGA could lead to more efficient large language models with improved inference capabilities. The ability to simplify the attention mechanism without sacrificing performance may have significant implications for the deployment of LLMs in real-world applications.
MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale
Large Language Models
NLP
Efficient ML
- MERA improves small model capabilities through iterative adaptation rather than just routing.
- The framework utilizes a SkillBook to capture and distill successful execution patterns.
- Empirical results show significant performance gains on benchmark tasks.
- Verifier-backed fallback mechanisms ensure quality during deployment.
Read more
MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale
Summary
The paper presents MERA, a novel framework designed to enhance the capabilities of smaller language models (LLMs) in agentic systems by integrating model evolution and routing with skill adaptation. Traditional routing methods assign tasks to models based on their complexity, often leading to limited improvements in smaller models. MERA addresses this by treating each model invocation as an adaptation unit, allowing for iterative improvements based on execution traces. The framework employs a SkillBook to distill successful procedures and fine-tunes a student model using supervised learning. A verifier ensures quality through fallback mechanisms, allowing for safe deployment of adapted models. Empirical results demonstrate that MERA significantly improves the performance of smaller models, achieving a pass rate increase from 28.7% to 49.7% on HumanEval+MBPP tasks after four adaptation cycles. The deployed policy maintains high performance at reduced costs, showcasing the effectiveness of multi-cycle adaptation in enhancing model capabilities rather than merely routing tasks to fixed models.
Methodology
MERA employs a multi-cycle adaptation protocol where each model invocation is treated as a unit for improvement. It utilizes online traces to update a SkillBook, fine-tune a student model via supervised learning, and train a routing mechanism. The adaptation process is verified through joint replay evaluation to ensure quality before deployment.
Results
The four-cycle adaptation process improved the Qwen2.5-Coder-1.5B model's pass rate from 28.7% to 49.7% on HumanEval+MBPP tasks. The deployed policy achieved an 88.3% pass rate at 60.8% of the cost of using a larger model, demonstrating the effectiveness of the MERA framework in enhancing small model performance.
Implications
MERA has significant implications for the deployment of cost-effective agentic systems, allowing for improved performance of smaller models in real-world applications. It can be applied in various domains requiring efficient model utilization, such as automated coding, natural language processing, and interactive AI systems.
Adversarial Resilience of Poisson-Process Submodular Maximization over Matroids: From Robust Offline Optimization to Full-Bandit Learning
Optimization
Theory
Efficient ML
- Establishes the resilience of the SGS-Poisson algorithm under controlled oracle errors.
- Achieves classical approximation factors for both monotone and non-monotone objectives in full-bandit settings.
- Introduces novel technical results, including adaptive potential-preservation and robust swap lemmas.
- Demonstrates the applicability of offline optimization techniques to online learning frameworks.
Read more
Adversarial Resilience of Poisson-Process Submodular Maximization over Matroids: From Robust Offline Optimization to Full-Bandit Learning
Summary
This paper investigates the robustness of nonnegative submodular maximization under matroid constraints when the offline algorithm is subject to controlled oracle errors. The author introduces the Spiteful Greedy Swap Poisson Process (SGS-Poisson) and demonstrates that it maintains its approximation guarantees (1/e for non-monotone and 1 - 1/e for monotone objectives) despite the presence of adversarially controlled oracle errors. The study establishes a resilience theorem that allows the SGS-Poisson algorithm to perform effectively under these conditions, yielding expected values close to optimal solutions while using a polynomial number of oracle calls. The results extend to full-bandit combinatorial multi-armed bandit (CMAB) settings, achieving classical approximation-regret factors. The paper also highlights the importance of controlled-oracle adaptive preprocessing and presents a robust almost-above-average swap lemma, which together ensure the algorithm's resilience. This work contributes to the understanding of how offline optimization techniques can be adapted for online learning scenarios, particularly in the context of submodular maximization with matroid constraints.
Methodology
The paper employs a theoretical approach to analyze the SGS-Poisson algorithm's performance under controlled oracle errors. It constructs an implementation that maintains approximation guarantees through controlled-oracle adaptive preprocessing and utilizes a black-box reduction from offline to online algorithms. Key technical contributions include the development of a robust drift inequality and a robust almost-above-average swap lemma.
Results
The main results indicate that the SGS-Poisson algorithm can achieve expected values of at least (1/e - Ξ΅)OPT for non-monotone objectives and (1 - 1/e - Ξ΅)OPT for monotone objectives, with a controlled oracle error term. The number of oracle calls required is polynomial in terms of the problem size and the desired accuracy. The findings also provide the first full-bandit guarantees for general matroids that achieve classical limiting factors.
Implications
The results have significant implications for applications in combinatorial optimization, particularly in scenarios where oracle evaluations are subject to errors. This work can enhance the performance of algorithms in fields such as influence maximization, sensor placement, and experimental design, by providing robust solutions that adapt to real-world uncertainties.
High-Order Liquid Evidence Encoding for Gradual GNSS Spoofing Detection in Autonomous Driving
Robotics
Time Series
Interpretability
- Introduction of a causal high-order liquid evidence framework for GNSS spoofing detection.
- Utilization of physics-guided residual evidence to model GNSS-motion inconsistencies.
- Separate adaptive liquid encoders for processing different orders of evidence variations.
- Achieved highest F1-scores among evaluated models on real-world datasets.
Read more
High-Order Liquid Evidence Encoding for Gradual GNSS Spoofing Detection in Autonomous Driving
Summary
This paper addresses the critical issue of GNSS spoofing detection in autonomous vehicles, where subtle and gradual spoofing attacks can manipulate vehicle position estimates without immediate detection. The authors propose a novel causal high-order liquid evidence framework that constructs a physics-guided GNSS-motion inconsistency residual by comparing GNSS-implied displacement with onboard motion-derived displacement. This framework generates separate evidence streams for the residual level and its first- and second-order variations, which are processed by adaptive liquid encoders. The hierarchical coupling of these streams allows for effective prediction of spoofing at the window endpoint, leveraging both current and past observations. The proposed method outperforms existing temporal models in experiments conducted on real-world AV-GPS datasets, demonstrating its effectiveness in detecting gradual spoofing attacks.
Methodology
The methodology involves constructing a GNSS-motion inconsistency residual by comparing GNSS and onboard motion displacements. This residual is then analyzed through separate evidence streams representing zeroth, first, and second-order variations. Each stream is processed by adaptive liquid encoders, and the resulting states are hierarchically coupled to predict spoofing probabilities at the endpoint of a fixed-length causal evidence window.
Results
The proposed method achieved F1-scores of 0.9535 and 0.9777 on two subsets of the AV-GPS dataset, outperforming existing temporal models. It successfully detected both labeled normal-to-attack transitions within four sampling steps on the third dataset.
Implications
The findings suggest that the proposed framework can significantly enhance the reliability of GNSS-based localization in autonomous driving, potentially leading to safer navigation and control systems. This approach may also be applicable in other domains where gradual anomalies need to be detected in time-series data.
Retrieval-Corrected Conformal Prediction for Time Series
Time Series
- RCCP improves upon traditional conformal prediction methods by using retrieval of similar past residuals for local calibration.
- The method constructs asymmetric prediction intervals that reflect local error behavior without relying on global residual quantiles.
- RCCP achieves target coverage levels and lower Winkler scores across various benchmarks, indicating improved performance.
- The approach maintains low calibration and inference overhead, enhancing its scalability for practical applications.
Read more
Retrieval-Corrected Conformal Prediction for Time Series
Summary
This paper introduces Retrieval-Corrected Conformal Prediction (RCCP), a novel method for constructing prediction intervals in time series forecasting that addresses the inefficiencies of traditional conformal prediction (CP) methods. Standard CP often struggles with time series data due to the temporal dependence and variability of forecast errors. While recent approaches have attempted to improve local calibration using recent or localized residuals, they can still dilute relevant evidence by incorporating broad residual weighting. RCCP overcomes this limitation by employing a retrieval strategy that selects similar past residuals as local evidence, followed by a conformal correction to address any coverage errors from the retrieval process. The method constructs asymmetric intervals based on retrieved one-sided residuals and calibrates the normalized retrieval error with a scalar correction factor. The authors provide a coverage-gap bound based on the stability of the normalized retrieval error distribution. Through extensive experiments on standard benchmarks, RCCP consistently achieves the target coverage levels and demonstrates lower Winkler scores, indicating fewer severe misses compared to existing methods. Additionally, RCCP shows low calibration and inference overhead, making it a scalable approach for uncertainty quantification in time series forecasting.
Methodology
RCCP utilizes a retrieval mechanism to select relevant past residuals from a time-ordered knowledge base, constructing asymmetric prediction intervals based on these residuals. It then calibrates the normalized retrieval error to correct for any coverage discrepancies, ensuring that the final intervals maintain the desired coverage levels.
Results
RCCP consistently meets target coverage levels across various time series forecasting benchmarks and achieves the lowest Winkler scores, indicating fewer severe prediction misses compared to existing methods. The method also demonstrates low overhead in calibration and inference, making it efficient for practical use.
Implications
The RCCP method has significant implications for decision-making in fields reliant on time series forecasting, such as finance, traffic management, and supply chain systems. By providing more accurate and reliable prediction intervals, it enhances the ability to quantify uncertainty and make informed decisions under varying conditions.
Dual-Model Sentiment Analysis of Consumer Reviews in the Retail Coffee Sector Using Machine Learning and Deep Learning Approaches
NLP
- Introduces a dual-model framework combining machine learning and deep learning for sentiment analysis.
- Evaluates five machine learning algorithms and five deep learning models on an imbalanced dataset.
- Implements a preprocessing pipeline using NLP techniques to enhance input quality.
- Demonstrates model performance on unseen real-world data, reflecting practical deployment scenarios.
Read more
Dual-Model Sentiment Analysis of Consumer Reviews in the Retail Coffee Sector Using Machine Learning and Deep Learning Approaches
Summary
This study presents a dual-model sentiment analysis framework applied to Starbucks customer reviews, utilizing both classical machine learning and advanced deep learning techniques. The dataset, comprising over 700 reviews from ConsumerAffairs, underwent extensive preprocessing to uncover trends and derive sentiment labels based on star ratings. Acknowledging the heavily imbalanced distribution of sentiments, the authors benchmarked traditional classifiers (Logistic Regression, SVM, Decision Tree, Random Forest, Naive Bayes) against deep learning models (LSTM, RNN, Bidirectional LSTM, GRU, CNN). Performance metrics included accuracy, precision, recall, and F1-score. The results indicated that SVM achieved the highest accuracy at 91.0%, while Bidirectional LSTM excelled in generalization on unseen data. The study emphasizes the impact of class imbalance on model performance, particularly in positive sentiment recall, and highlights the importance of tailored model selection and preprocessing strategies for effective sentiment analysis in the retail coffee sector. The findings contribute to understanding consumer sentiment and optimizing customer experience through automated analysis.
Methodology
The study employed a dual-model approach, comparing traditional machine learning classifiers (Logistic Regression, SVM, Decision Tree, Random Forest, Naive Bayes) with deep learning architectures (LSTM, RNN, Bidirectional LSTM, GRU, CNN). Extensive preprocessing techniques, including stopword removal, lemmatization, and TF-IDF vectorization, were applied to the dataset to prepare it for analysis.
Results
The SVM model achieved the highest accuracy of 91.0% among the machine learning models, while the Bidirectional LSTM outperformed other deep learning models in terms of generalization on unseen data. The study also highlighted the challenges posed by class imbalance, particularly affecting the recall of positive sentiments across various models.
Implications
The findings underscore the significance of selecting appropriate models and preprocessing strategies in sentiment analysis pipelines, which can lead to improved customer experience and brand loyalty in the retail coffee sector. This research also provides a framework for future studies in sentiment analysis, particularly in handling imbalanced datasets.
FunnelCausalNet: Funnel-aware Joint Conversion-Revenue Uplift for Multi-tier Coupon Allocation
Optimization
Theory
- FunnelCausalNet integrates conversion and revenue uplift modeling, addressing the limitations of decoupled approaches.
- The model incorporates a variance analysis that guides its implementation under high zero-inflation regimes.
- A budgeted multi-tier allocation strategy is proposed, allowing for efficient resource distribution across different coupon strengths.
- Empirical evaluations show significant improvements in GMV effect error reduction and ROI metrics compared to existing baselines.
Read more
FunnelCausalNet: Funnel-aware Joint Conversion-Revenue Uplift for Multi-tier Coupon Allocation
Summary
The paper introduces FunnelCausalNet, a novel uplift estimator designed to optimize coupon allocation in marketing campaigns by jointly estimating conversion and revenue uplift while accounting for the inherent funnel structure of gross merchandise value (GMV). Traditional approaches often treat conversion and revenue as separate entities, leading to inefficiencies and inaccuracies in uplift modeling. FunnelCausalNet addresses this by coupling a binary conversion head with a nonnegative conditional-value head, thereby respecting the deterministic relationship between conversion and GMV. The authors derive a leading-order mean squared error (MSE) ratio comparison under specific assumptions, which guides the implementation of the estimator. They also propose a budgeted multi-tier allocation strategy that utilizes RCT-anchored estimates for effective subsidy management. The model is evaluated against eleven baselines, demonstrating its effectiveness in reducing GMV effect error and achieving superior return on investment (ROI) metrics in various experimental settings, including semi-synthetic and industrial datasets.
Methodology
FunnelCausalNet employs a dual-head neural network architecture that simultaneously estimates binary conversion probabilities and nonnegative conditional order values. The model is built on assumptions derived from randomized controlled trials (RCTs) to ensure robust causal inference. It also includes a Lagrangian budget allocator for multi-tier coupon distribution and utilizes marginal split-conformal intervals for uncertainty quantification.
Results
FunnelCausalNet achieved a mean AUUC_GMV within one seed standard deviation of the leading baseline on a semi-synthetic dataset and reduced GMV effect error by 18-48% compared to direct GMV regression. In industrial applications, it attained the best seed-averaged mean ΞROI across multiple correlated expected-outcome anchors, demonstrating its effectiveness in real-world scenarios.
Implications
The findings suggest that FunnelCausalNet can significantly enhance the efficiency of marketing campaigns by providing a more accurate framework for coupon allocation. Its ability to jointly model conversion and revenue uplift while respecting the funnel structure can lead to better decision-making and resource allocation in e-commerce settings.
Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecasting
Time Series
- First systematic study of activation calibration for PTQ in financial forecasting.
- 4-bit activation quantization leads to significant predictive losses, recoverable through improved calibration.
- Activation range preferences evolve over time, necessitating dynamic calibration strategies.
- Practical deployment guidelines are provided for selecting among various quantization methods.
Read more
Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecasting
Summary
This paper investigates the impact of activation calibration on post-training quantization (PTQ) for financial time-series forecasting, specifically focusing on cross-sectional volatility forecasting of the S&P 500. The authors conduct a systematic study involving seven neural architectures and 560 trained models over eight years (2018-2025). They find that while 8-bit quantization and 4-bit weight-only quantization have minimal predictive degradation, 4-bit activation quantization can lead to significant losses (11-62% of the full-precision mean information coefficient) under default calibration strategies. The study reveals that improved calibration methods, such as percentile calibration, can recover a substantial portion of this degradation (53-94%) in the most affected architectures. Furthermore, the preferred activation range varies across different market conditions, emphasizing that activation calibration should be treated as a critical deployment decision rather than a static preprocessing step. The findings provide practical guidelines for selecting quantization strategies in financial forecasting.
Methodology
The authors employed a walk-forward evaluation protocol to assess the effects of activation calibration on PTQ. They systematically evaluated seven neural architectures across eight test years, comparing quantized models against their full-precision counterparts while varying the activation calibration strategy.
Results
The study found that 8-bit quantization and 4-bit weight-only quantization had minimal impact on predictive performance. However, 4-bit activation quantization resulted in significant losses, which could be mitigated through improved calibration methods. The preferred activation range was shown to vary with market conditions, indicating the need for adaptive calibration strategies.
Implications
The findings suggest that careful consideration of activation calibration is essential for deploying low-precision models in financial forecasting. The results can guide practitioners in selecting appropriate quantization strategies to maintain model performance while meeting computational constraints.
Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategies
Theory
Optimization
Time Series
- Introduces two strategies to mitigate error growth in long-time trajectory approximation using SA-NODEs.
- The model predictive strategy utilizes adaptive partitioning and state resets to maintain error tolerance.
- The Floquet strategy leverages stable limit cycles to ensure linear error growth without requiring data at deployment.
- Numerical experiments confirm the effectiveness of the proposed strategies and their theoretical guarantees.
Read more
Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategies
Summary
This paper addresses the challenge of approximating dynamical systems over long time horizons using semi-autonomous neural ordinary differential equations (SA-NODEs). The authors identify a significant issue where the error bound for a single network trained over a long horizon deteriorates double exponentially with the horizon length. To overcome this, they propose two innovative training strategies: a model predictive strategy and a Floquet strategy. The model predictive approach involves partitioning the time horizon into shorter windows, allowing for state resets from observed data, which helps maintain a uniform error tolerance across the entire horizon. The Floquet strategy is designed for autonomous targets with stable limit cycles, leveraging the inherent contraction properties of the learned return map to ensure that the error grows linearly with the number of periods rather than exponentially. The paper provides a theoretical foundation for these strategies and validates them through numerical experiments on various benchmarks, demonstrating that the proposed methods effectively control error propagation over long time horizons.
Methodology
The authors develop two main strategies: (1) a model predictive control approach that partitions the time horizon into manageable windows with state resets, and (2) a Floquet approach that utilizes the properties of limit cycles to control error growth. They also provide theoretical guarantees for these methods and conduct numerical experiments to validate their effectiveness.
Results
The proposed strategies successfully reduce the error growth associated with long-time trajectory approximation. The model predictive strategy allows for a linear parameter budget in relation to the horizon length, while the Floquet strategy ensures that errors grow linearly with the number of periods, rather than exponentially. Numerical experiments corroborate the theoretical predictions regarding error behavior.
Implications
The findings have significant implications for the modeling of dynamical systems in various fields, such as control systems, robotics, and biological systems, where long-term predictions are essential. The strategies could enhance the reliability of neural ODEs in practical applications requiring accurate long-term forecasting.
A Recommendation System Approach for Interference-Robust Sensor Subset Selection
Audio & Speech
Efficient ML
Robotics
- Formulates sensor activation as a recommendation problem using network observations as context.
- Introduces a Two-Tower MLP architecture for efficient scoring of sensor subsets.
- Demonstrates significant improvements in robustness to acoustic interference using frequency-band features.
- Achieves high accuracy (up to 98.4%) while maintaining low computational requirements (sub-millisecond to 1 ms).
Read more
A Recommendation System Approach for Interference-Robust Sensor Subset Selection
Summary
This paper presents a novel method for sensor subset selection aimed at improving tracking accuracy in environments with heterogeneous sensing modalities. Previous approaches relied on low-cost acoustic Received Signal Strength Indicator (RSSI) measurements to recommend sensor nodes for activation, but these methods faced challenges due to acoustic interference. The authors propose a recommendation-system-inspired framework that utilizes frequency-band acoustic features and a Two-Tower Multi-Layer Perceptron (MLP) architecture to score candidate sensor subsets more effectively. By leveraging richer acoustic information, the proposed method enhances robustness against interference while maintaining low computational overhead for real-time applications. Experimental evaluations on outdoor vehicle-tracking deployments demonstrate that the new approach can achieve approximately 20% improvement in accuracy over the RSSI baseline, showcasing its effectiveness in practical scenarios.
Methodology
The authors developed a Two-Tower MLP architecture that learns separate embeddings for the network's acoustic state and candidate sensor subsets. This architecture allows for efficient scoring of multiple sensor subsets based on richer acoustic features, moving beyond the limitations of scalar RSSI measurements.
Results
The proposed recommendation framework showed a substantial improvement in tracking accuracy, achieving up to 98.4% in one experiment, compared to an 80.4% accuracy with the RSSI baseline. The method also maintained a low computational cost, enabling real-time sensor activation.
Implications
The findings suggest that leveraging richer acoustic information can significantly enhance sensor management in tracking applications, particularly in environments where visual sensing is compromised. This approach could be applied in various fields, including autonomous vehicles, surveillance, and smart city infrastructure.
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Reinforcement Learning
Large Language Models
NLP
- Rubric-as-reward RL can lead to reward hacking due to fixed criteria exploitation.
- Rubric Dropout is a simple yet effective method to mitigate this issue by randomly dropping rubric criteria during training.
- Experiments show significant improvements in OOD performance and reductions in reward hacking metrics.
- The method is computationally efficient, requiring only a single hyperparameter adjustment.
Read more
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Summary
This paper addresses the issue of reward hacking in reinforcement learning (RL) when using rubrics as rewards, particularly in the context of training language models. The authors demonstrate that fixed rubrics can lead to a divergence between scores from a training judge and a stronger gold judge, indicating that the model is exploiting the rubric rather than genuinely improving. They introduce a novel technique called 'Rubric Dropout', which randomly drops a subset of rubric criteria during training to prevent the model from optimizing against the same fixed criteria repeatedly. This method is shown to effectively mitigate reward hacking without incurring additional costs in domain performance. The authors validate their approach through experiments on two benchmark datasets, showing that Rubric Dropout consistently improves out-of-distribution (OOD) performance and reduces measures of reward hacking compared to traditional methods.
Methodology
The authors implemented Rubric Dropout by randomly selecting a fraction of rubric criteria to drop at each training step. They measured the divergence between scores from a training judge and a gold judge to assess reward hacking. The methodology was tested on two independent training-evaluation pairs, using Group Relative Policy Optimization (GRPO) to maintain comparability across rollouts.
Results
The introduction of Rubric Dropout led to improvements in OOD gold scores, with increases of 1-2 points on HealthBench-Hard and 6-7 points on ResearchQA. The method also reduced the divergence between the training and gold judge scores, indicating a successful mitigation of reward hacking. The optimal dropout fraction was found to be between 30-50%, while traditional reweighting methods performed worse than no intervention.
Implications
The findings suggest that Rubric Dropout could be a valuable technique for enhancing the robustness of RL systems that rely on rubric-based rewards, particularly in open-ended tasks where quality is difficult to quantify. This approach may lead to more reliable and generalizable language models in various applications, including medical advice and research explanations.
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
NLP
Large Language Models
Theory
- OPD improves sampling efficiency but does not consistently expand the reasoning capability boundary of student models.
- OPD-trained models show better performance at small K values but are surpassed by pre-OPD models at larger K.
- More previously solvable problems become unsolvable after OPD training than vice versa.
- The study introduces the concept of 'illusory distillation,' indicating that apparent gains stem from better access to existing capabilities.
Read more
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Summary
This paper investigates the effectiveness of On-Policy Distillation (OPD) as a method for enhancing the reasoning capabilities of large language models (LLMs). The authors challenge the prevailing belief that OPD allows student models to acquire new reasoning capabilities from stronger teacher models. By employing a test-time scaling approach, they analyze the performance of OPD-trained models across varying sampling budgets (K) using metrics such as pass@K and avg@K. The findings reveal that while OPD-trained models excel in avg@K performance across different budgets, their advantage in pass@K diminishes and eventually reverses as K increases, indicating that OPD primarily enhances sampling efficiency rather than expanding the reasoning capability boundary of the student model. The study also highlights an asymmetry in problem solvability, where OPD leads to more previously solvable problems becoming unsolvable. This phenomenon is termed 'illusory distillation,' suggesting that the gains observed from OPD are largely due to improved access to existing capabilities rather than the acquisition of new reasoning skills.
Methodology
The authors conducted a systematic analysis of OPD by varying the sampling budget K and evaluating the performance of OPD-trained models against pre-OPD models using metrics like pass@K and avg@K across multiple reasoning benchmarks. They also performed a problem-level solvability analysis to assess the impact of OPD on the solvability of various problems.
Results
The results indicate that OPD-trained models achieve higher avg@K performance across sampling budgets, but their advantage in pass@K diminishes with larger K values. Additionally, the analysis shows that OPD tends to cause more previously solvable problems to become unsolvable, supporting the notion of illusory distillation.
Implications
These findings suggest that while OPD may enhance the efficiency of reasoning paths accessed by student models, it does not necessarily lead to the development of new reasoning capabilities. This has implications for the design and evaluation of distillation methods in machine learning, particularly in the context of LLMs.