AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
48
Papers today
8h
Update frequency
7
Days of history
Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals
Large Language Models
Optimization
Efficient ML
- PUST decouples update-signal exploration from distribution alignment, enhancing efficiency.
- A lightweight proxy model is used for exploration, reducing computational costs.
- Relative improvement signals are extracted and transferred to guide primary model updates.
- The framework supports asynchronous signal generation and cross-model transfer.
Read more
Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals
Summary
This paper introduces Proxy-guided Update Signal Transfer (PUST), a novel framework for post-training large language models (LLMs) that decouples the exploration of update signals from distribution alignment. Traditional methods tightly couple these processes, leading to high computational costs and inefficiencies. PUST employs a lightweight proxy model to conduct exploration, discovering high-reward behaviors without burdening the primary model. The framework extracts relative improvement signals from the proxy's performance and transfers these to the primary model for policy alignment. This decoupling allows for asynchronous generation and reuse of optimization signals, significantly reducing computational overhead. The authors demonstrate the effectiveness of PUST through systematic evaluations on Qwen3-family models across math and code domains, showing that signals from weaker proxies can enhance stronger primary models. The results indicate that PUST transforms post-training into a modular and reusable process, improving efficiency and scalability in model training.
Methodology
The methodology involves a three-step pipeline: proxy exploration using a lightweight model to discover high-reward behaviors, extraction of relative improvement signals from the proxy's performance, and transfer of these signals to the primary model for policy alignment. This modular approach allows for asynchronous signal generation and reuse across different training runs.
Results
The evaluations on Qwen3-family models reveal that update signals derived from significantly weaker proxies can effectively enhance the performance of stronger primary models. The transfer intensity of these signals can be adjusted, leading to robust performance across various settings.
Implications
The PUST framework has the potential to revolutionize post-training processes for LLMs, making them more efficient and scalable. It allows for the reuse of optimization signals across different models and configurations, which could lead to significant advancements in training methodologies and applications in diverse domains.
Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
Reinforcement Learning
Large Language Models
Theory
- Introduction of a hybrid verifier combining binary and continuous rewards for PDE solver generation.
- Creation of a multi-PDE solver dataset and a reproducible post-training pipeline.
- Demonstration of a single policy's ability to learn across diverse PDE families.
- Smaller RLVP-trained models outperform larger static prompting baselines.
Read more
Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
Summary
This paper presents Reinforcement Learning with Verifiable Physics (RLVP), a novel post-training framework aimed at enhancing the generation of code for solving partial differential equations (PDEs) using large language models (LLMs). Traditional methods for PDE solving often rely on expert knowledge and are labor-intensive, while existing LLM approaches primarily focus on inference-time improvements. RLVP addresses the limitations of binary verifiers in reinforcement learning by introducing a hybrid verification system that combines hard program-validity checks with continuous rewards based on physical accuracy and residual consistency. The authors curate a diverse dataset of PDE families and demonstrate that a single policy can be effectively trained across multiple PDE types. The results show that RLVP significantly outperforms both pre-trained and supervised-only baselines, achieving zero-shot transfer to unseen PDEs and indicating the model's ability to generalize and recombine learned numerical motifs. This work represents a significant step towards automating reliable PDE solver generation and broadening the accessibility of scientific computing.
Methodology
The authors employ a reinforcement learning framework with a hybrid verifier that integrates hard execution-validity checks and continuous rewards based on function-space accuracy and physics residual consistency. The model is post-trained on a curated dataset of PDE solvers, allowing it to learn from a variety of PDE families and improve its code generation capabilities.
Results
The RLVP framework shows an 8-13 percentage point improvement over binary-validity-only baselines and reduces normalized root mean square error (nRMSE) by 38%. The trained policy demonstrates the ability to generate accurate solvers for held-out PDEs, indicating effective zero-shot transfer and compositionality in numerical programming.
Implications
The RLVP framework could significantly streamline the process of PDE solver generation, making it more accessible to non-experts and enhancing the reliability of scientific computing applications. This approach may also inspire further research into the integration of reinforcement learning and LLMs for other complex code generation tasks.
OmniPMNet: Bridging discrete and gridded PM10 forecasts via omni-query neural processes
Graph Learning
Time Series
Multimodal
- OmniPM-Net effectively combines discrete and gridded PM10 forecasts using a shared spatial representation.
- The model outperforms both traditional chemical transport models and graph neural networks in accuracy and spatial resolution.
- Significant improvements are observed in high-concentration PM10 scenarios and during severe dust events.
- The approach provides a general framework for fusing heterogeneous atmospheric data sources.
Read more
OmniPMNet: Bridging discrete and gridded PM10 forecasts via omni-query neural processes
Summary
The paper introduces OmniPM-Net, a novel model designed to enhance PM10 forecasting by integrating discrete station-level predictions with continuous gridded forecasts. Traditional forecasting methods either excel at local accuracy or provide comprehensive spatial fields but struggle to combine both effectively. OmniPM-Net employs a Convolutional Conditional Neural Process (ConvCNP) framework to reconcile these two types of forecasts. The model utilizes a terrain-aware Gaussian set convolution to project irregular GNN station forecasts onto a regular grid, which is then enhanced by a multi-scale Spatial Source Attention (SSA) module that merges these forecasts with data from the Copernicus Atmosphere Monitoring Service (CAMS). The omni-query readout allows for consistent PM10 predictions at both station and grid levels over a 108-hour horizon. Evaluated across 1,618 air-quality monitoring stations in China throughout 2024, OmniPM-Net achieves station-level accuracy comparable to a strong GNN baseline while reducing CAMS mean absolute error by 30%. The model demonstrates significant improvements in high-concentration scenarios and during dust events, showcasing its ability to track evolving pollution patterns more effectively than existing methods.
Methodology
OmniPM-Net employs a Convolutional Conditional Neural Process (ConvCNP) framework, utilizing terrain-aware Gaussian set convolutions to map irregular GNN station forecasts onto a regular grid. It incorporates a multi-scale Spatial Source Attention (SSA) module to blend these forecasts with CAMS data, followed by an omni-query readout for generating consistent PM10 predictions at both station and grid levels.
Results
The model matches the station-level accuracy of the GNN baseline with a mean absolute error of 21.14 Β΅g/m3 compared to 22.00 Β΅g/m3 for the GNN. Additionally, it reduces the mean absolute error of CAMS forecasts by 30%. Notably, it shows a 9% improvement in the 90th-percentile MAE relative to the GNN and a 25% improvement relative to CAMS during high-concentration events.
Implications
OmniPM-Net's ability to reconcile point and gridded forecasts offers a promising approach for atmospheric data fusion, potentially enhancing air quality monitoring and forecasting capabilities. This model could be adapted for other environmental data fusion tasks, improving predictive accuracy and spatial resolution in various applications.
The Singularity Space: A Generative Diffusion Framework for Signal Representation
Generative Models
Audio & Speech
Theory
- Introduces a generative framework that uses complex-plane singularities for signal representation.
- Achieves interpretability by linking singularity configurations to physical parameters.
- Demonstrates structural stability, reducing artifacts at discontinuities.
- Shows significant improvements in reconstruction accuracy over traditional grid-based methods.
Read more
The Singularity Space: A Generative Diffusion Framework for Signal Representation
Summary
This paper introduces the Singularity Space, a novel generative framework designed to represent signals through complex-plane singularities, addressing the limitations of traditional dense grid representations that often blur critical transients in physical signals. The framework is built on the classical pole-residue representation of meromorphic functions, allowing for a latent space of singularity configurations that are physically constrained. Key properties of the framework include interpretability, structural stability, and resolution-free output reconstruction on arbitrary grids. The authors employ a transformer-based diffusion model to predict samples at singularity coordinates while adhering to geometric constraints. The framework is evaluated using 1D Burgers shocks, demonstrating its ability to maintain signal structure and achieve significantly lower reconstruction errors compared to grid-based methods. The results indicate that singularity-based representations could enhance the modeling of transient-dominated signals across various domains, including speech and biomedical signals.
Methodology
The Singularity Space framework utilizes a transformer-based diffusion model that predicts singularity configurations in the complex plane. It employs a score-based Stochastic Differential Equation (SDE) to transform a standard normal distribution into physically-consistent singularity configurations, optimizing the learning process to mitigate spectral bias and enhance signal representation.
Results
The framework successfully reconstructs 1D Burgers shocks using only 32 predicted singularities, achieving a 4.2Γ lower reconstruction error compared to a 1024-point grid signal. It maintains a total variation ratio close to 1 under unseen noise conditions and accurately recovers physical parameters with an absolute error of 10^-4.
Implications
The findings suggest that singularity-based representations could serve as a robust foundation for modeling various transient-dominated signals, potentially improving applications in fields such as audio processing, biomedical signal analysis, and fluid dynamics.
Differentiable Clone-Structured Causal Graphs for End-to-End Cognitive Map Learning from Image Sequences
Computer Vision
Graph Learning
Generative Models
- Introduction of gradCSCG, a differentiable version of the CSCG algorithm.
- Development of an end-to-end trainable pipeline combining gradCSCG and VQ-VAE.
- Implementation of loss-balancing mechanisms for stable joint training.
- Successful recovery of topological maps from aliased image sequences.
Read more
Differentiable Clone-Structured Causal Graphs for End-to-End Cognitive Map Learning from Image Sequences
Summary
This paper presents a novel approach to cognitive map learning through the introduction of the Clone-Structured Causal Graph (CSCG) algorithm, reformulated as a differentiable module called gradCSCG. The authors address the limitations of the original CSCG, which required a predefined discrete alphabet and was not easily integrated with neural networks. By coupling gradCSCG with a learned vector-quantized variational autoencoder (VQ-VAE), the authors create an end-to-end trainable pipeline that allows for the processing of raw image sequences. The methodology includes a soft emission forward pass that enables the learning objective to influence perception, alongside loss-balancing mechanisms to ensure stable joint training. The effectiveness of this approach is demonstrated through experiments on symbolic grid worlds and MNIST image sequences, showing that the model can recover the underlying adjacency graph from heavily aliased observations. This work provides a foundational step towards integrating CSCG into deep learning architectures, emphasizing its potential for creating interpretable cognitive maps from complex sensory inputs.
Methodology
The authors reformulate the Clone-Structured Causal Graph (CSCG) as a fully differentiable module (gradCSCG) that can be trained using gradient descent. This module is coupled with a vector-quantized variational autoencoder (VQ-VAE) to process raw image sequences. The training involves a soft emission forward pass and various loss-balancing techniques to maintain stability during joint training.
Results
The experiments demonstrate that the gradCSCG successfully reproduces the results of the original CSCG on symbolic grid worlds and effectively recovers the underlying adjacency graph from MNIST image sequences, achieving high edge precision and recall despite the presence of heavy aliasing.
Implications
This work suggests that the gradCSCG can serve as a composable building block in deep learning architectures, enabling the development of agents capable of constructing interpretable cognitive maps from complex sensory inputs. This has potential applications in robotics, navigation systems, and cognitive modeling.
Scale-Aware Attention for Scarce Neural Data: An RG-Flow Transformer on Sleep-EDF EEG
Time Series
Theory
Interpretability
- Introduction of the RG-Flow Transformer, which incorporates scale-aware attention mechanisms.
- Benchmarking against a vanilla transformer shows no significant accuracy advantage in sleep staging tasks.
- RG-Flow Transformer successfully recovers the spectral exponent Ξ², enhancing interpretability.
- Study emphasizes the challenges of working with scarce neural data and the importance of inductive biases.
Read more
Scale-Aware Attention for Scarce Neural Data: An RG-Flow Transformer on Sleep-EDF EEG
Summary
This paper introduces the RG-Flow Transformer, a novel architecture designed to leverage scale-invariance in neural data, particularly EEG signals from the Sleep-EDF dataset. The RG-Flow Transformer incorporates a renormalization-group (RG) inductive bias, which allows it to couple standard self-attention mechanisms with a scale-aware stream characterized by a learnable anomalous dimension. The study aims to determine whether this architecture outperforms a parameter-matched vanilla transformer in the context of scarce EEG data. The authors conduct a series of experiments, including a benchmark comparison against a vanilla transformer and a hierarchy-only ablation for 5-class AASM sleep staging, a data-budget sweep to identify performance crossover points in data-scarce scenarios, and an evaluation of the RG-Flow's ability to recover the spectral exponent Ξ², which tracks sleep depth. Despite the theoretical advantages of the RG-Flow Transformer, the results show that it performs similarly to the vanilla transformer in terms of accuracy (77.3% vs 77.0%) across five subjects, with no significant advantage in data-scarce conditions. However, RG-Flow demonstrates superior interpretability by successfully recovering the continuous spectral exponent out-of-sample, a feature absent in the vanilla model.
Methodology
The authors employed a strict leakage-free by-subject hold-out protocol to benchmark the RG-Flow Transformer against a vanilla transformer and a hierarchy-only ablation on the Sleep-EDF dataset. They conducted a per-subject data budget sweep to analyze performance under data scarcity and assessed the model's ability to recover the spectral exponent Ξ² out-of-sample.
Results
The RG-Flow Transformer achieved an accuracy of 77.3% compared to 77.0% for the vanilla transformer, with no statistically significant difference (p = 0.294). The anticipated crossover advantage in data-scarce conditions was not observed, as the vanilla transformer consistently outperformed RG-Flow across various data budgets. However, RG-Flow successfully recovered the spectral exponent Ξ² with an RΒ² of 0.416, indicating its interpretative strength.
Implications
The findings suggest that while the RG-Flow Transformer offers enhanced interpretability through the recovery of spectral properties, it may not provide a performance advantage over traditional models in practical applications involving scarce neural data. This highlights the need for further exploration of inductive biases in deep learning architectures for neural data analysis.
Data-Driven Telecom Marketing Optimization: A Machine Learning-Based Churn Prediction and Customer Segmentation Framework
Optimization
- Development of a comprehensive ML pipeline for churn prediction and customer segmentation.
- Utilization of advanced gradient boosting techniques for improved churn prediction accuracy.
- Implementation of K-Means clustering for actionable customer segmentation.
- Creation of a transparent ROI/CLV framework to assess marketing interventions.
Read more
Data-Driven Telecom Marketing Optimization: A Machine Learning-Based Churn Prediction and Customer Segmentation Framework
Summary
This paper addresses the critical issue of customer churn in the telecommunications sector, proposing a comprehensive, data-driven marketing optimization framework. Traditional retention strategies often fail to effectively target high-risk customers, leading to wasted marketing resources. The authors present a four-stage pipeline that integrates machine learning-based churn prediction, customer segmentation based on churn risk and customer value, and tailored marketing strategies with a focus on Return on Investment (ROI). Utilizing the IBM Telco Customer Churn dataset, the study employs three gradient-boosting algorithmsβXGBoost, LightGBM, and CatBoostβoptimized through rigorous hyperparameter tuning and cross-validation. CatBoost emerged as the best-performing model, achieving notable metrics including 77.68% accuracy and an F1-score of 0.6366. Following churn prediction, K-Means clustering was applied to segment customers into actionable clusters, which were then cross-tabulated with churn risk labels. The framework also includes a theoretical ROI/Customer Lifetime Value (CLV) model to quantify the financial impact of marketing interventions. The entire process is operationalized in an interactive Streamlit application, allowing marketing teams to visualize churn drivers and generate reports. The findings suggest that integrating predictive modeling with value-aware segmentation significantly enhances marketing effectiveness compared to traditional methods.
Methodology
The methodology involves training three gradient-boosting models (XGBoost, LightGBM, CatBoost) on the IBM Telco Customer Churn dataset, followed by K-Means clustering for customer segmentation. The models were optimized using randomized search and stratified 5-fold cross-validation. A theoretical ROI/CLV framework was also developed to evaluate the financial impact of marketing strategies.
Results
The CatBoost model achieved 77.68% accuracy, 73.53% recall, 56.12% precision, an F1-score of 0.6366, a PR-AUC of 0.6553, and a ROC-AUC of 0.8403 on the test set. The K-Means clustering resulted in four distinct customer segments, each with tailored marketing strategies, enhancing the potential for effective customer retention.
Implications
The proposed framework can significantly improve marketing strategies in the telecommunications industry by enabling targeted retention efforts, optimizing resource allocation, and ultimately increasing profitability. The interactive dashboard allows for real-time data analysis and decision-making, making it accessible for marketing teams.
From Global to Factor-Wise Expert Composition in Discrete Diffusion Models
Generative Models
Computer Vision
Theory
- Introduction of FactorDiff, a factor-wise composition framework for discrete diffusion models.
- Dynamic routing of factors to relevant experts improves performance over traditional global scalar weighting methods.
- Empirical validation on the ARC-AGI benchmark shows significant gains in logical consistency and spatial reasoning tasks.
- The method highlights the importance of spatial and functional independence among experts in generative modeling.
Read more
From Global to Factor-Wise Expert Composition in Discrete Diffusion Models
Summary
This paper introduces FactorDiff, a novel factor-wise composition framework for discrete diffusion models aimed at enhancing compositional generation in complex reasoning tasks. Traditional methods rely on global scalar weights for expert contributions, which can dilute the effectiveness of specialized models in specific domains. FactorDiff addresses this limitation by decomposing samples into smaller factors and dynamically routing each factor to the most relevant expert, allowing for more granular control over expert contributions. The authors validate their approach on the ARC-AGI benchmark, demonstrating that factor-specific routing consistently outperforms existing scalar weighting methods, particularly in tasks requiring logical consistency and spatial disentanglement. The proposed framework not only improves performance but also emphasizes the importance of recognizing the spatial and functional independence of different experts in generative tasks.
Methodology
The authors propose a factor-wise composition framework that decomposes generated samples into user-defined factors. Each factor is routed to the most relevant expert during the sampling process, allowing for heterogeneous contributions from multiple experts. The framework is instantiated with a position-level routing mechanism for grid-structured reasoning tasks, enhancing the model's ability to handle complex spatial relationships.
Results
FactorDiff was evaluated on the ARC-AGI benchmark, where it consistently outperformed existing methods such as SuperDiff, RNE, and Feynman-Kac correctors. The results indicate that the factor-wise routing approach leads to superior generalization, particularly in tasks that require strict logical consistency and the ability to manage spatial constraints effectively.
Implications
The findings suggest that adopting a factor-wise approach in discrete diffusion models can significantly enhance their effectiveness in complex reasoning tasks. This has potential applications in various fields requiring advanced generative modeling capabilities, such as computer vision and artificial intelligence, where understanding spatial and functional relationships is crucial.
Knowledge-Conditioned, Single-Pass LLM Synthesis of Executable Unity Game Scenes: A Compiler Error Census across 26 Goal Playable Concepts
Large Language Models
- Introduces a Grounding/Hygiene taxonomy for categorizing compiler errors in LLM-generated Unity scripts.
- Conducts a comprehensive error code census across 10,400 generated scripts, revealing patterns in model performance.
- Demonstrates that larger models and different generation modes do not lead to successful compilations.
- Links error profiles to gameplay design semantics, providing a framework for understanding model limitations in game design.
Read more
Knowledge-Conditioned, Single-Pass LLM Synthesis of Executable Unity Game Scenes: A Compiler Error Census across 26 Goal Playable Concepts
Summary
This paper investigates the capabilities of large language models (LLMs) in generating executable Unity C# scripts for game scenes without relying on iterative repair loops. The authors focus on a single-pass generation approach to isolate the intrinsic knowledge of the models, evaluating their performance across 10,400 generations involving four different models (ranging from 7B to 30B parameters) and 26 goal playable concepts. The study categorizes the resulting compiler errors into two main types: Grounding errors, which stem from incorrect or missing Unity API knowledge, and Hygiene errors, which are structural defects unrelated to Unity. The findings reveal that none of the generated scripts compiled successfully, highlighting a significant gap in the models' engine-specific knowledge. The paper also provides a detailed error code census, linking the error profiles to the semantic complexity of the goal patterns, thus offering insights into which gameplay concepts are challenging for current LLMs to realize.
Methodology
The authors employed a single-pass generation method using four open-weight LLMs to create Unity C# scripts based on 26 goal playable concepts. They categorized compiler errors into Grounding and Hygiene types and conducted a detailed analysis of 90,673 error occurrences across 10,400 generated scripts, examining the relationship between error types and gameplay design semantics.
Results
The study found that none of the generated C# scripts compiled into runnable Unity scenes. The error analysis revealed a total of 99 distinct error codes, with a significant portion classified as Hygiene errors. The distribution of errors varied by goal pattern, indicating that different gameplay concepts require varying levels of engine-specific knowledge.
Implications
The findings suggest that while LLMs can generate code, their current capabilities are limited when it comes to producing executable game content without human intervention. This highlights the need for further advancements in LLM training, particularly in understanding domain-specific knowledge like game engines. The insights gained can inform game designers about the limitations of LLMs and guide future research in improving code generation for game development.
An Agentic AI Scientific Community for Automated Neural Operator Discovery
Theory
Optimization
Large Language Models
- Introduces an agentic framework for neural operator discovery using a decentralized AI scientific community.
- Demonstrates the importance of LLM agency in preserving architectural diversity during the discovery process.
- Finds that no single neural operator architecture is universally superior, supporting a no-free-lunch theorem.
- Evaluates the framework on multiple PDE problems, showcasing its effectiveness in discovering efficient neural architectures.
Read more
An Agentic AI Scientific Community for Automated Neural Operator Discovery
Summary
This paper introduces an innovative framework for autonomous neural operator discovery through an AI scientific community composed of virtual laboratories. Each laboratory operates with three distinct agents: a large language model (LLM) planner that proposes neural architectures, a numerical worker that trains these architectures, and an LLM reviewer that conducts peer reviews across labs. The community functions under a citation-based economy, where well-cited labs influence the creation of new labs while non-performing labs are replaced. The authors evaluate this framework on five different problems, including various partial differential equations (PDEs), demonstrating its capability to discover high-accuracy, low-parameter-count neural operator architectures. The results indicate that LLM planners predominantly hybridize architectures, achieving a diverse range of solutions. An ablation study reveals that replacing LLM agents with rule-based alternatives leads to a collapse of diversity in architecture, underscoring the importance of LLM agency in maintaining a rich exploration of design space. The findings suggest a no-free-lunch theorem for neural operators, indicating that no single architecture universally outperforms others across all problems.
Methodology
The authors employ a decentralized network of virtual laboratories, each containing LLM planners, numerical workers, and reviewers. The framework utilizes a citation-based economy to guide the evolution of labs and architectures. An ablation study is conducted to compare the performance of LLM agents against rule-based agents in the architecture discovery process.
Results
The AI scientific community successfully discovered high-accuracy neural operator architectures with low parameter counts across five evaluated problems. The LLM planners favored hybrid architectures in 99.8% of their decisions, while the ablation study revealed that rule-based agents led to less diverse, single-family architectures. The results reinforce the idea that no universal neural operator architecture exists, as different architectures excel in different problem contexts.
Implications
This research has significant implications for the field of operator learning and neural architecture search, suggesting that autonomous systems can effectively explore design spaces without human intervention. The findings may influence future research in automated machine learning and the development of more efficient neural network architectures.
Graph-Constrained Policy Learning for Extreme Clinical Code Prediction
NLP
Graph Learning
- Introduces a graph-constrained traversal policy for clinical code prediction.
- Outperforms traditional flat multi-label classification methods.
- Demonstrates that a single shared policy can match complex architectures.
- Highlights the importance of increasing supervised training data.
Read more
Graph-Constrained Policy Learning for Extreme Clinical Code Prediction
Summary
This paper addresses the challenge of clinical code prediction, specifically mapping unstructured discharge summaries to ICD-10-CM codes, which is complicated by the large, sparse, and hierarchical nature of the label space. Traditional methods treat this as a flat multi-label classification problem, often neglecting rare labels due to insufficient training signals. The authors propose a novel approach using a graph-constrained traversal policy that reformulates the prediction task as a finite-horizon decision process over a pruned code hierarchy. This method allows a single language model to navigate the hierarchy level by level, selecting valid child nodes until reaching billable leaf codes. The results demonstrate that the proposed method, SFT-1+, significantly outperforms existing flat baselines, achieving a micro-F1 score of 0.709 on a curated subset and 0.527 on the full label space. The study also reveals that a shared traversal policy can match the performance of a more complex specialist cascade while avoiding context overflow issues. Additionally, increasing the amount of supervised data consistently improves performance, while reinforcement learning did not provide benefits over supervised methods. Overall, the findings suggest that graph-constrained policy learning is a promising approach for extreme clinical code prediction.
Methodology
The authors reformulate the ICD-10-CM code prediction task as a finite-horizon decision process using a graph-constrained traversal policy. A single decoder-only language model is employed to navigate the hierarchical structure of codes, selecting valid child nodes at each level until reaching billable leaf codes. This approach reduces the complexity of multi-label classification by focusing on relevant branches of the hierarchy.
Results
The proposed method, SFT-1+, achieved a micro-F1 score of 0.709 on a curated 50-code subset and 0.527 on the full 15,761-code label space, outperforming flat baselines such as CAML, LAAT, and PLM-ICD. The improvements were particularly notable in the full label space, where SFT-1+ exceeded the best flat baseline by +0.044 micro-F1 and +0.157 macro-F1. The study also found that increasing the amount of supervised data consistently improved performance, while GRPO reinforcement learning did not provide additional benefits.
Implications
The findings suggest that using a graph-constrained approach can effectively mitigate the challenges associated with rare clinical codes in automated coding systems. This methodology could enhance the accuracy of clinical coding, which is crucial for reimbursement, quality measurement, and epidemiological research. The insights regarding the importance of supervised data could inform future research and development in healthcare NLP applications.
Learning from Local Walks on Dynamic Graphs with Bandit Feedback
Graph Learning
Theory
Reinforcement Learning
- Introduces a novel framework for stochastic multi-armed bandits on dynamic graphs with local movement constraints.
- Establishes a structural condition (sliding-window mixing) that ensures stable exploration and navigation.
- Analyzes local explore-then-commit algorithms that achieve sublinear expected regret.
- Proposes a reward-aware strategy with formal guarantees on safety and performance.
Read more
Learning from Local Walks on Dynamic Graphs with Bandit Feedback
Summary
This paper investigates the stochastic multi-armed bandit problem in the context of dynamic graphs, where the learner can only move locally between nodes (arms) with time-varying edges. The authors highlight the challenge of identifying the best arm while being restricted to local movements, which can lead to situations where the optimal arm is unreachable even after identification. To address this, they introduce a structural condition based on sliding-window mixing that ensures the graph's intrinsic walk remains stable for exploration and navigation. The paper presents a family of local explore-then-commit algorithms that achieve sublinear expected regret under this condition. Additionally, the authors propose a reward-aware strategy and establish theorems regarding worst-case safety and performance gains. The findings emphasize the importance of maintaining navigability in dynamic environments for effective learning.
Methodology
The authors adopt an explore-then-commit framework, focusing on a canonical walk where the learner can either stay at its current node or move to an available neighbor. They analyze the implications of a sliding-window mixing condition on the graph's structure to ensure consistent long-term navigability and learning efficiency. The algorithms developed are evaluated based on their expected regret performance.
Results
The paper demonstrates that under the proposed sliding-window mixing condition, the local explore-then-commit algorithms can achieve sublinear expected regret. The reward-aware strategy is shown to provide formal safety guarantees while also yielding performance improvements, thus validating the effectiveness of the proposed methodologies.
Implications
The findings have significant implications for applications in dynamic networks, such as mobile sensor networks, peer-to-peer systems, and dynamic routing scenarios, where local movement constraints are prevalent. The results can inform the design of efficient learning algorithms that adapt to changing environments while ensuring optimal performance.
Reliability Scaling Laws for Quantized Large Language Models
NLP
Large Language Models
Efficient ML
- Introduces a reliability evaluation framework for quantized LLMs focusing on uncertainty, calibration, and robustness.
- Finds that reliability peaks at 4-bit quantization, suggesting an optimal trade-off between reliability and efficiency.
- Demonstrates that quantization improves robustness to semantically-preserving input perturbations.
- Conducts a comprehensive evaluation of six quantization techniques across models ranging from 1 billion to 70 billion parameters.
Read more
Reliability Scaling Laws for Quantized Large Language Models
Summary
This paper addresses the reliability of quantized large language models (LLMs), which are increasingly used for their efficiency in resource-limited settings. While quantization reduces the bitwidth of model parameters, leading to state-of-the-art performance on standard metrics, its impact on reliabilityβparticularly under perturbed inputsβhas been underexplored. The authors propose a comprehensive reliability evaluation framework that includes uncertainty assessment, calibration of uncertainty estimates, and robustness to semantically-preserving input perturbations. They evaluate quantized LLMs across various bit precisions (2, 3, 4, and 8 bits) and six quantization methods, revealing that while performance improves with increased bitwidth, reliability exhibits a non-linear scaling trend, peaking at 4-bit quantization. This indicates that moderate quantization offers the best reliability-efficiency trade-off. The study also finds that quantization enhances the models' robustness to natural input perturbations, which is crucial for real-world applications.
Methodology
The authors implemented a reliability evaluation framework consisting of three components: predictive uncertainty assessment, calibration of uncertainty estimates, and robustness testing against character-level and word-level input perturbations. They evaluated quantized models using six different quantization methods and analyzed the scaling trends of reliability with respect to the total number of model bits.
Results
The study revealed that while model performance scales monotonically with the total number of bits, reliability follows a non-monotonic trend, peaking at 4 bits. This suggests that quantized models at this precision provide the best trade-off between reliability and efficiency. Additionally, quantization was found to enhance the robustness of LLMs to natural input perturbations.
Implications
The findings have significant implications for deploying quantized LLMs in real-world applications, particularly in scenarios where input perturbations are common. Understanding the reliability of these models can lead to safer and more trustworthy AI systems, especially in resource-constrained environments.
ERP Data Provisioning Financial Control Testing
Optimization
- SEQ-FCT framework combines multiple data provisioning techniques to enhance financial control testing.
- Utilizes a synthetic dataset to evaluate the effectiveness of the proposed methods.
- Achieves high performance metrics in reconciliation and fraud detection while minimizing data leakage risk.
- Highlights the necessity of integrated governance in data provisioning for ERP systems.
Read more
ERP Data Provisioning Financial Control Testing
Summary
The paper presents a framework called Secure ERP Quality Provisioning for Financial Control Testing (SEQ-FCT) aimed at addressing the challenges of providing representative enterprise resource planning (ERP) data for financial control testing while ensuring data privacy. The framework integrates various techniques including deterministic masking, synthetic scenario expansion, referential tokenization, policy-based release approval, and automated validation to facilitate reconciliation, fraud-rule testing, and audit analytics. A synthetic dataset comprising 186,000 finance-process records from six subsidiaries is utilized for evaluation, demonstrating controlled internal consistency rather than production validation. The results indicate that SEQ-FCT outperforms traditional methods, achieving high scores in reconciliation F1, fraud-trigger recall, and control-failure F1, while maintaining a low leakage-risk score. The study emphasizes the importance of treating data masking, synthetic data generation, and governance checks as a cohesive release pipeline to enhance the reliability of financial process behavior preservation.
Methodology
The SEQ-FCT framework employs a hybrid approach that includes field-level classification, deterministic tokenization of entity identifiers, synthetic data generation for coverage gaps, policy-based release approval tied to validation outcomes, and comprehensive auditing of data refreshes. The methodology is evaluated using a synthetic dataset that simulates real-world financial processes.
Results
SEQ-FCT achieved a reconciliation F1 score of 0.932, a fraud-trigger recall of 0.887, and a control-failure F1 score of 0.914, with an estimated leakage-risk score of 0.018. These results indicate that the framework effectively preserves financial process behavior while ensuring data privacy.
Implications
The findings suggest that organizations can enhance their financial control testing processes by adopting the SEQ-FCT framework, which balances the need for realistic data with stringent privacy requirements. This approach can be particularly beneficial for cloud-hosted ERP systems and may influence future practices in data governance and financial automation.
Mathematics of Data Science
Theory
Optimization
Graph Learning
- Emphasizes the foundational role of mathematics in data science.
- Covers a wide range of topics including high-dimensional geometry, linear regression, and deep learning.
- Includes exercises to facilitate understanding and application of concepts.
- Aims to serve as a comprehensive resource for both researchers and practitioners in data science.
Read more
Mathematics of Data Science
Summary
The paper presents a comprehensive exploration of the mathematical foundations essential for understanding and advancing data science. It discusses the origins and evolution of data science, emphasizing the critical role of mathematics in its development. The authors provide a structured overview of various mathematical concepts and techniques that underpin data science, including high-dimensional geometry, linear regression, clustering, optimization, and deep learning. Each section is designed to build upon the previous one, offering exercises to reinforce learning. The work serves as both a textbook and a reference for researchers and practitioners, aiming to bridge the gap between theoretical mathematics and practical data science applications.
Methodology
The authors employ a structured approach to present mathematical concepts, integrating theoretical discussions with practical applications. They utilize exercises to engage readers and reinforce understanding, ensuring that the material is accessible and applicable to real-world data science problems.
Results
The paper successfully outlines the mathematical principles that are crucial for data science, providing a clear framework for understanding complex concepts. It highlights the importance of mathematical rigor in developing effective data science methodologies and offers insights into the interplay between theory and practice.
Implications
This work has significant implications for the education and training of data scientists, suggesting that a strong mathematical foundation is essential for effective practice in the field. It may also influence curriculum development in data science programs, advocating for a deeper integration of mathematical theory into practical training.
The Spectrum Is Not Enough: When Context Helps Time-Series Forecasting
Time Series
- Distinction between series-level predictability and configuration-level context value.
- Introduction of the 'coverage deficit' as a diagnostic tool for assessing context value.
- Demonstration that spectral indices do not capture the benefits of context due to phase randomization.
- Experimental validation showing that context can significantly alter forecasting outcomes.
Read more
The Spectrum Is Not Enough: When Context Helps Time-Series Forecasting
Summary
This paper addresses the limitations of current predictability measures in time-series forecasting, particularly those based on the power spectrum. The authors argue that while these measures can indicate how predictable a time series is, they fail to account for the value of context, such as longer lookback periods or retrieval mechanisms. The study introduces the concept of 'coverage deficit,' a diagnostic tool that distinguishes between series-level predictability and configuration-level context value. By using phase-randomized surrogate series, the authors demonstrate that traditional spectral indices are invariant under phase randomization, which obscures the true value of context. Through experiments on seven benchmarks, they show that the benefit of context can diverge significantly from predictability scores, leading to misleading conclusions about the effectiveness of additional context in forecasting. The findings emphasize the need for a more nuanced understanding of how context influences forecasting performance, independent of spectral predictability.
Methodology
The authors utilize phase-randomized surrogate series to analyze the limitations of spectral predictability indices. They introduce the coverage deficit diagnostic, which measures beyond-spectrum structure and assesses the value of context in forecasting. The methodology includes experiments on seven standard benchmarks to validate their theoretical claims.
Results
The results indicate that the value of context varies significantly across different series, with traditional spectral indices remaining unchanged while the benefits of context can collapse or change direction. Specifically, window-keyed retrieval's effectiveness was shown to drop from +33% to -35% across surrogate pairs, while spectral indices remained constant, highlighting the inadequacy of spectral measures in capturing the true value of context.
Implications
The findings suggest that practitioners should be cautious when interpreting predictability scores as indicators of the effectiveness of additional context in forecasting. The coverage deficit can serve as a valuable tool for making informed decisions about model deployment and configuration, ultimately leading to improved forecasting performance.
Machine Learning-based Correlation of Charpy Impact Properties Between Sub-sized and Standard-sized Specimens for Nuclear Structural Materials
- Introduces a machine learning framework for correlating Charpy impact properties between different specimen sizes.
- Achieves improved correlation performance with RΒ² values of 0.942 for USE and 0.892 for DBTT.
- Validates the framework using a comprehensive dataset of 389 matched tests on SA533B steel.
- The approach is applicable without needing full-sized Charpy data during inference.
Read more
Machine Learning-based Correlation of Charpy Impact Properties Between Sub-sized and Standard-sized Specimens for Nuclear Structural Materials
Summary
This paper presents a novel machine learning (ML)-based framework for correlating Charpy impact properties between sub-sized and standard-sized specimens, specifically for nuclear structural materials. The study addresses the challenges of limited material volume and spatial constraints that often necessitate the use of smaller specimens in testing. Traditional analytical methods for correlating impact properties, such as upper shelf energy (USE) and ductile-to-brittle transition temperature (DBTT), have shown limited accuracy and applicability. The proposed ML approach utilizes a temperature shift combined with scaled residual projection to align sub-sized test data with full-sized responses, effectively mapping absorbed energy values across the ductile-to-brittle transition region. The framework was validated using a dataset of 389 matched Charpy impact tests on SA533B steel, demonstrating superior correlation performance compared to conventional methods, achieving RΒ² values of 0.942 for USE and 0.892 for DBTT. Importantly, the trained ML models do not require access to full-sized Charpy data during inference, making this approach suitable for various applications, including material surveillance programs and accelerated irradiation testing.
Methodology
The methodology involves mapping absorbed energy values across the ductile-to-brittle transition region using a temperature shift and scaled residual projection. The correlated values for USE and DBTT are extracted by fitting the resulting temperature-energy profiles with a hyperbolic tangent model.
Results
The ML-based framework demonstrated significant improvements in correlation accuracy, with RΒ² values of 0.942 for upper shelf energy (USE) and 0.892 for ductile-to-brittle transition temperature (DBTT), outperforming conventional analytical methods.
Implications
The findings suggest that the ML framework can enhance material surveillance programs in nuclear applications, allowing for more reliable assessments of structural integrity using sub-sized specimens. It also opens avenues for accelerated irradiation testing and other applications where full-sized specimens are impractical.
Exploratory Analysis of Deep Learning Models for Forecasting Meteorological Parameters in the Agricultural Sector
Time Series
- Hybrid deep learning models (1D-CNN-GRU and 1D-CNN-LSTM) outperform traditional GRU and LSTM models in forecasting accuracy.
- The study utilizes a comprehensive dataset of 134,376 hourly meteorological observations from Ioannina, Greece.
- Convolutional feature extraction significantly enhances the performance of short-term forecasting tasks.
- The best-performing models achieved a WQS improvement of 1.22β1.63% for 24-hour forecasts and 0.44β0.45% for 168-hour forecasts compared to baseline models.
Read more
Exploratory Analysis of Deep Learning Models for Forecasting Meteorological Parameters in the Agricultural Sector
Summary
This paper presents a comparative evaluation of various deep learning architectures for forecasting key meteorological parameters essential for agricultural planning, including reference evapotranspiration (ET0), vapour pressure deficit (VPD), wind speed, and wind direction components. Utilizing a dataset of 134,376 hourly observations from Ioannina, Greece, spanning from January 2011 to April 2026, the authors explore single-layer and multi-layer GRU and LSTM networks alongside hybrid 1D-CNN-GRU and 1D-CNN-LSTM models. The study focuses on two forecasting tasks: a 24-hour next-day forecast and a 168-hour week-ahead forecast. Performance metrics include normalized root mean squared error, coefficient of determination, and a composite Weighted Quotient Score (WQS). The findings indicate that while hybrid models generally outperform purely recurrent models, the LSTM and GRU architectures also demonstrate significant effectiveness. The research highlights the importance of convolutional feature extraction for improving short-term forecasting accuracy, suggesting that hybrid models can provide better predictive capabilities with a trade-off in complexity.
Methodology
The authors conducted a controlled comparative evaluation of recurrent and hybrid neural network architectures, specifically GRU and LSTM networks, against hybrid CNN models. The analysis involved two forecasting scenarios (24-hour and 168-hour horizons) and utilized performance metrics such as normalized RMSE, R2, and a composite Weighted Quotient Score (WQS) to assess model effectiveness.
Results
The study found that the hybrid CNN-GRU models achieved the highest overall scores of 0.827535 for the 24-hour horizon and 0.782863 for the 168-hour horizon. The best purely recurrent models were a 64-unit LSTM for the 24-hour forecast (WQS of 0.816755) and a 1024-unit GRU for the 168-hour forecast (WQS of 0.779465). Hybrid models improved WQS by 1.22β1.63% at 24 hours and 0.44β0.45% at 168 hours compared to their recurrent counterparts.
Implications
The findings suggest that integrating deep learning models into agricultural decision-support systems can enhance the accuracy of meteorological forecasts, thereby improving irrigation management and crop planning. This research could inform the development of precision agriculture technologies that leverage real-time meteorological data.
Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels
Theory
Efficient ML
NLP
- Introduction of SFDA, enhancing KDA with phase-controlled dynamics.
- Establishment of a constructive chunk-WY theorem for efficient updates.
- Demonstration of compact realizations for various memory structures.
- Proof that every deterministic finite automaton can be realized by the proposed system.
Read more
Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels
Summary
The paper introduces Semidirect Fourier Delta Attention (SFDA), a novel attention mechanism that enhances the capabilities of Kimi Delta Attention (KDA) by incorporating phase-controlled dynamics into its recurrent memory structure. SFDA replaces KDA's purely real diagonal decay with a block-rotational Fourier control operator, allowing for improved long-context retrieval and algorithmic state tracking. The central technical contribution is a constructive chunk-WY theorem that enables explicit updates of factors in a left-to-right product, facilitating efficient computation while maintaining stability and cost bounds. The architecture is shown to compactly realize various memory structures, including cyclic counters and bounded stacks, through structured phase/control products. Additionally, the paper establishes that every deterministic finite automaton can be realized by a generalized tied-write affine memory system, highlighting the expressivity of the proposed framework. Empirical validation through numerical checks and toy experiments is provided, with future work aimed at optimizing performance in larger-scale applications.
Methodology
The methodology involves the development of a phase-controlled delta-rule layer that incorporates a block-rotational Fourier control operator into the recurrent update mechanism. The paper derives explicit recursions for the factors involved in the attention mechanism and characterizes the expressivity of the architecture through theoretical proofs and numerical experiments.
Results
The main results include the formalization of the SFDA architecture, the constructive chunk-WY theorem, and the proof of exact realizations for finite automata. The architecture demonstrates improved efficiency and expressivity compared to existing methods, with formal cost and stability bounds established.
Implications
The proposed SFDA framework has potential applications in sequence modeling tasks where long-context retrieval and precise memory tracking are critical. Its ability to compactly realize various memory structures may lead to advancements in efficient machine learning systems, particularly in natural language processing and other sequential data tasks.
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Large Language Models
Theory
Efficient ML
- Cross-model consensus outperforms traditional self-consistency and trained reward models in selecting correct answers.
- The LLM-jury mechanism relies on error decorrelation, allowing correct answers to accumulate agreement while wrong answers scatter.
- A closed-form predictive law quantifies consensus accuracy and establishes a ceiling for the method's effectiveness.
- The LLM-jury serves as a free selector that matches or exceeds the performance of trained verifiers across various benchmarks.
Read more
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Summary
This paper introduces a novel approach to selecting correct answers from candidate reasoning chains in large language models (LLMs) by leveraging cross-model consensus, termed as an LLM-jury. Traditional methods like self-consistency and trained reward models have inherent limitations, such as inheriting errors from single models or requiring labeled data. The proposed LLM-jury consists of independently trained models that each provide a solution without seeing each other's work. The agreement among these models serves as a verification signal, allowing for better selection of correct answers. The study demonstrates that this method significantly outperforms self-consistency and self-scoring models across seven benchmarks, particularly excelling in competition math tasks. The mechanism relies on error decorrelation, where independently trained models make different errors, allowing the correct answer to accumulate agreement while wrong answers scatter. A closed-form predictive law is established to quantify consensus accuracy based on panel statistics, achieving a mean absolute error of 0.03. The findings suggest that cross-model consensus is a robust, label-free verifier that generalizes well beyond the training distribution of traditional reward models.
Methodology
The study employs a comparative analysis of the LLM-jury against traditional methods like self-consistency and trained reward models. It utilizes independently trained models to generate candidate solutions and measures the degree of agreement among them to determine the final answer. A closed-form predictive law is derived to quantify the accuracy of consensus based on panel statistics.
Results
The LLM-jury significantly outperformed self-consistency and self-scoring models, achieving up to +20 points on AIME benchmarks. The predictive law demonstrated a mean absolute error of 0.03 in estimating consensus accuracy, with the method showing strong generalization across different benchmarks.
Implications
The findings suggest that cross-model consensus can serve as a reliable and efficient method for improving reasoning accuracy in LLMs, with potential applications in various domains requiring robust decision-making and verification processes.
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Multimodal
- PhysMRV provides a training-free framework for physical plausibility reasoning in VLMs.
- It organizes physical knowledge into a Hierarchical Memory Bank with three complementary levels.
- The framework improves physical reasoning without requiring model fine-tuning or parameter updates.
- Experimental results show consistent performance improvements across multiple benchmarks.
Read more
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Summary
The paper introduces PhysMRV, a novel framework designed to enhance physical plausibility reasoning in video-language models (VLMs). Despite advancements in VLMs for video understanding and visual question answering, they struggle with reasoning about physical interactions and causal dynamics. PhysMRV addresses this gap by creating a Hierarchical Memory Bank that organizes structured physical knowledge into three levels: scene descriptions, physical-event graphs, and physics-rule summaries. This framework operates without the need for training or parameter updates, allowing it to retrieve relevant physical memories during inference. The methodology involves a two-stage process: first, a physics-aware retrieval system identifies relevant memories based on semantic context and event-graph similarity; second, a verification stage uses these memories as evidence to assess the physical plausibility of observed events. The authors evaluate PhysMRV on three challenging benchmarksβImplausiBench, IntPhys2, and GRASP Level 2βdemonstrating significant improvements over existing methods that rely solely on direct prompting. The results indicate that structured physical memories effectively enhance the reasoning capabilities of VLMs, bridging the gap in physical commonsense reasoning.
Methodology
PhysMRV employs a two-stage approach consisting of physics-aware retrieval and evidence-centered verification. It first retrieves physically analogous memories using a coarse-to-fine strategy based on semantic context and event-graph similarity. Then, it verifies the physical plausibility of observed events by comparing them against the retrieved physical evidence.
Results
The evaluation of PhysMRV on the ImplausiBench, IntPhys2, and GRASP Level 2 benchmarks shows that it consistently outperforms direct prompting methods across various state-of-the-art VLMs, indicating that structured physical memories significantly enhance physical plausibility reasoning.
Implications
The development of PhysMRV suggests that integrating structured physical knowledge into VLMs can lead to more reliable physical reasoning capabilities, which is essential for real-world applications in robotics, autonomous systems, and interactive AI.
Adversarial Attacks on Online Handwriting using Salience-based Temporal Editing
Time Series
- Introduces a new adversarial attack framework for online handwriting recognition based on temporal editing.
- Demonstrates the inadequacy of spatial perturbations for time series data like handwriting.
- Achieves better transferability in black-box settings compared to traditional image-based attacks.
- Preserves the visual structure and kinematic smoothness of handwriting during adversarial attacks.
Read more
Adversarial Attacks on Online Handwriting using Salience-based Temporal Editing
Summary
This paper addresses the vulnerability of online handwriting recognition systems to adversarial attacks, which have predominantly been designed for image-based inputs. The authors propose a novel framework called Adversarial Iterative Temporal Editing (AITE), which generates adversarial examples by inserting and deleting points in the time series of pen trajectories based on temporal salience. This approach preserves the natural shape and smoothness of handwriting, contrasting with traditional methods that introduce high-frequency noise and artifacts. The salience is estimated using gradient-based activation mapping, guiding the edits towards time steps that significantly influence the original class prediction. The proposed method is evaluated on the Unipen and CASIA-OLHWDB datasets under both white-box and one-shot black-box attack settings. Results indicate that while conventional image-based attacks perform well in white-box scenarios, they lack transferability across models. In contrast, the temporal editing attack shows improved one-shot black-box transferability and maintains the visual integrity of the handwriting, highlighting its relevance as a threat model for online handwriting recognition.
Methodology
The authors developed the Adversarial Iterative Temporal Editing (AITE) method, which utilizes discrete editing operations (insertion and deletion of time steps) guided by the salience of those time steps, as determined by Gradient-weighted Class Activation Maps (Grad-CAM). This approach contrasts with traditional adversarial methods that apply additive noise in spatial dimensions.
Results
Experimental evaluations show that AITE achieves stronger visual similarity to original handwriting compared to conventional spatial perturbation attacks. Additionally, it demonstrates enhanced transferability in one-shot black-box attack scenarios, effectively attacking unseen models while maintaining the integrity of the handwriting.
Implications
The findings suggest that adversarial attacks on online handwriting recognition systems can be more effectively executed through temporal editing rather than spatial perturbations. This has significant implications for the security and reliability of handwriting recognition technologies in real-world applications, such as authentication and digital note-taking.
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Reinforcement Learning
Large Language Models
Optimization
- SCOPE-RL introduces a two-stage framework to enhance reasoning path optimization in RLVR.
- Adaptive Scaffolded RL (ASR) provides rewards for sub-question chains before achieving the final answer.
- Quality-Aware Process RL (QPR) refines reasoning quality after success using correctness-gated rewards.
- The Step-Quality Evaluation Protocol offers a comprehensive assessment of reasoning processes beyond final-answer accuracy.
Read more
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Summary
The paper introduces SCOPE-RL (Scaffolded Chain Optimization with Process Efficiency), a novel two-stage framework designed to enhance reinforcement learning with verifiable rewards (RLVR) for large language models (LLMs). Traditional RLVR methods rely on sparse final-answer rewards, which fail to provide feedback on the reasoning paths leading to success. This limitation results in two main issues: a lack of reward signals for prerequisite progress before achieving the final answer and an inability to distinguish between effective and flawed reasoning paths after success. SCOPE-RL addresses these challenges by incorporating Adaptive Scaffolded RL (ASR) and Quality-Aware Process RL (QPR). ASR provides prefix-decomposed verifiable rewards for sub-question chains before success, while QPR applies correctness-gated process-shape rewards to refine reasoning quality after success. The authors validate their approach using a Step-Quality Evaluation Protocol, which assesses useful-step density and error localization, among other metrics. Experimental results demonstrate significant improvements in accuracy and token efficiency across multiple benchmarks, indicating that the proposed densification of reward signals is effective and complementary to existing RLVR methods.
Methodology
The methodology involves a two-stage reinforcement learning framework: Adaptive Scaffolded RL (ASR) and Quality-Aware Process RL (QPR). ASR focuses on providing prefix-decomposed verifiable rewards for sub-questions before reaching the final answer, while QPR applies correctness-gated rewards to enhance the quality of reasoning paths after success. The effectiveness of the framework is evaluated using a Step-Quality Evaluation Protocol that measures various aspects of reasoning quality.
Results
SCOPE-RL achieved an average accuracy improvement of up to 11.2 percentage points and reduced reasoning tokens by up to 27.1% compared to outcome-only GRPO on the Qwen3-8B-Instruct model. The gains were consistent across different training corpora (DAPO-Math and Big-Math) and also held under a GSPO backend and on a smaller model (Qwen3-0.6B-Instruct).
Implications
The findings suggest that enhancing the reward signal density in reinforcement learning can lead to better reasoning quality in LLMs. This has implications for improving model performance in complex problem-solving tasks, particularly in areas requiring structured reasoning and verification.
Extractable Memorization From First Principles
NLP
Large Language Models
Theory
- Valid extraction claims require high probability generation of training sequences compared to non-training sequences.
- Matched comparisons are essential to distinguish memorization from predictability.
- The paper introduces two formal methods for conducting matched comparisons: a conformal test and a census.
- Experiments reveal that previous setups often overstate memorization claims due to validity issues.
Read more
Extractable Memorization From First Principles
Summary
This paper addresses the validity issues surrounding extractable memorization in language models (LLMs). The authors argue that previous studies either overstate extraction claims or imply that extraction is unreliable evidence of memorization. They emphasize that a valid extraction claim requires the model to generate a training sequence with a high enough probability, necessitating a matched comparison between training and non-training sequences. The authors formalize this concept through two methods: a conformal test for sampling from populations and a census for single documents. Their experiments demonstrate that matched comparisons provide rigorous evidence for memorization claims and reveal flaws in prior methodologies. For instance, they find that OLMo 2 32B reproduces non-training sequences at a rate that reflects false positives rather than memorization. The paper refines the definition of 'extractable memorization' to require both valid claims and near-certain generation within a realistic sampling budget.
Methodology
The authors formalize matched comparisons through two approaches: a conformal test that calibrates a threshold based on a chosen false-positive rate when sampling from populations, and a census that calibrates against a matched non-training document for specific cases. They conduct experiments comparing generation probabilities of training and non-training sequences to establish valid memorization claims.
Results
The experiments show that OLMo 2 32B reproduces non-training sequences at a rate of approximately 24% compared to training sequences, indicating that this rate reflects false positives rather than actual memorization. For Llama 3.1 70B, the calibrated thresholds for memorization claims are as low as 10^-27, suggesting that sequences can be memorized even when extraction is not feasible within a realistic budget.
Implications
The findings have significant implications for understanding the memorization capabilities of LLMs, particularly in the context of privacy and copyright concerns. By refining the criteria for extractable memorization, the paper contributes to more accurate assessments of LLM behavior and informs future research methodologies in this area.
From Preimage Search To Source-Grounded Feature Inversion
Interpretability
- Introduces source-grounded feature inversion for improved interpretability of neural networks.
- Conditions feature inversion on the local network geometry at the input that generated the feature.
- Utilizes closed-form matrix Wiener maps to correct adjoint signals during backpropagation.
- Demonstrates effectiveness across diverse architectures without requiring query-specific optimization.
Read more
From Preimage Search To Source-Grounded Feature Inversion
Summary
This paper introduces the concept of source-grounded feature inversion, which aims to enhance the interpretability of neural networks by providing a clearer understanding of what internal features extract from specific inputs. Traditional feature inversion methods often rely on iterative optimization to find an input that matches a target feature, but this approach can yield multiple valid inputs, making it difficult to ascertain the true source of the feature. The proposed method conditions the inversion on the local geometry of the network at the input that generated the feature, allowing for a more accurate representation of the feature in the input domain. The author employs a closed-form matrix Wiener map to repair the adjoint signal during backpropagation, ensuring that the inversion reflects the specific input being analyzed rather than a generic template. The methodology is validated across various neural network architectures, demonstrating its ability to produce calibrated feature inversions that are sensitive to the selected features and local operators. This approach not only facilitates a deeper inspection of the model's internal feature hierarchy but also aligns visualizations with independent interventions on the corresponding features, thereby linking the network's extraction processes to its decision-making evidence.
Methodology
The methodology involves formulating source-grounded feature inversion by conditioning the inverse on the local network geometry at the input that generated the feature. It employs closed-form matrix Wiener maps to repair signals during backpropagation, allowing for accurate estimation of upstream states along the computational DAG. This approach transforms the inversion problem into a series of local upstream-state estimation tasks.
Results
The results indicate that the proposed method successfully generates feature inversions that are architecture-specific and sensitive to the selected features and local operators. The outputs do not converge to a generic image template, and the method supports various inputs, depths, channels, and channel groups without the need for specific optimization.
Implications
The implications of this research extend to enhancing model interpretability in various applications of neural networks, allowing for better understanding and visualization of internal features. This could lead to improved trust and transparency in AI systems, particularly in critical domains such as healthcare and autonomous systems.
Sample Efficient Generative Optimization for Molecular Design
Optimization
Generative Models
Efficient ML
- Introduction of SEGO framework for molecular optimization.
- Combines Bayesian optimization and generative modeling for efficient search.
- Achieves state-of-the-art performance with significantly reduced oracle evaluations.
- Demonstrates effectiveness in both practical molecular optimization and multiparameter docking tasks.
Read more
Sample Efficient Generative Optimization for Molecular Design
Summary
The paper presents Sample Efficient Generative Optimization (SEGO), a novel framework designed to enhance sample efficiency in molecular optimization tasks such as drug discovery and materials design. SEGO integrates Bayesian optimization with generative modeling to navigate vast chemical spaces while minimizing the number of costly evaluations needed to identify promising molecular candidates. The framework employs a probabilistic surrogate model to hypothesize the locations of potential hits in chemical space, guiding a generative model to propose candidates in those regions. An acquisition function selects the most promising candidate for evaluation, and the feedback from this evaluation refines both the surrogate model and the generative model. SEGO demonstrates significant improvements in sample efficiency, achieving state-of-the-art performance on the practical molecular optimization benchmark, requiring only one-tenth of the oracle calls compared to existing methods. Additionally, in a multiparameter docking task, SEGO identifies successful candidates in approximately half the oracle calls of other state-of-the-art approaches. These advancements suggest that SEGO could facilitate more effective molecular optimization campaigns driven by direct experimental feedback.
Methodology
SEGO utilizes a tandem approach that combines a probabilistic surrogate model and a generative model. The surrogate model hypothesizes potential hit locations in chemical space, while the generative model is guided to produce candidate molecules in those regions. An acquisition function selects the most promising candidate for evaluation, and the results are used to refine the surrogate model and anchor the generative model in real rewards. The framework incorporates SMILES augmentation and deep-kernel learning for effective feature extraction.
Results
SEGO outperforms both pure Bayesian optimization and reinforcement learning methods, achieving state-of-the-art results on the practical molecular optimization benchmark with only one-tenth of the oracle calls required by other methods. In multiparameter docking tasks, it identifies successful candidates in about half the oracle calls compared to existing approaches.
Implications
The SEGO framework could significantly improve the efficiency of molecular optimization processes, enabling researchers to conduct more effective experimental campaigns with limited resources. This could lead to faster discoveries in drug development and materials science, where high-fidelity evaluations are often costly and time-consuming.
Condition-Stratified Robustness Analysis of Post-Hoc Calibration Methods for Probabilistic Classifiers
Theory
- Post-hoc calibration methods need to be evaluated across distinct conditions, not just aggregate performance.
- Temperature scaling (TEMP) consistently outperformed isotonic regression (ISO) in terms of calibration slope and stability across conditions.
- The analysis revealed that the relative advantages of TEMP and ISO are condition-dependent and metric-specific.
- Holm-adjusted multiplicity control was applied to ensure robust statistical comparisons.
Read more
Condition-Stratified Robustness Analysis of Post-Hoc Calibration Methods for Probabilistic Classifiers
Summary
This paper investigates the robustness of post-hoc calibration methods, specifically temperature scaling (TEMP) and isotonic regression (ISO), for probabilistic classifiers across different operating conditions. The study emphasizes the importance of evaluating calibration performance not just in aggregate but across distinct conditions within a dataset. A pre-registered condition-stratified robustness analysis was conducted, comparing the two methods under four controlled conditions (C1βC4). The analysis included four hypothesis groups focusing on discrimination deltas, Brier score differences, calibration slopes, and AUROC differences. Results indicated that TEMP and ISO exhibited condition-dependent behavior, with TEMP generally maintaining better calibration slopes and more consistent performance across conditions. The study highlights the necessity of understanding how calibration methods perform under varying conditions, as aggregate metrics can obscure significant differences in performance across subgroups.
Methodology
The methodology involved a pre-registered, condition-stratified analysis comparing TEMP and ISO across four controlled conditions. The evaluation was structured around four hypothesis groups assessing discrimination deltas, Brier scores, calibration slopes, and AUROC differences. The study utilized a systematic pipeline to partition raw classifier outputs into condition strata and apply recalibration methods independently, followed by statistical analysis with Holm-adjusted p-values.
Results
The results showed that TEMP-ISO discrimination deltas were small across all conditions, with Holm-adjusted p-values indicating no significant difference. TEMP consistently produced negative Brier score differences, while ISO showed variability. Calibration slopes for TEMP were closer to unity across conditions compared to ISO. AUROC differences varied, indicating a shift from near zero in C1 to positive in C4, underscoring the condition-dependent nature of the calibration methods.
Implications
The findings suggest that practitioners should consider condition-specific performance when applying calibration methods to probabilistic classifiers, as relying solely on aggregate metrics may lead to misinterpretations of model reliability. This research could inform future studies on calibration methods and their application in real-world scenarios where operating conditions vary.
Signal-Guided Optimization for Machine Unlearning
Optimization
Efficient ML
Theory
- GSUO introduces task-specific guidance signals for improved machine unlearning.
- The framework addresses issues of over-unlearning and under-unlearning by tailoring optimization based on sample characteristics.
- GSUO outperforms 14 existing methods in terms of unlearning effectiveness and generalization.
- The method achieves significant speedups, with up to 31Γ faster performance.
Read more
Signal-Guided Optimization for Machine Unlearning
Summary
This paper addresses the limitations of current machine unlearning methods, which typically employ global, coarse-grained strategies that lack precise guidance for the unlearning process. The authors introduce GSUO (Guidance-Signal-Aware Unlearning Optimization), a framework that utilizes task-specific fine-grained guidance signals to enhance the unlearning process for both random-subset and class-wise forgetting tasks. By recognizing the varying memorization strengths of samples, GSUO aims to mitigate the issues of over-unlearning and under-unlearning, which can degrade model performance or leave residual data vulnerable to privacy breaches. The framework employs diverse target distributions and intra-class dispersion loss to tailor the optimization process. Experimental results demonstrate that GSUO significantly outperforms 14 state-of-the-art baselines in terms of unlearning effectiveness, generalization, and efficiency, achieving substantial speedups and maintaining high accuracy.
Methodology
The GSUO framework employs guidance signals that are specifically designed for different forgetting tasks. For random-subset forgetting, it uses diverse target distributions to align forget samples based on their categories. For class-wise forgetting, it incorporates intra-class dispersion loss and alignment loss to maintain class boundaries. This approach allows for a more precise and efficient unlearning process compared to existing methods.
Results
GSUO consistently achieved the best performance across various metrics, including fidelity, generalization ability, and forgetting efficacy. In random-subset forgetting tasks, it reached a minimal accuracy gap of 0.17% between test and forget sets, with a test accuracy of 82.08%. It also demonstrated a significant speedup of up to 31Γ and the lowest AUC score of 0.749 under strong membership inference attacks. For class-wise forgetting, GSUO maintained high training and test accuracies while achieving complete unlearning.
Implications
The GSUO framework has significant implications for privacy-preserving machine learning, particularly in contexts where data removal is necessary due to legal requirements or ethical considerations. Its ability to efficiently and effectively unlearn specific data points while maintaining model performance makes it a valuable tool for organizations handling sensitive information.
Optimizing ARDL Models for Retail Sales Forecasting and Fair Pricing
Time Series
Optimization
- Integrates fairness constraints into retail sales forecasting models.
- Uses ARDL models to analyze the impact of pricing on sales elasticity.
- Demonstrates that Simulated Annealing can achieve fairer pricing compared to unconstrained optimization.
- Establishes a transparent framework for pricing that aligns with consumer welfare.
Read more
Optimizing ARDL Models for Retail Sales Forecasting and Fair Pricing
Summary
This paper addresses the challenge of balancing profitability with consumer welfare in retail food pricing, particularly under dynamic pricing strategies that often overlook fairness. The author proposes a novel methodology that integrates fairness constraints into retail sales forecasting by modeling total retail trade sales using a logβlog Autoregressive Distributed Lag (ARDL) specification. The pricing problem is framed as maximizing forecast sales while adhering to price bounds linked to the Consumer Price Index (CPI). The study employs Linear Programming (LP) and Simulated Annealing (SA) to solve this optimization problem in both single-product and multi-product contexts. Key findings reveal that nominal elasticities are positive, indicating that unconstrained sales maximization would lead to price increases to their upper limits. However, the SA approach results in more conservative pricing that reduces consumer costs while still achieving sales targets. The paper benchmarks the forecasting accuracy of the proposed model against several baselines, including naive and seasonal-naive models, ARIMA, and SARIMA, and finds that CPI-adjusted pricing enhances fairness and transparency in retail pricing strategies.
Methodology
The paper employs a logβlog Autoregressive Distributed Lag (ARDL) model for forecasting retail sales, incorporating fairness constraints related to the Consumer Price Index (CPI). It utilizes Linear Programming (LP) and Simulated Annealing (SA) to optimize pricing strategies under both single-product and multi-product scenarios.
Results
The study finds that nominal elasticities are positive, suggesting that unconstrained optimization would lead to excessive pricing. However, the use of Simulated Annealing results in more balanced pricing strategies that lower consumer costs while still meeting sales targets. The proposed model outperforms naive and traditional forecasting methods, demonstrating improved accuracy and fairness.
Implications
The findings suggest that retailers can adopt the proposed pricing framework to enhance consumer trust and equity in pricing strategies. By aligning prices with CPI trends, businesses can ensure fairness while optimizing sales, potentially leading to better customer relationships and increased loyalty.
AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation
Generative Models
Time Series
Theory
- Introduction of AdaPCLA framework for generating longitudinal EHR data.
- Focus on improving the representation of rare clinical events in synthetic data.
- Theoretical analysis of logit updates and prior-internalization dynamics.
- Significant performance improvements over existing generative models.
Read more
AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation
Summary
The paper presents AdaPCLA, a novel framework designed to enhance the generative modeling of longitudinal Electronic Health Records (EHR), particularly focusing on the challenges posed by long-tailed distributions of clinical events. Traditional autoregressive models often fail to accurately represent rare events, leading to synthetic data that lacks fidelity for underrepresented subpopulations. AdaPCLA addresses this issue through a data distribution-aware training strategy that incorporates simulated annealing to internalize data knowledge parameters, allowing for effective learning of rare codes. The framework also supports zero-shot adaptation to diverse clinical populations without the need for retraining. The authors provide a theoretical analysis of the model's performance, particularly in relation to rare-code logit updates and the effects of annealing speed on prior signal retention. Experimental results demonstrate that AdaPCLA significantly improves tail plausibility and downstream utility, outperforming existing models such as HALO and GPT-style generation in various metrics, including a notable increase in TailPairSeen scores on MIMIC-III and MIMIC-IV datasets. This work highlights the importance of preserving rare-event dependencies in synthetic EHR generation for better clinical applicability.
Methodology
AdaPCLA employs a prior-aware logit adjustment mechanism during training to enhance the learning of rare codes. It utilizes simulated annealing to gradually internalize these adjustments, allowing for effective inference post-training. The framework also incorporates zero-shot distribution control, enabling adaptation to different clinical populations without retraining.
Results
AdaPCLA achieved a 114.2% improvement in TailPairSeen over HALO on the MIMIC-III dataset and a 65.1% improvement on MIMIC-IV. In zero-shot generation tasks, it outperformed GPT-style generation by 3.5% F1 score, demonstrating its effectiveness in cross-population adaptation.
Implications
The findings suggest that AdaPCLA can significantly enhance the quality of synthetic EHR data, making it more suitable for privacy-preserving research and clinical applications. By improving the representation of rare events, the framework can facilitate better risk prediction, treatment recommendations, and decision support systems in healthcare.
LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation
Large Language Models
Optimization
Interpretability
- LLM-PDESR integrates LLMs with continuous numerical optimization for automated PDE discovery.
- The framework utilizes C4 continuous quintic splines and SWR evaluations to reduce noise impact on derivative calculations.
- A rigorous benchmark of 23 canonical PDEs and five novel equations validates the framework's discovery capabilities.
- LLM-PDESR successfully extracts interpretable models from noisy climate data, demonstrating real-world applicability.
Read more
LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation
Summary
The paper introduces LLM-PDESR, a novel framework designed to discover governing partial differential equations (PDEs) from noisy observational data. Traditional symbolic regression methods struggle with the vast search spaces and high-frequency noise, which distort the optimization landscape. LLM-PDESR addresses these challenges by integrating Large Language Models (LLMs) for hypothesis generation with a rigorous evaluation environment. It employs C4 continuous quintic splines for robust differentiation and subdomain weighted residuals (SWR) to act as low-pass filters, effectively mitigating noise effects. The framework features a Pareto-driven feedback loop that balances predictive accuracy and structural simplicity. Evaluated on 23 canonical PDEs and five novel equations, LLM-PDESR demonstrates superior performance in structural recovery and noise resilience, successfully extracting a 1D dynamical surrogate from noisy climate data. The results indicate that LLM-PDESR captures invariant dynamical mechanisms rather than mere statistical artifacts, showcasing its potential for real-world applications in scientific machine learning.
Methodology
LLM-PDESR combines structural hypothesis generation from LLMs with continuous quintic spline smoothing for noise-robust differentiation and subdomain weighted residuals for evaluation. This approach mitigates fitness landscape distortion and employs a Pareto-driven feedback loop for optimizing equation complexity and predictive accuracy.
Results
The framework was evaluated on a comprehensive benchmark, outperforming state-of-the-art methods in structural recovery and noise resilience. It successfully extracted a consistent structural skeleton for a 1D dynamical surrogate from noisy climate data, confirming its capability to capture invariant dynamical mechanisms.
Implications
LLM-PDESR has significant implications for scientific machine learning, particularly in automating the discovery of governing equations from empirical data. Its ability to handle noise and complex dynamics could enhance modeling in various scientific fields, including climate science and engineering.
Mechanical Analysis of Parachute Suspension Line Deployment with Binding Tapes Using PINN
Theory
Optimization
- Development of a PINN algorithm for predicting tension in parachute suspension lines.
- Outperformance of the PINN method compared to traditional numerical integration techniques.
- Investigation of the effect of binding tape parameters on dynamic tension.
- Validation of the model against real flight test data, demonstrating its effectiveness.
Read more
Mechanical Analysis of Parachute Suspension Line Deployment with Binding Tapes Using PINN
Summary
This paper presents a novel approach to analyze the mechanical dynamics of parachute suspension line deployment using a Physics-Informed Neural Network (PINN). The initial phase of parachute deployment, which involves the extraction and straightening of suspension lines, is critical for ensuring the successful inflation of the parachute canopy. Traditional methods, primarily based on numerical integration of ordinary differential equations, struggle with rapid tension calculations along the suspension lines. The authors propose a PINN framework that significantly enhances computational efficiency and accuracy in predicting line tension during deployment. The study also investigates the influence of binding tape parameters on dynamic tension, providing insights into the mechanical behavior of suspension lines. Validation against flight test data confirms the reliability of the proposed method, marking a significant advancement in parachute design and safety analysis.
Methodology
The authors constructed a force model for suspension line particles, representing them as a series of interconnected spring-damper units. A continuous extraction model was developed, and a new tension calculation model was established to account for the effects of binding tapes. The PINN framework was employed to learn solutions to the governing partial differential equations, allowing for interpolation and prediction of tension at arbitrary points in space and time.
Results
The PINN framework demonstrated superior performance in computational efficiency and numerical accuracy compared to traditional methods. The model successfully captured the dynamic tension variations during the extraction and straightening of suspension lines, and the results were validated against flight test data, confirming the model's reliability.
Implications
The findings of this research have significant implications for the design and optimization of parachute systems, particularly in enhancing safety and reliability. The innovative use of PINN could be applied to other engineering problems involving complex dynamic systems.
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
NLP
Large Language Models
- HyperSafe provides a model-specific, non-invasive approach to safety recovery in fine-tuned LLMs.
- The framework generates a Safe Side Network (SSN) using layer-wise activation fingerprints, ensuring tailored safety assessments.
- HyperSafe achieves harmful response rate reductions from 19-31% to below 1% without degrading task performance.
- The method requires no gradient updates or safety data at deployment time, making it efficient and practical.
Read more
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
Summary
The paper presents HyperSafe, a novel framework designed to restore safety in fine-tuned language models (LLMs) without requiring retraining or modification of model weights. Fine-tuning can degrade the safety alignment of LLMs, leading to increased harmful responses even with benign data. Existing methods either intervene during fine-tuning or modify model weights post-hoc, which can be costly and may negatively impact task performance. HyperSafe addresses these issues by introducing a Safe Side Network (SSN) that operates alongside the frozen fine-tuned model. This SSN is generated using layer-wise activation fingerprints from calibration prompts, allowing it to tailor its safety assessments to the specific fine-tuned model. During inference, the SSN classifies prompts as harmful or safe, routing harmful prompts to a refusal response while safe prompts are processed by the original model. The framework is evaluated on two model families, Qwen2-7B and LLaMA-3-8B, demonstrating significant reductions in harmful response rates while maintaining downstream task accuracy.
Methodology
HyperSafe employs a hypernetwork to generate a Safe Side Network (SSN) for each fine-tuned checkpoint. It captures layer-wise activation fingerprints from calibration prompts to map these to SSN parameters in a single forward pass. The SSN operates alongside the frozen fine-tuned model, classifying prompts at inference time without modifying the original model weights.
Results
HyperSafe significantly reduces harmful response rates from 19-31% to below 1% across all evaluated checkpoints while maintaining downstream task accuracy within 1% of the fine-tuned baseline on average.
Implications
The development of HyperSafe has important implications for the deployment of fine-tuned language models in sensitive applications, ensuring safety without sacrificing performance. This framework can be applied to various LLMs to enhance their safety alignment post-fine-tuning, making it a valuable tool for developers and researchers.
When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary
Reinforcement Learning
Theory
Optimization
- Introduces a white-box instrument for measuring latent state learning in RL agents.
- Establishes that reward success and latent-state learning are distinct and measurable.
- Identifies three axes influencing the coupling of reward and state learning: optimizer strength, task structure, and observation informativeness.
- Demonstrates the existence of perception and planning gaps in agent learning.
Read more
When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary
Summary
This paper addresses the question of whether a reinforcement-learning (RL) agent that achieves high rewards truly understands the latent state of its task or merely learns a reward-correlated shortcut. The author introduces a novel methodology using a hidden deterministic finite automaton (DFA) as a measurement instrument, allowing for the exact determination of the agent's latent state at every step and the optimal achievable return. This approach separates the concepts of reward success and latent-state learning, which are influenced by three controllable factors: optimizer strength, task structure, and observation informativeness. The findings reveal that weak RL often leads to decoupled reward and state learning, while stronger optimizers can couple them, except in cases of group-language automata. The paper also identifies a perception gap, where the latent state is not linearly recoverable, and a planning gap, where the state is recoverable but not utilized. The methodology is validated through a benchmark instance and a series of experiments, demonstrating its effectiveness in measuring the agent's understanding of the task state versus merely optimizing for reward.
Methodology
The methodology involves expressing the task as a hidden DFA, allowing the agent to observe a symbol stream and choose the next symbol under partial control. The agent receives a sparse terminal reward for acceptance, enabling the measurement of both reward success and latent-state learning. The study employs a series of experiments to analyze the effects of optimizer strength, task structure, and observation informativeness on the agent's learning outcomes.
Results
The results indicate that weak on-policy RL leads to a decoupling of reward and state learning, while stronger optimizers like PPO+GAE can recover the latent state, albeit partially and with high variance. The analysis of task structure reveals that group-language automata serve as a warning signal for perception gaps, with a precision of 0.86 in identifying such gaps across 153 automata. The study also distinguishes between perception and planning gaps, providing insights into the limitations of reward-based evaluations.
Implications
The findings have significant implications for the design and evaluation of RL agents, suggesting that high rewards do not necessarily indicate a true understanding of the task. This work encourages the development of more robust measurement methodologies to assess agent learning and could inform future research in RL, particularly in tasks with complex underlying structures.
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
NLP
Large Language Models
Interpretability
- Bias in LLMs as judges can be understood through their internal representation rather than just input-output interactions.
- The geometry of activation manifolds reveals how biases manifest and can be manipulated.
- Causal control over hidden states allows for effective steering of scoring outcomes.
- A linear projection onto bias features can predict judge performance on unseen data.
Read more
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
Summary
This paper investigates the biases present in large language models (LLMs) when used as judges, focusing on a representation-level understanding of these biases rather than the traditional input-output perspective. The authors present three main findings across seven judges, seven bias types, and nine benchmarks. First, they demonstrate that baseline judging inputs occupy a tight activation manifold, while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth. Second, they establish that steering hidden states along this subspace can control scoring outcomes, effectively reproducing biased scores on clean inputs and restoring baseline scores on biased ones. Third, they show that a simple linear projection onto these bias-direction features can predict judge failures on unseen benchmarks, outperforming text-based alternatives. By framing bias as activation geometry, the study unifies geometric structure, causal control, and operational prediction, offering a comprehensive account of LLM-as-judge bias.
Methodology
The authors conducted experiments using tightly controlled datasets that varied only in semantics-irrelevant surface framing across multiple bias types. They employed geometric analysis to characterize activation manifolds and used causal manipulation techniques to steer hidden states, assessing the impact on scoring outcomes. They also developed a linear projection method to predict judge failures on unseen benchmarks.
Results
The study found that biased inputs are consistently displaced along a low-dimensional subspace in the model's hidden states. Steering along this subspace allowed for both reproduction of biased scoring and restoration of baseline scoring. The linear projection method achieved an AUC of 0.82 in predicting judge failures on unseen benchmarks, significantly outperforming traditional text-based methods.
Implications
This research has significant implications for improving the fairness and reliability of LLMs used in evaluative roles. By understanding and manipulating the internal representations of these models, developers can create more robust systems that mitigate bias, enhancing the validity of automated assessments in various applications.
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
Interpretability
- Demonstrates the conflation of decoder-geometry alignment and encoder-activation behavior in SAE evaluations.
- Introduces a causal validation method revealing that many features deemed recovered are causally inert.
- Develops the sae-causal-audit tool for structured feature auditing across various models.
- Proposes a two-axis taxonomy of causal inertness, distinguishing between structural and competitive inertness.
Read more
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
Summary
This paper investigates the interpretability of Sparse Autoencoders (SAEs) by addressing the limitations of current evaluation practices that rely heavily on correlational recovery metrics. The author demonstrates that these metrics conflate two distinct empirical claims: decoder-geometry alignment and encoder-activation behavior. Through a controlled experimental setup, the paper reproduces the superposition phase diagram and the TopK-versus-L1 SAE comparison, revealing artifacts and new geometric regimes. The central contribution is a causal validation battery that shows a significant portion of features deemed recovered are causally inert, meaning they do not influence model outputs despite high correlation metrics. The author introduces the sae-causal-audit tool, which allows for a structured audit of any model's features, and proposes a taxonomy of causal inertness. Additionally, the paper discusses the challenges of reproducibility in machine learning experiments and suggests a framework for reporting reproducibility claims. The findings highlight the need for more rigorous validation methods in interpretability research.
Methodology
The study employs a controlled experimental setup to reproduce existing results in the field, including the superposition phase diagram and TopK-versus-L1 comparisons. A causal validation battery is applied to assess the causal efficacy of features, utilizing ablation and steering interventions. The methodology is packaged into the sae-causal-audit tool, which is model-agnostic and allows for reproducible audits of feature behavior.
Results
The findings indicate that up to 77% of features in a degraded SAE and 9% in a well-trained SAE are causally inert, despite high cosine similarity scores. The reproduction of the superposition phase diagram and the TopK-versus-L1 comparison provided new insights into feature behavior and the presence of convergence artifacts. The sae-causal-audit tool successfully identified causal relationships and provided a structured approach to feature validation.
Implications
The results underscore the importance of causal validation in the interpretability of machine learning models, suggesting that many features considered important may not contribute to model performance. This has implications for the design of interpretable models and the evaluation of their features, potentially leading to more robust and reliable AI systems.
SMETA-ZSL: Semantic Meta-Alignment for Zero-Shot Threat Classification
NLP
Large Language Models
Multimodal
- SMETA-ZSL utilizes semantic knowledge from CTI reports for zero-shot threat classification.
- The framework addresses challenges like semantic ambiguity and class imbalance in cybersecurity.
- It combines contrastive fine-tuning, episodic meta-learning, and adaptive routing for improved performance.
- SMETA-ZSL shows significant performance improvements over existing zero-shot learning methods.
Read more
SMETA-ZSL: Semantic Meta-Alignment for Zero-Shot Threat Classification
Summary
The paper presents SMETA-ZSL, a novel framework designed to enhance zero-shot threat classification in cybersecurity by leveraging Cyber Threat Intelligence (CTI) reports. Traditional machine learning models struggle to adapt to new threats due to the lack of labeled data, especially when these threats emerge. SMETA-ZSL addresses this challenge by employing generalized zero-shot learning (GZSL) techniques that utilize semantic knowledge from CTI reports to recognize unseen classes. The framework incorporates a contrastively fine-tuned large language model (LLM) to generate semantic prototypes from overlapping threat descriptions, aligns behavioral features through episodic meta-learning, and implements a parameter-free gating mechanism for adaptive inference. The authors highlight the unique challenges in cybersecurity, such as semantic ambiguity, cross-modal heterogeneity, class imbalance, and open-set classification, which SMETA-ZSL effectively navigates. The results demonstrate that SMETA-ZSL outperforms existing methods across seven benchmark datasets, achieving an average improvement of 10.8 points in generalized zero-shot performance, with gains reaching up to 18.1 points in some cases.
Methodology
SMETA-ZSL employs a three-pronged approach: (1) a contrastively fine-tuned LLM encoder to create discriminative semantic prototypes from CTI descriptions; (2) a cross-modal alignment framework using episodic meta-learning to simulate unseen-class emergence during training; and (3) a parameter-free gating mechanism for adaptive inference across seen and unseen classes.
Results
The framework was evaluated across seven benchmark datasets, achieving the strongest overall generalized zero-shot performance under strict inductive settings, surpassing prior methods by an average of 10.8 points, with some benchmarks showing improvements of up to 18.1 points.
Implications
The findings suggest that SMETA-ZSL can significantly enhance the adaptability of cybersecurity systems to emerging threats, enabling more timely and effective responses without the need for extensive labeled data. This has potential applications in real-time threat detection and response systems.
What Does Goodness Measure? A Likelihood-Ratio Account of Forward-Forward Learning
Theory
- The goodness measure is shown to be a sufficient statistic for a likelihood-ratio test, providing a theoretical foundation for FF learning.
- Generalizations to anisotropic and heavy-tailed distributions yield new insights into the behavior of the goodness measure.
- Proper normalization between layers is critical for maintaining effective learning dynamics in FF networks.
- Empirical results validate the theoretical predictions, although no significant performance improvements were observed.
Read more
What Does Goodness Measure? A Likelihood-Ratio Account of Forward-Forward Learning
Summary
This paper presents a theoretical framework for understanding the Forward-Forward (FF) learning algorithm, which trains neural network layers locally by optimizing a scalar goodness measure based on the sum of squared activations. The author argues that this goodness measure can be interpreted as a sufficient statistic in a likelihood-ratio test between two zero-mean populations differing in scale. The paper derives the optimal threshold for this goodness measure and explores generalizations to anisotropic and heavy-tailed populations. It also provides insights into the normalization process between layers, demonstrating that proper normalization is crucial for effective learning. The empirical study conducted on convolutional FF networks shows that the theoretical predictions align with observed behaviors in trained networks, although no significant improvements in representation quality were found compared to existing methods. Overall, the paper transforms the understanding of the FF learning framework from heuristic to a measurable and testable theory.
Methodology
The author employs a generative modeling approach to derive the goodness measure as a sufficient statistic for decision-making in FF learning. The analysis includes theoretical derivations for isotropic and anisotropic populations, as well as empirical validation through experiments on convolutional FF networks.
Results
The study finds that the goodness measure aligns with the theoretical predictions regarding the behavior of activations in trained networks. The empirical results indicate that the structured readout improves performance as predicted, while the normalization process is shown to be essential for effective learning. However, no substantial gains in representation quality were achieved over existing methods.
Implications
This work provides a deeper understanding of the FF learning framework, potentially guiding future research in local learning algorithms and their applications in neural network training. The insights into normalization and goodness measures may influence the design of more effective learning strategies in various machine learning contexts.
EvoClawBench: Can Agents Learn Reusable Skills from Their Own Runs?
Large Language Models
Reinforcement Learning
Robotics
- EvoClawBench is a novel benchmark focusing on agents' ability to learn reusable skills from their own runs.
- The evaluation includes three distinct strategies: BASELINE, PRESKILL, and POSTSKILL, to measure skill effectiveness.
- Results indicate that the effectiveness of self-authored skills is highly dependent on the runtime environment.
- The study highlights the non-monotonic nature of skill performance improvements, challenging assumptions about automatic benefits from skill authoring.
Read more
EvoClawBench: Can Agents Learn Reusable Skills from Their Own Runs?
Summary
The paper introduces EvoClawBench, a benchmark designed to evaluate whether agents can learn reusable skills from their own execution runs. Unlike existing benchmarks that primarily focus on task completion or tool use, EvoClawBench isolates the process of skill creation and reuse within the same agent runtime. The benchmark consists of 100 tasks with 502 sub-problems across various domains, allowing agents to recognize shared patterns and convert first-run evidence into reusable skills. The authors compare three strategies: BASELINE (direct execution), PRESKILL (skill authoring before execution), and POSTSKILL (skill summarization after the first run). Experiments reveal that performance varies significantly across different agent runtimes, with self-authored skills showing mixed results. The findings suggest that learning reusable skills is selective and cost-sensitive, challenging the assumption that adding skill authoring will always enhance performance. The paper contributes a new evaluation framework, empirical findings on runtime dependencies, and an open-source release of the benchmark materials.
Methodology
The authors developed EvoClawBench to evaluate agents on repeated tasks with multiple sub-problems. They implemented three evaluation conditions: BASELINE (no skill creation), PRESKILL (skills created before task execution), and POSTSKILL (skills created after the first task execution). The performance of agents was measured across different runtime environments, and the results were analyzed to understand the impact of self-authored skills.
Results
The experiments demonstrated that performance varied significantly between agent runtimes, with OpenClaw achieving below 20% performance across models, while nanobot ranged from 56.45% to 96.13%. Self-authored skills had mixed effects, with some models showing improvements while others experienced drastic performance drops. For instance, nanobot GPT-5.4 maintained high performance across all modes, while DeepSeek-V4-Pro's performance plummeted under certain conditions.
Implications
The findings suggest that while agents can potentially learn reusable skills, the process is complex and influenced by various factors, including the runtime environment. This has implications for the design of future agent systems and benchmarks, emphasizing the need for careful consideration of skill authoring strategies to enhance agent performance effectively.
DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms
Graph Learning
Theory
Efficient ML
- DAG-FM introduces a two-stage auto-regressive process for causal discovery, enhancing model performance.
- The Mixture-of-Leaf-Experts (MoLE) mechanism allows for dynamic adaptation to various causal mechanisms.
- The model guarantees the identification of a unique DAG from observational data, addressing limitations of traditional methods.
- DAG-FM demonstrates superior performance on synthetic and real-world datasets compared to existing approaches.
Read more
DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms
Summary
The paper presents DAG-FM, a novel foundation model designed for causal discovery from observational tabular data, addressing the challenges posed by heterogeneous causal mechanisms and the complexity of Directed Acyclic Graphs (DAGs). Traditional causal discovery methods often struggle with generalizability due to their reliance on homogeneous causal mechanisms. DAG-FM innovatively decomposes the causal discovery process into two auto-regressive stages: a leaf-node predictor and a parent-node predictor, both utilizing Transformer-based architectures. To enhance the model's adaptability to diverse Functional Causal Model (FCM) assumptions, the authors introduce a Mixture-of-Leaf-Experts (MoLE) mechanism, allowing dynamic routing to identifiable mechanism families. The model employs an iterative inference algorithm to extract causal orderings and construct valid DAGs. Extensive experiments demonstrate that DAG-FM outperforms existing classical algorithms and recent foundation models in terms of accuracy and scalability, achieving state-of-the-art results on both synthetic and real-world datasets. The paper bridges theoretical and practical aspects of causal discovery, providing a design condition for incorporating heterogeneous causal mechanisms into the prior space, ensuring the model's convergence to a unique identifiable DAG.
Methodology
DAG-FM employs a foundation model architecture that decomposes causal discovery into two sub-modules: a leaf-node predictor and a parent-node predictor, both based on Transformer architectures. It incorporates a Mixture-of-Leaf-Experts (MoLE) mechanism for dynamic routing to different causal mechanisms and utilizes an iterative inference algorithm to extract causal orderings and construct valid DAGs.
Results
DAG-FM achieves state-of-the-art performance on both synthetic benchmarks and complex real-world datasets, significantly outperforming traditional causal discovery algorithms and recent foundation models in accuracy and scalability. The model can process large datasets efficiently without relying on advanced optimizations.
Implications
The development of DAG-FM has significant implications for various fields that rely on causal inference, such as bioinformatics, epidemiology, and social sciences. Its ability to handle heterogeneous causal mechanisms could enhance the robustness and applicability of causal discovery methods in real-world scenarios.
Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse
Theory
- Independently trained neural networks lack a common neuron-index reference frame, complicating cross-trajectory comparisons.
- Neural Collapse creates a shared low-dimensional geometry, but does not eliminate functional differences between networks.
- The study successfully demonstrates the detectability of donor-specific functional fingerprints using a controlled empirical approach.
- An orthogonal Procrustes alignment and affine correction were employed to accurately map donor classifier heads into recipient coordinates.
Read more
Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse
Summary
This paper addresses the challenge of comparing independently trained neural networks, particularly in the context of Neural Collapse, where networks converge towards a shared low-dimensional geometry. The authors investigate whether functional variations specific to donor networks can be detected after this convergence. They separate the concepts of detectability, transplantability, and causal persistence, focusing on the first. Using five independently trained MLP-5 networks on the MNIST dataset, they apply an orthogonal Procrustes alignment followed by an affine correction to map donor classifier heads into recipient coordinates. The study finds that donor-specific functional fingerprints remain distinguishable even after baseline corrections, with all donor-recipient pairs correctly identified. The results indicate that while detectability is established, transplantability and causal persistence require further investigation. The paper highlights the importance of coordinate alignment, ambiguity diagnostics, and leakage control in testing for cross-network functional variation.
Methodology
The authors trained five independently initialized MLP-5 networks on the MNIST dataset using a two-phase protocol to induce Neural Collapse. They applied an orthogonal Procrustes alignment followed by an affine correction to map the donor classifiers into recipient coordinates, ensuring that donor logits were preserved. The quality of the representation fit was evaluated separately, and the alignment's underdetermined components were measured to assess their impact on identification results.
Results
The study confirmed that Neural Collapse occurred consistently across all five network seeds, with the affine transformation preserving donor logits. The analysis revealed that donor-specific functional fingerprints remained distinguishable after baseline correction, with a significant identification result (p=0.0083) for all donor-recipient pairs. The findings indicate that the low-dimensional structure necessary for identification was largely present before the completion of Neural Collapse.
Implications
The findings have implications for understanding the functional differences between independently trained neural networks and for developing protocols to assess cross-network variations. The methodology can be applied to other architectures and tasks, raising questions about the generalizability of the results.
From Many to Meaningful: Feature-Guided Zero-Shot Chronic Kidney Disease Screening Using Large Language Models
Large Language Models
NLP
- Introduces a feature-guided zero-shot framework for CKD screening using LLMs.
- Evaluates the performance of four LLMs without dataset-specific training.
- Demonstrates that a compact set of clinically relevant features can enhance screening accuracy.
- Validates the approach across three heterogeneous datasets from different countries.
Read more
From Many to Meaningful: Feature-Guided Zero-Shot Chronic Kidney Disease Screening Using Large Language Models
Summary
This paper addresses the challenge of early screening for chronic kidney disease (CKD), particularly in resource-limited settings where traditional machine learning methods may falter due to their dependence on large labeled datasets and complex clinical features. The authors propose a novel feature-guided zero-shot framework utilizing large language models (LLMs) for CKD screening without the need for dataset-specific training. By selecting a compact set of clinically meaningful features, the study evaluates the performance of four state-of-the-art LLMs (LLaMA-3, Qwen-3, Mistral, and GPT-4o-mini) across three diverse CKD datasets from Bangladesh, India, and the UAE. The results indicate that using a reduced feature set consistently improves balanced accuracy and probability estimates, demonstrating the potential of LLMs to facilitate effective CKD screening in community settings. This approach not only enhances the adaptability of screening methods to different populations but also emphasizes the importance of feature selection in improving model performance in zero-shot scenarios.
Methodology
The study employs a three-step workflow: (1) feature harmonization and selection guided by machine learning analysis to identify a compact set of clinically relevant variables, (2) serialization of tabular patient records into text using standardized prompt templates, and (3) inference using selected LLMs to assess their zero-shot performance across multiple datasets.
Results
The selected feature subset led to consistent and statistically significant improvements in balanced accuracy and probability estimates across all evaluated models and datasets. The findings indicate that LLMs can effectively support CKD screening using minimal community-accessible features, achieving performance levels suitable for practical screening applications.
Implications
The results suggest that LLMs can provide a practical solution for CKD screening in low-resource settings, potentially improving early detection rates and patient outcomes. This approach could complement traditional ML methods, making CKD screening more accessible and adaptable to diverse populations.
Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control
Reinforcement Learning
Large Language Models
Optimization
- Introduces a novel reinforcement learning approach with verifiable rewards for optimizing thermal energy storage scheduling.
- Achieves significant emission reductions in a controlled environment, demonstrating the effectiveness of the proposed method.
- Highlights the importance of reasoning capabilities in large language models for energy management tasks.
- Demonstrates robustness and generalization of planning patterns across different conditions and tasks.
Read more
Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control
Summary
This paper addresses the challenge of optimizing thermal energy storage (TES) scheduling in buildings, which is crucial for shifting cooling loads in response to grid conditions. The authors propose a novel approach that combines reinforcement learning with verifiable rewards (RLVR) to adapt an open-weight reasoning model for this task. By converting offline dynamic programming (DP) action values into dense rewards, the model is fine-tuned using only 30 training prompts to output hourly heat-pump setpoints based on text-based states and forecasts. The evaluation is conducted on a simple office-building TES benchmark where the optimal solution is known. The results show that reinforcement fine-tuning (RFT) significantly reduces emissions from 70.5 kg-CO2 to 61.2 kg-CO2, approaching the DP optimum of 60.8 kg-CO2. The study also finds that while GPT-5 nearly matches the performance of DP and model predictive control (MPC), a non-reasoning model (GPT-4o) performs worse than the no-storage baseline, highlighting the importance of inference-time reasoning. The analysis indicates that RFT stabilizes planning patterns rather than creating new strategies, and the reinforced patterns demonstrate robustness under various conditions. Overall, the findings suggest that DP-based verifiable rewards can effectively adapt reasoning models for building storage scheduling, paving the way for scalable energy management solutions.
Methodology
The authors utilize reinforcement learning with verifiable rewards (RLVR) to fine-tune an open-weight reasoning model. They convert exact offline dynamic programming action values into dense rewards for candidate actions and employ reinforcement fine-tuning (RFT) with a limited number of training prompts to optimize the model for scheduling tasks.
Results
The RFT approach reduces emissions from 70.5 kg-CO2 to 61.2 kg-CO2, closely approaching the dynamic programming optimum of 60.8 kg-CO2. The reasoning model (GPT-5) performs comparably to traditional methods (DP and MPC), while a non-reasoning model (GPT-4o) underperforms, indicating the significance of reasoning in decision-making.
Implications
The findings suggest that RLVR can effectively enhance reasoning models for practical applications in energy management, particularly in optimizing building operations. This approach could lead to more scalable and efficient energy control systems, contributing to sustainability efforts in urban environments.
Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination
Reinforcement Learning
Federated Learning
Optimization
- Proposes a constraint-aware aggregation framework for FedRL that enhances safety in energy coordination.
- Introduces DairyGridEnv as a benchmark for evaluating federated reinforcement learning in microgrids.
- Demonstrates that penalty-based aggregation consistently outperforms traditional FedAvg in terms of reward and safety.
- Shows that lightweight aggregation strategies can significantly improve empirical safety without modifying local training.
Read more
Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination
Summary
This paper addresses the challenges of Federated Reinforcement Learning (FedRL) in the context of distributed energy resources, specifically focusing on the safe coordination of microgrid energy systems. Traditional aggregation methods like FedAvg do not consider system-level constraints, which can lead to unsafe global behaviors. The authors propose a constraint-aware aggregation framework that incorporates local performance metrics and estimated constraint violations into the server-side update process. They introduce a penalty-based aggregation rule that balances reward and safety effectively without altering local training processes. The proposed methods are evaluated using DairyGridEnv, a benchmark for coordinating battery storage across multiple farms under stochastic demand and shared grid constraints. The authors also validate their approach using real-world load data from Finland and Germany. Results indicate that the penalty-based aggregation significantly reduces constraint violations while improving rewards compared to FedAvg, demonstrating the importance of incorporating system-level constraints into aggregation strategies for safe deployment in energy systems.
Methodology
The authors developed a constraint-aware aggregation framework that utilizes local performance (reward) and estimated constraint violations to reweight updates at the server. They introduced a penalty-based aggregation rule and evaluated their approach on both synthetic and real-world datasets, ensuring that local training processes remained unchanged.
Results
The penalty-based aggregation method achieved a significant reduction in constraint violations while improving rewards compared to the standard FedAvg method. The results were consistent across synthetic environments and real-world datasets, demonstrating the effectiveness of the proposed aggregation strategies.
Implications
The findings suggest that incorporating system-level constraints into aggregation methods is crucial for the safe deployment of federated reinforcement learning in energy systems. This approach can enhance the reliability and safety of decentralized energy resource management, making it applicable in various real-world scenarios where safety is a concern.
Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs
Large Language Models
Theory
Efficient ML
- The optimal training duration for merging models is dependent on the merging method used.
- Simple averaging degrades with overfitting, while sparsification-based methods benefit from overfitted experts.
- Bias-variance decomposition analysis provides insights into the performance of different merging strategies.
- The study emphasizes the need to jointly consider training duration and merging method for optimal model performance.
Read more
Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs
Summary
This paper investigates the impact of expert training duration on the quality of merged models in the context of multi-task learning for Large Language Models (LLMs). Traditionally, models are merged at their optimal validation loss, but the authors challenge this convention by exploring how varying the training duration of domain experts affects the performance of the merged model. They fine-tune experts across five domains (Math, Code, Instruction Following, Multilingual, and Safety) and three model sizes (Qwen 3.5 0.8B, 2B, and 4B), saving checkpoints from 25% to 500% of the optimal training steps. The study evaluates five merging methods and reveals that the optimal training duration is method-dependent. Simple averaging performs poorly with overfitting, while sparsification-based methods achieve better performance with overfitted experts. The authors formalize their findings through bias-variance decomposition analysis, drawing parallels to ensemble learning techniques. Their results suggest that training duration and merging method should be jointly optimized, providing practical guidance for model merging in industrial applications.
Methodology
The authors conducted a systematic study by fine-tuning expert models across five different domains and three model sizes. They saved checkpoints at various training durations (25% to 500% of optimal training steps) and evaluated five merging methods: Simple Averaging, Task Arithmetic, TIES-Merging, DARE+TIES, and Greedy Soup. The performance of the merged models was analyzed using bias-variance decomposition and mode connectivity analysis.
Results
The results indicate that different merging methods have varying optimal training durations. Simple Averaging peaks with under-trained experts, while sparsification-based methods (TIES, DARE+TIES) perform best with overfitted experts. This suggests that overfitting can enhance diversity in model predictions, benefiting certain merging strategies.
Implications
The findings have significant implications for the deployment of LLMs in industrial settings, particularly in optimizing the merging process of expert models. By understanding the relationship between training duration and merging methods, practitioners can improve the performance and efficiency of multi-task models, potentially reducing costs and enhancing model capabilities.
Vilya-1: An all-atom foundation model for macrocycle structure prediction and design
Generative Models
Optimization
- Vilya-1 significantly improves the geometric accuracy of macrocycle structure predictions compared to existing methods.
- The model operates on a uniform all-atom representation, allowing it to generalize across diverse chemical classes.
- Vilya-1 supports generative applications for designing novel macrocycles tailored to specific properties.
- The model demonstrates superior performance in conformational sampling, particularly for non-canonical and non-peptidic macrocycles.
Read more
Vilya-1: An all-atom foundation model for macrocycle structure prediction and design
Summary
The paper introduces Vilya-1, a deep learning model designed to enhance the prediction and design of macrocyclic peptides, which are increasingly recognized for their therapeutic potential. Existing computational methods struggle to generalize across diverse chemical spaces and accurately predict biologically relevant conformations. Vilya-1 addresses these challenges by employing a uniform all-atom representation that accommodates various chemistries, allowing it to learn from a wide range of structural datasets. The model significantly improves geometric accuracy in predicting macrocycle structures, outperforming traditional physics-based methods and existing deep learning approaches. Vilya-1 not only excels in conformational sampling but also supports generative applications, enabling the design of novel macrocycles with specific properties. The authors demonstrate Vilya-1's effectiveness through various benchmarks and real-world applications, establishing it as a foundational tool for the development of next-generation macrocycle therapeutics.
Methodology
Vilya-1 utilizes a deep learning architecture trained on a diverse set of structural datasets, employing a uniform all-atom representation that does not differentiate between peptide and small-molecule chemistries. This approach allows the model to learn generic structural principles applicable to a wide range of macrocycle designs. The training includes extensive internal and external datasets, enabling robust performance across various benchmarks.
Results
Vilya-1 achieves nearly double the success rate in accurate ring reconstruction (ring RMSD < 1ΛA) compared to leading physics-based methods. It demonstrates superior performance in conformational sampling for both canonical and non-canonical macrocycles, with significant improvements in predicting structures that align closely with experimental data. The model also effectively predicts key drug-like properties, facilitating the design of macrocycles with enhanced therapeutic potential.
Implications
The development of Vilya-1 has significant implications for drug discovery, particularly in the design and optimization of macrocyclic peptides. Its ability to accurately model diverse chemical spaces and predict important properties can accelerate the development of new therapeutics, potentially addressing challenging targets in drug discovery that traditional methods struggle to tackle.
How to Tame Grokking: Representation Geometry as a Control Signal
Theory
Optimization
- Grokking is characterized by delayed generalization in neural networks, where initial memorization is followed by improved test performance after prolonged training.
- GeomDR is introduced as a method to directly control the effective dimensionality of hidden representations, impacting grokking dynamics.
- Empirical results show that geometric interventions can accelerate grokking by up to 52 times, depending on the intervention schedule and target dimensionality.
- Changes in effective dimensionality consistently precede the transition from memorization to generalization, indicating its role as a controllable variable.
Read more
How to Tame Grokking: Representation Geometry as a Control Signal
Summary
This paper investigates the phenomenon of grokking in neural networks, where models initially memorize training data before achieving strong generalization after extended optimization. The study reveals that a collapse in representation dimensionality consistently precedes grokking. To address this, the author introduces Geometric Dimensionality Regularization (GeomDR), a spectral regularizer that modifies the effective dimensionality of hidden representations during training. The empirical analysis spans various tasks, including modular addition, division, and permutation composition, demonstrating that GeomDR can significantly alter grokking dynamics. The results indicate that grokking can be accelerated by up to 52 times compared to standard AdamW training, with similar effects observed in both multilayer perceptrons and transformers. This suggests that representation geometry serves as a controllable variable in influencing delayed generalization in neural networks.
Methodology
The paper employs a geometry-based regularization framework, Geometric Dimensionality Regularization (GeomDR), which modifies the effective dimensionality of hidden representations during training. The study includes a systematic investigation across various tasks, architectures, and intervention strategies to assess the impact of geometric interventions on grokking dynamics.
Results
The introduction of GeomDR leads to significant alterations in grokking dynamics, with acceleration of generalization observed in multiple settings. The study reports up to a 52-fold increase in the speed of grokking compared to standard training methods. Additionally, the results indicate that changes in effective dimensionality are a precursor to the transition from memorization to generalization.
Implications
The findings suggest that manipulating representation geometry can be a practical approach to influence generalization in neural networks. This could have implications for designing more efficient training protocols and understanding the underlying mechanisms of learning dynamics in deep learning models.
Learning-based Probabilistic Load Forecasting with Post-hoc and In-model Uncertainty
Time Series
- Developed a unified framework for one-day-ahead probabilistic load forecasting under feature-asymmetric conditions.
- Compared modular post-hoc and integrated in-model uncertainty placement methods using three deep learning architectures.
- Found that the Temporal Fusion Transformer outperformed other models in terms of accuracy and interval calibration.
- Demonstrated that reconstruction-induced uncertainty significantly impacts forecast quality.
Read more
Learning-based Probabilistic Load Forecasting with Post-hoc and In-model Uncertainty
Summary
This paper addresses the challenges of probabilistic load forecasting in smart buildings, particularly under conditions where input data is sparse and feature-limited due to sensor faults. The authors propose a unified framework for one-day-ahead forecasting that reconstructs unavailable inputs and evaluates the impact of uncertainty placement on forecast accuracy. They compare two approaches: a modular post-hoc residual-quantile scheme and an integrated in-model quantile-learning scheme, using three deep learning architectures: Bidirectional Long Short-Term Memory (BiLSTM), Bidirectional Gated Recurrent Unit-LSTM (BiGRU-LSTM), and Temporal Fusion Transformer (TFT). The study finds that the choice of uncertainty placement significantly affects forecast reliability, with the TFT model performing best in terms of accuracy and narrower prediction intervals. The results highlight the importance of properly accounting for reconstruction-induced uncertainty in forecasting models, revealing that reconstructed inputs can enhance forecast quality but do not automatically mitigate uncertainty. The paper concludes with a discussion on the operational reliability of the proposed methods and their implications for smart grid management.
Methodology
The authors created a probabilistic forecasting framework that harmonizes temporal resolution and reconstructs unavailable inputs. They evaluated three deep learning models under controlled conditions, comparing the effects of modular and integrated uncertainty placements on forecasting performance.
Results
The Temporal Fusion Transformer achieved the best performance with a Mean Absolute Percentage Error (MAPE) of 2.2-3.6% and a Root Mean Square Error (RMSE) of 28-83 W. The integrated quantile learning method produced prediction intervals that were approximately five times narrower than those from the modular approach. Reconstruction of inputs improved the Quantile Score by 106%, indicating significant benefits from addressing input uncertainty.
Implications
The findings suggest that accurately accounting for input uncertainty is crucial for effective load forecasting in smart buildings, which can enhance demand-response scheduling and resource management in smart grid operations.