AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
48
Papers today
8h
Update frequency
7
Days of history
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
NLP
Large Language Models
Reinforcement Learning
- Introduces the concept of non-hackable confidence reward schemes for LLMs.
- Demonstrates the phenomenon of confidence reward hacking in practical datasets.
- Establishes a spectrum of reward schemes to balance accuracy and confidence calibration.
- Suggests treating the choice of reward scheme as a hyperparameter for optimization.
Read more
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
Summary
This paper investigates the role of reward functions in reinforcement learning (RL) for calibrating the confidence of large language models (LLMs). The authors propose a reward scheme that differentiates between correct and incorrect answers, addressing the issue of 'confidence reward hacking,' where poorly designed rewards may lead LLMs to provide incorrect answers to maximize perceived confidence. They introduce the concept of non-hackable confidence reward schemes and define a spectrum of such schemes for RL training. The study empirically demonstrates that existing reward schemes can lead to confidence reward hacking in practical datasets. The authors emphasize that the optimal reward scheme for balancing accuracy and confidence calibration depends on the specific dataset and application, suggesting that the choice of reward scheme should be treated as a hyperparameter to optimize performance. The paper provides theoretical foundations for non-hackable reward schemes and highlights the importance of careful reward design in RL training for LLMs.
Methodology
The authors theoretically characterize non-hackable reward schemes and empirically test various reward functions in RL settings. They analyze the impact of these schemes on LLM performance, particularly focusing on the trade-offs between accuracy and confidence calibration.
Results
The study finds that LLMs trained with poorly designed reward schemes can exploit these to achieve higher confidence in incorrect answers. It also shows that different reward schemes yield varying accuracy-calibration trade-offs, indicating that the optimal scheme is context-dependent.
Implications
The findings suggest that careful design of reward functions is crucial for effective confidence calibration in LLMs, with potential applications in fields requiring high-stakes decision-making, such as healthcare and finance. The approach can enhance user trust in LLM outputs by reducing overconfidence in incorrect answers.
EquiFiLM: Charge-Conditioned Equivariant Force Fields via Feature-wise Linear Modulation
Graph Learning
Theory
Efficient ML
- EquiFiLM extends equivariant MLFFs to include continuous external conditioning, enhancing their applicability to non-equilibrium scenarios.
- The method utilizes a lightweight FiLM block that preserves E(3)-equivariance and requires minimal additional training data.
- E-MACE, the model developed using EquiFiLM, achieves substantial reductions in force and energy RMSE compared to traditional models.
- The approach is adaptable to other equivariant MLFFs and can incorporate various external scalar variables.
Read more
EquiFiLM: Charge-Conditioned Equivariant Force Fields via Feature-wise Linear Modulation
Summary
The paper introduces EquiFiLM, a novel extension to machine learning force fields (MLFFs) that enables them to handle external conditioning such as charge and applied fields, which is crucial for simulating driven processes like photoexcitation. Traditional MLFFs, while achieving near density functional theory (DFT) accuracy, are limited to equilibrium ground-state physics and cannot adapt to changes in electronic states. EquiFiLM employs a Feature-wise Linear Modulation (FiLM) block that integrates continuous external conditioning into any equivariant MLFF without altering the architecture. This method was demonstrated using the MACE-MatPES model on charged liquid water, resulting in significant improvements in force and energy accuracy while maintaining computational efficiency. The model, E-MACE, shows robust performance across various charge conditions, indicating its potential for broader applications in atomistic simulations under external control.
Methodology
EquiFiLM integrates a Feature-wise Linear Modulation (FiLM) block into the MACE-MatPES backbone, allowing the model to learn from external conditioning inputs while preserving the equivariance of the underlying architecture. The FiLM block modulates scalar channels in the interaction layers based on conditioning inputs processed by a multilayer perceptron (MLP). This setup enables the model to adapt to changes in electronic states without extensive retraining.
Results
The E-MACE model demonstrated a 3.1× reduction in force RMSE (from 21.3 to 6.96 meV/Å) and a 61× reduction in per-atom energy RMSE (from 6.1 to 0.1 meV/atom) compared to a baseline model without EquiFiLM. It maintained force RMSE within 18-61 meV/Å and energy RMSE within 0.7-5.4 meV/atom across seven held-out interpolation and extrapolation charges. The model also successfully performed stable molecular dynamics simulations across the tested charge range.
Implications
EquiFiLM's ability to incorporate external conditioning into MLFFs opens new avenues for simulating complex chemical processes under varying conditions, such as electrochemistry and photochemistry. Its lightweight design allows for efficient adaptation of existing models, potentially accelerating research in materials science and molecular dynamics.
Adversarial LassoNet: Robust Feature Selection via Stability-Driven Sparse Learning
Optimization
Theory
- AdLNet integrates adversarial perturbations into the hierarchical sparsity mechanism of LassoNet.
- The framework encourages feature selection that is both predictive and stable under local perturbations.
- Experiments show a 4.4% improvement in out-of-distribution robustness and a 6.3% increase in feature support reproducibility.
- AdLNet achieves a 5.3% test accuracy gain and a 6.0% AUC improvement on a lung cancer screening dataset compared to traditional methods.
Read more
Adversarial LassoNet: Robust Feature Selection via Stability-Driven Sparse Learning
Summary
The paper introduces Adversarial LassoNet (AdLNet), a novel framework for sparse feature selection that enhances robustness against observational noise and spurious correlations. Traditional â„“1-regularized methods often yield unstable feature supports, leading to poor generalization in high-dimensional settings. AdLNet integrates adversarial training with the hierarchical sparsity mechanism of LassoNet, allowing it to leverage local input perturbations as a training signal for feature selection. The authors derive a first-order approximation for local adversarial sensitivity and conduct a spectral analysis to demonstrate how this approach can reduce gradient concentration. Experiments on various datasets, including high-dimensional SERS data and ColoredMNIST, show that AdLNet not only maintains competitive sparse-selection performance but also improves out-of-distribution robustness and feature support reproducibility. Notably, on a lung cancer screening dataset, AdLNet achieves significant improvements in test accuracy and AUC compared to vanilla LassoNet.
Methodology
AdLNet employs a stability-driven approach by combining clean prediction loss with a perturbation-driven stability loss. It uses local adversarial sensitivity as an additional training signal, derived through a first-order approximation under local smoothness assumptions. The method modifies the training objective of LassoNet to include an adversarial stability term while preserving its hierarchical sparse optimization structure.
Results
AdLNet demonstrated competitive performance in sparse feature selection across multiple datasets. It improved out-of-distribution robustness by 4.4% and feature support reproducibility by 6.3% on ColoredMNIST. Additionally, on the high-dimensional lung cancer screening dataset, it achieved a 5.3% increase in test accuracy and a 6.0% improvement in AUC compared to vanilla LassoNet.
Implications
The findings suggest that integrating adversarial training with feature selection can lead to more robust models in high-dimensional settings, particularly in applications where data is prone to noise and spurious correlations, such as medical diagnostics and other critical decision-making domains.
Two Black Boxes, One Solver: Encoder Probing and Decoder Attribution for Neural Multi-Attribute VRP under Hard-Mask and Recourse Decoders
Optimization
Interpretability
Graph Learning
- Introduces a dual-protocol for probing encoder representations and attributing decoder decisions in MAVRP solvers.
- Demonstrates that graph inductive bias enhances representational predictability and decoder sanity.
- Finds that the Mixture-of-Experts encoder represents constraints in a distributed manner.
- Shows that the RECOURSE decoder can generate useful make-feasible counterfactuals, unlike the HARD-MASK decoder.
Read more
Two Black Boxes, One Solver: Encoder Probing and Decoder Attribution for Neural Multi-Attribute VRP under Hard-Mask and Recourse Decoders
Summary
This paper addresses the challenge of interpretability in neural autoregressive solvers for the Multi-Attribute Vehicle Routing Problem (MAVRP), which combines multiple constraints into a single optimization problem. The authors propose a comprehensive protocol that simultaneously probes the encoder and attributes decisions made by the decoder. On the encoder side, various metrics are employed to analyze how latent representations capture constraint families at different levels (graph, node, edge). On the decoder side, three attribution methods are utilized to provide insights into decision-making processes from different perspectives, including abduction and counterfactual reasoning. The study evaluates six combinations of encoder and decoder architectures, revealing that the Mixture-of-Experts encoder offers distributed representations of constraints, while the RECOURSE decoder effectively generates make-feasible counterfactuals that the HARD-MASK decoder fails to produce. The findings highlight the importance of understanding both representation and decision-making in neural solvers, emphasizing the need for a joint explainable AI (XAI) framework.
Methodology
The methodology involves a two-pillar protocol that probes the encoder for various metrics (predictability, spontaneous organization, richness, discovered directions) and attributes decoder decisions using three complementary methods (gradient, integrated gradients, DeepLIFT). The study evaluates six encoder-decoder combinations across multiple instances, scoring outputs based on five criteria of explainability.
Results
The results indicate that the Mixture-of-Experts encoder provides a more nuanced representation of constraints compared to traditional encoders. The RECOURSE decoder outperforms the HARD-MASK decoder in generating counterfactuals that explain decision-making processes. The study also reveals a significant sanity gap between different encoder architectures and highlights the representational differences in policies generated by the two decoder types.
Implications
The findings suggest that enhancing interpretability in neural solvers can improve operational efficiency in logistics and decision-making processes. The proposed framework can be applied to other combinatorial optimization problems, potentially leading to more transparent AI systems in various domains.
Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition
Time Series
Generative Models
Graph Learning
- MGCRL is the first framework to integrate masked generative and contrastive learning for EEG emotion recognition.
- The region-aware spatiotemporal encoder enhances feature learning by addressing varying EEG channel configurations.
- Generative learning via JEPA improves noise robustness and captures fine-grained emotional representations.
- Contrastive learning boosts emotion discrimination and generalization across subjects.
Read more
Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition
Summary
This paper introduces Masked Generative-Contrastive Representation Learning (MGCRL), a self-supervised learning (SSL) framework tailored for EEG-based emotion recognition. The authors identify significant challenges in existing SSL methods, such as inadequate spatiotemporal modeling, insufficient noise resilience, and limited generalization across subjects. MGCRL addresses these issues through three innovative designs: a region-aware spatiotemporal encoder that captures localized spatial relationships, a generative learning mechanism based on the Joint Embedding Predictive Architecture (JEPA) for robust feature extraction, and a contrastive learning strategy that enhances emotion discrimination. The framework effectively combines generative and contrastive learning paradigms, utilizing a masking mechanism for both contextual reconstruction and data augmentation. Extensive experiments on the FACED dataset and SEED series datasets demonstrate MGCRL's superior performance in cross-subject emotion recognition tasks, showcasing its ability to learn universal representations from unlabeled EEG data.
Methodology
The methodology involves a novel SSL framework (MGCRL) that combines a region-aware spatiotemporal encoder, a generative learning mechanism using JEPA, and a contrastive learning strategy. The spatiotemporal encoder captures localized relationships in EEG data, while the generative mechanism focuses on noise resilience. The contrastive approach leverages masked features to enhance representation stability across subjects.
Results
MGCRL consistently outperformed competitive baseline methods in emotion recognition tasks across different datasets, demonstrating its effectiveness in learning robust and transferable representations from unlabeled EEG data.
Implications
The findings suggest that MGCRL can significantly improve the performance of EEG-based emotion recognition systems, potentially enhancing applications in affective brain-computer interfaces (aBCIs) and other emotion-sensitive technologies.
Learning Sparsest Linear Causal DAGs with Latent Confounders via Higher-Order Cumulants
Graph Learning
Theory
Interpretability
- Introduces a finite-sample algorithm for recovering the sparsest DAG in canonical LvLiNGAMs.
- Implements a new update rule that directly residualizes observed variables, enhancing performance.
- Develops a sequential procedure for identifying exact parent-child relationships.
- Demonstrates superior finite-sample performance through simulations and real data analyses.
Read more
Learning Sparsest Linear Causal DAGs with Latent Confounders via Higher-Order Cumulants
Summary
This paper addresses the challenge of recovering the sparsest directed acyclic graph (DAG) in linear non-Gaussian acyclic models with latent confounders (LvLiNGAM). While existing methods provide asymptotic consistency, they lack explicit finite-sample procedures and do not accommodate an arbitrary number of latent confounders. The authors propose a novel finite-sample algorithm that builds on the ReLVLiNGAM framework, allowing for the recovery of the sparsest DAG without local restrictions. Key innovations include a new update rule that directly residualizes observed variables, improving finite-sample performance, and a sequential procedure for identifying parent-child relationships among observed variables. The proposed method demonstrates superior performance in simulations and real data analyses compared to existing approaches, particularly in scenarios where local restrictions are violated.
Methodology
The authors develop a finite-sample algorithm that builds on the top-down framework of ReLVLiNGAM. The method involves directly residualizing observed variables instead of recursively updating higher-order cumulants, which mitigates error propagation. Additionally, a sequential procedure is introduced to identify parent-child relationships, enabling direct recovery of the sparsest DAG from finite samples.
Results
The proposed method outperforms existing approaches in both synthetic and real data experiments, particularly when local restrictions of previous methods are not met. The results indicate that the new algorithm effectively recovers the sparsest DAG in various scenarios, demonstrating its robustness and efficiency.
Implications
This work has significant implications for causal discovery in complex systems where latent confounders are present. The ability to recover the sparsest DAG can enhance understanding of causal relationships in various fields, including epidemiology, social sciences, and economics, where latent variables often play a crucial role.
Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation
Reinforcement Learning
Large Language Models
- SRRL integrates a self-review mechanism into reinforcement learning episodes to enhance learning from sparse feedback.
- The framework uses policy gradients to optimize self-reviews and internalizes improvements into the base policy.
- Cross-episode memory allows the model to reuse successful self-reviews for similar tasks, improving learning efficiency.
- SRRL outperforms traditional RLVR methods in final reward performance on the GSM8K benchmark.
Read more
Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation
Summary
The paper introduces Self-Review Reinforcement Learning (SRRL), a novel framework designed to enhance the learning efficiency of large language models (LLMs) in environments with sparse or delayed feedback. Traditional reinforcement learning approaches struggle with pinpointing the actions that lead to success or failure due to the lack of immediate feedback. SRRL addresses this by incorporating a self-review step within each reinforcement learning episode. When a model's initial response fails, it generates a self-review to analyze the error, which informs a revised attempt. This self-review process is optimized using policy gradients and is internalized into the model's base policy through selective distillation, ensuring that improvements are retained across future episodes. Additionally, SRRL employs a cross-episode memory that stores successful self-reviews for reuse in similar tasks, enhancing the model's ability to learn from past experiences. The framework was evaluated against a standard RLVR baseline using the GRPO optimizer on two language models, Qwen 3-4B and OLMo-3-7B, on the GSM8K benchmark. The results demonstrate that SRRL consistently outperforms RLVR in terms of final reward performance and achieves greater learning efficiency by effectively transforming feedback into behavioral improvements.
Methodology
The SRRL framework incorporates a self-review step after each failed response, allowing the model to analyze errors and adjust its behavior. It employs policy gradients for optimizing the self-review process and uses selective distillation to internalize improvements into the base policy. A cross-episode memory is utilized to store and reuse successful self-reviews for similar tasks.
Results
The evaluation of SRRL against the RLVR baseline showed that SRRL consistently achieved higher final rewards and demonstrated improved learning efficiency on the GSM8K benchmark, effectively translating feedback into behavioral enhancements.
Implications
The SRRL framework has the potential to significantly improve the training of large language models in environments with delayed feedback, making them more effective in decision-making tasks. It could be applied in various domains where LLMs are utilized, enhancing their adaptability and learning capabilities.
EntroPath: Maximum Entropy Path Ensemble Embedding for Manifold Learning
Graph Learning
Theory
Optimization
- EntroPath utilizes maximum entropy random walks to enhance manifold learning by aggregating multiple diffusion paths.
- The method provides a free-energy dissimilarity formulation that effectively approximates geodesic distances.
- EntroPath shows significant advantages over traditional methods, especially in non-uniformly sampled manifolds.
- The paper introduces scalable extensions for practical applications in trajectory inference and out-of-sample embedding.
Read more
EntroPath: Maximum Entropy Path Ensemble Embedding for Manifold Learning
Summary
This paper introduces EntroPath, a novel manifold learning method that leverages maximum entropy random walks (MERW) to recover geodesic geometry from data graphs through ensembles of diffusion paths. Unlike traditional graph-based embeddings that rely on locally normalized random walks or shortest-path distances, which can distort the representation of the underlying manifold, EntroPath aggregates information from multiple paths, thereby enhancing robustness against noise and spurious edges. The method formulates dissimilarities using a free-energy approach that converges to squared geodesic distances in the short-time limit, effectively balancing local and global manifold structures. The paper also presents scalable extensions for out-of-sample embedding and trajectory inference, and introduces a two-stage evaluation protocol to assess performance at both distance and embedding levels. Experimental results demonstrate that EntroPath consistently matches or outperforms existing methods, particularly in scenarios involving non-uniform sampling densities and branching trajectories, showcasing its potential for accurate manifold learning.
Methodology
EntroPath employs maximum entropy random walks to compute dissimilarities based on k-step path ensembles. It formulates a free-energy dissimilarity that aggregates path costs, ensuring robustness against graph errors. The method includes a geodesic recovery theorem and scalable extensions for practical applications.
Results
EntroPath was evaluated on synthetic manifolds and biological single-cell datasets, demonstrating performance that matches or exceeds existing methods like PHATE, HeatGeo, DTNE, and Isomap. Its advantages were particularly pronounced in scenarios with non-uniform sampling and well-separated branching trajectories.
Implications
The development of EntroPath has significant implications for manifold learning in various fields, including single-cell transcriptomics and molecular dynamics, where accurate representation of underlying geometries is crucial. Its robustness to sampling density and branching structures makes it a valuable tool for analyzing complex high-dimensional data.
CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion
Efficient ML
Theory
Generative Models
- CollabEval treats model evaluation as a matrix completion problem, leveraging historical data to improve efficiency.
- The method guarantees unbiased estimates and valid confidence intervals using control variates derived from imputed scores.
- Empirical results show that CollabEval can reduce confidence interval widths by up to 30% compared to classical methods.
- The approach requires minimal additional cost, as it avoids the need for generating model outputs for skipped prompts.
Read more
CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion
Summary
This paper introduces CollabEval, a novel method for evaluating generative AI models that aims to improve statistical efficiency by leveraging historical evaluation data. Traditional model evaluation is resource-intensive, requiring extensive data collection and scoring, which can be costly and time-consuming. CollabEval reframes the evaluation process as a matrix completion problem, where the evaluation scores of multiple models across various prompts are organized in a matrix format. By focusing on a subset of models with limited annotated scores, CollabEval uses matrix completion techniques to impute unobserved scores based on correlations with historical models. This approach allows for the reduction of evaluation variance and the generation of unbiased estimates with valid confidence intervals. The authors empirically validate CollabEval across diverse datasets and demonstrate its effectiveness in reducing mean confidence interval sizes and mean squared errors compared to baseline methods, achieving significant improvements in statistical efficiency without incurring additional costs from model generation.
Methodology
CollabEval employs a matrix completion framework to impute missing evaluation scores for target models based on historical scores from other models. It utilizes control variates derived from these imputed scores to reduce variance in estimates, ensuring unbiased results and valid confidence intervals. The method combines collaborative filtering with cross-prediction-powered inference techniques to enhance the accuracy of model evaluations.
Results
The empirical evaluation of CollabEval across five diverse text generation benchmarks demonstrated a significant reduction in mean confidence interval sizes and mean squared errors compared to traditional evaluation methods. Specifically, the method achieved up to a 30% reduction in confidence interval widths while maintaining accuracy within the same annotation budget.
Implications
CollabEval has the potential to streamline the evaluation process for generative AI models, making it more efficient and cost-effective. This method can be particularly beneficial in scenarios where multiple models are evaluated simultaneously, allowing for better resource allocation and faster iterations in model development.
Statistically Meaningful Geometry and Gauge Symmetry Breaking: A Geometric Foundation for Scientific Discovery and Intelligence Emergence
Theory
- Introduction of Statistically Meaningful Geometry (SMG) as a framework for understanding over-parameterized models.
- Demonstration of gauge symmetry breaking (GSB) as a critical factor for genuine intelligence emergence.
- Non-parametric deconstruction methods for analyzing core variables in machine learning systems.
- Mathematical formalization of connections that relate geometry to intelligence and scientific discovery.
Read more
Statistically Meaningful Geometry and Gauge Symmetry Breaking: A Geometric Foundation for Scientific Discovery and Intelligence Emergence
Summary
This paper addresses the epistemological crisis in the context of over-parameterized machine learning models, particularly Large Language Models (LLMs), questioning whether these systems exhibit true intelligence or merely function as advanced statistical pattern matchers. The authors introduce Statistically Meaningful Geometry (SMG), a geometric framework that conceptualizes over-parameterized learning systems as infinite-dimensional non-parametric Orlicz fiber bundles. They demonstrate that persistent exposure to out-of-distribution (OOD) environmental stimuli can lead to significant gauge symmetry breaking (GSB), which is essential for genuine scientific discovery and the emergence of intelligence. The paper outlines a non-parametric deconstruction of core variables and introduces mathematical constructs such as δ-connections and ∆-connections to formalize the relationships between geometry and intelligence. The authors also explore the implications of GSB for understanding the structural challenges in achieving true intelligence and propose future directions for integrating SMG into AI for scientific discovery and artificial general intelligence (AGI).
Methodology
The authors employ a non-parametric approach to construct empirical information velocity and horizontal lift vectors, alongside mathematical formalizations of δ-connections and ∆-connections. They utilize rigorous geometric frameworks to analyze the implications of gauge symmetry breaking in the context of machine learning.
Results
The paper establishes that the introduction of SMG allows for a clearer distinction between statistical pattern matching and genuine intelligence. It proves that GSB can facilitate the discovery of novel causal laws in over-parameterized systems, providing a mathematical foundation for understanding the dynamics of intelligence emergence.
Implications
The findings suggest that by applying SMG and understanding GSB, researchers can enhance the capabilities of AI systems in scientific discovery, potentially leading to breakthroughs in generative AI and AGI. This framework could also help address issues such as hallucinations and catastrophic forgetting in LLMs.
Unsupervised Anomaly Detection of Information Operations Users via Behavioral and Language Patterns
NLP
Large Language Models
Multimodal
- TENSOR is an unsupervised approach that formulates IO user detection as an anomaly detection problem.
- The method leverages both temporal behavioral patterns and language patterns from user post timelines.
- A Temporal Point Process (TPP) is utilized to capture abnormal user behaviors associated with IOs.
- The introduction of a novel evidence function enhances the detection accuracy by refining TPP outputs.
Read more
Unsupervised Anomaly Detection of Information Operations Users via Behavioral and Language Patterns
Summary
This paper addresses the challenge of detecting Information Operations (IOs) on social networks, which pose significant threats to democracy. Traditional supervised methods struggle to adapt to the evolving nature of IO user behavior, while existing unsupervised approaches often rely on oversimplified assumptions. To tackle these issues, the authors propose TENSOR (Temporal-bEhavior-laNguage Signals for information Operation Recognition), an unsupervised anomaly detection framework that utilizes multimodal data, including temporal user behavior and textual content. The approach employs a Temporal Point Process (TPP) to identify abnormal behavioral patterns indicative of IO users, who typically exhibit coordinated actions. Additionally, a novel evidence function is introduced to refine TPP outputs using quantitative scores derived from responses generated by large language models (LLMs). Experimental results demonstrate that TENSOR significantly outperforms baseline methods across five real-world IO datasets, showcasing its effectiveness in detecting IO users based on their unique behavioral and language patterns.
Methodology
The authors developed TENSOR, which employs a Temporal Point Process (TPP) to model and identify abnormal temporal behavioral patterns of IO users. The approach also includes a novel evidence function that converts LLM-generated responses into quantitative scores to adjust TPP outputs, enhancing the detection of IO users amidst contaminated data.
Results
TENSOR outperformed baseline methods on five real-world datasets, demonstrating its capability to effectively detect IO users by leveraging unique behavioral and language patterns.
Implications
The findings suggest that TENSOR could be a valuable tool for social media platforms and governmental organizations in identifying and mitigating the impact of IOs, thereby protecting democratic processes and public opinion from manipulation.
Hyperparameter Transfer in Graph Neural Networks
Graph Learning
Optimization
Theory
- Developed a framework for hyperparameter transfer in GNNs, enabling optimization of large models based on smaller counterparts.
- Identified graph-dependent first-layer correction factors for SGD, enhancing early training performance.
- Explored the effects of message passing normalizations on transfer behavior in GNNs, advocating for a global normalization strategy.
- Adapted AdamW parameterization for joint transfer of weight decay and learning rate, improving training efficiency.
Read more
Hyperparameter Transfer in Graph Neural Networks
Summary
This paper addresses the challenge of hyperparameter tuning in Graph Neural Networks (GNNs), which is crucial for their performance but often computationally expensive, especially for large models. The authors propose a framework for hyperparameter transfer that allows for the optimization of larger GNNs by leveraging the hyperparameters identified for smaller models. They derive and validate learning rate transfer scalings for GNNs trained with different optimization algorithms (SGD, Adam, and AdamW). The study highlights the importance of first-layer correction factors for SGD in graph regression tasks and explores the impact of message passing normalizations on training dynamics. The authors also adapt the weight decay parameterization for AdamW to facilitate effective hyperparameter transfer. Through theoretical analyses and empirical experiments, they demonstrate that their proposed methods lead to improved performance and stability in GNNs as model size increases.
Methodology
The authors conducted theoretical scaling analyses and controlled experiments to derive learning rate transfer scalings for GNNs. They examined the effects of different optimization algorithms (SGD, Adam, AdamW) and message passing normalizations on training dynamics. Empirical validation was performed using various datasets to demonstrate the effectiveness of their proposed hyperparameter transfer techniques.
Results
The proposed hyperparameter transfer framework yielded stable feature updates and improved performance as the width and depth of GNNs increased. The first-layer correction factors for SGD significantly accelerated early training in sparse graph datasets. For Adam, the study showed that message passing normalizations are crucial for robust transfer behavior. The adaptations made for AdamW allowed for effective joint transfer of weight decay and learning rate, contributing to enhanced training outcomes.
Implications
The findings suggest that hyperparameter transfer can significantly reduce the computational burden associated with tuning large GNNs, making it feasible to apply GNNs to larger and more complex datasets. This work lays the groundwork for future research in optimizing GNN architectures and could lead to broader applications in fields such as social network analysis, recommendation systems, and biological network modeling.
Co-Adaptive Multi-Task LoRA: Transfer-Aware, Label-Free Control of Domain Participation
NLP
Large Language Models
Efficient ML
- Introduces CODA, a co-adaptive controller for multi-task LoRA that optimizes domain participation.
- Utilizes label-free signals to assess domain competence and cross-domain transfer affinities.
- Demonstrates improved performance over traditional methods while using half the data.
- Proves that the competence signal effectively tracks domain risk and informs participation decisions.
Read more
Co-Adaptive Multi-Task LoRA: Transfer-Aware, Label-Free Control of Domain Participation
Summary
This paper presents CODA (Co-adaptive Domain Adaptation), a novel approach for fine-tuning low-rank adapters (LoRA) in multi-task learning settings. Traditional methods often overlook the dynamic nature of domain participation, leading to inefficiencies in training. CODA addresses this by introducing a forward-only controller that utilizes unlabeled probe sets to estimate each domain's competence and cross-domain affinities without requiring additional labeled data. The controller formulates an entropy-regularized quadratic program to optimize domain participation, balancing loss weights and data shares based on the domains' learning states. The authors demonstrate that CODA outperforms existing methods across five heterogeneous domains, achieving better performance with reduced data usage and lower cross-domain gradient conflicts. The findings highlight the importance of adaptive domain participation in multi-task learning, providing a framework that can be integrated into various LoRA pipelines.
Methodology
The methodology involves a forward-only controller that probes each domain using unlabeled data to estimate competence (headroom and learning speed) and cross-domain affinities (transfer potential). These signals are integrated into an entropy-regularized quadratic program that determines optimal domain participation, adjusting both loss weights and data shares dynamically throughout training.
Results
CODA was tested across five heterogeneous domains and two backbone models, outperforming uniform mixing, learned mixtures, gradient-surgery multi-task optimizers, and online data selection methods. It achieved better performance while utilizing only half the data and reduced cross-domain gradient conflicts, demonstrating the effectiveness of its adaptive participation strategy.
Implications
The findings suggest that adaptive control of domain participation can significantly enhance the efficiency and effectiveness of multi-task learning frameworks, particularly in scenarios involving diverse and competing tasks. This approach could be applied to various domains, including natural language processing and computer vision, where multi-task learning is prevalent.
Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System
Reinforcement Learning
Robotics
Optimization
- Integration of a differentiable physics model into the PPO algorithm enhances safety in reinforcement learning.
- The approach allows for anticipatory safety measures without relying solely on complex reward shaping.
- Evaluation on a 1-DoF helicopter system shows a trade-off between safety constraint satisfaction and task performance.
- The method demonstrates potential for real-world applications in industrial control systems.
Read more
Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System
Summary
This work-in-progress paper addresses the challenge of ensuring safety in deep reinforcement learning (DRL) applications for industrial cyber-physical systems (ICPSs). The authors propose a novel approach that integrates a differentiable physics model into the proximal policy optimization (PPO) algorithm's loss function. By simulating short-horizon future trajectories during training, the policy is penalized for anticipated safety violations, independent of the task-reward signal. The proposed method is evaluated on a simulated 1-degree-of-freedom helicopter testbed with strict pitch constraints. The results demonstrate that the physics-informed soft regularizations significantly reduce constraint violations while maintaining reliable target tracking, showcasing a scalable mechanism for safe RL deployment.
Methodology
The authors utilize the PPO algorithm, embedding a physics-informed neural network (PINN) into the loss function. They model system dynamics using differential equations and employ a 4th order Runge-Kutta (RK4) integration method to simulate future trajectories. The loss function combines the standard PPO actor loss with a safety penalty derived from the differentiable physics model, allowing for gradient flow from anticipated safety violations back to the actor network.
Results
The experiments reveal that while all configurations converge, there is a clear trade-off between target tracking performance and safety constraint satisfaction. The naive baseline achieves high task rewards but violates safety limits, while the over-penalized model maintains safety at the cost of tracking performance. The balanced configuration offers a middle ground, demonstrating the effectiveness of the proposed method.
Implications
This research has significant implications for the deployment of reinforcement learning in safety-critical applications, such as industrial automation and robotics, where ensuring compliance with safety constraints is paramount. The proposed method could lead to more robust and reliable control systems that can operate safely in complex environments.
GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech
Audio & Speech
- GRAFT enables per-word pronunciation control in TTS without additional parameters.
- It utilizes voice conversion to disentangle pronunciation from speaker identity.
- The system significantly improves pronunciation accuracy for rare and difficult words.
- GRAFT outperforms existing zero-shot TTS systems in multiple languages.
Read more
GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech
Summary
The paper introduces GRAFT, a novel mechanism for controlling per-word pronunciation in text-to-speech (TTS) systems, particularly in zero-shot scenarios. Traditional TTS systems often struggle with the accurate pronunciation of rare words, proper nouns, and technical terms due to the ambiguity of text representation. GRAFT addresses this issue by allowing the pronunciation of a word to be conditioned on a short spoken sample, which is integrated into the model's existing architecture without requiring additional parameters. The approach utilizes voice conversion techniques to separate the pronunciation from the speaker identity, enabling the model to synthesize speech in a target voice while preserving the intended pronunciation. The authors conducted a blind listening study, where GRAFT was rated significantly higher than other systems for its accuracy in rendering difficult words. Additionally, GRAFT demonstrated a 22-39% reduction in phoneme error rates across five languages compared to traditional text-only models and outperformed competitive zero-shot systems in pronunciation accuracy while maintaining speaker similarity and naturalness.
Methodology
GRAFT integrates a per-word audio conditioning mechanism into a neural codec language model for TTS. It uses a spoken example of a word, encoded with the model's speech tokenizer, to guide pronunciation. The training process employs voice conversion techniques to ensure that the pronunciation can be applied to any target voice, effectively decoupling the hint speaker from the target speaker.
Results
In a blind listening study, GRAFT was rated first by human raters for its pronunciation accuracy. On a five-language benchmark, it achieved a 22-39% reduction in target-word phoneme error rates compared to a text-only backbone and outperformed other competitive zero-shot systems in terms of pronunciation accuracy while preserving the naturalness of the synthesized speech.
Implications
GRAFT has significant implications for improving the accuracy of TTS systems, particularly in applications requiring the synthesis of rare or specialized vocabulary. This advancement can enhance user experience in various domains, including virtual assistants, audiobooks, and language learning tools, where precise pronunciation is crucial.
Self-Supervised Implicit CEST Reconstruction via Physics-Informed Lorentz Encoding
Theory
Optimization
Efficient ML
- Introduction of Lorentz Encoding (LE) as a physics-informed framework for CEST MRI reconstruction.
- Decoupled hybrid architecture combining spatial and spectral encoding to enhance reconstruction quality.
- Demonstrated significant performance improvements over state-of-the-art methods under extreme sampling conditions.
- Achieved high PSNR (57.58 dB) and SSIM (0.9994) metrics in in vivo experiments.
Read more
Self-Supervised Implicit CEST Reconstruction via Physics-Informed Lorentz Encoding
Summary
This paper presents a novel approach for reconstructing high-resolution Z-spectra in Multi-Pool Chemical Exchange Saturation Transfer (CEST) MRI using a self-supervised, physics-informed framework known as Lorentz Encoding (LE). Traditional methods struggle with sparse sampling, leading to ill-posed inverse problems and artifacts in the reconstructed spectra. The authors propose LE to reformulate CEST reconstruction as a self-supervised task, leveraging implicit continuous coordinate learning while embedding physical constraints into the reconstruction process. LE utilizes a decoupled hybrid architecture that combines Multi-resolution Hash Encoding for spatial features and Lorentz Encoding for spectral features, effectively filtering out noise and ensuring physical validity. Experimental results on in vivo human brain data demonstrate that LE significantly outperforms state-of-the-art methods, achieving high PSNR and SSIM values even with limited sampling points. This work highlights the potential of integrating physical models into deep learning frameworks for improved accuracy in medical imaging.
Methodology
The proposed method employs a decoupled hybrid architecture that utilizes Multi-resolution Hash Encoding for high-frequency spatial textures and Lorentz Encoding for embedding physical spectral priors. This architecture allows for the reconstruction of CEST Z-spectra by mapping spatial-spectral coordinates to normalized signal intensities while maintaining physical constraints to filter out artifacts and noise.
Results
The experiments conducted on in vivo human brain data showed that the Lorentz Encoding approach significantly outperformed existing methods, achieving a PSNR of 57.58 dB and an SSIM of 0.9994 under a 39-point sampling strategy. The method also demonstrated robust performance with as few as 21 sampling points, ensuring accurate metabolite mapping.
Implications
The integration of physics-informed models into deep learning frameworks for medical imaging can lead to more accurate and reliable reconstructions from sparse data, potentially enhancing the clinical utility of CEST MRI in metabolic imaging and diagnostics.
Learning When to Automate: Queue Control in Human-AI Service Systems
Theory
Optimization
- Introduces a novel queueing model for human-AI service systems that couples automation and human scheduling decisions.
- Proposes the UCB-DPP policy, which effectively learns unknown parameters while managing queue dynamics.
- Demonstrates theoretical guarantees of sublinear regret and stability in human-service queues.
- Shows through simulations that UCB-DPP outperforms existing baseline policies in various scenarios.
Read more
Learning When to Automate: Queue Control in Human-AI Service Systems
Summary
This paper investigates the dynamics of a human-AI service system where tasks are processed through a two-stage architecture involving an automated chatbot and human agents. The authors model a scenario where tasks arrive sequentially and belong to various types, each with distinct processing requirements. The decision-maker must allocate resources to the chatbot, which has type-dependent success probabilities that are initially unknown. Tasks that the chatbot cannot resolve are sent to human agents, who also have unknown service rates. The study highlights the trade-off between automation and human intervention, where increased reliance on automation can reduce human congestion but raises costs for the chatbot. The authors propose the UCB-DPP policy, which integrates Upper Confidence Bounds with Drift-Plus-Penalty control to learn system parameters while making informed decisions about resource allocation. The theoretical analysis demonstrates that UCB-DPP achieves a sublinear regret bound and ensures stability in human-service queues. Simulations indicate that this policy outperforms baseline approaches, showcasing its effectiveness in managing the complexities of hybrid service systems.
Methodology
The authors develop a queueing model that captures the interaction between automated chatbot decisions and human service scheduling. They propose the UCB-DPP policy, which combines optimistic estimates of unknown parameters with a weighted drift-plus-penalty objective to make queue-aware decisions. The theoretical analysis employs concentration bounds and Lyapunov-drift arguments to establish performance guarantees.
Results
The UCB-DPP policy achieves a regret bound of order O(K√T) and ensures mean-rate stability of human-service queues. Simulations on synthetic instances demonstrate that UCB-DPP outperforms natural baseline policies, indicating its practical effectiveness in managing human-AI service systems.
Implications
The findings suggest that effective coordination between automation and human agents can enhance service efficiency and customer satisfaction in various applications, such as customer support, IT helpdesks, and fraud detection. The proposed framework can inform the design of hybrid service systems that optimize resource allocation and improve operational performance.
Heckman-Corrected Epistemic Uncertainty: Selection on Unobservables Defeats Importance Weighting
Theory
- Importance weighting fails to correct for selection on unobservables, leading to biased predictions.
- Heckman's two-equation model provides a robust framework for addressing selection bias in machine learning.
- The proposed deep Heckman UQ method significantly improves predictive coverage in selected-against regions.
- The stability of the joint maximum likelihood estimator requires careful warm-up scheduling.
Read more
Heckman-Corrected Epistemic Uncertainty: Selection on Unobservables Defeats Importance Weighting
Summary
This paper addresses the challenge of selection bias in machine learning models, particularly when training data is collected through processes that the model does not observe. Traditional methods like importance weighting and covariate-shift correction assume that selection is ignorable given observable variables. However, this paper argues that selection on unobservables can lead to systematic biases that these methods cannot correct. The author applies Heckman's two-equation model, which jointly estimates a selection equation and an outcome equation, to correct for this bias. The proposed methodology includes a deep outcome network and a linear selection head, allowing for a joint bivariate-normal likelihood estimation. The results demonstrate that the Heckman-corrected predictive distribution significantly improves coverage in selected-against regions compared to traditional importance weighting methods. The paper also discusses the importance of using instruments in the selection equation to maintain identification and presents empirical validations using real-world data, highlighting the limitations of existing methods in the presence of unobserved selection.
Methodology
The paper employs Heckman's two-equation model to jointly estimate a selection equation and an outcome equation, incorporating a deep outcome network and a linear selection head. The methodology includes both a two-step estimator and a joint maximum likelihood estimator, with a focus on correcting for selection bias using a bivariate-normal likelihood over all units.
Results
The Heckman-corrected predictive distribution achieved a coverage of 88.9% in selected-against regions, compared to only 43.1% for oracle-propensity importance weighting. The method's performance degrades gracefully without an instrument, with coverage dropping to 40.3%. The empirical validation on real tabular data showed that the corrected intervals were the best-calibrated among non-oracle methods.
Implications
The findings suggest that practitioners should consider selection on unobservables when developing machine learning models, particularly in fields like economics and healthcare where such biases are prevalent. The proposed methodology offers a more reliable approach to uncertainty quantification in these contexts.
Deep Reinforcement Learning for Dynamic Battery Management of Autonomous Order Pickers
Reinforcement Learning
Robotics
Optimization
- Development of a multi-agent DRL framework using PPO for optimizing AMR routing and charging in warehouses.
- Incorporation of a comprehensive action space for charging decisions, including when to recharge, which station to use, and duration of charging.
- Dynamic modeling of order drop-off decisions to minimize unnecessary travel time for AMRs.
- Explicit modeling of stochastic order arrivals and queuing dynamics at charging stations.
Read more
Deep Reinforcement Learning for Dynamic Battery Management of Autonomous Order Pickers
Summary
This paper addresses the critical challenge of battery management for Autonomous Mobile Robots (AMRs) in warehouse environments, particularly under conditions of stochastic order arrivals. Traditional fixed-rule heuristics for charging often lead to inefficiencies, particularly in multi-AMR scenarios. To tackle this issue, the authors propose a Deep Reinforcement Learning (DRL) framework based on Proximal Policy Optimization (PPO) that enables AMRs to learn optimal charging decisions dynamically. The model focuses on two primary decisions: selecting charging stations and determining optimal charging durations while considering anticipated queuing times. Through extensive numerical experiments, the proposed DRL model is benchmarked against both state-of-the-art DRL methods and traditional heuristics. The results indicate that the PPO framework can enhance order-completion rates by up to 6% compared to the best baseline, while also significantly reducing the total time spent on recharging operations. The robustness of the model is validated across various warehouse configurations and stochastic arrival rates, providing insights into its operational advantages over conventional methods.
Methodology
The authors developed a DRL framework utilizing Proximal Policy Optimization (PPO) to optimize the charging and routing decisions of AMRs in a multi-block warehouse setting. The framework incorporates a comprehensive action space that includes decisions on when to recharge, which charging station to use, and how long to charge, while also modeling the stochastic nature of order arrivals and queuing at charging stations.
Results
The proposed PPO-based DRL model showed a significant increase in order-completion rates by up to 6% compared to the strongest baseline. Additionally, it achieved a notable reduction in the total time spent on recharging operations, demonstrating its effectiveness in enhancing warehouse efficiency.
Implications
The findings suggest that implementing DRL frameworks for battery management in AMRs can lead to more efficient warehouse operations, particularly in environments with high variability in order arrivals. This approach can be applied to improve logistics and operational efficiency in various automated settings, potentially influencing future designs of warehouse management systems.
Scalable Perturbation Learning for Online Self-Supervised Echo State Networks
Time Series
Efficient ML
Theory
- Introduces a perturbation-based learning rule for online self-supervised learning in Echo State Networks.
- Addresses variance scaling issues in high-dimensional systems by focusing on input-dependent components.
- Demonstrates improved scalability and efficiency for online learning in ESNs.
- Provides a theoretical foundation through orthogonal decomposition of the self-supervised cost function.
Read more
Scalable Perturbation Learning for Online Self-Supervised Echo State Networks
Summary
This paper addresses the challenges of creating intelligent systems that can adapt to real-world dynamics using Echo State Networks (ESNs). The authors propose a novel perturbation-based learning rule for online self-supervised learning in ESNs, which mitigates the variance issues that arise with high-dimensional systems. By employing an orthogonal decomposition of the self-supervised learning cost, the method separates the input-dependent component from a redundant component determined by fixed ESN parameters. This allows the learning process to focus solely on the input-dependent component, effectively reducing the perturbation dimension from the reservoir dimension to the input dimension. Consequently, the proposed approach maintains the benefits of self-supervised adaptation and online learning while avoiding the variance growth associated with larger reservoirs. The study demonstrates that this method enhances the scalability and efficiency of online self-supervised learning in ESNs, making it more suitable for hardware implementations with limited memory and feedback routing capabilities.
Methodology
The authors derive a perturbation-based learning rule from an orthogonal decomposition of the self-supervised learning cost function, which separates the input-dependent component from a redundant component. This allows for perturbation learning to be applied only to the input-dependent term, thus reducing the effective perturbation dimension and variance scaling.
Results
The proposed method was validated through numerical experiments, showing that the variance of the gradient estimate is reduced from O(nr) to O(nin), where nr is the reservoir dimension and nin is the input dimension. This improvement leads to a better signal-to-noise ratio in the learning process, making it more effective for large reservoirs.
Implications
The findings suggest that the proposed learning rule can significantly enhance the performance of ESNs in real-world applications where online adaptation and memory efficiency are critical. This could lead to advancements in various fields such as robotics, autonomous systems, and other intelligent systems that require real-time learning capabilities.
Teacher Supervision over Representation Equivalence Classes
NLP
Large Language Models
Theory
- Knowledge distillation should focus on matching output functions rather than internal representations.
- Pretrained representations are identifiable only up to an equivalence class, making absolute feature matching ill-posed.
- Capability transfer is determined by class invariants, not by the specific coordinates of features.
- Empirical results show that matching output distributions restores model capability, while matching internal representations does not.
Read more
Teacher Supervision over Representation Equivalence Classes
Summary
This paper presents a novel perspective on knowledge distillation in machine learning, arguing that traditional approaches to matching teacher and student representations are fundamentally flawed. The author posits that a pretrained representation is only identifiable up to an equivalence class defined by orthogonal and isotropic scaling, meaning that students should learn the teacher's equivalence class rather than specific features. The core contribution is the assertion that the capability of a model is determined by its output function, which is invariant to the representation's coordinate system. The paper critiques existing methods of distillation, which typically focus on matching logits, hidden features, or sample relations, and instead emphasizes the importance of targeting class invariants such as Gram structure and principal subspaces. Empirical validation is provided through experiments on Qwen2.5 and Llama-3.1, demonstrating that while matching internal representations can yield high similarity scores (CKA ≈0.99), it does not necessarily restore model capability. In contrast, matching output distributions effectively recovers capability, highlighting the distinction between internal representation alignment and functional output matching. The findings suggest a unified geometric framework for understanding various distillation techniques, positioning capability transfer as a function of the output invariant rather than the internal representation.
Methodology
The author employs a theoretical framework to analyze knowledge distillation, focusing on the geometric properties of representations and their equivalence classes. Empirical validation is conducted using two language models (Qwen2.5 and Llama-3.1) to demonstrate the effects of different matching strategies on model capability.
Results
The experiments reveal that while matching internal representations can achieve high similarity scores (CKA ≈0.99), it fails to restore model capability, as indicated by significantly worse perplexity and agreement scores. In contrast, matching the output distribution leads to a near-complete recovery of capability, with perplexity ratios close to 1.01 and top-1 agreement at 0.98.
Implications
This work challenges the conventional wisdom in knowledge distillation, suggesting that future research should prioritize output function matching over internal feature alignment. It has potential implications for improving model transferability and efficiency in various machine learning applications, particularly in natural language processing.
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
Federated Learning
Optimization
Theory
- Introduction of the 'Spectral Bias of Drift' concept in Federated Learning.
- Development of SpecGradFilter to suppress low-frequency gradient components.
- Demonstration of superior performance in Non-IID settings compared to existing methods.
- Flexibility of SpecGradFilter allows integration into existing federated learning pipelines.
Read more
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
Summary
This paper addresses the challenges posed by statistical heterogeneity in Federated Learning (FL), particularly the issue of 'client drift' caused by non-IID data distributions. The authors introduce a novel perspective by examining client drift through a frequency-domain lens, identifying a phenomenon termed the 'Spectral Bias of Drift.' They find that inter-client gradient divergence is primarily concentrated in low-frequency components, which represent client-specific distributional shifts, while high-frequency components remain consistent across clients. To mitigate this issue, the authors propose SpecGradFilter, a unified framework that suppresses discordant low-frequency signals during local updates. This framework can be implemented through both precise FFT-based truncation and spatial approximations like Gaussian detrending. The effectiveness of SpecGradFilter is demonstrated through extensive experiments on benchmark datasets such as CIFAR-10/100 and Tiny-ImageNet, showing significant improvements in performance and convergence speed in highly Non-IID settings, while maintaining negligible communication overhead. This work establishes a new paradigm for robust federated optimization by leveraging spectral analysis.
Methodology
The authors analyze client drift from a frequency-domain perspective, identifying that low-frequency gradient components are primarily responsible for divergence among clients. SpecGradFilter is proposed as a framework to suppress these low-frequency components, implemented through both FFT-based truncation and spatial methods like Gaussian detrending. The methodology includes extensive empirical testing on various datasets to validate the effectiveness of the proposed framework.
Results
The experiments conducted on CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly outperforms a range of strong federated learning baselines, particularly in highly Non-IID scenarios. The results indicate improved convergence speed and overall model performance, establishing the framework as a robust solution for federated optimization challenges.
Implications
SpecGradFilter has the potential to enhance the performance of federated learning systems, particularly in applications where data is distributed across clients with varying distributions. This could lead to more effective collaborative machine learning solutions in fields such as healthcare, finance, and any domain requiring privacy-preserving data analysis.
Low-Overhead Error-Corrected QCNNs Using Bivariate Bicycle Codes
Theory
Efficient ML
Optimization
- Introduces a low-overhead QEC technique for QCNNs using bivariate bicycle codes.
- Demonstrates that unprotected QCNNs struggle with convergence under realistic noise levels.
- Shows that the distance-4 BB code can improve learning rates and convergence in QCNNs.
- Validates the effectiveness of the proposed QEC method through simulations.
Read more
Low-Overhead Error-Corrected QCNNs Using Bivariate Bicycle Codes
Summary
This paper addresses the challenges of implementing Quantum Convolutional Neural Networks (QCNNs) on noisy intermediate-scale quantum (NISQ) devices, particularly focusing on high error rates that hinder practical execution. The authors propose a novel quantum error correction (QEC) technique utilizing bivariate bicycle (BB) codes, which are characterized by a constant encoding rate and linear code distance. The study demonstrates that a 4-qubit unprotected QCNN fails to converge under realistic noise conditions, highlighting the necessity for effective error correction. By integrating the distance-4 BB code with a Feed-Forward Neural Network (FFNN), the authors show that their approach significantly improves the learning rate and convergence of the QCNN. The results indicate that the proposed QEC method can effectively mitigate the effects of noise, allowing for more reliable deployment of QCNNs on NISQ hardware, thus paving the way for practical applications in quantum machine learning.
Methodology
The authors conducted simulations of a 4-qubit QCNN with and without the proposed distance-4 BB code under various NISQ error rates. They compared the performance metrics, including convergence and learning rates, to evaluate the effectiveness of the error correction technique.
Results
The results revealed that the QCNN utilizing the distance-4 BB code achieved satisfactory performance, demonstrating improved convergence and reduced learning loss compared to the unprotected QCNN. The proposed QEC method maintained a constant encoding rate and linear code distance, allowing for low-overhead scaling.
Implications
The findings suggest that integrating effective error correction techniques like BB codes can enhance the viability of QCNNs for real-world applications in quantum machine learning, potentially enabling advancements in fields such as signal processing, healthcare, and beyond.
Canopy: A Heterograph Foundation Model for Metabolic Engineering
Graph Learning
Multimodal
Optimization
- CANOPY integrates ten diverse data sources into a unified knowledge graph for metabolic engineering.
- The model utilizes domain-specific foundation models for multi-modal feature encoding.
- A Heterogeneous Graph Transformer is employed for self-supervised pretraining.
- CANOPY outperforms traditional tabular models and homogeneous GNNs in fermentation titer prediction.
Read more
Canopy: A Heterograph Foundation Model for Metabolic Engineering
Summary
The paper introduces CANOPY, a novel heterogeneous graph foundation model designed to enhance metabolic engineering by integrating multiple data sources into a comprehensive knowledge graph. This graph encompasses 6.9 million nodes across 13 types and 34 edge types, representing various biological entities such as genes, proteins, metabolites, and fermentation experiments. CANOPY employs domain-specific foundation models to encode node features, creating a multi-modal representation. The model is pretrained using a Heterogeneous Graph Transformer (HGT) with advanced techniques such as SignNet positional encodings and Jumping Knowledge aggregation. The authors demonstrate that CANOPY significantly improves fermentation titer prediction, achieving an R2 score of 0.41, which surpasses traditional tabular machine learning approaches and homogeneous graph neural networks. This work highlights the potential of integrating relational biological knowledge into machine learning frameworks for more effective strain design in metabolic engineering.
Methodology
The authors developed a heterogeneous graph knowledge model that integrates various biological data sources. They utilized domain-specific foundation models (ESM-2, MoLFormer, and PubMedBERT) for encoding node features and pretrained a Heterogeneous Graph Transformer with multiple self-supervised learning objectives, including link prediction and masked node modeling.
Results
CANOPY achieved an R2 score of 0.41 in fermentation titer prediction using frozen embeddings, significantly outperforming the best tabular baseline score of 0.24 and homogeneous graph neural network variants.
Implications
The development of CANOPY has significant implications for metabolic engineering, as it provides a powerful tool for predicting fermentation outcomes and optimizing strain design, potentially accelerating the bioeconomy by improving the efficiency of microbial strain development.
Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations
Computer Vision
Theory
Generative Models
- Introduces a mesh-free auxiliary loss based on Rice level-crossing density for INRs.
- Addresses the spectral bias problem in neural representations by focusing on high-frequency content.
- Validates the proposed loss against existing methods, showing significant improvements in performance.
- Demonstrates robustness to irregular supervision and gradient-target quality.
Read more
Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations
Summary
This paper addresses the issue of spectral bias in coordinate-MLP implicit neural representations (INRs), where low-frequency structures are learned quickly while high-frequency details are neglected. The author proposes a novel auxiliary loss based on the Rice level-crossing density, which serves as a differentiable training objective that does not require a regular sampling grid. By estimating the level-crossing density of the output field using a smoothed Monte-Carlo method, the proposed loss can be evaluated at arbitrary sample locations, making it mesh-free and robust against irregular supervision. The paper validates the effectiveness of this loss by comparing it to existing methods like the Focal Frequency Loss (FFL) and Sobolev supervision, demonstrating that it significantly improves performance in scenarios with scarce and irregular supervision. The results indicate that the crossing-density loss matches or exceeds the performance of FFL in certain contexts, particularly on statistically homogeneous textures, while remaining structurally distinct by not relying on a sampling grid or gradient targets. The author provides code and experimental scripts to facilitate further research.
Methodology
The methodology involves deriving a differentiable training objective from the Rice level-crossing density, which estimates the density of level crossings in the output field of the INR. A smoothed Monte-Carlo estimator is constructed to evaluate this density at arbitrary sample locations, allowing for a mesh-free approach. The performance of this auxiliary loss is compared against traditional loss functions like FFL and Sobolev supervision under various sampling conditions.
Results
The proposed crossing-density loss shows transformative effects in scenarios with scarce supervision, achieving improvements of 2.3-3.0 dB over MSE-only training on PE-MLP and 1.4-1.8 dB on SIREN. It matches the performance of FFL on edge-dominated natural images and outperforms it by 0.6 dB on statistically homogeneous textures. The loss is uniquely insensitive to the quality of gradient targets, making it a robust choice for training INRs.
Implications
The findings suggest that the crossing-density loss can be a valuable tool for improving the training of implicit neural representations, particularly in applications where supervision is limited or irregular. This could enhance the quality of generated images and signals in various domains, including computer graphics and computer vision.
Reward Granularity in RLVR: Comparing Process and Outcome Reward Structures for Mathematical Reasoning in Small Language Models
NLP
Large Language Models
Reinforcement Learning
- Process-only rewards significantly improve accuracy and reasoning trace validity compared to outcome-only rewards.
- Hybrid reward configurations can yield conflicting optimization signals, particularly in low-process/high-outcome setups.
- Error analysis reveals different failure modes for process and outcome models, affecting the quality of reasoning outputs.
- The study emphasizes the importance of reward granularity as a critical design decision in RLVR frameworks.
Read more
Reward Granularity in RLVR: Comparing Process and Outcome Reward Structures for Mathematical Reasoning in Small Language Models
Summary
This paper explores the impact of reward granularity in Reinforcement Learning with Verifiable Rewards (RLVR) on the mathematical reasoning capabilities of small language models, specifically Qwen2.5-0.5B. The authors investigate the effectiveness of different reward structures: process-only, outcome-only, and hybrid combinations of both. The study reveals that process-only supervision significantly outperforms outcome-only supervision, achieving a test accuracy of 63.73% compared to 53.75%. Additionally, process rewards lead to higher validity in reasoning traces and lower deviations from the ground-truth chain length. The authors conduct a thorough analysis of five reward conditions using Group Relative Policy Optimization (GRPO) on the GSM8K dataset, which is well-suited for evaluating stepwise reasoning. They also identify an anomaly in hybrid reward configurations, where a low-process/high-outcome setting underperforms compared to pure outcome supervision, indicating conflicting optimization signals. Error analysis using GPT-4o highlights distinct failure modes for each reward regime, emphasizing the importance of reward granularity in enhancing reasoning quality in small language models.
Methodology
The authors conducted a systematic comparison of five reward conditions applied to the Qwen2.5-0.5B model fine-tuned with Group Relative Policy Optimization (GRPO) on the GSM8K dataset. They evaluated the models based on test accuracy, reasoning trace validity, chain-length deviation, and process step ratio. An error analysis was performed using GPT-4o to characterize failure modes across different reward regimes.
Results
The results showed that process-only supervision achieved a test accuracy of 63.73%, significantly higher than the 53.75% accuracy of outcome-only supervision. Process rewards also led to higher step validity and lower deviations from the ground-truth chain length. Hybrid rewards generally correlated positively with process weight, but the low-process/high-outcome configuration (λ=0.1) underperformed compared to pure outcome supervision.
Implications
The findings suggest that incorporating process-level supervision in RLVR can substantially enhance the reasoning capabilities of small language models, which may have broader applications in educational tools, automated reasoning systems, and AI-assisted problem-solving.
Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependence
Time Series
- TSFMs demonstrate strong performance in electricity price forecasting, often surpassing general-purpose models.
- The effectiveness of TSFMs is significantly influenced by the availability of covariate information.
- A two-dataset benchmarking framework is proposed to mitigate contamination risks in model evaluation.
- Ensemble methods combining TSFMs and domain-specific approaches may yield improved forecasting results.
Read more
Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependence
Summary
This paper investigates the performance of Time Series Foundation Models (TSFMs) in the context of electricity price forecasting (EPF), a domain characterized by complex temporal dependencies and non-stationarity. The authors propose a two-dataset benchmarking framework to address contamination risks and facilitate a fair evaluation of TSFMs. They assess various aspects of EPF, including point and probabilistic forecasting performance, tail behavior, and price spikes, while comparing TSFMs against both general-purpose and domain-specific methods. The findings reveal that TSFMs are competitive and often outperform general-purpose baselines, although their effectiveness is highly dependent on the availability of covariate support. Furthermore, simple ensembles of TSFMs and domain-specific methods show promise, indicating that these approaches can capture complementary predictive information. The study emphasizes the need for a dedicated benchmarking framework to evaluate the unique characteristics of electricity pricing signals and the role of exogenous covariates in enhancing forecasting accuracy.
Methodology
The authors developed a two-dataset benchmarking framework to evaluate TSFMs in EPF. They benchmarked four TSFMs against classical time series models, deep learning approaches, and domain-specific methods. The evaluation focused on point and probabilistic forecasting, tail behavior, and robustness to price spikes, while addressing contamination risks associated with pretraining data.
Results
The results indicate that TSFMs are highly competitive in EPF, often outperforming general-purpose baselines. However, their performance is contingent on the presence of relevant covariates. Additionally, simple ensembles of TSFMs and domain-specific methods show potential for enhanced forecasting accuracy.
Implications
The findings suggest that TSFMs can be effectively utilized in electricity price forecasting, but their integration with domain-specific methods may lead to better performance. The proposed benchmarking framework can serve as a valuable tool for future research in time series forecasting, particularly in non-stationary environments.
Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion
Graph Learning
Theory
Efficient ML
- Linear attention is suboptimal for graph denoising due to its averaging of spectral properties.
- Spectral Attention provides a theoretical framework that outperforms linear attention based on spectral diversity.
- Graph Convolutional Attention (GCA) is introduced as a permutation-equivariant mechanism that implements spectral denoising effectively.
- The softmax operation enhances denoising by approximating projections onto clean eigenspaces.
Read more
Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion
Summary
This paper addresses the challenge of graph denoising, a critical task in graph learning and diffusion models. The authors critique existing attention-based architectures, particularly linear attention, which they find suboptimal for denoising due to its inability to adapt to varying spectral properties across graphs. To overcome this limitation, they introduce Spectral Attention, which leverages the input graph spectrum, and propose Graph Convolutional Attention (GCA) as a practical implementation that maintains permutation equivariance. GCA is shown to match the performance of the idealized Spectral Attention in stochastic block models. The authors also analyze the role of the softmax operation in enhancing denoising by projecting noisy eigenvectors onto a clean eigenspace. Empirical results demonstrate that GCA consistently improves graph denoising and diffusion across various datasets, with performance gains correlating with spectral diversity. Additionally, GCA integrates seamlessly into existing models like DiGress, enhancing efficiency by avoiding costly computations associated with structural features.
Methodology
The authors analyze linear attention's limitations in graph denoising, introduce Spectral Attention as a more effective alternative, and develop Graph Convolutional Attention (GCA) that operates as a graph convolutional filter. They also investigate the softmax operation's role in denoising. Empirical validation is conducted on synthetic and real datasets to demonstrate the effectiveness of GCA compared to standard attention mechanisms.
Results
The introduction of GCA leads to consistent improvements in graph denoising and diffusion performance across various datasets. The gains are strongly correlated with the spectral diversity of the graphs. GCA matches the performance of existing models like DiGress without the need for expensive structural feature computations, resulting in faster inference times.
Implications
The findings suggest that GCA can be effectively integrated into modern graph learning frameworks, enhancing both performance and efficiency. This has potential applications in various domains where graph data is prevalent, such as social networks, biological networks, and recommendation systems.
DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics
Multimodal
Computer Vision
Large Language Models
- DynaVieW enhances visual dynamic understanding in multimodal LLMs by learning interleaved state-transition sequences.
- The model employs a mixture-of-experts architecture with selective attention for robust learning.
- DynaVieW achieves superior performance in visual narrative generation and world simulation tasks.
- The schema-guided approach allows for better controllability and consistency in generated visual outputs.
Read more
DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics
Summary
DynaVieW addresses the limitations of multimodal large language models (LLMs) in understanding and simulating visual dynamics in videos and multi-image sequences. The proposed model utilizes a dynamic schema-guided world model to learn interleaved state-transition sequences, where states represent visual scenes and transitions capture hierarchical dynamic constituents. DynaVieW employs a mixture-of-experts architecture with selective attention mechanisms and a schema token re-weighted loss to enhance learning efficiency. The model is pre-trained on diverse datasets that encompass a wide range of visual dynamics, allowing it to predict transitions and simulate visual states effectively. DynaVieW demonstrates improved performance in visual narrative creation and world simulation tasks, achieving greater consistency, controllability, and adherence to instructions compared to baseline models. This advancement signifies a step forward in the capability of LLMs to understand and generate complex visual narratives.
Methodology
DynaVieW utilizes a mixture-of-Transformer-experts architecture to learn interleaved state-transition sequences from keyframes of videos. It incorporates a cross-expert selective attention mechanism and a schema token re-weighted loss function to balance learning between transition schemas and specific slot values. The model is pre-trained on a comprehensive dataset that includes various human activities and dynamic scenes.
Results
DynaVieW outperforms baseline models in visual narrative generation, achieving higher consistency and quality in outputs. It demonstrates improved controllability when generating narratives based on varying scene descriptions and excels in world simulation tasks, showing better instruction-following capabilities.
Implications
The advancements made by DynaVieW could lead to more reliable and controllable multimodal applications, such as enhanced visual storytelling, interactive simulations, and improved human-computer interaction in visual contexts.
InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programs
Theory
- INVWEAVER effectively synthesizes invariants for multi-loop programs, overcoming limitations of previous methods focused on single loops.
- The framework utilizes a loop-level call graph to manage inter-loop dependencies and enhance context-aware reasoning.
- A WP-guided refinement mechanism allows for the propagation of proof obligations, improving the accuracy of synthesized invariants.
- Experimental results show INVWEAVER solves 72 out of 82 multi-loop benchmark problems, outperforming competitors by a significant margin.
Read more
InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programs
Summary
This paper introduces INVWEAVER, a novel neuro-symbolic framework designed to synthesize loop invariants for multi-loop programs, addressing the limitations of existing methods that primarily excel with single-loop structures. The authors highlight the challenges posed by multi-loop programs, which are common in real-world applications but often lead to local under-specification issues in invariant inference. INVWEAVER leverages a loop-level call graph (LCG) to expose inter-loop dependencies and employs a bi-directional, top-down inference strategy using large language models (LLMs). This approach allows for a context-aware reasoning process that captures global constraints across loops. Additionally, a weakest precondition (WP)-guided refinement mechanism is introduced to propagate proof obligations across loop boundaries, enhancing the robustness of the invariant synthesis. The framework is evaluated on a comprehensive benchmark suite, including a newly curated dataset derived from classic algorithms, demonstrating significant improvements over existing state-of-the-art methods.
Methodology
INVWEAVER employs a neuro-symbolic approach, integrating a loop-level call graph for inter-loop dependency management with a bi-directional, top-down inference strategy using LLMs. It incorporates a WP-guided refinement mechanism to propagate proof obligations across loops, ensuring a coherent inductive proof chain.
Results
INVWEAVER successfully solved 72 out of 82 multi-loop benchmark problems, outperforming the strongest competitor by 32 problems. It also maintained superior performance on single-loop tasks, demonstrating its effectiveness across various program structures.
Implications
The advancements made by INVWEAVER in invariant synthesis can significantly enhance program verification processes, particularly for complex algorithms in software development. This framework could be applied in various domains requiring rigorous correctness proofs, such as safety-critical systems and automated program analysis.
Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks
NLP
Large Language Models
Efficient ML
- Introduction of Tiny Language Models (TLMs) for consumer device deployment.
- Classification-head fine-tuning significantly outperforms label generation methods.
- TLMs can achieve state-of-the-art performance on multiple benchmarks despite their smaller size.
- The study highlights the importance of fine-tuning approaches in maximizing TLM performance.
Read more
Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks
Summary
This paper introduces the concept of Tiny Language Models (TLMs), defined as models with fewer than 3 billion parameters that can be effectively deployed on mainstream consumer devices. The authors investigate the adaptation of TLMs for verifiable multiple-choice tasks, focusing on commonsense reasoning. They compare three fine-tuning paradigms using LoRA: label generation, gold only, and a novel discriminative classification head. The study employs a unified setup across various Qwen3 models (0.6B to 8B parameters) and evaluates performance on five benchmarks: HellaSwag, WinoGrande, PIQA, SciQ, and ARC-C. The findings reveal that classification-head fine-tuning consistently outperforms label generation, particularly at smaller model sizes, and that TLMs fine-tuned with the discriminative method achieve competitive performance compared to much larger models like GPT-3, PaLM, and GPT-4. The results challenge the notion that TLMs are inherently inadequate for commonsense reasoning tasks, suggesting that previous performance assessments may stem from suboptimal fine-tuning practices.
Methodology
The authors conducted a systematic comparison of three fine-tuning paradigms (label generation, gold only, and discriminative classification head) on TLMs across various model sizes and benchmarks. They utilized LoRA-based fine-tuning techniques and evaluated the models on five well-established commonsense reasoning tasks.
Results
The study found that TLMs fine-tuned with the classification head outperformed those fine-tuned with label generation by 2-3% at the 0.6B and 1.7B scales. Additionally, TLMs fine-tuned using the discriminative method demonstrated competitive performance against larger models like GPT-3, PaLM, and GPT-4, achieving state-of-the-art results on HellaSwag, WinoGrande, and PIQA.
Implications
The findings suggest that TLMs can be effectively utilized for commonsense reasoning tasks on consumer devices, potentially broadening the accessibility and application of language models in various practical scenarios. This research encourages further exploration of fine-tuning strategies and evaluation methods for smaller models.
PDEFlow: Autonomous Agentic PDE Pipelines for Neural Operator Learning and Solver-Free Inference
Optimization
Theory
Efficient ML
- PDEFlow automates the conversion of user-defined PDEs into neural operator pipelines.
- The framework integrates problem specification, data generation, training, and inference in a single workflow.
- Utilizes a stateful input graph to handle user inputs and modifications effectively.
- Demonstrates the ability to generate solver-backed datasets and perform solver-free inference.
Read more
PDEFlow: Autonomous Agentic PDE Pipelines for Neural Operator Learning and Solver-Free Inference
Summary
PDEFlow is introduced as an autonomous framework designed to streamline the process of transforming user-defined ordinary and partial differential equations (ODEs and PDEs) into neural operator pipelines that are solver-backed. The framework encompasses a comprehensive workflow that integrates problem specification, data generation, operator training, and checkpoint-based inference. A key feature of PDEFlow is its stateful input graph, which processes multi-turn natural language inputs and user modifications into validated problem specifications. The data generation module samples parameters and utilizes the FEniCSx finite-element backend to solve the governing equations, storing the results as tensors ready for operator training. The training and inference stages utilize a registry-based interface, allowing for the flexible training and deployment of various neural operators without altering the pipeline. The current implementation employs a multi-branch Bayesian DeepONet. Experimental results demonstrate PDEFlow's capability to construct valid specifications, generate datasets, train neural operators across various problem classes, and deliver solver-free predictions from saved checkpoints. This framework aims to facilitate repeatable scientific and engineering workflows, minimizing manual intervention while ensuring accuracy in simulations.
Methodology
PDEFlow employs a stateful input graph to process natural language inputs into validated JSON specifications. It utilizes a data generation module that samples parameters and solves PDEs using the FEniCSx backend, generating operator-ready datasets. The framework incorporates a registry-based interface for training and deploying neural operators, specifically implementing a multi-branch Bayesian DeepONet for the current experiments.
Results
The experiments conducted on benchmark ODE and PDE tasks confirmed that PDEFlow can effectively construct valid problem specifications, generate datasets, train neural operators, and provide accurate solver-free predictions from previously saved checkpoints. The framework demonstrated robustness across various steady and transient problem classes.
Implications
PDEFlow has significant implications for scientific and engineering workflows, particularly in scenarios requiring repeated evaluations of governing physics under varying conditions. It simplifies the specification and simulation processes, making it easier for researchers to explore large design spaces with minimal manual intervention, thereby enhancing productivity and reliability in computational tasks.
How Far is Too Far? Defining the Distance Threshold for Verification Siamese Networks
Computer Vision
- Proposes an unsupervised method for estimating verification thresholds in Siamese networks.
- Models embedding distances as a bimodal function to identify the optimal threshold.
- Eliminates the need for labeled data, allowing for dynamic updates in deployment.
- Achieves an average verification accuracy of 94% across multiple datasets.
Read more
How Far is Too Far? Defining the Distance Threshold for Verification Siamese Networks
Summary
This paper addresses the challenge of defining a distance threshold for Siamese verification networks, which are commonly used for comparing items like faces and signatures. The authors propose an unsupervised method to determine this verification threshold by modeling the distribution of distances produced by the network as a bimodal function. The first mode corresponds to pairs of similar items, while the second mode corresponds to dissimilar items. The threshold is defined as the minimum point between these two modes, allowing for dynamic updates in deployment without requiring labeled data. The method is evaluated on four datasets: MNIST, CIFAR-10, LFW, and PKLot, demonstrating an average verification accuracy of 94%, which is comparable to traditional methods that rely on labeled data, such as the Equal Error Rate method. This approach not only simplifies the threshold-setting process but also enhances the adaptability of Siamese networks in real-world applications.
Methodology
The authors model the distribution of distances from a Siamese verification network as a bimodal function. They identify the threshold as the minimum point between the two modes of this distribution, which represent similar and dissimilar item pairs. This method operates without labeled data, allowing for threshold updates based on test data.
Results
The proposed method was tested on four datasets: MNIST, CIFAR-10, LFW, and PKLot, achieving an average verification accuracy of 94%. This performance is comparable to traditional methods that require labeled data, such as the Equal Error Rate approach.
Implications
The findings suggest that the proposed unsupervised method can significantly improve the deployment of Siamese networks in various verification tasks, such as face recognition and signature verification, by allowing for real-time updates to the verification threshold without the need for manual labeling.
Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
Large Language Models
Reinforcement Learning
- The execution harness of LLM agents is proposed as a learnable control layer.
- A Harness MDP framework is introduced to formalize the control decisions made by a lightweight controller.
- The methodology employs offline reinforcement learning with advantage-weighted regression, focusing on terminal rewards.
- A distinction is made between final task quality and harness process quality, with implications for learning behavior.
Read more
Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
Summary
This paper introduces a novel approach to enhance the performance of large language model (LLM) agents by treating the execution harness as a learnable control layer. The authors formalize this harness operation as a finite-horizon Harness Markov Decision Process (MDP), where a lightweight controller selects structural execution actions while the LLM executor remains unchanged. The controller is trained using offline rollouts through advantage-weighted regression, focusing on terminal task-rubric rewards. A key contribution is the separation of final task quality from a post-hoc Harness Maturity Score (HMS), which evaluates the reliability of execution patterns. The study demonstrates that while final task quality improvements require high-return support in the offline buffer, process behavior can adapt more readily based on advantage weights. Empirical evaluations across six controlled domains and two public-benchmark adapters reveal that the learned controller consistently enhances verification behavior and selectively improves final quality, particularly in specific tasks. The results indicate that harness control can be effectively learned, suggesting significant implications for the design of LLM agents.
Methodology
The authors define a Harness MDP where a controller selects structural actions based on the current trajectory state. The controller is trained using offline rollouts and advantage-weighted regression, focusing on terminal task-rubric rewards without updating the LLM parameters. The study evaluates the harness process quality separately from final task quality using the Harness Maturity Score (HMS).
Results
The learned controller significantly improves verification behavior before submission in all evaluation settings. Aggregate HMS scores improved in five out of six controlled domains and both benchmark evaluations, primarily driven by specific process behaviors like 'CheckBeforeSubmit'. Final task quality improvements were observed selectively, with notable gains in coding tasks and specific benchmark adapters.
Implications
This research suggests that harness control can be a critical component in the design of LLM agents, allowing for more adaptive and efficient execution strategies. The findings could influence future developments in agent-based systems, particularly in enhancing their reliability and performance in complex tasks.
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Optimization
Theory
Efficient ML
- EISAM improves upon SAM by incorporating an extragradient-inspired two-step update process.
- The optimizer enhances generalization performance and reduces sensitivity to hyperparameters.
- Extensive experiments show EISAM consistently outperforms traditional optimizers like SGD and Adam.
- Theoretical analysis confirms that EISAM achieves flatter minima, leading to better generalization.
Read more
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Summary
This paper addresses the challenge of generalization in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting. The authors propose a novel optimizer called Extragradient-Inspired Sharpness-Aware Minimization (EISAM), which builds on Sharpness-Aware Minimization (SAM) to seek flatter minima associated with improved generalization. EISAM employs a two-step update process: a prediction step that explores the geometry of the loss landscape and a perturbation step that refines updates using a base optimizer. This approach not only enhances generalization performance compared to SAM but also reduces sensitivity to the perturbation radius, making it more robust and easier to tune across different settings. The authors conduct extensive experiments on benchmark datasets, demonstrating that EISAM outperforms SGD, Adam, and SAM in terms of test accuracy and training efficiency across various architectures. Theoretical analysis supports the efficacy of EISAM by showing that it tightens the generalization bound by guiding parameters toward flatter minima with reduced curvature. The paper provides practical hyperparameter tuning guidance, establishing EISAM as a scalable and broadly applicable optimization solution that advances both theoretical and practical aspects of deep learning.
Methodology
EISAM utilizes a two-step update mechanism: a prediction step that investigates the local geometry of the loss landscape followed by a perturbation step that refines updates using a base optimizer. This method is inspired by the extragradient technique, which allows for more robust optimization trajectories.
Results
EISAM demonstrated superior performance in terms of test accuracy and training efficiency on various benchmark datasets, including CIFAR-10, CIFAR-100, ImageNet-1K, and others, outperforming SGD, Adam, and SAM. The theoretical analysis indicated that EISAM effectively tightens the generalization bound by steering parameters toward flatter minima.
Implications
EISAM offers a robust and efficient optimization tool for deep learning practitioners, balancing performance and computational efficiency. Its design allows for broader applicability across various machine learning tasks, potentially improving generalization in real-world applications.
Heterogeneous Graph Condensation via Role-Aware Clustering
Graph Learning
Efficient ML
Optimization
- HGC-RC is designed specifically for heterogeneous graphs, addressing the limitations of existing condensation methods.
- The framework utilizes role-aware clustering to differentiate between target and non-target nodes, enhancing classification performance.
- HGC-RC achieves high compression rates while maintaining task performance, outperforming optimization-heavy baselines.
- The method is computationally efficient, avoiding expensive iterative optimization processes.
Read more
Heterogeneous Graph Condensation via Role-Aware Clustering
Summary
The paper presents HGC-RC, a novel framework for heterogeneous graph condensation that addresses the challenges of training Heterogeneous Graph Neural Networks (HGNNs) on large-scale heterogeneous graphs. Traditional graph condensation methods are primarily designed for homogeneous graphs and often rely on complex optimization techniques, which are not suitable for heterogeneous settings. HGC-RC introduces a role-aware approach that recognizes the differing roles of target and non-target nodes in preserving classification utility. The framework first generates semantically enhanced node embeddings through lightweight propagation. It then employs a hybrid clustering strategy: class-partitioned clustering for labeled target nodes to maintain class distributions, and type-wise unsupervised clustering for non-target nodes to preserve essential cross-type connectivity. Finally, a compact heterogeneous graph is reconstructed based on the cluster assignments. Experimental results demonstrate that HGC-RC significantly outperforms state-of-the-art methods, achieving competitive performance with reduced computational costs, thus providing an efficient pathway for HGNN training on large heterogeneous graphs.
Methodology
HGC-RC employs a role-aware framework for heterogeneous graph condensation. It begins with the extraction of semantic embeddings using SeHGNN preprocessing. The framework then applies class-partitioned clustering for target nodes and type-wise unsupervised clustering for non-target nodes. Based on the resulting cluster assignments, it aggregates original node features and reconstructs a compact heterogeneous graph through inter-cluster connectivity summarization.
Results
The experimental evaluation on three heterogeneous benchmarks shows that HGC-RC achieves competitive performance with significantly lower condensation costs compared to existing optimization-based methods. The results indicate that HGC-RC can effectively condense heterogeneous graphs while preserving essential structural and semantic information.
Implications
HGC-RC provides a practical solution for accelerating the training of HGNNs on large-scale heterogeneous graphs, making it applicable in various domains such as social networks, recommendation systems, and knowledge graphs. The role-aware approach can also inspire future research in graph learning and optimization techniques.
GraphBU: MILP Instance Generation with Graph-Native Block Units
Optimization
Graph Learning
Theory
- GraphBU is the first graph-native block unit generator for MILP instance generation.
- It preserves the structural properties of the original MILP instances, enhancing feasibility and statistical similarity.
- The methodology includes interface detection and compatibility-checked replacement of block units.
- The generated instances improve the performance of downstream learning-based solvers.
Read more
GraphBU: MILP Instance Generation with Graph-Native Block Units
Summary
The paper introduces GraphBU, a novel generator for Mixed-Integer Linear Programming (MILP) instances that addresses the challenges of generating representative instances for solver development. Traditional instance generators often fail to maintain the structural integrity required by solvers, leading to issues in feasibility and performance. GraphBU innovatively employs graph-native block units that consist of local subproblems and their interfaces, allowing for better coupling of constraints and variables. The methodology involves identifying and promoting nodes that act as interfaces, ensuring that the generated instances closely resemble the source family in terms of graph statistics and feasibility. The authors provide theoretical guarantees for the construction of these block units, including interface separation and invariance to row-column permutations. Experimental results demonstrate that GraphBU-generated instances achieve an average graph-statistical similarity of approximately 0.934 and maintain a feasibility rate of around 96.7%. Furthermore, the generated data enhances downstream Predict-and-Search training, indicating its utility in improving solver performance.
Methodology
GraphBU represents MILP as a constraint-variable bipartite graph, identifies coupling nodes, and decomposes the graph into local subproblems with explicit interfaces. The generator replaces target units with source units based on compatibility checks regarding shape, interface dimensions, and metadata.
Results
GraphBU-generated instances exhibited an average graph-statistical similarity of 0.934 and a feasibility rate of 96.7%. Additionally, the generated data led to an approximate 8.0% improvement in the main index of downstream Predict-and-Search training.
Implications
The development of GraphBU has significant implications for the generation of MILP instances, particularly in enhancing the training of learning-based solvers and improving the efficiency of solver development processes. It could facilitate better benchmarking and testing of optimization algorithms in various applications such as logistics, scheduling, and planning.
Intuitionistic Fuzzy Graph Embedded Random Vector Functional Link with Multiview Learning
Graph Learning
Theory
Efficient ML
- Introduction of intuitionistic fuzzy sets into the RVFL framework for better uncertainty handling.
- Incorporation of graph embedding to preserve geometric relationships within data.
- Utilization of multiview learning to combine information from multiple feature sets.
- Statistical analyses confirm significant improvements in classification performance over existing models.
Read more
Intuitionistic Fuzzy Graph Embedded Random Vector Functional Link with Multiview Learning
Summary
The paper presents the Intuitionistic Fuzzy Graph Embedded Random Vector Functional Link with Multiview Learning (IFGRVFL-MV) model, which aims to enhance the capabilities of Random Vector Functional Link (RVFL) networks by addressing their limitations in preserving geometric relationships and effectively utilizing multiple feature views. The proposed model integrates three key components: intuitionistic fuzzy sets for managing uncertainty, graph embedding to capture intrinsic geometric structures, and multiview learning to leverage complementary information from various feature spaces. By assigning intuitionistic fuzzy membership and non-membership values to data points, the model demonstrates robustness against outliers. The graph embedding framework helps maintain topological structures, thereby improving generalization performance. Experimental evaluations on benchmark datasets from UCI and KEEL repositories indicate that IFGRVFL-MV significantly outperforms existing models in classification accuracy, establishing it as a promising advancement in handling uncertainty and multiview environments.
Methodology
The IFGRVFL-MV model combines intuitionistic fuzzy logic, graph embedding, and multiview learning. It assigns membership and non-membership values based on the distance from class centers and neighbor heterogeneity. The graph embedding framework captures geometric relationships by defining intrinsic and penalty graphs over concatenated feature matrices. Multiview learning is employed to integrate information from different feature sets, enhancing the model's ability to capture complex patterns.
Results
The experiments conducted on benchmark datasets revealed that the IFGRVFL-MV model outperformed existing classification models, achieving statistically significant improvements in accuracy. The results were validated through various statistical tests, including Friedman, Wilcoxon, and win-tie-loss tests, confirming the model's superiority.
Implications
The IFGRVFL-MV model has potential applications in various fields requiring robust classification in noisy environments, such as healthcare, image classification, and forecasting. Its ability to handle uncertainty and leverage multiple feature views can lead to improved decision-making processes in complex data scenarios.
$\mathbf{\lambda}$-VAE: Variance Equalization for Posterior Collapse
Generative Models
Theory
Computer Vision
- Identifies two causes of posterior collapse in VAEs: gradient imbalance and information gap.
- Proposes λ-VAE, which modifies the reparameterization step to achieve variance equalization.
- Demonstrates significant reductions in collapsed dimensions and improvements in reconstruction quality across multiple datasets.
- Establishes a closed-form optimal exponent for scaling noise, controlled by a single hyperparameter.
Read more
$\mathbf{\lambda}$-VAE: Variance Equalization for Posterior Collapse
Summary
This paper addresses the issue of posterior collapse in Variational Autoencoders (VAEs), where the approximate posterior converges to the prior, leading to uninformative latent codes. The author identifies two independent yet coupled causes of this phenomenon: gradient imbalance and information gap. Gradient imbalance occurs when the decoder's reconstruction signal diminishes faster than the KL regularization as the posterior widens, while the information gap arises from stochastic sampling that discards significant encoder information, making collapse easier. The paper introduces the λ-VAE, which modifies the reparameterization step by scaling the sampling noise per dimension while retaining the KL penalty on the original posterior variance. This approach, termed variance equalization, helps shift the training dynamics away from the collapsed state. The method is validated on standard datasets, demonstrating significant improvements in reconstruction quality and information capacity, effectively reducing the number of collapsed dimensions.
Methodology
The paper formalizes the causes of posterior collapse and introduces λ-VAE, which adjusts the reparameterization noise while keeping the KL penalty intact. This asymmetry is designed to equalize variance across latent dimensions, thus preventing collapse. The approach is validated through experiments on various benchmark datasets, including Binary MNIST, Binary Omniglot, CIFAR-10, and CelebA-64.
Results
The λ-VAE significantly reduced the number of collapsed dimensions from 16 to 1 on Binary MNIST and from 13 to 0 on Binary Omniglot. For RGB images, it achieved an information capacity increase of up to 2.8× nats and improved the BPD by +0.33 on CIFAR-10. The findings indicate that the model's performance can vary significantly even with similar BPD scores, highlighting the importance of latent code utilization.
Implications
The findings suggest that addressing the causes of posterior collapse can enhance the performance of VAEs in various applications, including image generation and representation learning. The λ-VAE framework may lead to more robust generative models that maintain informative latent representations, which could be beneficial in fields such as computer vision and natural language processing.
SplineNet: An Isogeometric Deep Learning Method for Complex Shells
Theory
Interpretability
Optimization
- Introduction of SplineNet for integrating CAD and CAE in deep learning frameworks.
- Utilization of watertight spline representations for exact geometric descriptions.
- Ability to operate in both data-free and data-driven modes for enhanced flexibility.
- Demonstration of effectiveness through numerical examples involving complex geometries.
Read more
SplineNet: An Isogeometric Deep Learning Method for Complex Shells
Summary
This paper introduces SplineNet, a novel isogeometric deep learning method designed for the seamless design and analysis of shell structures with complex geometries. SplineNet utilizes watertight spline representations, specifically analysis-suitable unstructured T-splines, to provide exact geometric descriptions of CAD models within neural networks. The architecture employs B´ezier extraction, with Bernstein polynomials serving as nonlinear activation functions. SplineNet can operate in both data-free and data-driven modes. In the data-free scenario, energy-based formulations are integrated as loss terms, allowing for accurate calculations of mechanical behaviors using the Kirchhoff–Love model. This integration facilitates a direct coupling of CAD and CAE processes within a deep neural network, eliminating the need for time-consuming model/data exchanges. In the data-driven approach, SplineNet functions as the trunk net of Deep Operator Networks (DeepONet), enabling immediate predictions for unseen input data without retraining. The effectiveness of SplineNet is demonstrated through various numerical examples, particularly highlighting its performance with real-world complex geometries.
Methodology
SplineNet employs watertight spline representations and B´ezier extraction to construct its neural network architecture. It incorporates both data-free and data-driven approaches, utilizing energy-based formulations and the Kirchhoff–Love model for mechanical analysis. The network architecture leverages Bernstein polynomials as activation functions, facilitating the integration of CAD and CAE processes.
Results
The results indicate that SplineNet effectively bridges the gap between CAD and CAE, allowing for accurate predictions of mechanical behaviors in shell structures with complex geometries. The numerical examples provided demonstrate the method's capability to handle real-world applications without the need for extensive retraining or data exchange.
Implications
SplineNet has significant implications for engineering design, particularly in fields requiring high geometric fidelity and efficient numerical analysis. Its ability to integrate CAD and CAE processes can streamline workflows in structural engineering, potentially reducing computational costs and improving design optimization processes.
AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference
NLP
Large Language Models
Efficient ML
- AdaptiveSD integrates a closed-loop control architecture for adaptive speculative decoding.
- The framework prioritizes resource preservation over raw throughput, addressing the limitations of fixed draft depths.
- Achieves 68-82% speculative efficiency while maintaining low levels of wasted compute.
- Effectively manages latency variance and resource usage, preventing system failures.
Read more
AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference
Summary
The paper introduces AdaptiveSD, a runtime-adaptive speculative decoding framework designed to enhance the performance of quantized GGUF-based language models during inference on CPU-constrained hardware. Traditional speculative decoding methods often suffer from performance degradation due to fixed draft depths, which do not adapt to varying workload conditions. AdaptiveSD addresses this issue by integrating four components: a Runtime Monitoring Engine that tracks various computational signals, an Adaptive Draft Controller that prioritizes resource preservation, a Dynamic Policy Engine that adjusts policies based on workload behavior, and a KV Cache Coordination Layer that manages cache states effectively. The framework aims to maintain a balance between throughput and resource utilization, ensuring reliable execution without exceeding safety margins for CPU usage and memory bandwidth. Experimental results demonstrate that AdaptiveSD achieves a speculative efficiency of 68-82%, keeping wasted compute below 32% and effectively managing latency variance spikes, thus providing a robust solution for on-device inference tasks.
Methodology
The methodology involves a closed-loop control architecture comprising four components: a Runtime Monitoring Engine for tracking computational metrics, an Adaptive Draft Controller that enforces a hierarchy of policies, a Dynamic Policy Engine that utilizes heuristic and reinforcement learning techniques for policy adjustment, and a KV Cache Coordination Layer for managing cache states with fine-grained control.
Results
AdaptiveSD consistently achieves a speculative efficiency of 68-82%, with wasted compute levels maintained below 32% of drafted tokens. The framework successfully clamps down on potential latency variance spikes and manages CPU usage and memory bandwidth within predefined safety margins.
Implications
The proposed framework has significant implications for deploying language models in resource-constrained environments, particularly for on-device AI applications. It enhances the reliability and efficiency of inference tasks, making it suitable for edge computing scenarios where memory bandwidth is limited.
Physics-Informed Graph Learning with Uncertainty Awareness for Open-Set Domain Generalization in Fault Diagnosis
Graph Learning
Time Series
Optimization
- Introduction of PGU-OD framework for open-set domain generalization in fault diagnosis.
- Development of PISA-Net for robust feature extraction that addresses frequency shifts.
- Implementation of an uncertainty-aware adaptive graph learning mechanism to manage information propagation.
- Creation of a dual-criteria decision-making strategy for effective unknown fault rejection.
Read more
Physics-Informed Graph Learning with Uncertainty Awareness for Open-Set Domain Generalization in Fault Diagnosis
Summary
This paper addresses the challenges of fault diagnosis in rotating machinery, particularly under conditions of unknown fault types and domain shifts, which are framed as the open-set domain generalization (OSDG) problem. The authors propose a novel framework, PGU-OD, which integrates physics-informed graph learning with uncertainty awareness. The framework consists of three main components: a physics-informed feature extraction module (PISA-Net) that utilizes spectral attention to extract robust fault features while mitigating perceptual uncertainty from frequency shifts; an uncertainty-aware adaptive graph learning mechanism that adjusts edge weights based on class-scale Gaussian distribution parameters to reduce the propagation of uncertainty; and a Gaussian-distribution-based adaptive boundary loss function combined with a dual-criteria open-set inference strategy to optimize decision boundaries and effectively reject unknown faults. The proposed method is evaluated on two public datasets, demonstrating superior performance compared to state-of-the-art baselines in both known fault classification and unknown fault rejection under varying operational conditions.
Methodology
The PGU-OD framework employs an end-to-end probabilistic diagnostic approach that includes a physics-informed feature extraction module (PISA-Net) for robust feature representation, an adaptive graph learning mechanism that incorporates uncertainty awareness, and a Gaussian-distribution-based loss function for decision boundary optimization. The architecture is designed to mitigate uncertainty propagation across all stages of the fault diagnosis process.
Results
Extensive experiments on two widely used rotating machinery fault datasets show that PGU-OD outperforms existing state-of-the-art methods in both known fault classification and the rejection of unknown faults, particularly under conditions of domain shifts.
Implications
The proposed framework has significant implications for intelligent industrial maintenance systems, enhancing the reliability of fault diagnosis in dynamic environments where unknown fault types and varying operational conditions are prevalent. This could lead to improved maintenance strategies and reduced downtime in industrial applications.
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Multimodal
Large Language Models
Efficient ML
- OmniFocus mitigates modality bias in token compression by using query-guided importance estimation for both audio and video.
- The method preserves inter-modal alignment and modality-specific salient evidence, enhancing performance in audio-visual tasks.
- Experimental results show that OmniFocus outperforms existing token compression methods, achieving favorable accuracy-efficiency trade-offs.
Read more
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Summary
The paper introduces OmniFocus, a novel query-guided token compression method designed for Omni-modal Large Language Models (OmniLLMs) that process audio and video inputs. Traditional token compression methods often rely on unimodal guidance, which can lead to modality bias and overlook the temporal locality of relevant evidence in audio-visual data. OmniFocus addresses these limitations by performing independent importance estimation for both audio and video modalities, allowing for a modality-symmetric compression approach. This method preserves salient evidence from each modality while maintaining inter-modal alignment. The authors conducted extensive experiments on the Qwen2.5-Omni model family across four audio-visual benchmarks, demonstrating that OmniFocus achieves superior performance at low token retention ratios compared to existing methods. Notably, at a 25% token retention rate on the DailyOmni benchmark, OmniFocus achieved an accuracy of 59.40 and provided significant speedups in inference time, highlighting its effectiveness in balancing accuracy and efficiency in audio-visual tasks.
Methodology
OmniFocus employs a training-free, query-guided approach to assess the importance of temporal chunks in audio and video inputs. It computes query-token similarity to determine the significance of different temporal segments, allowing for independent importance estimation for each modality. The method then selects retained tokens based on inter-modal associations and intra-modal peak scores, ensuring both audio-visual correspondence and the retention of salient evidence.
Results
In experiments on the Qwen2.5-Omni model family, OmniFocus demonstrated strong performance across four audio-visual benchmarks. At a 25% token retention rate on the DailyOmni benchmark, it achieved an accuracy of 59.40 and provided a 1.38× prefill speedup compared to the full-token baseline, indicating a significant improvement in efficiency without sacrificing accuracy.
Implications
The findings suggest that OmniFocus can enhance the practical application of OmniLLMs in resource-constrained environments by reducing inference costs while maintaining high performance in audio-visual tasks. This has potential implications for various applications, including real-time audio-visual understanding and interactive systems.
Exogenous Dropout: A Simple, Strong Baseline for Corruption-Robust Time Series Forecasting with Covariates
Time Series
- Exogenous dropout significantly improves robustness in time series forecasting models against corrupted covariates.
- The proposed method outperforms complex architectures designed for robustness, demonstrating that simpler interventions can be more effective.
- The study establishes a comprehensive benchmark for evaluating corruption robustness across multiple domains and corruption types.
- Architectural boundedness is shown to be unnecessary for achieving robustness, as unbounded models with dropout perform better.
Read more
Exogenous Dropout: A Simple, Strong Baseline for Corruption-Robust Time Series Forecasting with Covariates
Summary
This paper addresses the vulnerability of time series forecasting models that incorporate exogenous covariates, particularly when these covariates are corrupted. The authors demonstrate that existing state-of-the-art models suffer significant performance degradation (32-62%) when faced with corrupted inputs. In contrast to the trend of developing complex architectures to mitigate this issue, the authors propose a simple yet effective training intervention called 'exogenous dropout.' This technique involves randomly zeroing out entire exogenous channels during training, which enhances model robustness without sacrificing clean accuracy. The authors validate their approach across three domains—electricity prices, reservoir hydrology, and meteorology—using a comprehensive corruption-robustness benchmark. They find that models trained with exogenous dropout outperform a purpose-built bounded architecture (BoundEx) designed to handle corrupted covariates. The findings suggest that architectural complexity is not necessary for achieving robustness against exogenous corruption, as simpler models with dropout can achieve superior performance.
Methodology
The authors introduce exogenous dropout as a model-agnostic training intervention that randomly zeros out entire exogenous channels during training. They apply this method to five different forecasting architectures and evaluate their performance against a corruption-robustness benchmark that includes Gaussian noise, temporal misalignment, and missing channels. Additionally, they construct a bounded architecture (BoundEx) for comparative analysis.
Results
The application of exogenous dropout resulted in significant improvements in robustness across all tested architectures, with notable performance gains under various corruption scenarios. For instance, the dual-correlation network (DAG) trained with dropout achieved a +32% improvement under Gaussian noise, a +25% improvement under misalignment, and a +24% improvement under missing channels, while maintaining near-best clean accuracy. In contrast, the BoundEx architecture did not outperform DAG with dropout in any corruption scenario.
Implications
The findings suggest that simpler training interventions like exogenous dropout can serve as strong baselines for future research in corruption-robust time series forecasting. This could lead to more efficient model designs that prioritize robustness without the need for complex architectures, potentially impacting various applications in energy forecasting, meteorology, and supply chain management.
Decision-Focused Scenario Generation and Selection for Efficient and Robust Grid Dispatch
Generative Models
Optimization
- Introduces a decision-focused generative framework for scenario generation in DRO-based grid dispatch.
- Optimizes generated scenarios based on their impact on operational costs rather than just accuracy.
- Compatible with mainstream generative models, allowing for flexibility in application.
- Includes a differentiable scenario selector to enhance computational efficiency.
Read more
Decision-Focused Scenario Generation and Selection for Efficient and Robust Grid Dispatch
Summary
This paper addresses the challenges posed by increasing uncertainties in power systems due to flexible demand and renewable energy generation. It critiques traditional scenario generation methods used in distributionally robust optimization (DRO), which often focus on accuracy rather than operational cost efficiency. The authors propose a novel decision-focused generative framework that optimizes scenario generation based on their impact on downstream operational costs, rather than solely fitting historical uncertainty distributions. This framework is compatible with various generative models, including variational autoencoders, generative adversarial networks, and diffusion models, and it effectively captures the joint distribution of uncertainties across different buses in the power grid. Additionally, the authors introduce a differentiable scenario selector that identifies decision-relevant scenarios from a generated pool, enhancing computational efficiency. Through case studies, the proposed framework demonstrates a reduction in operational costs by 0.80% to 2.02% compared to traditional accuracy-oriented methods, showcasing its effectiveness in improving grid dispatch operations under uncertainty.
Methodology
The authors develop a decision-focused generative framework that optimizes scenario generation by considering the downstream operational costs. This framework is designed to work with various generative models, including VAEs, GANs, and diffusion models, and incorporates a differentiable scenario selector to streamline the selection of relevant scenarios.
Results
The proposed framework effectively reduces operational costs by 0.80% to 2.02% across different generative models when compared to conventional accuracy-oriented scenario generation methods, indicating its potential for enhancing efficiency in grid dispatch.
Implications
The findings suggest that integrating decision-focused approaches in scenario generation can lead to more robust and cost-effective power system operations, particularly in environments with high uncertainty due to renewable energy sources. This could influence future practices in energy management and optimization strategies.
Compressed Computation under $L^4$ Loss is likely Computation in Superposition
Theory
Optimization
- The paper introduces a toy model of compressed computation that operates under L4 loss, demonstrating computation in superposition.
- The trained network assigns sparse binary codewords to features, allowing for efficient decoding and recovery of performance.
- A three-scalar parameterization can approximate the network's output, showcasing the potential for simplified representations.
- The findings validate the use of L4 loss in eliciting computation in superposition, contrasting with traditional L2 loss.
Read more
Compressed Computation under $L^4$ Loss is likely Computation in Superposition
Summary
This paper investigates the concept of computation in superposition (CiS) within neural networks, specifically focusing on a compressed computation model. The authors propose that neural networks can compute more functions than they have neurons by leveraging superposition. They introduce a toy model consisting of a single-hidden-layer ReLU network with 50 neurons tasked with computing the ReLU of 100 sparse input features. By training this model under an L4 loss function, which emphasizes outlier errors more than the traditional L2 loss, the authors demonstrate that the network can effectively compute all features in superposition. They reverse-engineer the trained network, revealing that it assigns each feature a sparse binary codeword over neurons and decodes it using a pseudoinverse of the encoder. Notably, the authors find that a simple three-scalar parameterization can recover most of the network's performance, validating their approach by constructing equivalent networks using hand-designed codes. This work contributes to the understanding of how neural networks can utilize superposition for computation, providing insights into their representational capabilities.
Methodology
The authors trained a single-hidden-layer ReLU network with 50 neurons on a 100-dimensional sparse input dataset, using L4 loss instead of L2 loss. They employed the Adam optimizer with a learning rate of 0.01 and a batch size of 8,192 over 100,000 training steps. The network architecture was designed to compute the element-wise ReLU of the input, and the authors reverse-engineered the trained model to analyze its internal representations.
Results
The results indicate that the network trained under L4 loss successfully computes all input features in superposition. The reverse-engineering process revealed that the network uses sparse binary codewords for feature representation, and a three-scalar parameterization was able to recover approximately 1.1 times its loss. The validation of this approach was confirmed by constructing equivalent networks using hand-designed codes, which yielded similar performance.
Implications
This research has implications for understanding the computational capabilities of neural networks, particularly in scenarios where the number of functions to compute exceeds the number of neurons. It suggests that modifying loss functions can enhance the ability of networks to leverage superposition, potentially leading to more efficient architectures in various applications.
CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents
Reinforcement Learning
Large Language Models
- CompactionRL integrates context compaction into long-horizon reinforcement learning for LLMs.
- The framework optimizes both task execution and summarization actions under a shared task-level reward.
- Significant performance gains were observed on coding benchmarks, demonstrating the effectiveness of the approach.
- The method allows LLMs to operate effectively within fixed context budgets, enhancing their utility in long-horizon tasks.
Read more
CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents
Summary
The paper introduces CompactionRL, a novel reinforcement learning strategy designed to enhance long-horizon agentic large language models (LLMs) by integrating context compaction. As LLMs face limitations due to finite context windows during extended interactions, context compaction serves as a solution by summarizing previous states to continue task execution within a compressed context. The authors propose a framework that jointly optimizes task execution and summary generation, utilizing token-level loss normalization and cross-trajectory generalized advantage estimation to effectively train LLMs on compacted long-horizon trajectories. The approach is evaluated on open models, demonstrating significant performance improvements on coding tasks, with the GLM-4.5-Air model achieving Pass@1 scores of 66.8% on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, while the GLM-4.7-Flash model shows similar gains. CompactionRL thus extends the effective training horizon of agentic LLMs without increasing the maximum context length, marking a significant advancement in reinforcement learning for LLMs.
Methodology
CompactionRL employs a PPO-based reinforcement learning framework that incorporates context compaction into the rollout collection process. It optimizes task execution and summarization jointly, utilizing token-level loss normalization to mitigate length-induced biases and cross-trajectory generalized advantage estimation to maintain temporal credit assignment across compaction boundaries.
Results
The implementation of CompactionRL led to notable improvements in performance metrics: GLM-4.5-Air achieved Pass@1 scores of 66.8% on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, with gains of 7.0 and 3.1 points, respectively. Similarly, GLM-4.7-Flash improved by 5.5 points on SWE-bench Verified and 6.8 points on Terminal-Bench 2.0, reaching scores of 56.0% and 20.2%, respectively.
Implications
The findings suggest that integrating context compaction into reinforcement learning can significantly enhance the capabilities of LLMs in long-horizon tasks, making them more efficient and effective in real-world applications such as software engineering and interactive problem-solving.
Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQL
NLP
Large Language Models
Efficient ML
- Introduces a method for adaptive stopping in Text-to-SQL execution based on consistency convergence.
- Develops lightweight models that outperform fixed-budget and principled stopping rules.
- Demonstrates robustness against label noise, indicating practical applicability in production settings.
- Utilizes run-order permutation as a training augmentation to improve model performance.
Read more
Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQL
Summary
This paper addresses the challenge of determining when to stop executing multiple runs of a Text-to-SQL query to assess the reliability of the results generated by large language models (LLMs). The authors propose a convergence-prediction framework that utilizes lightweight 1-D models to monitor the consistency of execution outcomes over repeated runs. By analyzing the trajectory of consistency, the model predicts when further executions are unlikely to alter the consistency significantly. The proposed method is benchmarked against a Beta-Bernoulli stopping rule and a learned run-count baseline across various datasets, including the BIRD benchmark and two production datasets. The results demonstrate that the method can adaptively determine stopping points based on the specific user question, optimizing the number of runs required. Additionally, the authors explore the impact of weak serial correlation between runs, allowing for run-order permutation as a training augmentation. The robustness of the method is further validated by introducing noise to simulate an imperfect judge, showing that convergence can still be reliably predicted. Overall, this work enhances the efficiency of Text-to-SQL pipelines by reducing unnecessary executions while maintaining trust in the results.
Methodology
The authors formulate the problem of determining when to stop executing SQL queries as a convergence-prediction task. They train a family of lightweight 1-D models that analyze the consistency trajectory of execution results. The models classify whether the consistency has converged and identify the first converged run. The performance of these models is compared against a fixed-budget stopping rule and a Beta-Bernoulli stopping rule. Additionally, the authors introduce noise to simulate an imperfect judge to test the robustness of their method.
Results
The proposed method successfully adapts its stopping point based on the user question, leading to earlier halts when consistency converges quickly and longer execution for more complex queries. The performance remains consistent across multiple datasets, and the method effectively predicts convergence even when noise is injected into the correctness judgments.
Implications
This research has significant implications for optimizing Text-to-SQL pipelines in production environments, potentially reducing latency and computational costs while maintaining high reliability in query results. The adaptive stopping mechanism can be integrated into existing systems to enhance efficiency and user experience.