AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
57
Papers today
8h
Update frequency
7
Days of history
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
NLP
Large Language Models
Optimization
- Introduces a Bayesian domain weighting method to optimize data mixtures for LLMs.
- Addresses limitations of traditional manual heuristics and function-fitting approaches.
- Demonstrates stable and efficient learning of domain weights using less data.
- Empirical results show improved performance over existing methods.
Read more
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
Summary
This paper addresses the challenge of optimizing data mixtures for Large Language Models (LLMs) by proposing a Bayesian domain weighting method. Traditional approaches often rely on manual heuristics or function-fitting techniques that assume stable relationships between domain weights and validation losses. However, these assumptions can lead to significant estimation biases, especially as data complexity increases. The authors introduce a novel method that utilizes a Dirichlet distribution with Gamma prior information to infer domain weights, allowing for more stable and efficient learning. Experimental results indicate that this Bayesian approach not only stabilizes the optimization process but also identifies optimal domain mixtures using substantially less data compared to existing search-based methods. The findings suggest that the proposed method revitalizes domain weighting optimization for large-scale applications, enhancing the performance of LLMs.
Methodology
The authors propose a Bayesian domain weighting method that infers domain weights from a Dirichlet distribution, incorporating Gamma prior information learned from observations. This approach contrasts with traditional methods that rely on strong structural assumptions and function-fitting techniques, which can introduce biases and instability in weight learning.
Results
The experimental results show that the proposed Bayesian method achieves stable convergence in domain weight learning, significantly outperforming existing methods in terms of efficiency and accuracy. The method identifies optimal domain mixtures while consuming less data, demonstrating its effectiveness in large-scale applications.
Implications
The findings suggest that the Bayesian domain weighting method can enhance the training of LLMs by providing a more robust framework for optimizing data mixtures. This could lead to improved model performance across various applications in natural language processing and other domains that rely on multi-domain data.
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
Multimodal
NLP
Computer Vision
- Introduces CoT-Mediate framework for auditing reasoning in medical VLMs.
- Demonstrates that prefix-forced continuation improves mediation faithfulness compared to re-prompting.
- Identifies model-specific sycophancy behaviors based on the source of evidence.
- Establishes a new methodology for evaluating the influence of reasoning on model predictions.
Read more
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
Summary
This paper addresses the challenge of determining whether the reasoning provided by medical vision-language models (VLMs) genuinely influences their predictions or merely decorates pre-existing conclusions. The authors introduce a novel framework called CoT-Mediate, which evaluates the faithfulness of reasoning in medical VLMs by perturbing clinically relevant attributes within the model's generated reasoning. The study employs a dual-arm protocol to compare the effects of presenting edits as either external evidence or as the model's own prior tokens. Additionally, the authors investigate the impact of varying the attributed source of evidence, such as self-generated content versus notes from authoritative figures like senior radiologists. The experiments conducted on models LLaVA-Med and MedGemma using the VQA-RAD dataset reveal significant differences in mediation outcomes based on the injection method and the model's reliance on its own reasoning versus external authority. The findings indicate that prefix-forced continuation enhances mediation faithfulness compared to re-prompting, and that the models exhibit varying degrees of sycophancy based on the source of evidence. This work contributes to a deeper understanding of how medical VLMs process reasoning and highlights the importance of distinguishing between genuine reasoning and sycophantic behavior.
Methodology
The authors developed a behavioral audit framework (CoT-Mediate) that perturbs clinically significant attributes in the model's reasoning and measures the impact on predictions. A dual-arm protocol was used to compare the effects of presenting edits as external evidence versus the model's own prior tokens. The study also varied the attributed source of evidence to isolate authority-deference from content-tracking.
Results
The results showed that prefix-forced continuation significantly increased the Mediation Faithfulness Score for both models, with LLaVA-Med improving by 39.8 points and MedGemma by 4.9 points. The study found that the models' responses were influenced by the method of evidence injection and that sycophancy was model-specific, with LLaVA-Med showing a significant drop in adoption rates when self-attribution was involved.
Implications
This research has implications for the development of more reliable medical VLMs, emphasizing the need for models to demonstrate genuine reasoning capabilities rather than simply deferring to authoritative text. It also provides a framework for future studies to assess the reasoning processes of AI in clinical settings, potentially improving trust and transparency in AI-assisted medical decision-making.
Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory
Optimization
Generative Models
Graph Learning
- Introduction of short-term graph memory as a plug-in module for molecular optimization.
- Demonstrated improvements in mean top-10 scores across multiple generators without additional oracle calls.
- Analysis of how the module interacts with different generator architectures and their search dynamics.
- Characterization of the conditions under which the benefits of the module are retained or diminished.
Read more
Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory
Summary
This paper addresses the challenge of molecular optimization under a limited oracle budget, where the cost of evaluating candidate molecules is significantly higher than generating them. The authors introduce a novel module called short-term graph memory, which acts as an online graph neural surrogate that learns from previously evaluated molecules to prioritize future oracle queries. This module allows for more efficient allocation of the fixed oracle budget by pre-screening candidate pools, ensuring that the budget is spent on molecules with higher predicted utility. The approach was tested on a fragment-based generator using a standard molecular optimization benchmark, demonstrating improvements in the mean top-10 score without incurring additional oracle costs. The results indicate that the short-term graph memory module enhances the performance of various generators, particularly in tight budget scenarios, and provides insights into the interaction between surrogate-guided selection and the exploration-exploitation dynamics of different generators.
Methodology
The authors developed a short-term graph memory module that continuously fine-tunes a graph neural network based on previously evaluated molecules. This module is integrated into existing generator architectures, allowing it to pre-screen candidate pools and select the most promising candidates for evaluation by the oracle. The methodology maintains the original generator's architecture and update rules, ensuring compatibility across various types of generators.
Results
The implementation of short-term graph memory led to a significant increase in the mean top-10 score from 0.648 to 0.784 on the GenMol benchmark, with improvements observed across all four tested generators at a budget of 1,000 oracle calls. The module proved effective in reallocating the oracle budget without increasing the number of evaluations, thus enhancing the efficiency of the molecular optimization process.
Implications
The findings suggest that incorporating short-term memory mechanisms into molecular optimization frameworks can lead to more efficient use of limited evaluation resources, potentially accelerating drug discovery processes. This approach could be applied to other domains where evaluation costs are high relative to generation costs, enhancing the overall efficiency of search algorithms.
FunL2O: LLM-Guided Feature Function Design for Learning to Optimize
Optimization
Large Language Models
- FunL2O automates feature function design for L2O using LLMs.
- The framework evaluates feature functions based on their impact on optimization performance.
- Evolved features consistently outperform traditional hand-crafted representations.
- The approach is applicable across various optimization tasks, including linear, quadratic, and mixed-integer programming.
Read more
FunL2O: LLM-Guided Feature Function Design for Learning to Optimize
Summary
The paper introduces FunL2O, a novel framework that automates the design of feature functions for Learning-to-Optimize (L2O) methods using Large Language Models (LLMs). Traditional L2O approaches rely on manually crafted features, which can limit their adaptability and effectiveness across different optimization tasks. FunL2O addresses this limitation by employing an LLM to propose executable feature functions in a search loop, where the performance of these features is evaluated based on their impact on downstream optimization outcomes. The framework ensures that the proposed features adhere to specific semantic contracts, allowing for a structured evaluation of their effectiveness. The authors demonstrate the efficacy of FunL2O through extensive experiments on various optimization tasks, including linear and quadratic programming, as well as mixed-integer optimization. The results show that the features evolved through FunL2O consistently outperform hand-crafted features, leading to improved solution quality and solver efficiency. This work highlights the potential of LLM-driven feature evolution as a robust approach for enhancing representation design in L2O.
Methodology
FunL2O employs a search process where an LLM generates executable feature functions that are evaluated based on their performance in an L2O pipeline. Each proposed feature function is retrained within the existing L2O framework, and its effectiveness is measured through downstream optimization metrics. The framework incorporates semantic contracts to ensure that the generated features are valid and executable.
Results
The experiments conducted on linear and quadratic programming tasks, as well as mixed-integer optimization, demonstrate that the features evolved through FunL2O significantly enhance solution quality, feasibility, and solver efficiency compared to hand-crafted features. This improvement is consistent across different optimization scenarios and LLMs used in the feature generation process.
Implications
The findings suggest that automating feature function design can lead to more effective optimization solutions, reducing the reliance on manual feature engineering. This approach could be beneficial in various fields where optimization plays a critical role, such as operations research, logistics, and resource allocation.
Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs
Reinforcement Learning
Theory
Optimization
- Introduces a hierarchical MLMC neural critic to resolve the bias-cost tradeoff in neural critic estimation.
- Achieves first order-optimal convergence guarantees for infinite-horizon average-reward CMDPs with neural critics.
- Develops a primal-dual Natural Actor-Critic algorithm that does not require knowledge of the mixing time.
- Establishes global convergence and constraint-violation rates of O(T^(-1/2)).
Read more
Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs
Summary
This paper addresses the challenge of achieving order-optimal convergence in reinforcement learning using neural critics within Constrained Markov Decision Processes (CMDPs). The authors identify a fundamental bias-cost tradeoff that hinders the performance of existing neural actor-critic methods. To overcome this, they propose a hierarchical Multilevel Monte Carlo (MLMC) neural critic that simultaneously debiases trajectory sampling and critic optimization. This innovative approach allows the estimator to inherit the bias of a long critic optimization run while only incurring logarithmic expected sample costs. The authors then develop a primal-dual Natural Actor-Critic algorithm that achieves convergence and constraint violation rates of O(T^(-1/2)), marking the first instance of order-optimal guarantees for infinite-horizon average-reward CMDPs with neural critics. Notably, their method does not require prior knowledge of the mixing time of the CMDP, enhancing its practicality for large-scale applications.
Methodology
The authors utilize a hierarchical Multilevel Monte Carlo (MLMC) approach that incorporates two layers of debiasing: one for trajectory sampling and another for critic optimization. This dual-layer method allows for efficient bias reduction while maintaining low sample costs. They then construct a primal-dual Natural Actor-Critic algorithm based on this estimator, ensuring optimal convergence rates.
Results
The proposed algorithm achieves an optimality gap and constraint violation rate of O(T^(-1/2)), which is the first of its kind for neural critics in CMDPs. The method also demonstrates significant improvements over existing approaches, particularly in terms of sample efficiency and practical applicability without the need for mixing time knowledge.
Implications
This work has significant implications for reinforcement learning in safety-critical applications, such as robotics, healthcare, and transportation, where both performance and compliance with constraints are crucial. The ability to achieve order-optimal convergence with neural critics opens new avenues for deploying advanced reinforcement learning techniques in real-world scenarios.
THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
Graph Learning
- THGFM combines cross-type transfer and relation-aware specialization in a unified model.
- The model introduces a novel adaptive fusion mechanism (TC-NGSF) that allows independent feature amplification.
- Rotary Temporal Attention is used to incorporate relative time directly into the attention scores.
- THGFM shows consistent performance improvements over existing graph transformer models.
Read more
THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
Summary
The paper introduces THGFM, a novel model designed for learning on temporal heterogeneous graphs, which are complex structures that evolve over time and contain diverse types of nodes and relations. Existing methods struggle to balance cross-type transfer efficiency with relation-aware specialization. THGFM addresses these challenges through a dual-path architecture that includes a Shared-Space Temporal Attention (SSTA) branch for efficient cross-type transfer and a Relational Type-Partitioned Temporal Attention (RTTA) branch for specialization. The model employs a unique Type-Conditioned Non-Competitive Gated Sum Fusion (TC-NGSF) mechanism, allowing independent amplification or suppression of features from both branches. Additionally, THGFM incorporates Rotary Temporal Attention to directly integrate relative time into the attention mechanism, enhancing the model's ability to prioritize relevant neighbors based on temporal proximity. The experimental results demonstrate that THGFM outperforms baseline graph transformer models across various academic graph benchmarks, achieving significant performance gains in multiple tasks.
Methodology
THGFM employs a dual-branch architecture consisting of a Shared-Space Temporal Attention branch for cross-type transfer and a Relational Type-Partitioned Temporal Attention branch for specialization. It utilizes Type-Conditioned Non-Competitive Gated Sum Fusion to adaptively combine features from both branches and incorporates Rotary Temporal Attention to enhance temporal relevance in the attention mechanism.
Results
THGFM outperformed baseline models on academic graph benchmarks, achieving a +3.25% mean gain across six tasks. Notable peak gains included +12.37% on OAG-CS PV, +4.87% on PF-L2, and +1.18% on PF-L1, as well as +4.24%, +3.73%, and +4.61% on OGBN-MAG, HTAG-ArXiv, and HTAG-DBLP, respectively.
Implications
The THGFM model has significant implications for applications involving dynamic relational systems, such as academic research, e-commerce, and knowledge graphs, where understanding the interplay of diverse node types and temporal dynamics is crucial for effective representation learning.
QAdapt: A Noise-Adaptive Neural Pre-Decoding Framework for Quantum Error Correction
Theory
- QAdapt enables sequential adaptation to changing noise conditions while preserving knowledge from previous regimes.
- The framework shows robustness under distribution shifts, evaluated across diverse noise configurations.
- QAdapt improves both logical accuracy and reduces backend decoding latency in quantum error correction.
- The approach serves as an adaptive interface between quantum hardware and classical decoding systems.
Read more
QAdapt: A Noise-Adaptive Neural Pre-Decoding Framework for Quantum Error Correction
Summary
The paper introduces QAdapt, a novel noise-adaptive neural pre-decoding framework aimed at enhancing quantum error correction (QEC) in fault-tolerant quantum computing (FTQC). QEC is essential for maintaining logical information in the presence of physical errors, but conventional decoding methods face challenges due to the heterogeneous and nonstationary nature of quantum noise, as well as the latency involved in processing syndrome data. QAdapt addresses these issues by capturing local spatiotemporal correlations in syndrome data and adapting to evolving noise conditions while mitigating catastrophic forgetting. This framework operates as a pre-decoder that reduces the complexity of the residual syndrome before passing it to a conventional global decoder. The authors evaluate QAdapt across 110 synthetic out-of-distribution noise configurations and on Google's Willow benchmark data, demonstrating significant improvements in logical error rates and backend decoding latency. The results indicate that QAdapt effectively enhances the robustness and efficiency of quantum error correction systems under varying hardware noise conditions, providing a practical solution without the need for extensive retraining or modifications to existing decoders.
Methodology
QAdapt employs a neural network to learn local spatiotemporal correlations in syndrome data, adapting sequentially to different noise conditions. It utilizes techniques to prevent catastrophic forgetting, ensuring that knowledge from previously encountered noise regimes is retained. The residual syndrome produced by QAdapt is then processed by a conventional global decoder, allowing for a hybrid decoding strategy that balances local adaptation with global consistency.
Results
The implementation of QAdapt resulted in a reduction of up to 5.79% in logical error rates and 9.32% in backend decoding latency on the residual syndrome when tested on Google's Willow benchmark data, without requiring target-domain fine-tuning. These results were consistent across 110 synthetic out-of-distribution noise configurations, demonstrating the framework's effectiveness in improving quantum error correction performance.
Implications
QAdapt's adaptive pre-decoding framework has the potential to significantly enhance the efficiency and reliability of quantum error correction systems, making them more robust to varying hardware conditions. This could facilitate the scalability of fault-tolerant quantum computing, allowing for more practical applications in quantum information processing.
Subtract or Replay? Exact Deletion from Language-Model Memory
NLP
Large Language Models
Theory
- Exact deletion from language model memory depends on the representation of records.
- Two approaches are explored: algebraic decrement for addressable influences and rebuilding for shared recurrent states.
- Gemma model utilizes support-vector memory for efficient deletion, achieving high accuracy in next-token predictions.
- Kimi model highlights challenges in deletion due to the delta rule, necessitating checkpointing and replaying for accurate memory editing.
Read more
Subtract or Replay? Exact Deletion from Language-Model Memory
Summary
This paper addresses the challenge of exact deletion from the persistent memory of language models, emphasizing that the approach depends on how memory represents records. The author distinguishes between two scenarios: when a record's influence retains an address, allowing for algebraic decrement, and when later writes transform that influence within a shared recurrent state, necessitating a rebuild from prior states. The study operationalizes this distinction in two pretrained language models, Gemma and Kimi, and audits deletion against a reference that omits the record. For Gemma, a memory layer with support-vector memory is introduced, achieving agreement in next-token output after deletions. In Kimi, the delta rule is isolated, revealing that certain writes complicate deletion. The findings indicate that exact deletion is contingent on memory representation, advocating for subtraction where influence is addressable and rebuilding where it is interwoven into the state.
Methodology
The paper employs two pretrained language models, Gemma and Kimi, to explore memory deletion strategies. For Gemma, a support-vector memory layer is integrated, allowing for algebraic decrement of memory influences. In Kimi, the delta rule is analyzed to understand how recurrent state transformations affect deletion. The models are tested against various deletion scenarios, measuring next-token outputs and memory integrity.
Results
The results demonstrate that for Gemma, the next-token output remains consistent with a median KL divergence of 5.4 Γ 10β15 over multiple deletions, indicating effective memory management. In Kimi, the delta rule complicates deletion, with 12-49% of a record's contribution being dependent on subsequent inputs. Checkpointing and replaying methods successfully delete clinical records while maintaining accuracy across extensive contexts.
Implications
The findings have significant implications for applications requiring memory management in language models, such as clinical documentation, regulatory compliance (e.g., GDPR), and scenarios where precise record deletion is critical. The study highlights the need for thoughtful memory architecture in AI systems to facilitate effective unlearning.
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
Reinforcement Learning
Large Language Models
Optimization
- LSPO effectively recovers gradients on cliff prompts that typically yield zero rewards in standard RL methods.
- The method utilizes a transient low-rank adapter to fit ground-truth solutions without modifying the overall loss function.
- Empirical results show LSPO outperforms the DAPO baseline on all evaluated benchmarks, with notable improvements in performance metrics.
- The approach demonstrates a high conversion rate of cliff prompts into usable gradient signals, enhancing learning efficiency.
Read more
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
Summary
The paper introduces LoRA Scaffolded Policy Optimization (LSPO), a novel approach designed to address a critical limitation in reinforcement learning from verifiable rewards (RLVR) when dealing with 'cliff' promptsβsituations where all sampled rollouts fail, leading to a zero reward and consequently no gradient for policy updates. LSPO employs a sampling-time mechanism that integrates a low-rank adapter to recover lost gradients by fitting it to ground-truth solutions during training. The method involves detecting cliff prompts, briefly training the adapter on these prompts, re-sampling with the adapted model, and then integrating successful rollouts back into the RL batch while maintaining a base-only model. The empirical evaluation on the DeepMath-103K dataset demonstrates that LSPO significantly outperforms the DAPO baseline across multiple benchmarks, effectively converting a substantial percentage of cliff prompts into usable gradient signals, thus enhancing the learning process for challenging problems.
Methodology
LSPO operates within a standard policy-gradient RL framework, introducing a low-rank adapter that is trained briefly on cliff prompts' ground-truth solutions. It detects cliff prompts, updates the adapter, re-samples successful rollouts, and integrates these into the RL batch while ensuring that the base model remains unchanged. The method employs a two-optimizer split to route gradients appropriately, allowing for effective learning from cliff prompts without altering the loss structure.
Results
In experiments conducted on the DeepMath-103K dataset, LSPO achieved superior performance compared to the DAPO baseline across all 16 benchmark cells, with significant gains such as +10.7 points on AIME24/pass@4 and +6.7 points on AIME24 and AIME26 at pass@16. Overall, LSPO averaged an improvement of +3.8 points across all benchmarks, successfully converting approximately 43% of cliff prompts into usable gradient signals.
Implications
The LSPO method has the potential to enhance the training of reinforcement learning models, particularly in scenarios where traditional methods struggle with zero-reward prompts. This could lead to improved performance in applications requiring mathematical reasoning and problem-solving capabilities, particularly in large language models.
TAPO: Transition-Aware Policy Optimization for LLM Agents
Reinforcement Learning
Large Language Models
Optimization
- TAPO leverages dense environmental feedback for policy optimization, enhancing agent performance.
- The framework alternates between standard policy optimization and transition supervision.
- Empirical results show consistent improvements across different environments and model scales.
- TAPO requires no additional data or inference overhead, making it a cost-effective solution.
Read more
TAPO: Transition-Aware Policy Optimization for LLM Agents
Summary
The paper introduces TAPO (Transition-Aware Policy Optimization), a novel framework designed to enhance the post-training of Large Language Model (LLM) agents through Reinforcement Learning (RL). Traditional RL methods primarily focus on sparse task rewards for policy optimization, neglecting the rich, dense supervisory signals provided by environmental feedback after action execution. TAPO addresses this gap by alternating between policy optimization and transition supervision, utilizing existing rollout data to predict the next observation conditioned on the executed action. This dual approach not only optimizes the policy but also improves the model's understanding of environmental dynamics and action consequences. TAPO is presented as a lightweight, plug-and-play enhancement that does not require additional expert data or increased sampling costs. The authors conduct extensive experiments on two benchmarks, WebShop and ALFWorld, demonstrating that TAPO consistently outperforms standard policy optimization methods across various model scales and RL algorithms. The findings underscore the importance of transition feedback in improving long-horizon decision-making capabilities of LLM agents.
Methodology
TAPO employs a unified training framework that alternates between two main processes: standard policy optimization based on task rewards and transition supervision that predicts the next observation based on the current state and executed action. This method utilizes existing rollout data from RL interactions to enhance the model's sensitivity to environmental dynamics.
Results
The experiments conducted on WebShop and ALFWorld benchmarks reveal that TAPO significantly improves task performance compared to traditional policy optimization baselines. The results indicate that the integration of transition supervision leads to better long-horizon decision-making and a more robust understanding of environmental consequences.
Implications
The findings suggest that incorporating transition feedback into RL training can enhance the capabilities of LLM agents in complex, long-horizon tasks. This approach could be applied to various domains where LLM agents are utilized, such as web navigation and embodied control, potentially leading to more efficient and effective autonomous systems.
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Efficient ML
- ROCS improves inference efficiency by sharing computation across candidates in recommendation systems.
- Generalized Layer Masking (GLM) and Deep Cross Attention (DCA) are key innovations enabling this approach.
- ROCS achieves significant performance improvements without sacrificing prediction quality.
- The method has been successfully implemented in Meta's large-scale recommendation systems.
Read more
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Summary
The paper introduces ROCS (Request-Oriented Compute Sharing), a novel paradigm aimed at enhancing the efficiency of large-scale recommendation systems. Traditional recommendation models often face challenges in balancing prediction quality with production costs, particularly as they scale. ROCS leverages the unique nature of recommendation inference, where a single user request is evaluated against multiple candidates, allowing for shared computation across candidates. By deferring request-candidate interactions and isolating candidate-dependent representations, ROCS enables significant portions of the model to be evaluated once per request, rather than for each candidate. This approach is realized through two key innovations: Generalized Layer Masking (GLM), which preserves candidate isolation in feature-interaction architectures, and Deep Cross Attention (DCA), which extends request-oriented sharing to sequence models. Additionally, the In-Kernel Broadcast Optimization (IKBO) is co-designed to enhance GPU deployment efficiency. Experimental results demonstrate that ROCS consistently improves the quality-efficiency tradeoff across various recommendation backbones, achieving up to a 3Γ increase in query per second (QPS) for retrieval models without degrading quality, and a 0.5% relative LogLoss improvement with a 50% QPS gain for short-form video ranking models. ROCS has been successfully deployed in large-scale recommendation systems at Meta, leading to substantial online performance gains and reduced infrastructure costs.
Methodology
The methodology involves a modeling-driven approach that delays request-candidate interactions to maximize reusable computation. ROCS employs Generalized Layer Masking (GLM) to maintain candidate isolation and Deep Cross Attention (DCA) for sequence models, alongside In-Kernel Broadcast Optimization (IKBO) for efficient GPU execution.
Results
ROCS shows up to a 3Γ improvement in query per second (QPS) for retrieval models and a 0.5% relative LogLoss improvement with a 50% QPS gain for short-form video ranking models, all while maintaining or enhancing prediction quality.
Implications
The ROCS paradigm can significantly enhance the efficiency of recommendation systems across various applications, including ads and organic content, while reducing infrastructure costs. Its successful deployment at Meta suggests potential for broader adoption in other large-scale systems.
Good Rankers, Bad Objectives: Bilinear Contrastive Critics under Expressive Policy Search
Reinforcement Learning
Theory
Optimization
- Bilinear contrastive critics can rank actions well but may lead to poor selection outcomes due to norm drift and value decalibration.
- The study reveals that aggressive candidate maximization can convert small ranking errors into significant policy errors.
- Experiments show that contrastive critics exhibit weak or inverted value ordering despite strong compatibility across tasks.
- A Bellman-trained TD-Q approach effectively restores action ordering, indicating the necessity of value-calibrated objectives.
Read more
Good Rankers, Bad Objectives: Bilinear Contrastive Critics under Expressive Policy Search
Summary
This paper investigates the limitations of bilinear contrastive critics in reinforcement learning, particularly in the context of maximizing action rankings. The authors argue that while these critics can effectively rank actions based on compatibility, they often fail to provide a value-calibrated selection, leading to suboptimal decision-making. The study identifies norm drift and value decalibration as key issues that cause critics to misorder actions, even when they perform well in in-support ranking tasks. Through a series of experiments across four OGBench navigation tasks, the authors demonstrate that contrastive critics can incur significant regret during candidate maximization, despite showing strong retrieval capabilities. The findings suggest that a Bellman-trained TD-Q approach can restore proper action ordering, highlighting the need for value-calibrated objectives in action selection. Overall, the paper emphasizes the importance of separating compatibility ranking from value-based selection in reinforcement learning frameworks.
Methodology
The authors conducted a theoretical analysis of bilinear critics, focusing on their performance in ranking and selection tasks. They performed experiments on four OGBench navigation tasks to assess the critics' effectiveness in action ranking and selection, comparing results with a Bellman-trained TD-Q approach. The methodology included a training/readout decomposition to trace the sources of lost ordering and evaluate the impact of norm drift and value calibration.
Results
The experiments revealed that contrastive critics often misorder actions, leading to regret during candidate maximization. Despite strong retrieval performance, the critics showed weak or inverted value ordering in the top score decile. The Bellman-trained TD-Q method successfully restored the correct action ordering, demonstrating the importance of value calibration in action selection.
Implications
The findings suggest that while contrastive critics can be useful for ranking actions in navigation and manipulation tasks, their application in action selection requires careful consideration of value calibration. This has implications for the design of reinforcement learning systems, particularly those employing expressive policies that sample from large candidate sets.
EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis
Large Language Models
Graph Learning
Interpretability
- EvoCause integrates LLMs for refining causal graphs based on expert feedback.
- Introduces TeleRCA, a large-scale expert-annotated RCA benchmark dataset.
- Demonstrates significant improvements in RCA performance metrics over traditional methods.
- Highlights the importance of human-readable alarm titles in graph refinement.
Read more
EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis
Summary
EvoCause addresses the challenge of root cause analysis (RCA) in modern telecommunication and cloud systems, where correlated alarm cascades occur during component failures. Traditional methods rely on fixed causal graphs derived from observational data, which do not adapt based on expert feedback from historical incidents. EvoCause introduces a novel framework that integrates large language models (LLMs) to propose semantically plausible edits to causal graphs based on expert constraints, while deterministic code ensures structural validity. The authors also present TeleRCA, a benchmark dataset containing expert-annotated RCA data from a production telecommunication network, which aids in evaluating RCA methods. Experimental results demonstrate that EvoCause significantly improves RCA performance metrics and graph reconstruction accuracy compared to baseline methods, highlighting the importance of incorporating human-readable alarm information. This work not only refines existing causal graphs but also provides a comprehensive dataset for future research in RCA.
Methodology
EvoCause employs a two-step approach: first, it uses a large language model to suggest semantically plausible edits to a causal graph based on expert constraints. Second, deterministic procedures validate these edits to ensure acyclicity and structural integrity, retaining the best-performing graph based on a labeled alignment set. The framework is designed to work with graphs produced by different causal discovery algorithms without altering their internal learning processes.
Results
EvoCause outperformed the unrefined PC causal discovery algorithm on synthetic data, achieving improvements in Node F1, Case EM, and Graph F1 by 11.59, 9.40, and 4.59 percentage points, respectively, while reducing nSHD by 0.2379. On the TeleRCA dataset, anonymizing alarm titles led to a decrease in Node F1 and Case EM by 6.12 and 8.04 percentage points, indicating the critical role of alarm-name information in the refinement process.
Implications
The findings suggest that incorporating expert feedback into causal graph refinement can significantly enhance the accuracy of root cause analysis in complex systems. The TeleRCA dataset provides a valuable resource for future research, enabling the development and evaluation of more effective RCA methods. This approach could be applied in various domains where causal relationships are critical for diagnosing failures.
NMINE: Normalized Mutual Information Neural Estimation
Theory
Efficient ML
- Introduces a fully neural estimator for normalized mutual information between continuous multidimensional variables.
- Combines MINE-based mutual information estimation with MI-NEE-inspired entropy estimation in a unified framework.
- Derives entropy estimates from neural divergence estimates relative to uniform reference distributions.
- Demonstrates improved accuracy over traditional k-nearest-neighbor-based NMI estimation methods.
Read more
NMINE: Normalized Mutual Information Neural Estimation
Summary
This paper introduces NMINE, a fully neural estimator for normalized mutual information (NMI) aimed at addressing the challenges of estimating mutual information for continuous and multidimensional variables. Traditional methods for estimating mutual information, particularly those based on nearest-neighbor statistics, struggle with numerical stability and accuracy as dimensionality increases. NMINE combines the Mutual Information Neural Estimation (MINE) framework with neural marginal entropy estimation inspired by Neural Entropic Estimation (NEE). By utilizing the DonskerβVaradhan variational representation for mutual information and learning the divergence between marginal distributions and a uniform reference distribution for entropy, NMINE provides a robust alternative to existing methods. The authors demonstrate the effectiveness of their approach through experiments on Gaussian data across one to eight dimensions, showing improved accuracy over a k-nearest-neighbor-based NMI baseline. This work highlights the potential of neural estimation techniques for normalized dependency measurement in complex, high-dimensional settings.
Methodology
The proposed NMINE framework utilizes the DonskerβVaradhan variational representation to estimate mutual information and employs neural divergence estimation to derive marginal entropies. This approach replaces traditional nearest-neighbor methods, enhancing stability and accuracy in high-dimensional settings.
Results
Experiments conducted on Gaussian data from one to eight dimensions indicate that NMINE outperforms a k-nearest-neighbor-based normalized mutual information baseline, demonstrating its effectiveness in estimating normalized mutual information accurately.
Implications
The findings suggest that NMINE can be applied in various fields requiring accurate dependency measurement, such as molecular dynamics, interpretable machine learning, and other applications involving continuous and high-dimensional data.
GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios
Generative Models
Reinforcement Learning
Computer Vision
- Introduction of DocMeetSVG-100K, a large-scale dataset for complex SVG generation.
- Development of GVR-Coder, which integrates curriculum-driven fine-tuning and layout constraints.
- Implementation of reinforcement learning for dual rendering feedback to optimize output quality.
- Establishment of a generate-verify-repair loop for iterative improvement of SVG generation.
Read more
GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios
Summary
The paper presents GVR-Coder, a novel framework designed to enhance the generation of structured Scalable Vector Graphics (SVG) from lengthy professional texts, particularly in complex document and meeting scenarios. The authors identify three major challenges in current Text-to-SVG generation: the lack of datasets for complex diagrams, the absence of explicit layout priors leading to chaotic arrangements, and insufficient visual feedback to validate outputs. To address these issues, they introduce DocMeetSVG-100K, a large-scale dataset tailored for document authoring and meeting reviews. GVR-Coder employs a curriculum-driven rejection sampling fine-tuning approach to progressively improve the model's ability to handle complex structures while incorporating layout constraints during training. Additionally, the framework utilizes reinforcement learning from dual rendering feedback to optimize both structural complexity and visual aesthetics. A generate-verify-repair agent loop is also designed to enhance generation quality through targeted refinement based on fine-grained feedback. Experimental results demonstrate that GVR-Coder outperforms existing baselines, producing diagrams that are both logically coherent and visually appealing, thus facilitating better information communication in professional settings.
Methodology
The authors developed GVR-Coder, which employs a combination of curriculum-driven rejection sampling for fine-tuning, reinforcement learning for dual feedback optimization, and a generate-verify-repair agent loop to iteratively enhance the quality of generated SVGs. The framework incorporates layout constraints and utilizes a large-scale dataset specifically designed for complex document and meeting scenarios.
Results
GVR-Coder significantly outperformed existing methods in generating logically coherent and visually appealing SVG diagrams. The framework's innovative approaches to dataset creation, model training, and feedback mechanisms led to improved structural complexity and aesthetic quality in the generated outputs.
Implications
The advancements presented in GVR-Coder have the potential to streamline information communication in professional environments, making it easier to distill complex content into clear visual representations. This can enhance productivity in document authoring and meeting reviews, ultimately improving decision-making processes.
MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
Multimodal
Generative Models
Efficient ML
- MUGEN integrates motion understanding and generation using a single continuous representation without the need for discrete codebooks.
- The framework employs depth-routed hidden states to enhance the distinctiveness of latent slots.
- MUGEN achieves state-of-the-art performance on various metrics while significantly reducing computational costs.
- The use of a calibrated low-rank factor head allows for efficient text-conditional variation across latent slots.
Read more
MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
Summary
The paper introduces MUGEN, a unified framework designed to enhance the efficiency of motion understanding and generation by integrating language and motion representations without the limitations of traditional quantization methods. Existing systems often rely on discrete codebooks that compromise generation quality and require extensive computational resources for decoding. MUGEN circumvents these issues by employing a single adaptive-length autoencoder (ALAE) that compresses motion into a few continuous latent slots, which serve as the sole representation of motion. This allows the language model to generate and interpret motion data efficiently. The framework utilizes depth-routed hidden states to enable each latent slot to access relevant information from different layers of the transformer architecture, ensuring that each slot carries distinct information. Additionally, a calibrated low-rank factor head predicts a joint distribution over the latent slots, allowing for a single draw to encapsulate the necessary variation for both generation and understanding tasks. MUGEN demonstrates superior performance on benchmark datasets, achieving high scores in generation fidelity and retrieval precision while significantly reducing inference time compared to existing methods.
Methodology
MUGEN utilizes an adaptive-length autoencoder (ALAE) to compress motion into continuous latent slots, which are then processed by a language model for both generation and understanding tasks. Depth-routed hidden states allow each slot to access information from different transformer layers, while a calibrated low-rank factor head predicts a joint distribution over the latent slots, enabling efficient variation representation.
Results
MUGEN outperforms existing motion-language baselines on the HumanML3D and SnapMoGen datasets, achieving the best FID, CIDEr, and BLEU@4 scores. It also shows improved retrieval precision and matching distance, all while maintaining a significantly lower inference time (9 ms per motion) compared to traditional methods.
Implications
The MUGEN framework has the potential to advance the development of physical AI systems that require efficient interaction between language and motion. Its efficient architecture could be applied in robotics, animation, and virtual reality, where real-time motion generation and understanding are critical.
AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization
Reinforcement Learning
Optimization
- AutoPref automates the design of task-specific preference objectives for neural combinatorial optimization.
- The framework factorizes the preference objective into a pairwise loss program and a set-aware weighting program.
- A staged conditional search strategy is employed to efficiently explore the objective space.
- AutoPref outperforms traditional hand-designed objectives across multiple combinatorial optimization problems.
Read more
AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization
Summary
The paper introduces AutoPref, a novel framework for the automatic discovery of task-specific preference objectives in neural combinatorial optimization (NCO). Traditional approaches to NCO often rely on manually designed objectives, which can be inefficient and not adaptable across different combinatorial optimization problems (COPs). AutoPref addresses this limitation by factorizing the preference objective into two components: a pairwise loss program that determines the learning signal from individual solution comparisons, and a set-aware weighting program that adjusts the contribution of each comparison based on the entire sampled set. This factorization allows for a more expressive and adaptable objective space. The authors propose a staged conditional search strategy to efficiently explore this space, first optimizing the pairwise loss and then the weighting program, while employing behavioral screening to ensure the validity of candidate programs. The effectiveness of AutoPref is demonstrated through experiments on various COPs, including the Traveling Salesman Problem (TSP) and Capacitated Vehicle Routing Problem (CVRP), where the discovered objectives consistently outperform existing hand-designed baselines, showcasing the advantages of automated objective discovery in NCO.
Methodology
AutoPref employs a factorization approach to preference objectives, separating the learning signal from pairwise comparisons and the weighting of these comparisons. It utilizes a staged conditional search strategy to optimize these components sequentially, along with behavioral screening to filter out invalid candidates before training and evaluation.
Results
The experiments conducted on various combinatorial optimization problems, including TSP, CVRP, Flexible Flow Shop Scheduling Problem (FFSP), and Job Shop Scheduling Problem (JSSP), demonstrate that the objectives discovered by AutoPref consistently outperform the strongest hand-designed baselines across different problem scales.
Implications
The findings suggest that automated discovery of preference objectives can significantly enhance the efficiency and effectiveness of neural combinatorial optimization methods, making them more adaptable to various real-world applications in routing, scheduling, and other decision-making scenarios.
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Computer Vision
Theory
Efficient ML
- Introduction of a dual-teacher model to enhance the robustness/accuracy tradeoff in DNNs.
- Improvement in clean sample classification accuracy compared to the original IBD framework.
- Maintained robust accuracy against adversarial samples.
- Competitive performance with state-of-the-art methods like B-MTARD.
Read more
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Summary
This paper addresses the vulnerability of deep neural networks (DNNs) to adversarial attacks, which compromise their reliability in real-world applications. The authors build upon the Information Bottleneck Distillation (IBD) framework, which typically enhances robustness against adversarial inputs at the cost of clean input accuracy. They propose an extension that incorporates a dual-teacher model approach, where one teacher is trained on clean inputs (clean teacher) and the other on adversarial inputs (robust teacher). This dual-teacher setup allows for the transfer of features through a cross-layer attention mechanism, aiming to improve classification accuracy on clean samples while maintaining robustness against adversarial samples. The experimental evaluation on CIFAR-10 and CIFAR-100 datasets demonstrates that the proposed method outperforms the original IBD in terms of clean accuracy while preserving robust accuracy. The results also indicate competitiveness with state-of-the-art methods, particularly in balancing clean and robust accuracy through the harmonic mean. Additionally, the paper explores the effects of different training configurations on the attention module's performance.
Methodology
The authors propose a dual-teacher distillation framework that utilizes two teacher models: a robust teacher trained on adversarial examples and a clean teacher trained solely on clean inputs. The student model learns from both teachers through a cross-layer attention mechanism, employing two strategies for feature distillation: double distillation and joint distillation.
Results
The proposed method shows a significant improvement in clean accuracy over the original IBD while maintaining similar robust accuracy. It also demonstrates competitive performance against existing state-of-the-art frameworks, particularly in achieving a favorable harmonic mean between clean and robust accuracy.
Implications
The findings suggest that leveraging dual-teacher models can effectively enhance the robustness and accuracy of DNNs, making them more reliable for real-world applications where adversarial attacks are a concern. This approach could be beneficial in various domains, including security-sensitive applications in computer vision and autonomous systems.
Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus
Graph Learning
- Introduces an ensemble consensus approach for semi-supervised learning in molecular graphs.
- Demonstrates significant improvements in predictive accuracy across diverse datasets and tasks.
- Eliminates the performance gap between individual models and full ensembles, with individual models often outperforming traditional ensembles.
- Reduces calibration error and enhances model robustness.
Read more
Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus
Summary
This paper addresses the challenges of semi-supervised learning (SSL) in molecular sciences, where acquiring labeled data is expensive and time-consuming, while large amounts of unlabeled data are available. Traditional SSL methods often rely on label-preserving data augmentations, which are difficult to implement in the molecular domain due to the sensitivity of molecular properties to minor structural changes. The authors propose a novel SSL approach that utilizes ensemble consensus instead of explicit data augmentations. By training an ensemble of models, where each member learns from labeled data using a supervised loss and from unlabeled data using a consensus loss, the method enhances predictive accuracy across various molecular datasets and tasks. The ensemble consensus approach not only improves individual model performance but also reduces calibration error, leading to more robust models. The paper presents theoretical foundations for the method, demonstrating its effectiveness in achieving ensemble-level performance with individual models and eliminating the performance gap between them and the full ensemble.
Methodology
The proposed method involves training a model ensemble where each member is trained on labeled data using a standard supervised loss and on unlabeled data using a consensus loss that promotes agreement among the ensemble members. This approach does not require explicit data augmentations, making it suitable for the molecular domain.
Results
The ensemble consensus method robustly improved predictive accuracy across various molecular datasets and architectures for both regression and classification tasks. Individual models trained with this method consistently outperformed traditional ensemble models, and the approach also led to reduced calibration error.
Implications
The findings suggest that the ensemble consensus approach can significantly enhance the application of machine learning in molecular sciences, facilitating better predictions in drug discovery and materials science by leveraging abundant unlabeled data more effectively.
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Reinforcement Learning
Large Language Models
Optimization
- CRPO enhances OPSD by addressing exposure bias through contrastive learning.
- The method utilizes predictive entropy to classify positions for effective distillation.
- Extensive evaluations show CRPO outperforms existing methods across diverse reasoning tasks.
- The theoretical foundation of CRPO aligns with logit-wise credit assignment.
Read more
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Summary
This paper introduces Contrastive Reinforced Policy Optimization (CRPO), a novel approach to On-Policy Self-Distillation (OPSD) that addresses the challenges of exposure bias in multi-turn agentic settings. Traditional OPSD methods suffer from unreliable position-level supervision due to the privileged information of the self-teacher, leading to reasoning route convergence and unclear optimization directions. CRPO reformulates OPSD through a contrastive learning lens, utilizing predictive entropy to differentiate between positive (reflective exploration) and negative (exposure bias) positions. By conducting group-wise contrast, CRPO preserves fine-grained optimization signals, enhancing training stability and generalization. The authors validate their approach through extensive evaluations across 13 benchmarks, demonstrating that CRPO consistently outperforms existing reinforcement learning and self-distillation methods, particularly in long-horizon interactions.
Methodology
CRPO reformulates the OPSD framework by treating the original context and the context with privileged information as two views. It employs predictive entropy to classify positions and conducts group-wise contrast to enhance optimization signals. The method penalizes positions with exposure bias while encouraging advantageous positions for better generalization.
Results
CRPO was evaluated on 13 challenging benchmarks, consistently outperforming existing reinforcement learning and self-distillation baselines. The results indicate significant improvements in training stability and generalization, particularly in long-horizon interactions.
Implications
The findings suggest that CRPO can be applied to enhance the performance of large language models and other agentic systems that require reliable supervision in multi-turn interactions. This approach may lead to more robust models in various applications, including dialogue systems, automated reasoning, and interactive AI.
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Large Language Models
Optimization
Efficient ML
- Introduces SDO, a dynamic data organization framework for LLM post-training.
- Implements an exposure-driven feedback mechanism to optimize mini-batch composition.
- Operates on frozen embeddings, eliminating the need for warm-up training.
- Demonstrates improved convergence and gradient coherence across multiple training paradigms.
Read more
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Summary
The paper introduces SDO (Structure-Aware Data Organization), a novel framework designed to enhance the efficiency of post-training for large language models (LLMs). Traditional methods for improving post-training efficiency primarily focus on selecting informative samples or optimizing training schedules, often treating data organization as a static preprocessing step. This static approach leads to suboptimal sample exposure during optimization, resulting in redundant updates for some samples while others remain under-optimized. SDO addresses this issue by implementing an exposure-driven feedback mechanism that dynamically organizes mini-batch composition and sample exposure based on the structure of the representation space. The framework operates on frozen external embeddings, avoiding the overhead of model warm-up training. SDO employs locality-aware batching through KNN neighborhood traversal to form coherent mini-batches and exposure-balanced scheduling to manage sample participation across epochs. The empirical results demonstrate that SDO accelerates convergence across various post-training paradigms, including supervised fine-tuning (SFT), direct preference optimization (DPO), and gradient-based reinforcement policy optimization (GRPO), with significant improvements observed particularly in the early-to-mid training phases.
Methodology
SDO utilizes a closed-loop data organization approach that combines locality-aware batching through KNN traversal of frozen embeddings and exposure-balanced scheduling across epochs. This allows for dynamic reorganization of the active data pool based on actual sample visitation, enhancing training efficiency without altering the learning objective or requiring warm-up training.
Results
The implementation of SDO resulted in accelerated convergence rates across SFT, DPO, and GRPO, with the most significant gains observed during the early-to-mid phases of training. The framework produced more coherent gradients and balanced accuracy across different question types, as confirmed by gradient diagnostics and component ablation studies.
Implications
SDO has the potential to significantly improve the efficiency of post-training processes for large language models, making it applicable in various contexts where LLMs are utilized. Its dynamic data organization approach could lead to better resource utilization and faster model adaptation to new tasks.
Compression-Based Behavioral Similarity for Open-World Sybil Discovery on Ethereum
Graph Learning
- Introduces a compression-based approach for detecting Sybil attackers on Ethereum.
- Utilizes a symbolic Transaction Grammar to model wallet behavior.
- Employs Normalized Compression Distance (NCD) for measuring behavioral similarity.
- Demonstrates robustness against adversarial tactics and temporal changes.
Read more
Compression-Based Behavioral Similarity for Open-World Sybil Discovery on Ethereum
Summary
This paper addresses the challenge of detecting Sybil attackers in the Ethereum blockchain, who create multiple pseudonymous wallets to exploit airdrops and manipulate governance. Traditional detection methods rely heavily on explicit financial interactions and supervised machine learning, which are inadequate in the dynamic landscape of blockchain where attack strategies frequently evolve. The authors propose a novel approach that utilizes compression-based behavioral similarity to differentiate between Sybil bots, organic users, and arbitrage bot wallets without requiring direct financial links. They introduce a symbolic Transaction Grammar that captures transaction rhythm, execution structure, and functional intent, allowing for the measurement of behavioral similarity using Normalized Compression Distance (NCD). The framework is validated against supervised machine learning baselines and demonstrates robustness against temporal drift and adversarial perturbations. The proposed method is positioned as a training-free local discovery tool for open-world blockchain audits, emphasizing its capability to expand suspicious seed wallets without explicit funding links.
Methodology
The authors synthesize a symbolic Transaction Grammar from Ethereum Virtual Machine (EVM) traces to capture transaction behavior. They apply Gzip-based Normalized Compression Distance (NCD) to construct a behavioral graph for Sybil discovery, filtering high-signal contracts using their Blind-Spot Protocol. The framework is validated through comparisons with supervised machine learning methods and stress tests.
Results
The proposed framework successfully identifies Sybil candidates without requiring labeled training data and demonstrates resilience to changes in attack strategies. It outperforms traditional methods that rely on explicit financial connections, showcasing its effectiveness in open-world scenarios.
Implications
This research has significant implications for blockchain security, particularly in enhancing the detection of Sybil attacks in decentralized finance (DeFi) and governance contexts. The training-free nature of the method allows for timely responses to emerging threats, making it a valuable tool for blockchain auditors and security analysts.
Learning features from Newton's algorithm: a way to accelerate nonlinear parametrized PDE solvers
Optimization
Theory
Efficient ML
- Introduces a two-stage Newton initial guess strategy to accelerate nonlinear PDE solvers.
- Utilizes learned feature spaces from previous Newton computations to enhance initialization.
- Demonstrates significant reductions in Newton iterations and CPU time.
- Method is weakly intrusive, requiring minimal access to high-fidelity residual fields.
Read more
Learning features from Newton's algorithm: a way to accelerate nonlinear parametrized PDE solvers
Summary
This paper presents a novel two-stage strategy to enhance the efficiency of solving nonlinear parametrized partial differential equations (PDEs) by leveraging features learned from Newton's algorithm. The authors propose constructing two complementary reduced spaces: a solution feature space derived from converged states and a corrective search direction feature space based on intermediate Newton increments. For unseen parameters, a regression model predicts a surrogate solution, followed by a residual-minimizing correction using a GMRES-based approach. This method serves as an initial guess for the high-fidelity Newton method, significantly reducing the number of iterations required and overall CPU time. The approach is shown to be weakly intrusive, requiring only residual evaluations and a small least-squares problem solution, making it applicable to a wide range of large-scale nonlinear problems. Numerical experiments demonstrate quantifiable speedups compared to standalone surrogate initializations, highlighting the method's effectiveness in accelerating convergence.
Methodology
The methodology involves learning two complementary feature spaces from offline Newton computations: a solution space for predicting initial guesses and a corrective search direction space for refining these guesses. A regression model predicts surrogate solutions for unseen parameters, followed by a GMRES-based correction step to minimize residuals. This process is designed to be computationally efficient and requires only residual evaluations.
Results
Numerical experiments on representative PDE problems indicate that the proposed method achieves significant speedups in convergence compared to traditional standalone surrogate initializations. The reduction in Newton iterations and overall computational time is quantifiable, demonstrating the method's effectiveness in practical applications.
Implications
The proposed approach has the potential to streamline the solution of complex nonlinear PDEs in various fields, including engineering, physics, and applied mathematics. By improving the efficiency of existing solvers, it may facilitate faster simulations and optimizations in design and uncertainty quantification tasks.
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Reinforcement Learning
Interpretability
- Incorporating predicted class information in RL state representation enhances CFE generation.
- Class-aware RL improves convergence speed, reward optimization, and reduces episode length.
- The method generates significantly more valid CFEs compared to class-blind RL.
- Class-based features are influential in the action-selection process of the RL model.
Read more
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Summary
This paper presents a novel approach to generating counterfactual explanations (CFEs) using class-aware reinforcement learning (RL). CFEs are crucial for enhancing the interpretability of black-box machine learning models by providing alternative instances that yield different outcomes with minimal feature adjustments. The authors argue that incorporating the predicted class of an instance into the RL state representation can improve exploration efficiency and policy optimality. The study contrasts the proposed class-aware RL method with a class-blind RL method, which does not utilize class information. Experiments conducted on seven diverse datasets demonstrate that the class-aware approach accelerates convergence, optimizes rewards, and reduces episode lengths during training. Additionally, it generates a significantly higher number of valid CFEs compared to the class-blind method. The findings highlight the importance of class-awareness in RL for CFE generation, leading to clearer explanations and more effective counterfactual generation across various domains.
Methodology
The authors frame the CFE generation task as a Markov Decision Process (MDP), defining states, actions, and rewards. They enhance the RL state representation by including the predicted class of the instance along with features from the supervised dataset. The RL model interacts with a designed environment for each dataset, treating the CFE generation as an episodic task.
Results
The class-aware RL method showed improved performance metrics, including faster convergence, better reward optimization, and a higher number of valid CFEs compared to the class-blind RL approach. The inclusion of class information consistently ranked among the most influential predictors in the RL's action-selection process.
Implications
This research has significant implications for the field of Explainable AI (XAI), particularly in applications requiring transparent decision-making, such as healthcare and finance. The findings suggest that class-aware methods can enhance the interpretability of machine learning models, fostering greater trust and understanding among users.
Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadata
Federated Learning
- Introduction of the CFL trilemma, highlighting trade-offs in privacy, communication, and computation.
- Proposal of FLAMECHE, a framework that reformulates metadata-based clustering using distributed EM.
- Ensures compatibility with secure federated learning mechanisms by limiting server updates to additive operations.
- Demonstrates improved model effectiveness through extensive experiments on various datasets.
Read more
Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadata
Summary
This paper addresses the challenges of Clustered Federated Learning (CFL), particularly in the context of data heterogeneity among clients. The authors introduce the concept of the CFL trilemma, which highlights the trade-offs between privacy preservation, communication cost, and computational efficiency in existing CFL methods. The paper critiques current approaches that rely on metadata for clustering, as they often compromise privacy by sharing low-dimensional dataset representations in plaintext. To overcome these limitations, the authors propose FLAMECHE, a novel framework that reformulates metadata-based CFL as a distributed Expectation-Maximization (EM) procedure. This approach restricts server updates to additive operations, ensuring compatibility with secure federated learning schemes. The framework utilizes randomly initialized neural networks to compute metadata, thus avoiding the need for prior knowledge of client data distributions. Extensive experiments demonstrate that FLAMECHE enhances the effectiveness of client models while maintaining encryption compatibility, thereby improving its position within the CFL trilemma.
Methodology
The authors developed FLAMECHE by reformulating the clustering process as a distributed Expectation-Maximization (EM) procedure. This method limits server updates to additive operations, which are compatible with secure federated learning protocols. Metadata is generated using randomly initialized neural networks, avoiding the need for specific client data distribution knowledge.
Results
The experimental results indicate that FLAMECHE significantly improves the effectiveness of client models in clustered federated learning scenarios. The framework successfully maintains encryption compatibility while addressing the CFL trilemma, demonstrating enhanced performance across various heterogeneous datasets.
Implications
FLAMECHE has the potential to advance the field of federated learning by enabling more secure and efficient clustering of clients with heterogeneous data. Its compatibility with encryption mechanisms could facilitate broader adoption of federated learning in privacy-sensitive applications, such as healthcare and finance.
Latent-Kernel Discrete Flow Maps for Few-Step Generation
NLP
Generative Models
Large Language Models
- Introduction of Latent-Kernel Discrete Flow Maps (LKF) for few-step generation.
- LKF allows for correlated token generation by using a shared latent variable across factorized components.
- Demonstrated significant improvements in generative perplexity over existing models without relying on teacher models.
- Performance scales positively with the number of components (M) used in the model.
Read more
Latent-Kernel Discrete Flow Maps for Few-Step Generation
Summary
This paper introduces Latent-Kernel Discrete Flow Maps (LKF), a novel approach to few-step generation in discrete diffusion and flow-matching models. Traditional models often struggle with generating correlated tokens, such as subjects and verbs that must agree, due to their independent token updates. LKF addresses this issue by employing a mixture of factorized components tied by a shared latent variable, allowing for correlated sampling in a single step. The authors demonstrate that LKF can achieve significant improvements in generative perplexity on text generation tasks, outperforming existing methods that rely on teacher models for distillation or rectification. The model's performance is validated on the One-Billion-Word (LM1B) and WikiText-103 benchmarks, showing a 2.1Γ to 3.3Γ improvement over likelihood baselines, with gains increasing as the number of components (M) increases. The results indicate that LKF effectively captures the necessary correlations for few-step generation without the limitations imposed by teacher models.
Methodology
The authors propose a flow-map kernel that combines M factorized components, each conditioned on a shared latent variable. This allows for efficient sampling of correlated tokens in a single step. The model's performance is evaluated through experiments on synthetic datasets and standard text generation benchmarks, comparing its results against likelihood models and few-step accelerators.
Results
The LKF model achieved a generative perplexity improvement of 2.1Γ to 3.3Γ over likelihood baselines on the LM1B and WikiText-103 datasets. At M=8, LKF surpassed distilled and rectified few-step samplers, demonstrating that the model effectively captures correlations without the need for a teacher.
Implications
The findings suggest that LKF could enhance the efficiency and quality of text generation tasks in NLP, particularly in scenarios where token dependencies are critical. This approach may lead to advancements in generative models and their applications in various language processing tasks.
Multi-channel Uplift Policy Learning
Optimization
- Introduces a compositional uplift framework for multi-channel budget allocation.
- Proposes ReAlloc, which separates causal response learning from decision-making.
- Demonstrates significant improvements in marketing outcomes through online deployment.
- Addresses the limitations of traditional PTO methods in multi-channel settings.
Read more
Multi-channel Uplift Policy Learning
Summary
The paper addresses the challenge of allocating fixed marketing budgets across multiple channels in e-commerce to maximize business utility, which traditional predict-then-optimize (PTO) methods struggle to handle due to observational confounding and extrapolation issues. The authors propose a novel framework called ReAlloc, which formulates the problem as a simplex-constrained uplift decision problem. ReAlloc employs a dual approach: an Orthogonal Teacher that extracts unbiased local gradients from short-term logs and an Explanation-Guided Student that distills these gradients into a structured marginal field for long-term decision-making. This design allows for support-aware and conservative decisions that account for cross-channel substitutions. The effectiveness of ReAlloc is demonstrated through extensive simulations and large-scale online A/B tests on the Taobao platform, showing significant improvements in both pay order and income.
Methodology
The methodology involves a two-part framework where an Orthogonal Teacher is used to derive unbiased local gradients from short-term data, and an Explanation-Guided Student distills these gradients into a structured marginal field. This approach allows for conservative decision-making that respects the budget constraints of multi-channel marketing.
Results
The results from extensive offline experiments and large-scale online A/B tests on the Taobao platform indicate that ReAlloc significantly outperforms traditional uplift and PTO baselines, achieving a 3.53% increase in pay order and a 3.26 percentage point increase in income.
Implications
The findings suggest that ReAlloc can be effectively utilized in real-world e-commerce systems to optimize marketing resource allocation, leading to improved business outcomes. This approach could be adapted to other domains requiring budget-constrained decision-making across multiple channels.
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Multimodal
Robotics
Efficient ML
- Identifies decode latency as a critical factor causing delays in GUI agents' responses.
- Introduces Adaptive Anticipatory Policy Trees (AAPT) to pre-compile actions and reduce decision-time delays.
- Demonstrates a significant improvement in action success rates through controlled experiments.
- Finds that the effectiveness of AAPT is dependent on specific capabilities such as observer decoding and branch routing.
Read more
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Summary
This paper addresses the issue of delays in GUI agents' responses to transient events, which often results in correct actions being executed too late. The authors identify that the primary cause of this delay is the expensive autoregressive decoding that occurs on the decision-time critical path. To mitigate this issue, they propose a novel approach called Adaptive Anticipatory Policy Trees (AAPT). AAPT allows for the pre-compilation of a conditional policy tree during idle periods, which includes observable guards, pre-authorized actions, and deadlines specific to branches. This preparation enables the agent to quickly match incoming events to the appropriate action without needing to generate new text at the moment of the event. The authors conducted experiments that demonstrated AAPT significantly improved the success rate of actions from 0.50 to 0.79 in contested decision windows, while maintaining accuracy with no incorrect actions. The study also reveals that the performance gains are contingent upon three key factors: fast observer decoding, valid tree planning, and accurate branch routing. The findings suggest that AAPT is particularly effective when candidate actions can be enumerated in advance, contrasting with reactive methods that excel in scenarios where this is not possible.
Methodology
The authors developed the Adaptive Anticipatory Policy Trees (AAPT) framework, which involves pre-compiling a conditional policy tree during idle periods. This tree is designed to match incoming events quickly, allowing for immediate action execution without the need for real-time decoding. The methodology includes controlled experiments with pre-registered endpoints and exact McNemar tests to validate the effectiveness of AAPT against traditional reactive methods.
Results
The implementation of AAPT resulted in a success rate increase from 0.50 to 0.79 in contested decision windows (p = 1.8 Γ 10β3), with no incorrect actions produced. Additional experiments confirmed that the performance improvements were linked to the latency-based sizing of the policy tree and the capabilities of the observer and routing mechanisms. The findings were replicated on an independent multimodal model, further validating the approach.
Implications
The results suggest that AAPT could be utilized in various applications where timely responses to transient events are critical, such as in user interface automation, gaming, and real-time decision-making systems. The insights gained from this research could inform the design of more efficient GUI agents that can anticipate and react to user actions more effectively.
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Large Language Models
NLP
Efficient ML
- Quantization to 4-bit weights amplifies existing failures in LLMs rather than introducing new ones.
- Standard metrics can mask the true performance degradation due to the error budget in benchmarks.
- Tightening the error budget reveals significant performance gaps that are otherwise hidden.
- The propensity for failure at full precision is a critical factor in how quantization affects model performance.
Read more
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Summary
This paper investigates the impact of post-training quantization of large language models (LLMs) to 4-bit weights, particularly in multi-turn, tool-calling agents. The authors challenge the prevailing notion that such quantization is nearly lossless, especially in agentic settings where models interact with environments. Through experiments on the Ο 2-bench benchmark, they find that while standard metrics show no significant degradation in performance, quantization amplifies existing failures, particularly in tool-name hallucination, by up to 2.5 times. The study reveals that the flat scores reported are misleading due to the error budget of the benchmark, which allows for multiple failures without penalizing the overall score. When the error budget is tightened, the true extent of the damage becomes apparent. The authors propose that diagnostics such as per-channel error rates and success under varying error budgets should be reported alongside task rewards to provide a clearer picture of model performance. They conclude that the propensity of a model to fail at full precision is a key factor in determining the extent of damage caused by quantization.
Methodology
The authors conducted experiments using the Ο 2-bench benchmark, comparing the performance of full-precision and quantized LLMs across multiple domains and model families. They analyzed task success rates, error rates, and the impact of varying error budgets on performance metrics.
Results
The results indicated that while quantization appeared to have no significant effect on overall task success rates, it significantly increased the frequency of existing errors, particularly in tool-name hallucination. When the error budget was reduced, the performance gap became evident, highlighting the limitations of relying solely on standard metrics.
Implications
The findings suggest that practitioners should be cautious when interpreting performance metrics of quantized models, as they may not accurately reflect the models' operational capabilities. The proposed diagnostics could lead to more robust evaluations of model performance in real-world applications.
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
Reinforcement Learning
Large Language Models
Optimization
- Identifies HPC task heterogeneity as a significant challenge for RL post-training.
- Proposes HARGO, which utilizes confidence-modulated advantage for adaptive response weighting.
- Demonstrates that uniform-weight RL methods are inadequate for heterogeneous tasks.
- Achieves best performance across multiple metrics in a comprehensive evaluation of RL methods.
Read more
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
Summary
The paper introduces HARGO, a novel approach for optimizing large language models (LLMs) post-training using reinforcement learning (RL) specifically tailored for high-performance computing (HPC) tasks. Traditional supervised fine-tuning (SFT) methods equip LLMs with domain knowledge but fail to ensure task-appropriate behavior, leading to verbose and imprecise outputs. HARGO addresses this issue by recognizing the extreme heterogeneity in HPC tasks, which vary significantly in answer length, reward distributions, and SFT accuracy. The proposed method employs per-response importance weighting through a confidence-modulated advantage mechanism, allowing for adaptive learning without requiring task-type labels. HARGO was evaluated across four HPC tasks and outperformed existing methods, achieving superior performance metrics while demonstrating the importance of both discrimination and confidence signals in enhancing model alignment and response quality.
Methodology
HARGO employs a per-response importance weighting strategy that computes a modulated advantage based on group-level reward contrast and confidence signals derived from reference model log-probabilities. This method does not require task-type labels and is designed to enhance learning efficiency by adapting the weight of responses based on their informativeness.
Results
HARGO achieved the best performance on three primary metrics: WinRate of 54.62%, Data Race F1 score of 91.30%, and PLP Similarity of 0.8558, outperforming the baseline HPC-GPT and other RL methods. HARGO also demonstrated strong performance across individual tasks, leading in data race detection and achieving perfect accuracy in PLP tasks.
Implications
The findings suggest that HARGO can significantly improve the application of LLMs in HPC tasks, enhancing their utility in real-world scenarios where task heterogeneity is prevalent. This approach could lead to more efficient training methods for LLMs in various domains requiring precise and concise outputs.
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
Optimization
Federated Learning
Theory
- Introduces F2CTO, the first method for distributed robust coreset selection.
- Formulates coreset selection as a constrained trilevel optimization problem.
- Achieves a non-asymptotic convergence rate of O(Ο΅β3/2).
- Demonstrates effectiveness through extensive empirical evaluations.
Read more
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
Summary
This paper addresses the challenge of robust coreset selection in distributed networks, particularly in the context of the Internet of Things (IoT), where vast amounts of data are generated. The authors propose a novel framework called Federated First-order Constrained Trilevel Optimization (F2CTO) to tackle the problem of selecting a representative subset of data (coreset) while ensuring model robustness and maintaining data privacy. The work formulates the coreset selection as a constrained trilevel optimization problem, which is more complex than traditional bilevel optimization approaches. The F2CTO framework integrates a hierarchical composite value-function reformulation with a distributed alternating projected gradient algorithm, allowing for efficient optimization across distributed nodes without sharing raw data. The authors prove that F2CTO achieves a non-asymptotic convergence rate of O(Ο΅β3/2) for finding an Ο΅-stationary point, demonstrating its theoretical robustness. Empirical evaluations on continual learning tasks show that F2CTO is effective and efficient, marking a significant advancement in the field of distributed robust coreset selection.
Methodology
The authors develop the F2CTO framework, which reformulates the coreset selection problem into a trilevel optimization format with level-wise constraints. They employ a hierarchical composite value-function approach and a distributed alternating projected gradient algorithm to solve the optimization problem efficiently across distributed networks.
Results
The proposed F2CTO framework is shown to converge to an Ο΅-stationary point with a non-asymptotic rate of O(Ο΅β3/2). Empirical results indicate that F2CTO significantly improves the efficiency and effectiveness of coreset selection in continual learning scenarios, outperforming existing methods.
Implications
This work has significant implications for the development of robust machine learning models in distributed environments, particularly in IoT applications where data privacy and computational efficiency are critical. The proposed framework can enhance model training processes by enabling effective data selection without compromising privacy.
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
Time Series
- ZUNA1.1 allows for flexible reconstruction of EEG signals with variable lengths and arbitrary channel configurations.
- The model significantly outperforms traditional EEG denoising methods, showcasing its effectiveness in real-world applications.
- Innovative training techniques, including implicit augmentation and quality-aware preprocessing, enhance model robustness.
- The expanded training corpus and diverse dropout strategies improve the model's ability to handle realistic scenarios of data corruption.
Read more
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
Summary
The paper introduces ZUNA1.1, a 380M-parameter diffusion autoencoder designed for flexible EEG signal reconstruction. This model enhances the capabilities of its predecessor, ZUNA1, by allowing for variable-length sequence reconstruction of EEG data, accommodating up to 30 seconds of data from an arbitrary number of channels and scalp locations. ZUNA1.1 addresses limitations in the original model, such as fixed window sizes and inadequate dropout strategies, by implementing variable-length training, implicit data augmentation, quality-aware preprocessing, and diverse dropout mixtures. The model demonstrates superior performance compared to standard EEG denoising methods, including spherical spline interpolation, and is trained on a significantly larger dataset of approximately 3.5 million channel-hours. The open-source release of ZUNA1.1 under the Apache 2.0 license promotes accessibility for further research and application in EEG signal processing.
Methodology
ZUNA1.1 employs a diffusion autoencoder architecture, trained on randomly cropped EEG signal windows ranging from 0.5 to 30 seconds. It incorporates implicit data augmentation techniques, quality-aware preprocessing for channel data, and a mixture of dropout schemes to simulate various real-world data corruption scenarios. The model is trained on a large dataset of approximately 3.5 million channel-hours, enhancing its generalization capabilities.
Results
ZUNA1.1 achieves performance on par with or better than the previous ZUNA1 model and significantly outperforms standard EEG reconstruction methods like spherical spline interpolation. The model's ability to reconstruct EEG signals accurately across diverse conditions demonstrates its effectiveness and flexibility.
Implications
The advancements presented in ZUNA1.1 have significant implications for both clinical and consumer applications of EEG technology. Its flexibility and robustness in handling noisy and corrupted data can enhance the usability of EEG in various settings, including mental health monitoring, cognitive research, and brain-computer interfaces.
The Convergence Behavior of Adam under Heavy-Tailed Noise
Optimization
Theory
- Establishes the first convergence guarantees for Adam under heavy-tailed noise.
- Generalizes the online-to-nonconvex conversion framework to accommodate heavy-tailed noise.
- Demonstrates that Adam converges to (Ο, Ο΅)-stationary points but with suboptimal iteration complexity.
- Improves convergence rates when the domain radius is known and used in the optimization process.
Read more
The Convergence Behavior of Adam under Heavy-Tailed Noise
Summary
This paper presents the first convergence guarantees for the Adam optimizer in the presence of heavy-tailed stochastic noise, a scenario increasingly relevant in modern deep learning. The author identifies a gap in the theoretical understanding of Adam-type algorithms when stochastic gradients exhibit only a bounded p-th central moment for p in (1, 2]. To address this, the paper generalizes the online-to-nonconvex conversion framework to include heavy-tailed martingale-difference noise. A comprehensive discounted regret analysis of the vector-form Adam update is conducted without imposing restrictive parameter couplings. The findings indicate that while Adam can converge to (Ο, Ο΅)-stationary points under heavy-tailed noise, it demonstrates suboptimal iteration complexity that is p-dependent. However, when the domain radius is known and utilized to control the output of the online learner, the convergence rate can be improved to achieve optimal complexity. This work provides new theoretical insights into the robustness and limitations of the Adam optimizer in heavy-tailed environments, highlighting the need for further exploration in this area.
Methodology
The paper employs a generalized online-to-nonconvex conversion framework to analyze the Adam optimizer under heavy-tailed stochastic noise. It conducts a discounted regret analysis of the vector-form Adam update without restrictive parameter coupling, allowing for a more accurate representation of the algorithm's behavior in practice.
Results
The analysis reveals that Adam converges to (Ο, Ο΅)-stationary points under heavy-tailed noise, although the iteration complexity is suboptimal and p-dependent. When the domain radius is known, the convergence rate can be optimized to match the best-known rates for non-smooth stochastic optimization.
Implications
The findings suggest that while Adam is effective in heavy-tailed noise scenarios, its suboptimal convergence characteristics may necessitate the development of new optimization strategies or modifications to existing algorithms to enhance performance in such environments. This research could influence future work on adaptive optimization methods in deep learning.
Latent States in Neural Networks: Recovering the Temporal Structure of Drifting Data from Model Weights
NLP
Time Series
Multimodal
- Latent states in neural networks can be recovered from model weights using HMMs.
- Improved generalization occurs when training and test data share the same latent state.
- The study demonstrates the applicability of this framework in misinformation detection and sentiment analysis.
- The within-state transfer advantage exceeds that of naive equal-size partitions.
Read more
Latent States in Neural Networks: Recovering the Temporal Structure of Drifting Data from Model Weights
Summary
This paper investigates whether discrete regimes in temporally drifting data streams can be recovered from the weights of neural network models trained on such data. The author employs a hidden Markov model (HMM) to analyze the trajectory of model weights over time, focusing on two domains: multimodal misinformation detection using the Fakeddit dataset and sentiment analysis using the Yelp dataset. By training classifiers on consecutive temporal windows and aligning their weights, the study finds that the latent states identified by the HMM correspond to coherent phases in the data, leading to improved generalization when test data falls within the same latent state as the training data. This within-state transfer advantage persists even after controlling for temporal proximity and surpasses a naive partitioning approach. The results indicate that the estimated states correlate more strongly with shifts in class distribution than with weight-space geometry, suggesting that the latent states capture relevant structural information for transfer learning beyond mere data distribution shifts.
Methodology
The author trains classifiers on consecutive temporal windows of data, aligns and compresses the resulting weight vectors, and fits a hidden Markov model (HMM) to the ordered sequence of these weights. This approach allows for the identification of latent states that segment the timeline into coherent phases.
Results
The study finds that classifiers trained on data within the same latent state generalize better to test data than those trained across state boundaries. This effect is robust against controls for temporal proximity and is more pronounced than results from a naive partitioning of the data. The latent states also show a strong correlation with shifts in class distribution.
Implications
The findings suggest that understanding the latent states in neural networks can enhance model performance in dynamic environments, particularly in applications involving temporal data shifts. This approach could be beneficial for developing more robust models in fields such as misinformation detection and sentiment analysis.
Information Bottleneck Learning for Faithful Time Series Forecasting Explanations
Time Series
Interpretability
- IB-Forecast provides a self-interpretable framework for multivariate time-series forecasting.
- The model decomposes forecasts into a learned periodic profile and a gated deviation readout.
- It incorporates an information bottleneck that allows users to control the amount of historical data used for predictions.
- IB-Forecast guarantees high explanation fidelity while matching the forecasting accuracy of leading black-box models.
Read more
Information Bottleneck Learning for Faithful Time Series Forecasting Explanations
Summary
The paper introduces IB-Forecast, a novel multivariate time-series forecasting framework that emphasizes both predictive accuracy and interpretability. As forecasting plays a critical role in decision-making across various sectors, understanding the historical data influencing predictions is essential. Existing interpretable forecasting models often fail to provide faithful explanations, while faithfulness-oriented methods are typically designed for post-hoc analysis in classification tasks. IB-Forecast addresses this gap by integrating an information bottleneck approach that allows for end-to-end optimization, enabling users to control explanation sparsity. The framework decomposes forecasts into a learned periodic component and a residual component, utilizing explainable masks over input tokens. The authors demonstrate that IB-Forecast achieves competitive forecasting accuracy compared to leading black-box models while ensuring high explanation fidelity. Extensive experiments reveal that IB-Forecast consistently outperforms traditional explanation methods across various datasets, requiring only a fraction of historical observations to maintain low forecasting error.
Methodology
IB-Forecast employs an information bottleneck approach to balance the retention of forecasting-relevant information and the limitation of unnecessary historical data. It decomposes predictions into a periodic profile and a gated deviation readout, where the gating mechanism acts as an information bottleneck. This allows the model to prioritize the most informative historical observations while generating explanations during the forecasting computation.
Results
The experimental results indicate that IB-Forecast matches the forecasting accuracy of state-of-the-art black-box models without incurring additional inference costs. Under matched sparsity budgets, the explanations provided by IB-Forecast consistently outperform gradient-based, occlusion-based, and optimization-based baselines across all evaluated datasets. The model requires only 14-20% of the observations to achieve low-error predictions.
Implications
The findings suggest that IB-Forecast can be effectively utilized in critical decision-making domains such as energy, transportation, and healthcare, where understanding the rationale behind predictions is as important as the predictions themselves. The framework's ability to provide faithful explanations while maintaining accuracy could enhance trust and transparency in automated forecasting systems.
TopoFormer: Topology Meets Attention for Graph Learning
Graph Learning
- TOPOFORMER introduces a scalable method for encoding topological structures into attention-ready sequences.
- The Topo-Scan module allows for efficient processing of graphs by Transformers, bypassing traditional persistent homology pipelines.
- The framework demonstrates strong performance on various graph learning benchmarks, matching or exceeding existing models.
- Theoretical guarantees on the robustness of topological encodings are provided, ensuring practical applicability.
Read more
TopoFormer: Topology Meets Attention for Graph Learning
Summary
The paper introduces TOPOFORMER, a novel framework for graph representation learning that integrates topological data analysis with attention mechanisms. Central to TOPOFORMER is the Topo-Scan module, which transforms graphs into short, ordered sequences of topological tokens by utilizing node or edge filtrations. This approach captures multi-scale structural patterns, enabling the processing of graphs by Transformers without the need for extensive preprocessing or node embeddings. The authors provide theoretical guarantees on the stability of their topological encodings and demonstrate that TOPOFORMER achieves state-of-the-art performance on graph classification and molecular property prediction tasks. The framework is designed to be lightweight and scalable, addressing the computational challenges associated with traditional persistent homology methods. By bridging topological data analysis and deep learning, TOPOFORMER paves the way for more generalizable graph learning models that leverage topological inductive biases.
Methodology
TOPOFORMER employs the Topo-Scan module to convert graphs into sequences of topological tokens, which are then processed by a Transformer architecture. This method avoids the computationally intensive steps of traditional persistent homology, such as persistence diagram computations, while retaining essential multi-resolution topological information. The framework is designed to be parallelizable and integrates seamlessly with existing deep learning architectures.
Results
The authors report that TOPOFORMER achieves state-of-the-art results on graph classification and molecular property prediction tasks. The framework's performance is competitive with strong graph neural network (GNN) and topology-based baselines, demonstrating both efficiency and effectiveness in handling graph data.
Implications
TOPOFORMER's integration of topological data analysis with attention mechanisms could lead to more robust and generalizable models for graph learning. Its lightweight and scalable nature makes it suitable for large-scale applications in various domains, including biology, chemistry, and social networks.
Learning-Augmented and Randomized Algorithms for Line Aggregation with Delays
Theory
Optimization
- Introduction of a deterministic learning-augmented algorithm for line aggregation with delays.
- Development of a randomized algorithm with improved competitive ratio against adversarial models.
- Establishment of a new lower bound for randomized algorithms in the context of line aggregation.
- Combination of learning augmentation and randomization to enhance algorithm performance.
Read more
Learning-Augmented and Randomized Algorithms for Line Aggregation with Delays
Summary
This paper investigates learning-augmented and randomized algorithms for online aggregation with delays on a line metric. The authors propose a deterministic learning-augmented Balance algorithm that achieves (4/Ξ» + 1/Ξ»2)-robustness and (4 + Ξ»)-consistency, where Ξ» is a parameter indicating the trust in the advice provided. Additionally, they introduce a randomized algorithm that operates in the classical adversarial model, achieving an (e + 1)-competitive ratio, which is an improvement over the deterministic 5-competitive benchmark. The paper also establishes a lower bound of e for randomized algorithms, enhancing the previous lower bound of e/(e - 1). Furthermore, a randomized learning-augmented algorithm is developed, yielding (e/Ξ» + 1/Ξ»2)-robustness and (e + Ξ»)-consistency. The authors conduct numerical experiments to validate their theoretical findings, demonstrating that their algorithms outperform existing online algorithms and maintain stability as the advice replacement rate increases.
Methodology
The authors utilize a combination of deterministic and randomized approaches to develop algorithms for line aggregation with delays. They analyze the performance of these algorithms through worst-case competitive analysis and establish robustness and consistency metrics based on the trust in provided advice. Numerical experiments complement the theoretical analysis to evaluate the algorithms' empirical performance.
Results
The deterministic learning-augmented algorithm achieves (4/Ξ» + 1/Ξ»2)-robustness and (4 + Ξ»)-consistency. The randomized algorithm achieves an (e + 1)-competitive ratio, surpassing the deterministic benchmark. The randomized learning-augmented algorithm achieves (e/Ξ» + 1/Ξ»2)-robustness and (e + Ξ»)-consistency. The numerical experiments confirm that the proposed algorithms outperform existing online algorithms and remain stable with varying advice replacement rates.
Implications
The findings suggest that integrating learning augmentation with randomized strategies can significantly enhance the performance of online algorithms in real-world applications, particularly in scenarios involving delays and uncertain request patterns. This approach could be beneficial in logistics, data transmission, and other domains requiring efficient online decision-making.
Certifying when decision-time information justifies adaptive experimentation
Theory
Optimization
Robotics
- Opal framework determines the justification for enabling adaptive experimentation before it occurs.
- Establishes an impossibility boundary for non-trivial authorization under certain conditions.
- Demonstrates effectiveness through empirical studies, achieving a low false-activation rate.
- Links operational gain to information and reuse cost, ensuring technically safe policies are also adoptable.
Read more
Certifying when decision-time information justifies adaptive experimentation
Summary
This paper introduces the Opportunity-aware Policy Authorization for Laboratories (Opal), a framework designed to determine whether adaptive experimentation should be enabled based on decision-time information. Unlike existing methods that assume adaptation is permitted, Opal evaluates the justification for adaptation before it occurs. The framework incorporates a precommitted contract that mandates non-trivial adaptation, controlled risk, and positive executed value after costs. The authors establish an impossibility boundary indicating that source outcomes and unlabelled target covariates cannot uniformly support non-trivial authorization under unrestricted conditional outcome shifts. They derive a target-calibrated recovery method to address this issue. Empirical studies demonstrate Opal's effectiveness, particularly in a Cell Painting experiment involving 11,265 compounds, where it successfully selected 595 compounds with a 5.18% false-activation upper bound, remaining below the 7.5% limit. The paper emphasizes the importance of distinguishing between policy misalignment and non-certifiability, establishing a new layer of authorization for safe adaptive scientific practices.
Methodology
The authors developed the Opal framework, which integrates a precommitted contract to assess the necessity of adaptation based on decision-time information. They formalized the impossibility boundary for non-trivial authorization and derived a target-calibrated recovery method. Empirical validation was conducted using a Cell Painting dataset, where they applied the framework to select compounds and measure performance metrics.
Results
In the Cell Painting experiment, Opal activated 595 compounds, with 384 identified as positive opportunities. The false-activation upper bound was 5.18%, which is below the established limit of 7.5%. The framework successfully recovered a positive-value active branch, demonstrating its capability to certify adaptive opportunities effectively.
Implications
The Opal framework has significant implications for autonomous laboratories and adaptive experimentation, providing a structured approach to decision-making that enhances safety and efficiency. It can be applied in various scientific fields where adaptive experimentation is critical, ensuring that decisions are based on robust evidence and risk management.
TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
Multimodal
- Introduces a risk-guided mixture-of-experts model for multimodal fusion.
- Defines modality reliability based on expected prediction loss from unimodal predictors.
- Combines estimated risk with expert-specific compatibility for improved routing.
- Demonstrates superior performance on multiple biomedical classification tasks.
Read more
TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
Summary
The paper introduces TIER-MoE, a novel framework for multimodal biomedical classification that addresses the challenges of modality reliability and complementarity in predictive modeling. Traditional multimodal models often fail to accurately assess the reliability of different modalities, leading to suboptimal predictions. TIER-MoE innovatively defines sample-specific modality reliability as the expected prediction loss of unimodal predictors, which is learned from out-of-fold predictions. This approach allows for a more nuanced routing of modalities to specialized experts based on their reliability and compatibility. The model maintains an always-active shared path to ensure multimodal complementarity. Evaluated on four public biomedical datasets related to Alzheimer's disease, skin-lesion malignancy, and retinal classification, TIER-MoE outperformed state-of-the-art methods in predictive performance and probability calibration, demonstrating significant improvements in Macro-F1 scores and Brier scores, along with strong zero-shot generalization capabilities.
Methodology
TIER-MoE employs a hierarchical framework that estimates the conditional risk of each modality, using out-of-fold predictions to gauge reliability. It integrates this risk with low-rank expert subspaces to facilitate sparse routing of modalities to specialized experts, while also maintaining a shared path for multimodal interaction.
Results
The results indicate that TIER-MoE significantly enhances predictive performance compared to existing methods, achieving higher Macro-F1 and Brier scores across the evaluated datasets. The model also demonstrated robust zero-shot generalization, indicating its effectiveness in handling unseen data.
Implications
The TIER-MoE framework has the potential to improve diagnostic accuracy and patient stratification in biomedical applications by effectively leveraging multimodal data. Its approach to modality reliability could be adapted to other domains where multimodal data is prevalent.
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Multimodal
NLP
Large Language Models
- Introduces a Structured Evidence Ledger for tracking multimodal reasoning.
- Identifies and addresses four recurring failure patterns in multimodal agentic reasoning.
- Implements a Three-Layer Grounding Protocol to ensure evidence integrity.
- Demonstrates improvements in answer accuracy and trajectory-level faithfulness.
Read more
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Summary
The paper presents LedgerMind, a novel framework for multimodal agentic reasoning that addresses the limitations of current evaluation methods in visual question answering systems. Traditional approaches primarily focus on final-answer accuracy, which fails to provide insights into the reasoning process behind the answers. LedgerMind introduces a Structured Evidence Ledger that normalizes tool outputs and serves as a provenance-constrained state machine. This ledger allows for the tracking of evidence and ensures that claims made during reasoning are auditable against the evidence that supports them. The framework incorporates a Three-Layer Grounding Protocol, an Adaptive Dual-Path Dispatcher, and an Event-Triggered Verification-and-Repair engine to enhance the reliability of reasoning processes. The authors identify four common failure patternsβunsupported claims, Phantom Grounding, over-reasoning, and repair-time amplificationβthat can obscure the accuracy of final answers. By employing LedgerMind, the authors demonstrate improvements in both answer accuracy and trajectory-level faithfulness across various multimodal reasoning benchmarks.
Methodology
The authors developed LedgerMind as a structured framework that normalizes tool outputs into a ledger format, ensuring that all claims made during reasoning can be traced back to valid evidence. They implemented a Three-Layer Grounding Protocol to verify the integrity of cited evidence, an Adaptive Dual-Path Dispatcher to optimize reasoning depth, and an Event-Triggered Verification-and-Repair engine to manage repairs without introducing unsupported claims.
Results
Experiments conducted across multiple multimodal reasoning benchmarks indicated that LedgerMind significantly improved both the accuracy of answers and the faithfulness of the reasoning trajectories, effectively addressing the identified failure patterns.
Implications
The findings suggest that LedgerMind can enhance the reliability of multimodal agents in applications such as visual question answering, where understanding the reasoning process is crucial. This framework could be applied to improve transparency and accountability in AI systems that rely on multimodal reasoning.
Recursive transformers for semiconductor thermo-mechanical reliability
Efficient ML
- Conventional transformers are over-parameterized for small engineering datasets, leading to overfitting.
- Three recursive transformer architectures are proposed to optimize parameter efficiency and computational cost.
- Recursive weight-sharing allows deeper models without increasing the number of parameters.
- The models were validated on thermo-mechanical reliability and capacitance field prediction tasks.
Read more
Recursive transformers for semiconductor thermo-mechanical reliability
Summary
This paper addresses the challenges of using conventional transformer architectures in engineering design, particularly in semiconductor thermo-mechanical reliability analysis. Traditional transformers are often over-parameterized for the small, low-dimensional datasets typical in this field, leading to overfitting and increased computational costs. The author proposes three recursive transformer paradigms: Tiny Recursive Model, Depth Recursive Model, and a simple recursive transformer, focusing on reducing parameter count while maintaining predictive performance. The study systematically evaluates these models based on metrics such as Recall@K and Mean Reciprocal Rank, alongside computational complexity measured in FLOPs. The proposed recursive weight-sharing transformers demonstrate a significant trade-off between prediction accuracy, parameter efficiency, and computational cost, making them suitable for resource-constrained environments. The models are validated through two engineering tasks: thermo-mechanical reliability analysis of semiconductor packages and a Laplace PDE iterative numerical solver for capacitance fields, showcasing their effectiveness in small-data surrogate modeling.
Methodology
The paper introduces recursive weight-sharing transformers, which utilize a reusable transformer template (RECUR) to reduce the number of parameters while allowing for deeper computation. The Tiny Recursive Model and Depth Recursive Model are specifically designed to handle low-dimensional datasets in engineering applications, focusing on computational efficiency and predictive performance. The models are evaluated using metrics such as Recall@K and Mean Reciprocal Rank, and their computational complexity is assessed in terms of FLOPs.
Results
The recursive transformer architectures demonstrated improved predictive performance and reduced parameter counts compared to conventional transformers. The models effectively handled the thermo-mechanical reliability analysis and capacitance field modeling tasks, providing a viable alternative for small-data engineering applications. The results indicate that recursive weight-sharing transformers can achieve a balance between accuracy and computational efficiency.
Implications
The findings suggest that recursive transformers can be effectively utilized in engineering design workflows, particularly in environments with limited computational resources. This approach can enhance the efficiency of surrogate modeling in various applications, including electronic design automation and real-time design optimization.
ECG-InterpBench: Benchmarking the Interpretability of ECG Foundation Models with Matched-Scale Sparse Autoencoders
Interpretability
Time Series
- ECG-InterpBench provides a systematic approach to evaluate ECG model interpretability.
- The benchmark uses matched-scale sparse autoencoders for controlled comparisons across models.
- Distinct interpretability profiles were identified among the evaluated ECG foundation models.
- The benchmark includes a comprehensive interpretability atlas with 450 evaluation cells.
Read more
ECG-InterpBench: Benchmarking the Interpretability of ECG Foundation Models with Matched-Scale Sparse Autoencoders
Summary
The paper introduces ECG-InterpBench, a benchmark aimed at evaluating the interpretability of electrocardiogram (ECG) foundation models (FMs) beyond traditional predictive performance metrics. Existing benchmarks primarily focus on downstream accuracy, which limits insights into the internal representations of these models. ECG-InterpBench employs matched-scale sparse autoencoders (SAEs) to provide a standardized framework for assessing representation interpretability across six frozen ECG FMs. The benchmark evaluates three key dimensions: sparse reconstruction fidelity, single-feature accessibility related to clinically meaningful ECG measurements, and cross-seed reproducibility of features. A comprehensive evaluation was conducted across five encoder depths, five matched dictionary widths, and three random seeds, resulting in a 450-cell interpretability atlas. The findings reveal distinct interpretability profiles among the ECG FMs, with different models excelling in reconstruction fidelity and clinical accessibility. The benchmark's implementation, including evaluation code and metrics, is publicly available, facilitating reproducible research in ECG model interpretability.
Methodology
The methodology involves using matched-scale sparse autoencoders to evaluate six frozen ECG foundation models across various depths and dictionary widths. The interpretability is assessed through metrics that measure reconstruction fidelity, clinical feature accessibility, and reproducibility of features across different random seeds.
Results
The evaluation produced a 450-cell interpretability atlas, revealing that different ECG foundation models exhibit unique interpretability profiles. The matched replication on MIMIC-IV-ECG confirmed that reconstruction fidelity and clinical accessibility metrics identify different leading models, highlighting the importance of representation interpretability in clinical applications.
Implications
The ECG-InterpBench framework enhances the understanding of ECG foundation models' internal representations, which is crucial for clinical applications. By providing a reproducible and capacity-controlled benchmark, it encourages further research into model interpretability, potentially leading to improved diagnostic tools and better patient outcomes.
Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs
Large Language Models
Efficient ML
- Prox leverages the SwiGLU intermediate state for effective channel selection without incurring high computational costs.
- The framework consists of two stages: a lightweight proxy construction followed by exact sparse computation.
- Prox achieves up to a 1.99Γ speedup in end-to-end decoding at 70% FFN sparsity with minimal accuracy loss.
- The method is compatible with quantization and sparse attention, allowing for synergistic efficiency improvements.
Read more
Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs
Summary
The paper introduces Prox, a novel two-stage training-free framework aimed at enhancing activation sparsity in feed-forward networks (FFNs) used in large language models (LLMs). FFNs are identified as significant contributors to memory traffic and computation during LLM inference, making them prime candidates for optimization. Existing training-free methods often lead to model quality degradation at high sparsity levels due to ineffective channel-selection strategies. The authors observe that the intermediate state of the SwiGLU activation function can provide a robust channel-selection signal, although obtaining it typically requires expensive dense computations. Prox addresses this by constructing a channel mask based on the magnitude ranking of the intermediate state entries rather than their exact values. The framework consists of two stages: the first stage constructs a shared mask using input sparsity and quantized proxy weights, while the second stage computes the selected channels accurately, allowing for sparse execution across all projections. The evaluation of Prox across ten LLMs from six model families demonstrates its superiority over existing training-free baselines, achieving significant speedups in decoding while maintaining accuracy.
Methodology
Prox employs a two-stage approach: Stage 1 constructs a shared channel mask using input sparsity and quantized proxy weights, while Stage 2 computes the selected channels exactly using the original weights. This method allows for efficient sparse execution of the SwiGLU projections without the need for extensive training.
Results
Prox consistently outperformed state-of-the-art training-free baselines across ten LLMs, particularly at 60-70% FFN sparsity, achieving up to a 1.99Γ speedup in decoding with minimal accuracy degradation.
Implications
The findings suggest that Prox could significantly enhance the efficiency of LLM inference in resource-constrained environments, making it a valuable approach for deploying large models in practical applications.
Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
Time Series
- Introduces a structured evaluation protocol for EEG emotion recognition that separates target quantity, development procedure, and reporting rules.
- Demonstrates significant differences in reported accuracies based on evaluation protocols, particularly between subject-dependent and subject-disjoint evaluations.
- Finds that checkpoint selection based on repeated test evaluations can substantially improve accuracy metrics.
- Highlights the importance of considering subject identity and other factors in model evaluation to avoid overestimating performance.
Read more
Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
Summary
This paper addresses the critical issue of evaluation protocols in EEG emotion recognition, emphasizing that reported accuracy is influenced by the entire evaluation process rather than just the classifier used. The authors propose a structured approach to separate target quantities, development procedures, and reporting rules, which is illustrated through the application of a dynamical graph convolutional neural network (DGCNN) on the SEED and SEED-IV datasets. The study highlights the discrepancies in accuracy results due to different evaluation protocols, particularly in subject-dependent and subject-disjoint evaluations. The authors conduct a series of experiments to assess the impact of checkpoint selection and target-data access on performance metrics. They find that while subject-dependent evaluations yield high accuracies, the performance drops significantly when evaluating entirely held-out participants, indicating the importance of considering subject identity and other factors in model evaluation. The paper concludes with a proposed EEG-specific evaluation methodology and an auditable reporting checklist to enhance reproducibility and clarity in future studies.
Methodology
The authors utilized a dynamical graph convolutional neural network (DGCNN) to analyze EEG data from the SEED and SEED-IV datasets. They implemented various evaluation protocols, including subject-dependent trial-disjoint evaluation and subject-disjoint evaluation without target-data access. The study involved repeated test-based checkpoint selection and comparisons of performance metrics across different evaluation settings.
Results
The results indicated that the SEED dataset achieved a mean window accuracy of 0.8892 after checkpoint selection, while the accuracy for entirely held-out participants was 0.5348. The SEED-IV dataset showed a lower accuracy of 0.3954, which was reported as secondary sensitivity evidence due to unresolved protocol compatibility. The findings underscored the significant gaps in performance when transitioning from training to held-out subjects, challenging the assumption of model generalization.
Implications
The proposed evaluation methodology and checklist can enhance the reproducibility and reliability of EEG emotion recognition studies. By clarifying the evaluation process, researchers can better understand the limitations of their models and improve the comparability of results across different studies.
Rethinking EEG-Based Disease Diagnosis: Decoupling Instance Representation Learning from Subject-Level Supervision
Time Series
- Introduces BridgeMIL, a two-stage framework for EEG-based disease diagnosis.
- Decouples instance representation learning from subject-level supervision to improve diagnostic accuracy.
- Achieves significant performance improvements over traditional methods in EEG datasets.
- Highlights the variability in inherited-label reliability across instances.
Read more
Rethinking EEG-Based Disease Diagnosis: Decoupling Instance Representation Learning from Subject-Level Supervision
Summary
This paper addresses the challenges in EEG-based disease diagnosis by proposing a novel framework called BridgeMIL, which decouples instance representation learning from subject-level supervision. Traditional methods often segment EEG recordings into short instances and assign the same subject label to all instances, leading to potentially misleading instance-level supervision. The authors highlight the limitations of both inherited-label training and end-to-end multiple instance learning (MIL), which struggles with the scarcity of labeled subjects. BridgeMIL consists of two stages: the first stage pretrains an encoder without inherited labels by aligning temporally nearby windows and independently sampled within-subject sub-bags, while employing variance and covariance regularization to prevent representation collapse. The second stage fine-tunes the encoder within an attention-based MIL model, applying supervision only to subject-level predictions and using feature retention to limit representation drift. The proposed method is evaluated on three EEG disease datasets and demonstrates superior performance compared to existing methods, achieving the highest mean accuracy in 14 out of 15 dataset-backbone settings. The findings emphasize the importance of aligning supervision with subject-level objectives while effectively utilizing abundant EEG instances.
Methodology
The methodology involves a two-stage framework: Stage 1 pretrains an encoder without inherited labels by aligning temporally nearby instances and independently sampled sub-bags, utilizing variance and covariance regularization. Stage 2 fine-tunes the encoder in an attention-based MIL model, applying supervision only to subject-level predictions and employing feature retention to maintain representation consistency.
Results
BridgeMIL achieved the highest mean accuracy of 76.57% across three EEG disease datasets, outperforming the strongest baseline by 4.28 percentage points. It excelled in 14 out of 15 dataset-backbone configurations, indicating robust performance across various settings.
Implications
The findings suggest that decoupling instance-level learning from subject-level supervision can enhance the reliability of EEG-based disease diagnosis. This approach may lead to better diagnostic tools in clinical settings, improving the accuracy of neurological and psychiatric disorder diagnoses.
SE(3)-MeanFlow: Few-Step Protein Backbone Generation on Lie Groups
Generative Models
Robotics
Optimization
- Introduces SE(3)-MeanFlow, a few-step generative model for protein backbone design.
- Derives closed-form average-velocity identities for efficient training without numerical integration.
- Achieves competitive performance against existing methods while significantly reducing inference steps.
- Demonstrates practical applications in high-throughput protein design and drug discovery.
Read more
SE(3)-MeanFlow: Few-Step Protein Backbone Generation on Lie Groups
Summary
This paper presents SE(3)-MeanFlow, a novel generative framework for protein backbone generation that operates on the Lie group SE(3). Traditional methods for generating protein backbones often require extensive numerical integration, making them inefficient for high-throughput design. SE(3)-MeanFlow addresses this by extending the MeanFlow approach from Euclidean spaces to the Lie group geometry of protein frames. The authors derive closed-form average-velocity identities for both rotations and translations, allowing for simulation-free training targets. They introduce an SE(3) Ξ±-Flow objective that simplifies the training process by eliminating the Jacobian-vector product from the rotation branch. The model demonstrates significant improvements in generating protein backbones, matching or exceeding the performance of existing flow-matching baselines while using fewer sampling steps. This is particularly advantageous in scenarios where rapid generation is critical, such as drug discovery campaigns.
Methodology
The methodology involves extending the MeanFlow framework to the Lie group SE(3) by deriving average-velocity identities in the Lie algebra so(3) and in R3. The SE(3) Ξ±-Flow objective is introduced to facilitate training, and numerical stabilization techniques are employed to ensure reliable few-step generation of protein backbones.
Results
The SE(3)-MeanFlow model matches or exceeds the performance of flow-matching baselines that require multiple sampling steps, particularly excelling in the few-step regime. The results indicate that the model can generate high-quality protein backbones efficiently, with a modest trade-off in diversity.
Implications
The findings suggest that SE(3)-MeanFlow can significantly enhance the efficiency of protein design processes, making it a valuable tool for high-throughput drug discovery and other applications in computational biology.
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Reinforcement Learning
Large Language Models
NLP
- Introduces an expand-then-compress framework for reasoning policy learning.
- Proposes Residual GRPO for expanding the reasoning solution manifold with complementary teachers.
- Develops Reliability-Gated Teacher-Union OPD for effective compression of teacher knowledge.
- Demonstrates that a student model can outperform individual teachers by leveraging a diverse teacher union.
Read more
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Summary
This paper presents a novel framework for enhancing reasoning capabilities in reinforcement learning (RL) models by addressing the limitations of traditional single-teacher approaches. The authors argue that RL-trained policies often represent only a subset of valid reasoning solutions, leading to incomplete teaching. To overcome this, they propose an 'expand-then-compress' framework that consists of two main stages: expansion and compression. In the expansion stage, Residual Group Relative Policy Optimization (RGRPO) is employed to train multiple teachers that cover different regions of the reasoning solution manifold, ensuring that each subsequent teacher focuses on areas not yet addressed by the existing teacher union. In the compression stage, Reliability-Gated Teacher-Union On-policy Distillation (TU-OPD) is used to distill the knowledge from these teachers into a single student model, allowing it to learn from the most reliable teachers based on per-example performance. The framework also incorporates Consensus-Residual Decomposition to maintain the unique preferences of the best-performing teachers during the aggregation process. Experiments demonstrate that the resulting student model, Qwen3-1.7B, outperforms the strongest individual teacher across various tasks, including mathematical reasoning, code generation, and instruction following, highlighting the effectiveness of constructing and compressing a diverse teacher union.
Methodology
The methodology involves two main processes: (1) Expansion using Residual Group Relative Policy Optimization (RGRPO) to train multiple teachers that focus on different uncovered areas of the reasoning solution manifold, and (2) Compression through Reliability-Gated Teacher-Union On-policy Distillation (TU-OPD) that selectively distills knowledge from the most reliable teachers while preserving unique preferences through Consensus-Residual Decomposition.
Results
The experiments show that the Qwen3-1.7B student model consistently outperforms the strongest individual teacher by 2.0%, 8.3%, and 6.9% across mathematical reasoning, code generation, and instruction following tasks, respectively, demonstrating the effectiveness of the proposed framework.
Implications
The findings suggest that constructing a diverse set of teachers and effectively compressing their knowledge can lead to significantly improved performance in reasoning tasks. This approach could be applied to various domains where complex reasoning is required, potentially enhancing the capabilities of AI systems in real-world applications.
Strategies for Milestone-driven Start-ups in Multi-activity Settings
Optimization
Theory
- Introduces a stochastic control model for milestone-driven start-ups.
- Characterizes the optimal policy using an efficient frontier curve.
- Demonstrates that the optimal strategy varies based on boundary conditions.
- Shows significant differences in expected payoffs between optimal and myopic policies.
Read more
Strategies for Milestone-driven Start-ups in Multi-activity Settings
Summary
This paper investigates strategies for new venture start-ups operating in milestone-oriented environments, where firms must navigate multiple stages to achieve specific targets. The author models the state of a start-up as a diffusion process, allowing entrepreneurs to choose from various costly activities that influence the drift and variance of this process. The study presents a continuous-time stochastic model with multiple controls, overcoming challenges typically associated with such models. The optimal policy is characterized by an efficient frontier curve, which ranks activities based on riskiness (drift-to-volatility ratio) and cost-effectiveness (drift-to-cost ratio). The findings reveal that the structure of the optimal policy varies depending on the distance between fixed upper and lower boundaries, leading to qualitatively different strategies. This work provides a foundational framework for understanding entrepreneurial decision-making in milestone-driven contexts.
Methodology
The paper employs continuous-time stochastic modeling to analyze a multi-control setting, utilizing an efficient frontier curve to determine the optimal policy for start-ups. The model captures the dynamics of start-up activities and their impact on reaching milestone targets.
Results
The study successfully derives a closed-form solution for the optimal policy, demonstrating that the efficient frontier can take different forms based on the proximity of the upper and lower boundaries. The results indicate that the optimal policy shifts from riskier to less risky controls depending on these boundaries, highlighting the importance of context in entrepreneurial decision-making.
Implications
The findings offer valuable insights for start-up firms in evaluating their activities and adapting strategies based on specific scenarios. This research contributes to the understanding of entrepreneurial decision-making and provides a structured approach for navigating milestone-driven environments.
ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow
Generative Models
Robotics
Time Series
- Introduction of Physical-Time Flow (PT-Flow) for continuous latent modeling.
- ODEWorld effectively addresses representation collapse in latent world models.
- Supports high-fidelity image reconstruction and arbitrary temporal resolution.
- Demonstrates superior performance in video generation and robotic control tasks.
Read more
ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow
Summary
The paper introduces ODEWorld, a novel continuous-time latent world model that leverages Physical-Time Flow (PT-Flow) to address the limitations of existing discrete-time predictive architectures in machine learning. Traditional models struggle with the continuous nature of physical dynamics, often resulting in inefficiencies and representation collapse. ODEWorld overcomes these challenges by employing a continuous latent velocity field governed by an ordinary differential equation (ODE), allowing for temporal integration in a compressed latent space. This architecture enables high-quality image reconstruction, arbitrary temporal resolution, and backward prediction, which are not feasible with discrete-time models. The authors demonstrate the effectiveness of ODEWorld through comprehensive experiments on various simulation and real-world datasets, showcasing its ability to reconcile planning-oriented dynamics with visual realism. The results indicate that ODEWorld excels in video generation and robotic control, providing rich, interpretable representations that facilitate downstream policy learning.
Methodology
The authors developed PT-Flow, which parameterizes the dynamics of sequential data using an ODE in a structured latent representation space. ODEWorld is built on this framework, employing direct supervision of the latent velocity field and decoupling dynamical representation from static context. This allows for efficient learning and high-quality predictions through temporal integration using ODE solvers.
Results
Empirical evaluations show that ODEWorld achieves high-fidelity image reconstruction and effective long-horizon predictions across diverse datasets, including LIBERO and Agibot-World. The model successfully reconciles dynamics abstraction with visual realism, demonstrating robust performance in video generation and robotic control tasks.
Implications
ODEWorld's continuous modeling approach has significant implications for applications requiring high-quality video synthesis and effective robotic policy learning. Its ability to handle irregularly sampled data and provide interpretable dynamics can enhance various AI systems in real-world scenarios.
Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing
NLP
Large Language Models
Efficient ML
- Introduces a new framework for analyzing geometric complementarity in MoE models.
- Demonstrates that expert subspaces overlap significantly while maintaining coherent routing.
- Finds that selected candidates outperform rivals in explaining token representations, but this advantage is context-dependent.
- Shows that adding later experts can improve performance despite low marginal geometric novelty.
Read more
Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing
Summary
This paper investigates the routing mechanisms in sparse mixture-of-experts (MoE) language models, challenging the conventional understanding of geometric complementarity. The authors introduce a diagnostic framework that distinguishes between route coherence, candidate quality, and candidate-by-context interaction using an Expert Subspace Separation Index (ESSI) and a prefix-controlled 2 Γ 2 factorial design. Through empirical analysis across six MoE architectures, the study reveals that expert subspaces exhibit significant overlap, yet the actual routing of tokens leads to better representation than matched alternatives. The findings indicate that selected candidates explain more of the residual representation than their strongest unselected rivals, although the advantage diminishes with actual prefixes. Notably, adding later experts can enhance next-token prediction despite low geometric novelty, leading to the concept of 'coherent overlap' where routing is effective without necessitating disjoint linear coverage. This work clarifies the limitations of relying solely on geometric similarity to assess redundancy or pruning value in MoE models.
Methodology
The authors employed an Expert Subspace Separation Index (ESSI) to measure between-expert distances normalized by local within-expert geometry. They conducted a prefix-controlled 2 Γ 2 factorial analysis to isolate candidate quality, contextual opportunity, and candidate-by-context interactions. The study involved empirical evaluations across multiple MoE architectures, including OLMoE, Mixtral, and DeepSeek, to assess the functional value of routing mechanisms.
Results
The analysis revealed that expert subspaces in MoE models overlap significantly, yet the actual routes provide better token representation than matched alternatives. Selected candidates consistently explained more of the residual representation than their strongest unselected rivals across various contexts. Additionally, the introduction of later experts improved next-token prediction in a majority of cases, supporting the notion of coherent overlap in expert routing.
Implications
The findings suggest that understanding the interactions between selected experts and their contexts is crucial for optimizing MoE models. This could lead to improved design principles for sparse computation, expert pruning strategies, and enhanced performance in language modeling tasks.
It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
Theory
Efficient ML
- Introduction of einx as a universal notation for tensor operations.
- Revisiting vectorization to simplify tensor operation transformations.
- Reduction of complex APIs to a few elementary operations for better readability.
- Seamless integration of einx with existing tensor frameworks.
Read more
It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
Summary
The paper introduces einx, a novel universal notation for tensor operations aimed at addressing the limitations of existing tensor frameworks that utilize Numpy-like notations. The authors argue that these notations are often difficult to read, write, and prone to shape errors due to inconsistent rules across various operations. By revisiting vectorization as a fundamental function for transforming tensor operations, einx provides a declarative and pointful expression style analogous to loop notation. This notation simplifies the complex APIs of existing frameworks into a small set of elementary operations, ensuring consistent rules and enhancing code readability. The authors also present an implementation of einx that integrates seamlessly with popular tensor frameworks, allowing users to leverage its capabilities in their existing workflows. The paper emphasizes the need for a universal model of tensor programming that can accommodate a wide range of operations while maintaining clarity and ease of use.
Methodology
The authors developed einx by analyzing existing tensor operation notations and identifying their shortcomings. They proposed a vectorization-based approach to transform and decompose tensor operations, leading to the creation of a declarative notation that is both pointful and analogous to loop notation. The einx implementation was embedded in Python to ensure compatibility with existing frameworks.
Results
The paper demonstrates that einx can effectively represent a wide range of tensor operations with a consistent and simplified notation. It provides examples showing how various Numpy-like function calls can be mapped to einx expressions, highlighting the reduction in complexity and the clarity of the new notation. The implementation of einx is shown to integrate well with existing tensor frameworks, facilitating its adoption in practical applications.
Implications
The introduction of einx has the potential to streamline tensor programming, making it more accessible and less error-prone for researchers and practitioners in scientific computing and deep learning. By providing a universal notation, it could foster better collaboration and understanding across different domains that utilize tensor operations.
Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning
Reinforcement Learning
Large Language Models
Efficient ML
- KGPS reformulates prompt selection as a dynamic state estimation problem, improving adaptability to policy changes.
- The method uses a Kalman filter to maintain a Gaussian posterior over prompt difficulty, enhancing selection accuracy.
- KGPS achieves state-of-the-art performance in online prompt selection, reducing rollout costs significantly while improving accuracy.
- Extensive experiments validate KGPS across multiple reasoning tasks and RL algorithms, showcasing its robustness and efficiency.
Read more
Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning
Summary
This paper addresses the challenge of selecting prompts of appropriate difficulty during reinforcement learning (RL) finetuning of large language models (LLMs). The authors introduce a novel method called Kalman-Guided Prompt Selection (KGPS), which reformulates prompt selection as a dynamic state estimation problem rather than a static difficulty prediction. KGPS employs a linear-Gaussian state-space model to track each prompt's latent success rate, adjusting for the non-stationary nature of RL training. By utilizing a Kalman filter, KGPS maintains a calibrated Gaussian posterior over prompt difficulty, selecting prompts that maximize expected training utility while favoring intermediate-difficulty prompts. The method is adaptive to policy changes and does not require additional rollouts beyond standard training. Experimental results demonstrate that KGPS significantly improves both accuracy and rollout efficiency across various reasoning benchmarks and RL algorithms, outperforming existing online prompt selection methods.
Methodology
The authors developed KGPS, which models prompt difficulty using a linear-Gaussian state-space framework. A Kalman filter is employed to update the posterior distribution of prompt difficulties based on rollout observations, allowing for dynamic adjustments as the RL policy evolves. The selection process maximizes a posterior-expected training utility that favors prompts of intermediate difficulty.
Results
KGPS demonstrated a reduction of 83% in rollouts compared to the Dynamic Sampling (DS) method while achieving a 0.12 point improvement in average performance across six math reasoning benchmarks. The method consistently outperformed both evaluation-based and prediction-based approaches, establishing new state-of-the-art results in online prompt selection.
Implications
The findings suggest that KGPS can enhance the efficiency of RL finetuning processes in LLMs, making it a valuable tool for improving model performance in various reasoning tasks. The dynamic nature of prompt selection could lead to more effective training strategies in RL applications, potentially benefiting areas such as instruction following and complex reasoning.
Expanding Data-Agnostic Pivotal Instances Selection Models with Proximity Trees and Ensemble Learning
Interpretability
- Introduction of PivotTree, a hierarchical model for selecting pivotal instances to enhance interpretability.
- Incorporation of proximity and oblique splits to refine decision-making processes.
- Data modality-agnostic approach allows application across diverse datasets.
- Experimental results show superior performance compared to traditional instance selection methods.
Read more
Expanding Data-Agnostic Pivotal Instances Selection Models with Proximity Trees and Ensemble Learning
Summary
This paper presents a novel approach to enhancing interpretability in machine learning models through a hierarchical, interpretable pivot selection model called PivotTree. The authors argue that traditional decision-making processes in machine learning often lack transparency, making it difficult for users to understand model predictions. PivotTree addresses this by selecting pivotal instances based on their similarity to input data, thereby creating a more intuitive framework for decision-making. The model not only serves as a pivot selection technique but also functions as a standalone predictive model. The authors extend the concept of single pivots to pairs of pivots, utilizing proximity and oblique trees, as well as ensemble methods to improve performance. The proposed method is data modality-agnostic, allowing it to be applied across various types of data, including tabular data, text, images, and time series. Experimental results demonstrate that PivotTree outperforms existing instance selection strategies and achieves competitive results compared to state-of-the-art interpretable models while maintaining a minimal number of pivots.
Methodology
The authors developed PivotTree, which utilizes a similarity-based decision tree structure for selecting pivotal instances. The model incorporates both single and pairs of pivots, employing proximity and oblique splits to improve decision-making. Additionally, PivotTree is integrated into ensemble models, specifically Random Forests, to enhance predictive performance while maintaining interpretability.
Results
The experiments conducted across various datasets demonstrate that PivotTree outperforms alternative instance selection strategies and achieves competitive results against state-of-the-art interpretable models. The model effectively maintains a minimal number of pivots while ensuring robust predictions.
Implications
The findings suggest that PivotTree can significantly improve the interpretability of machine learning models, making them more accessible for decision-makers in various fields, including healthcare, finance, and beyond. Its data-agnostic nature broadens its applicability, potentially transforming how interpretable models are utilized in practice.
Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture Decoding
Time Series
- Introduces a montage-agnostic encoder for sEMG gesture recognition that adapts across recording sessions.
- Demonstrates significant performance improvement over traditional per-user LDA classification methods.
- Identifies feature-statistic alignment as the only effective label-free adaptation method across subjects.
- Highlights the limitations of batch-normalization re-estimation in adapting to new recording sessions.
Read more
Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture Decoding
Summary
This paper addresses the challenge of maintaining recognition accuracy in surface electromyography (sEMG) gesture decoding across different recording sessions. The authors highlight that variations in electrode placement, skin condition, and muscle activation patterns can lead to significant day-to-day variability, which hampers the effectiveness of myoelectric control systems. The study introduces a montage-agnostic encoder that is trained on data from one recording session and tested on data from a different session without requiring recalibration. The performance of this encoder is compared to a per-user linear discriminant analysis (LDA) classification pipeline and two existing methods that rely solely on data from the same session. The results show that the encoder achieves a macro-F1 score of 0.688, outperforming the per-user pipeline score of 0.540. Additionally, the paper explores five label-free adaptation methods, finding that aligning feature statistics to the new session consistently improves performance across all subjects, while batch-normalization re-estimation fails. This work demonstrates the potential for robust sEMG gesture recognition systems that can adapt to variability without extensive recalibration, thus enhancing usability in practical applications.
Methodology
The study employs a montage-agnostic encoder trained on data from a specific recording session and evaluates its performance on data from subsequent sessions. The encoder's performance is compared against a per-user LDA pipeline and two published methods that utilize data from the same session. Additionally, various label-free adaptation techniques are tested to improve recognition accuracy without requiring labeled data.
Results
The montage-agnostic encoder achieved a macro-F1 score of 0.688, significantly higher than the 0.540 score of the per-user LDA pipeline. Among the adaptation methods tested, only feature-statistic alignment improved performance for all subjects, while batch-normalization re-estimation was ineffective.
Implications
The findings suggest that it is feasible to develop sEMG gesture recognition systems that can maintain performance across different recording sessions without the need for extensive recalibration. This could lead to more practical and user-friendly myoelectric control systems for prosthetics and assistive devices.
QQWorld: Quantile-Quantile Matching for World Model Regularization
Reinforcement Learning
Generative Models
Optimization
- QQWorld replaces the EppsβPulley regularization with a quantile-quantile matching objective.
- The new approach maintains effective corrective gradients for tail deviations, addressing heavy-tailed behavior.
- Cross-batch QQ strategy enhances the effective ranking pool while minimizing memory usage.
- QQWorld significantly improves planning success rates and Gaussian alignment in latent distributions.
Read more
QQWorld: Quantile-Quantile Matching for World Model Regularization
Summary
The paper introduces QQWorld, a novel approach to regularizing latent world models by addressing the limitations of the EppsβPulley (EP) objective, which is used in the LeWorldModel (LeWM). The authors identify that the EP regularizer's corrective gradients diminish for isolated tail samples, leading to insufficient control over heavy-tailed deviations in the learned latent distribution. To overcome this, QQWorld employs a quantile-quantile matching objective that aligns projected latent samples with rank-matched Gaussian quantiles, ensuring effective corrective gradients even in the tails. The authors also propose a cross-batch QQ strategy to enhance the ranking pool without significant memory overhead. Extensive experiments demonstrate that QQWorld not only improves the average planning success rate of LeWM but also achieves better Gaussian alignment and reduces the heaviness of latent tails, indicating a more robust latent representation for planning tasks.
Methodology
The authors analyze the limitations of the EppsβPulley regularizer by interpreting it as a kernel discrepancy and demonstrate its rapid decay of corrective gradients for tail samples. They then propose the quantile-quantile matching objective for aligning latent values with Gaussian quantiles, ensuring stronger corrective signals for extreme values. Additionally, they develop a memory-efficient cross-batch strategy to enhance the ranking pool for the QQ objective.
Results
QQWorld shows a reduction in the mean QQ RMSE from 0.157 to 0.121, indicating improved alignment with the isotropic Gaussian target. The tail rate of the latent distribution is also reduced from 0.315 to 0.123, approaching the Gaussian reference. Overall, QQWorld consistently enhances downstream planning performance across four control environments.
Implications
The findings suggest that QQWorld can be effectively utilized in various planning tasks within latent world models, potentially leading to more accurate predictions and improved decision-making in environments requiring dynamic planning. Its straightforward integration into existing models without additional hyperparameters makes it an attractive option for researchers and practitioners in the field.
Towards joint scaling laws with optimal batch size schedules
Optimization
Large Language Models
Efficient ML
- Dynamic batch size scheduling significantly improves training efficiency compared to static batch sizes.
- The optimal batch size schedule is independent of peak learning rate and model architecture.
- A closed-form solution for batch size scheduling is derived, applicable to various learning rate schedules.
- Joint scaling laws are established, enhancing compute efficiency by 6-15% without additional computational overhead.
Read more
Towards joint scaling laws with optimal batch size schedules
Summary
This paper addresses the optimization of batch size schedules in deep learning, which is often kept static during training. The authors explore the interplay between learning rate and batch size, deriving a joint characterization of loss that applies to various optimizers and model architectures. They present a closed-form solution for optimal batch size schedules corresponding to any learning rate schedule, demonstrating that dynamic batch size scheduling can significantly enhance training efficiency. The study reveals that the optimal batch size is determined by the total computational budget and the shape of the learning rate schedule, rather than being influenced by peak learning rates or model specifics. The authors validate their findings through empirical experiments on large-scale models, showing that their proposed dynamic batch size schedules outperform static ones, leading to improved compute efficiency and maintaining robust training dynamics.
Methodology
The authors utilize stochastic convex optimization theory to analyze deep learning training dynamics. They derive a sequence-to-sequence characterization of loss based on learning rate and batch size schedules, formalizing the problem as a constrained optimization task. The optimal batch size schedule is computed based on total computational budgets and learning rate schedule shapes.
Results
The study shows that dynamic batch size schedules consistently outperform static batch sizes across various models and configurations. Empirical results indicate improvements in compute efficiency by 6-15%, with no additional computational costs. The proposed method preserves universal training dynamics, contributing to a more robust training process.
Implications
The findings suggest that adopting dynamic batch size schedules can lead to more efficient training of large-scale models, potentially reducing the computational resources required for hyperparameter tuning. This approach could be widely applicable in the training of large language models and other deep learning applications, enhancing performance while minimizing costs.
Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control
Reinforcement Learning
Large Language Models
Optimization
- The use of a large language model (LLM) for VAV control can achieve competitive results without building-specific training.
- TD3-guided reinforcement fine-tuning (RFT) can transfer control knowledge but has limitations in action ranking reliability.
- The study found that while TD3 improved energy efficiency, the LLM showed a larger reduction in energy use but compromised ventilation quality.
- The authors propose that transition-focused supervised fine-tuning is essential for improving the effectiveness of RFT.
Read more
Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control
Summary
This paper addresses the challenges of controlling multi-zone variable-air-volume (VAV) systems, which must balance thermal comfort, indoor air quality, and energy consumption. The authors investigate the use of a frontier reasoning model, specifically a large language model (LLM), to achieve effective VAV control without the need for building-specific training. They explore the application of TD3-guided reinforcement fine-tuning (RFT) to transfer control knowledge into a deployable model. The study evaluates five controllers over three summer days using a physics-based emulator. Results show that while TD3 reduced HVAC electricity consumption by 4.5% and improved compliance with temperature and CO2 standards, the LLM (GPT-5) achieved a greater reduction of 6.2% but at the cost of reduced ventilation margins. The authors highlight the limitations of the learned critic in the RFT process, which failed to reliably rank actions within states, and suggest that transition-focused supervised fine-tuning is necessary before value-based RFT can be effective.
Methodology
The authors employed a frontier reasoning model (LLM) to control a multi-zone VAV system without building-specific training. They utilized TD3-guided reinforcement fine-tuning (RFT) to adapt control knowledge into an open-weight model. The evaluation was conducted in a physics-based four-zone emulator over three summer days, comparing the performance of five controllers against a baseline based on ASHRAE Guideline 36.
Results
The results indicated that TD3 reduced HVAC electricity consumption by 4.5% while improving temperature and CO2 compliance. The LLM (GPT-5) achieved a 6.2% reduction in energy use but negatively impacted ventilation margins. The RFT process revealed that the learned critic was unreliable in action ranking, with only 50% accuracy in selecting the best candidate actions during rollouts.
Implications
This research suggests that while LLMs can be effective in HVAC control, there is a critical need for methods that ensure reliable action evaluation and transition modeling. The findings could lead to more scalable and interpretable control systems in HVAC applications, potentially reducing energy consumption and improving indoor air quality.