AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
48
Papers today
8h
Update frequency
7
Days of history
An optimal control approach for neural network architecture adaptation with a posteriori error estimation
Optimization
Theory
Efficient ML
- Introduces a continuous-time optimal control framework for neural network architecture adaptation.
- Derives rigorous a posteriori error estimates to guide layer insertion based on maximum estimated error.
- Proposes a novel architecture representation with piecewise linear weights and biases.
- Utilizes dual weighted residual methodology to compute error bounds and refine architecture.
Read more
An optimal control approach for neural network architecture adaptation with a posteriori error estimation
Summary
This paper introduces a novel method for adapting the architecture of neural networks, specifically focusing on depth adaptation, through a continuous-time optimal control framework. By treating neural network training as an optimal control problem, the authors derive rigorous a posteriori error estimates that quantify how approximation errors are distributed across different layers of the network. This allows for a principled strategy for inserting new layers at locations where the estimated error is maximal, thereby enhancing the network's ability to capture complex, nonlinear variations in the underlying data. The proposed architecture treats weights and biases as piecewise linear functions that vary across layers, and the error estimator provides bounds on the discrepancy between this discrete representation and the true continuous optimal control solution. The methodology leverages dual weighted residual techniques from finite element analysis to compute upper bounds on functional errors. The theoretical contributions include explicit error bounds that decompose total approximation errors into contributions from specific intervals, facilitating targeted architecture refinement. The effectiveness of the proposed method is demonstrated through experiments on scientific datasets, including the observable-to-parameter map for the Navier-Stokes equation, showing that it consistently outperforms existing architecture adaptation methods in terms of generalization performance.
Methodology
The authors reformulate neural network training as a continuous-time optimal control problem, deriving error estimates that allow for the identification of optimal layer insertion points based on error distribution. They utilize dual weighted residual techniques from finite element analysis to compute upper bounds on functional errors, enabling a systematic approach to architecture adaptation.
Results
The proposed method was tested on scientific datasets, including the Navier-Stokes equation, and showed consistent improvements in generalization performance over existing architecture adaptation techniques. The error bounds derived provided a clear basis for targeted refinement of the network architecture.
Implications
This work has significant implications for the design of neural networks, particularly in fields requiring precise modeling of complex phenomena, such as fluid dynamics and other scientific computations. The systematic approach to architecture adaptation could lead to more efficient training processes and better-performing models.
Unlocking Temporal Generalization in Hamiltonian Video Dynamics Models
Generative Models
Reinforcement Learning
Robotics
- Traditional world models are limited to fixed time steps, hindering temporal generalization.
- Hamiltonian Generative Networks (HGN) can predict dynamics based on continuous-time energy functions but struggle in non-conservative settings.
- The paper identifies failure modes in HGN rollouts when extrapolating beyond training step sizes.
- Targeted fixes are proposed for each identified failure mode, enhancing stability in predictions.
Read more
Unlocking Temporal Generalization in Hamiltonian Video Dynamics Models
Summary
This paper addresses the limitations of traditional world models in predicting physical dynamics at variable temporal resolutions. Most existing models are constrained to a fixed time step during training, which hampers their ability to generalize across different time scales. The authors propose an enhancement to Hamiltonian Generative Networks (HGN) by incorporating port-Hamiltonian structures to better handle externally forced, dissipative environments. They identify specific failure modes that occur when extrapolating beyond the training step size, such as latent magnitude growth and global truncation error. The authors provide targeted solutions to these issues, enabling stable dynamics prediction across a broader range of temporal resolutions. Their analysis includes strategies for improving temporal generalization in continuous-time video generation, ultimately contributing to more robust hierarchical planning and sim-to-real transfer applications.
Methodology
The authors extend Hamiltonian Generative Networks by integrating port-Hamiltonian structures to address issues in externally forced, dissipative environments. They analyze the failure modes of HGN rollouts when predicting at step sizes outside the training regime and propose targeted solutions to mitigate these failures.
Results
The proposed enhancements to HGN allow for stable dynamics prediction at temporal resolutions significantly different from those seen during training. The targeted fixes effectively address the identified failure modes, leading to improved performance in generating continuous-time video dynamics.
Implications
The findings have significant implications for applications requiring hierarchical planning and sim-to-real transfer, such as robotics and scientific simulations, where querying dynamics at multiple time scales is essential.
ALER-TI: Aligned Latent Embedding Retrieval for Time Series Imputation
Time Series
- ALER-TI enhances time series imputation by integrating historical patterns through a retrieval-augmented framework.
- Latent Embedding Alignment (LEA) allows for efficient retrieval while addressing representation mismatches between corrupted queries and clean candidates.
- The framework is model-agnostic and can be integrated with various existing imputation models.
- Extensive experiments show significant improvements in imputation performance across different datasets and missing rates.
Read more
ALER-TI: Aligned Latent Embedding Retrieval for Time Series Imputation
Summary
The paper introduces ALER-TI, a novel framework for time series imputation that enhances the reconstruction of missing values by leveraging historical patterns through a retrieval-augmented approach. Traditional deep learning methods for time series imputation often rely solely on localized temporal context, which can be inadequate in real-world scenarios characterized by non-stationary dynamics and weak temporal correlations. ALER-TI addresses these limitations by employing a mechanism called Latent Embedding Alignment (LEA), which mitigates the representation mismatch between corrupted queries and complete historical candidates. This is achieved through post-hoc masking in the latent space, allowing for efficient retrieval of pre-computed historical embeddings. The framework is model-agnostic, enabling integration with various imputation backbones, and has been tested across six real-world datasets with varying missing rates. The results demonstrate that ALER-TI consistently outperforms strong baseline models, enhancing robustness and imputation accuracy across diverse settings.
Methodology
The methodology involves the development of ALER-TI, which utilizes Latent Embedding Alignment (LEA) to align corrupted queries with historical candidates. The framework employs a mask-agnostic encoding strategy, allowing for the pre-computation and caching of historical embeddings, while applying post-hoc masking to align these embeddings with the query's missingness pattern during retrieval. This approach preserves efficiency and enhances the accuracy of missing-value reconstruction.
Results
The experiments conducted on six real-world datasets indicate that ALER-TI consistently improves upon strong baseline models in terms of imputation accuracy and robustness. The framework demonstrates enhanced performance across various missing rates and sequence lengths, validating its effectiveness in real-world applications.
Implications
The implications of this research are significant for fields that rely on time series data, such as healthcare monitoring, financial forecasting, and industrial anomaly detection. By improving the reliability of time series imputation, ALER-TI can enhance the performance of downstream analytical models that depend on complete data.
The Key to Going Linear: Analysis-Driven Transformer Linearization
NLP
Large Language Models
Efficient ML
- Isolated analysis of linearization methods in a frozen-backbone setting reveals key-dependent dynamics of softmax attention.
- Delta-style updates outperform gated accumulation methods in approximating softmax attention.
- Structural interventions like sink tokens and short convolutions effectively reduce performance gaps in linearized models.
- The proposed linearization approach scales effectively across large models (up to 32B parameters) and outperforms prior baselines.
Read more
The Key to Going Linear: Analysis-Driven Transformer Linearization
Summary
This paper addresses the computational bottleneck of causal self-attention in transformers, particularly when scaling to long contexts. The authors focus on post hoc linearization methods that convert full-attention models into linear-time architectures without the need for retraining from scratch. They isolate the effects of various state update designs in a strict frozen-backbone setting, where only new parameters are trained. The analysis reveals that softmax attention utilizes key-dependent, rank-1 orthogonal projections, which delta-style networks can effectively replicate, outperforming purely gated accumulation methods. The authors introduce structural interventions, such as sink tokens, short convolutions, and fixed-budget cache routing, to mitigate approximation errors. Their approach is validated on LLaMA and Qwen models, demonstrating superior performance on MMLU benchmarks and matching the retrieval capabilities of complex adaptive-caching frameworks, while remaining competitive across various downstream tasks.
Methodology
The authors employ an analysis-driven approach to transformer linearization, focusing on the approximation of softmax attention through linear state updates. They conduct empirical comparisons of various linear attention mechanisms, including kernelized, gated, and delta-based methods, in a strict frozen-backbone regime. The study introduces practical design choices to enhance performance, such as disjoint sliding-window paths and projection adaptations.
Results
The proposed linearization methods consistently outperform previous post hoc baselines on the MMLU benchmark and match the long-context retrieval capabilities of advanced adaptive-caching frameworks. The interventions introduced help narrow the performance gap, demonstrating effectiveness across LLaMA and Qwen models with up to 32 billion parameters.
Implications
This research has significant implications for the efficient deployment of transformer models in applications requiring long-context processing, such as natural language understanding and generation tasks. The findings could lead to more scalable and efficient architectures in various NLP applications.
Federated Deep Learning for Privacy-Preserving Cardiovascular Disease Risk Prediction
Federated Learning
- Federated learning enables collaborative model development without sharing sensitive patient data.
- The study integrates two heterogeneous cohorts to improve cardiovascular disease risk prediction.
- Deep survival models trained via federated learning showed superior performance compared to local models.
- C-statistic improvements indicate enhanced predictive accuracy for both cohorts.
Read more
Federated Deep Learning for Privacy-Preserving Cardiovascular Disease Risk Prediction
Summary
This study presents a federated deep learning approach aimed at improving cardiovascular disease (CVD) risk prediction while preserving patient privacy. Traditional models often rely on data from single institutions, which can limit their applicability due to privacy regulations. The authors integrate two distinct population-based cohorts: Lifelines, with 148,230 participants and self-reported outcomes, and the Rotterdam Study, with 10,155 participants and clinically linked outcomes. The federated learning framework allows for collaborative model training without sharing sensitive patient data. The performance of deep survival models trained using this federated approach was evaluated, primarily on the Rotterdam Study cohort. Results indicated that federated models outperformed locally trained models, with the C-statistic for the Rotterdam Study increasing from 0.728 to 0.739, and for Lifelines from 0.783 to 0.787. These findings demonstrate that federated deep learning can enhance CVD risk prediction across heterogeneous datasets while maintaining data privacy.
Methodology
The study utilized a federated learning framework to train deep survival models on two distinct cohorts without transferring individual-level data. The Lifelines cohort provided a large sample with self-reported outcomes, while the Rotterdam Study offered a smaller cohort with clinically linked outcomes. The federated learning process involved local model training at each site, followed by aggregation of model updates to create a global model.
Results
The federated deep learning models achieved a C-statistic of 0.739 (95% CI: 0.728โ0.749) for the Rotterdam Study, an increase from 0.728. For the Lifelines cohort, the C-statistic improved from 0.783 (95% CI: 0.775โ0.791) to 0.787 (95% CI: 0.780โ0.792), indicating enhanced predictive performance across both datasets.
Implications
The findings suggest that federated deep learning can be a viable solution for developing predictive models in healthcare, particularly in scenarios where data privacy is a concern. This approach could facilitate the use of diverse datasets to improve risk prediction models, ultimately aiding in better clinical decision-making and patient outcomes.
Spectral Analysis of Dueling Q-Learning
Reinforcement Learning
Theory
- Introduces a centered tabular decomposition of the Q-function for dueling Q-learning.
- Establishes convergence guarantees for unregularized dueling Q-learning with constant step sizes.
- Derives a finite-time error bound for the sampled stochastic version of dueling Q-learning.
- Clarifies the roles of value and advantage updates in the learning process.
Read more
Spectral Analysis of Dueling Q-Learning
Summary
This paper presents a theoretical analysis of dueling Q-learning, a reinforcement learning algorithm that enhances the standard Q-learning approach by decomposing the Q-function into a value function and an advantage function. While dueling Q-learning has shown empirical success, its theoretical foundations remain underexplored. The author builds upon previous work that analyzed tabular dueling Q-learning with regularization, addressing the gap in understanding the unregularized version. The paper introduces a centered tabular decomposition of the Q-function, establishing convergence guarantees for the unregularized, constant step-size recursion. The analysis employs a switching linear system (SLS) framework to derive an exact representation for deterministic dueling Q-learning and provides a finite-time error bound for its stochastic counterpart. The findings clarify the distinct roles of value and advantage updates in the learning process, leading to convergence to a neighborhood of the optimal Q-function, with the neighborhood size diminishing as the common scalar gain approaches zero.
Methodology
The paper employs a theoretical framework based on switching linear systems to analyze the convergence of dueling Q-learning. It interprets the Q-function through an orthogonal decomposition into value and advantage components, allowing for a detailed examination of the update mechanisms and their effects on learning efficiency.
Results
The analysis demonstrates that the deterministic dueling Q-learning algorithm converges to a first-moment neighborhood of the optimal Q-function. The size of this neighborhood decreases as the common scalar gain approaches zero, indicating improved learning stability and efficiency.
Implications
The findings provide a deeper theoretical understanding of dueling Q-learning, which could enhance its application in complex reinforcement learning tasks. By clarifying the roles of value and advantage functions, this work may lead to more efficient learning algorithms and improved performance in high-dimensional environments.
Distributed Sketching on Data Partitions for OLS Regression
Theory
Efficient ML
Optimization
- Introduces a distributed sketching method for OLS regression that operates on data partitions.
- Characterizes the exact excess loss of the averaged OLS estimator, showing it can be comparable to traditional methods.
- Demonstrates that the performance of the proposed method is influenced by the divergence of subset covariances.
- Highlights the computational efficiency gained by reducing the size of data subsets handled by each machine.
Read more
Distributed Sketching on Data Partitions for OLS Regression
Summary
This paper explores a novel approach to distributed sketching for ordinary least squares (OLS) regression, focusing on the computational efficiency of handling large datasets. Unlike traditional methods that sketch the entire dataset, the authors propose sketching on partitioned subsets of data distributed across multiple machines. This method reduces the computational cost associated with mapping the data while maintaining the accuracy of the OLS estimators. The authors characterize the exact excess loss of the averaged OLS estimator under a fixed design setting, demonstrating that this loss is comparable to that of sketching on the entire dataset, particularly when the covariance divergence among subsets is minimal. The findings suggest that partitioned sketching can be advantageous in scenarios where data is collected from homogeneous sources, while caution is advised in heterogeneous data contexts where covariance divergence may be significant.
Methodology
The authors utilize a fixed design setting to analyze OLS estimators built from sketches of partitioned subsets of data. They derive the exact excess loss associated with the averaged estimator and compare it to the loss from traditional sketching methods that use the entire dataset. The analysis involves mathematical characterization of covariance divergence and its impact on estimator performance.
Results
The study finds that the excess loss for the averaged estimator from partitioned sketches is comparable to that from the whole dataset when the subset covariances exhibit low divergence. Specifically, the results indicate that the performance of the partitioned sketching method can outperform traditional methods if the data is i.i.d. sampled, while it may underperform in cases of high covariance divergence.
Implications
This research has significant implications for large-scale data analysis in distributed computing environments, particularly in scenarios where data is naturally partitioned. It suggests that practitioners can achieve efficient and accurate OLS regression without the computational burden of handling entire datasets, thus enabling more scalable machine learning applications.
Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization
Optimization
Theory
- First theoretical guarantee for vanilla SGD with momentum under heavy-tailed noise for various objective types.
- Convergence rates established are slightly inferior to those of normalized SGD, highlighting limitations of vanilla methods.
- Results do not require bounded gradients, offering a more general theoretical framework.
- Experiments confirm the importance of the condition relating the Hรถlder continuity parameter and tail index for stable convergence.
Read more
Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization
Summary
This paper investigates the performance of vanilla Stochastic Gradient Descent (SGD) with momentum in the presence of heavy-tailed noise, a common issue in modern optimization scenarios. Unlike previous studies that often rely on gradient clipping or normalization to ensure convergence, this work provides a comprehensive theoretical analysis of vanilla SGD with momentum for strongly convex, convex, and nonconvex objectives without employing any gradient control mechanisms. The authors refine existing convergence results and establish that while the convergence rates of vanilla SGD with momentum are inferior to those of modified variants, they still provide a fundamental baseline for understanding the algorithm's behavior under heavy-tailed noise. The findings are supported by experiments on synthetic functions, demonstrating the critical role of the relationship between the Hรถlder continuity parameter and the tail index of the noise in achieving stable convergence.
Methodology
The authors refine convergence results for vanilla SGD with momentum and analyze its performance under heavy-tailed noise conditions. They establish theoretical guarantees for convergence rates across different types of objective functions and validate their findings through experiments on synthetic functions, focusing on the relationship between the Hรถlder continuity parameter and the tail index of the noise.
Results
The paper presents convergence rates for vanilla SGD with momentum, specifically O(T^(-p-1)/2p) for known tail index p and O(T^(-p-1)/4) for unknown p in nonconvex settings. The results indicate that while these rates are not optimal compared to normalized variants, they still demonstrate the algorithm's ability to converge under heavy-tailed noise. The experiments further validate the theoretical findings, emphasizing the critical condition for stable convergence.
Implications
The findings have significant implications for the design of optimization algorithms in machine learning, particularly in scenarios where heavy-tailed noise is prevalent. The results suggest that vanilla SGD with momentum can serve as a reliable baseline for future algorithmic improvements and adaptations in noisy environments.
Ensemble Diversity Optimization for Subjective Supervision
NLP
Optimization
Theory
- EDO framework optimizes ensemble weights and structure to handle annotator disagreement in subjective NLP tasks.
- Introduces a signed diversity regularizer to control the balance between preserving and suppressing disagreement.
- Implements a multi-objective optimization approach that integrates predictive utility, calibration, and diversity.
- Demonstrates significant improvements in probabilistic calibration and alignment with annotator distributions across multiple benchmarks.
Read more
Ensemble Diversity Optimization for Subjective Supervision
Summary
This paper introduces Ensemble Diversity Optimization (EDO), a novel framework designed to address the challenges of subjective NLP tasks characterized by annotator disagreement. Traditional supervised learning methods often collapse diverse annotations into a single target, leading to overfitting and loss of valuable distributional information. EDO optimizes ensemble weights, effective cardinality, and calibration through a unified differentiable objective, allowing for end-to-end learning of ensemble composition and size. The framework employs a signed diversity regularizer that can either preserve or suppress disagreement based on validation data, thus enabling controlled navigation of the utility-calibration trade-off. EDO integrates a soft F1 surrogate, class-weighted cross-entropy for imbalance handling, and reliability-weighted diversity to manage intra-ensemble variability. Experimental results on four subjective text-classification benchmarks demonstrate that EDO significantly enhances probabilistic calibration, achieving a reduction in cross-entropy by 40-78% and lowering Brier scores compared to existing methods, while maintaining competitive F1 scores and better alignment with annotator distributions. This work highlights the importance of jointly optimizing ensemble structure and diversity in modeling human subjectivity in supervised learning.
Methodology
The EDO framework utilizes GumbelโSoftmax relaxation for differentiable learning of ensemble structure and cardinality. It incorporates a signed diversity regularizer, which is tuned on validation data to guide the optimization process. The framework balances multiple objectives, including predictive utility (micro-F1), calibration (class-weighted cross-entropy), and internal diversity, allowing for effective handling of subjective supervision.
Results
Experiments conducted on four subjective text-classification benchmarks (ArMIS, ConvAbuse, HS-Brexit, MD-Agreement) show that EDO achieves a 40-78% reduction in cross-entropy and lowers Brier scores compared to baseline methods like Soft-CE, Soft-MD, Top-5 Voting, and WEL. EDO maintains competitive F1 scores while providing better alignment with the distributions of annotator responses.
Implications
The EDO framework offers a principled approach to modeling human subjectivity in supervised learning, making it applicable to various NLP tasks where annotator disagreement is prevalent. Its ability to optimize ensemble diversity can enhance the robustness and interpretability of models in fields such as content moderation, sentiment analysis, and other subjective classification tasks.
FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention
NLP
Large Language Models
- FFT-based spectral preprocessing of query-key projections significantly improves transformer attention.
- Achieved a 79% reduction in validation loss over standard dot-product attention on the TinyShakespeare dataset.
- Results are reproducible across multiple random seeds, confirming the reliability of the approach.
- The improvements are attributed to global sequence mixing in the frequency domain, not positional artifacts.
Read more
FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention
Summary
This paper introduces FourierQK, a novel approach that applies FFT-based frequency-domain preprocessing to the query and key projections in transformer models, significantly enhancing their attention mechanisms. The study focuses on character-level language modeling using the TinyShakespeare dataset. The author demonstrates that a fixed random spectral filter can achieve a validation loss of 1.031, while a single learned frequency filter initialized at the paragraph scale improves this to 0.608, and a multi-frequency spectral attention model with four learned frequencies achieves a remarkable 0.309, marking a 79% reduction in validation loss compared to standard dot-product attention. The results are reproducible across multiple random seeds, confirming the robustness of the findings. The paper also highlights that the improvements stem from the spectral preprocessing rather than from other methods like random orthogonal rotations or non-orthogonal projections. Additionally, the study identifies a clear architectural distinction between its approach and previous works like FNET, which entirely replaces attention with Fourier mixing of token embeddings. The findings suggest that spectral preprocessing can lead to better sequence learning by mixing information in the frequency domain before computing attention scores.
Methodology
The methodology involves applying frequency-domain filters to the learned query and key projections before computing attention scores. The author tests various filter designs, including fixed random filters, learned frequency filters, and causal filters, to evaluate their impact on attention performance. The effectiveness of the approach is validated using a shuffled validation diagnostic to ensure that improvements are due to genuine sequence learning.
Results
The results indicate that the spectral preprocessing leads to substantial improvements in validation loss, with the best-performing model achieving a loss of 0.309. The study confirms that the single-frequency filter yields consistent results across different random seeds, and the multi-frequency model converges to a near-geometric ordering of frequencies corresponding to different text scales. The findings also demonstrate that causal time-domain filters do not outperform standard attention, highlighting the unique advantages of the proposed spectral approach.
Implications
The implications of this work suggest that integrating spectral preprocessing into transformer architectures can enhance their performance in language modeling tasks. This approach may open new avenues for improving attention mechanisms in various NLP applications, potentially leading to more efficient and effective models.
path_boost: A Python Package for Interpretable Graph-Level Prediction using Path-Based Gradient Boosting
Graph Learning
Interpretability
- PathBoost provides an interpretable alternative to graph neural networks for graph-level predictions.
- The algorithm automatically discovers and utilizes predictive paths within graph structures.
- path_boost supports both regression and binary classification tasks.
- The package is compatible with scikit-learn, facilitating integration into existing workflows.
Read more
path_boost: A Python Package for Interpretable Graph-Level Prediction using Path-Based Gradient Boosting
Summary
The paper introduces 'path_boost', an open-source Python package designed for interpretable supervised learning on graph-structured data. It implements the PathBoost algorithm, which identifies predictive labeled paths within graphs during the learning process, contrasting with traditional graph neural networks that lack interpretability. PathBoost iteratively selects and extends paths based on their predictive power, combining weak learners into a robust ensemble through boosting. The package supports regression and binary classification, integrates seamlessly with scikit-learn workflows, and offers features such as automatic anchor node selection and parallel training. The authors demonstrate the effectiveness of PathBoost on molecular property prediction tasks, specifically for transition metal compounds, and benchmark its performance against established graph neural networks and graph kernel methods across multiple datasets. The package is available on PyPI and GitHub under an open-source license.
Methodology
The PathBoost algorithm employs a boosting framework that iteratively selects and extends labeled paths based on their predictive power. It uses a boosting matrix to track path occurrences and an extended boosting matrix to incorporate node and edge attributes. The model employs a selector model to identify informative paths and a base learner to produce updates based on selected paths. The implementation includes a multi-anchor parallel architecture for training independent sub-models and an enhanced variable importance framework.
Results
PathBoost was benchmarked against a graph neural network and a graph kernel method across six molecular datasets, showing competitive performance. The package effectively predicted molecular properties, demonstrating its capability to handle graph-structured data while maintaining interpretability.
Implications
The development of path_boost has significant implications for fields that utilize graph-structured data, such as molecular chemistry and materials science. Its interpretability can enhance understanding of model predictions, making it a valuable tool for researchers and practitioners in these domains.
The Importance of Encoder Choice:A Tabular-Image Study
Multimodal
- Multimodal rankings vary significantly across different tabular encoders.
- Combining modalities can degrade performance below unimodal baselines in certain datasets.
- Stronger unimodal encoders may provide a better estimate of the benefits of multimodal learning.
- A simple bilinear fusion method can perform comparably to complex multimodal methods when using strong encoders.
Read more
The Importance of Encoder Choice:A Tabular-Image Study
Summary
This paper investigates the role of encoder choice in multimodal learning, specifically focusing on the integration of tabular and image data. Traditionally, a simple Multi-Layer Perceptron (MLP) has been used as the encoder for tabular data, but this study argues that stronger tabular models could enhance multimodal fusion. The authors highlight the challenges posed by In-Context Learning (ICL) models, which require labels for processing instances, complicating the embedding of training and test instances uniformly. The research systematically evaluates various state-of-the-art tabular models as encoders in the image-tabular setting, revealing that multimodal rankings are unstable across different encoders and that combining modalities does not always yield better performance than unimodal approaches. The study also identifies a context-query representation shift in ICL models, which can negatively impact downstream tasks. The findings suggest that a stronger unimodal encoder may provide a more reliable estimate of the benefits of multimodal integration, and that simple bilinear fusion methods can match the performance of complex multimodal architectures when paired with robust encoders.
Methodology
The authors conducted a systematic evaluation of various tabular encoders in the context of image-tabular multimodal learning. They analyzed the performance of In-Context Learning Tabular Foundation Models (ICL TFMs) and assessed the impact of context-query representation shifts on model performance. The study involved comparative experiments to determine the effectiveness of different encoders and fusion methods.
Results
The study found that multimodal rankings are not stable across different tabular encoders, and that performance can degrade when combining modalities in datasets with low predictive signal. It was also observed that the context-query representation shift negatively affects model performance, and that simpler bilinear fusion methods can achieve competitive results when paired with strong encoders.
Implications
The findings underscore the importance of selecting appropriate encoders in multimodal learning tasks, particularly when integrating tabular and image data. This research could inform future developments in multimodal systems, encouraging the exploration of stronger tabular models and simpler fusion techniques to enhance performance.
Architecture Generalization with MetaNCA
Efficient ML
Theory
Graph Learning
- Introduction of MetaNCA, a framework for self-organizing neural network weights using local rules.
- Utilization of a Weight Transformer architecture for efficient weight updates based on local interactions.
- Demonstration of architecture generalization capabilities to unseen neural network configurations.
- Scalability to large networks with millions of parameters without backpropagation.
Read more
Architecture Generalization with MetaNCA
Summary
The paper introduces Meta Neural Cellular Automata (MetaNCA), a novel framework that learns local rules to self-organize the weights of artificial neural networks. Inspired by the adaptability and local interactions of biological neurons, MetaNCA employs a Weight Transformer architecture that utilizes linear attention to aggregate signals from neighboring weights and hidden states. This approach allows for the generation of diverse neural network architectures without the need for backpropagation. The authors demonstrate MetaNCA's capability to generate weights for various architectures, including feedforward MLPs, CNNs, and ResNets, on datasets like MNIST and CIFAR-100, scaling up to networks with 2 million parameters. Importantly, MetaNCA exhibits generalization to unseen architectures, with architectural diversity during training enhancing this capability. The work addresses limitations of traditional gradient-based optimization methods, emphasizing the need for more flexible and adaptable models that can learn generalized rules for network development rather than specific architectures.
Methodology
MetaNCA employs a graph neural cellular automaton approach, where a local rule network iteratively updates the weights of a task network based solely on local information from the computational graph. The Weight Transformer architecture aggregates signals from neighboring weights and hidden states using linear attention, allowing for the generation of diverse neural network architectures.
Results
MetaNCA successfully generates weights for various neural network architectures, including MLPs, CNNs, and ResNets, achieving scalability to networks with 2 million parameters. The framework demonstrates strong generalization to architectures not encountered during meta-training, with improved performance linked to the diversity of architectures used during the training phase.
Implications
The findings suggest that MetaNCA could lead to more efficient and adaptable neural network training methods, reducing reliance on traditional backpropagation and enabling the development of models that can generalize across a wider range of architectures. This has potential applications in various fields requiring flexible and scalable machine learning solutions.
The Rank-One Corner: How Much Value Equivalence Does a Task Need from a World Model?
Reinforcement Learning
Theory
- The concept of 'closure' is introduced, focusing on the specific predictive coordinates needed for task performance.
- A scalar reward signal captures only a limited aspect of the task's structure, termed the 'rank-one corner.'
- The dimensionality of the objective directly affects the model's ability to represent the closure, with higher dimensions allowing for richer representations.
- The study provides empirical evidence that the objective's dimensionality governs the model's predictive capabilities.
Read more
The Rank-One Corner: How Much Value Equivalence Does a Task Need from a World Model?
Summary
This paper investigates the relationship between the dimensionality of objectives in reinforcement learning and the structure that learned world models can represent. The author argues that traditional evaluations of world models focus on overall fidelity, but what is crucial is the model's ability to capture the specific predictive coordinates necessary for the task, termed the 'closure.' The study utilizes a DreamerV3 stack in a controlled environment to measure how much of the closure a latent representation can capture based on the dimensionality of the objective it is trained against. The findings reveal that a scalar value signal, which is commonly used in reinforcement learning, only captures a one-dimensional projection of the closure, while a full multi-dimensional objective can capture significantly more (from Rยฒ = 0.10 to 0.76). The results suggest that the dimensionality of the objective directly influences the predictive capabilities of the model, indicating that value equivalence is not binary but rather dimensional. The paper concludes that to effectively represent a task's structure, the model's objective must match the rank of the closure it aims to predict.
Methodology
The research employs a DreamerV3 stack in a controlled environment to analyze the relationship between the dimensionality of training objectives and the closure representation in learned world models. The study compares the performance of models trained with scalar objectives against those trained with full multi-dimensional objectives, measuring the installed closure through linear probes.
Results
The results indicate that a scalar objective installs only a fraction (0.10) of the closure compared to a full objective (0.76). As the dimensionality of the objective increases from one to four, the model's ability to capture predictive directions also increases correspondingly. The findings confirm that the structure a latent representation can capture is directly tied to the dimensionality of the objective it is trained against.
Implications
These findings suggest that in reinforcement learning, using a richer, multi-dimensional objective could enhance the effectiveness of learned world models, potentially leading to better performance in complex tasks. This has implications for the design of training objectives in model-based reinforcement learning.
Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data
Efficient ML
Theory
- Proposes an adaptive evaluation framework to replace fixed-size benchmarks.
- Utilizes sequential testing to balance efficiency and reliability in model evaluation.
- Demonstrates significant computational cost savings (up to 80%) while maintaining statistical significance.
- Allows users to define specific evaluation needs, enhancing transparency and decision-making.
Read more
Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data
Summary
This paper addresses the inefficiencies of fixed-size benchmarks in model evaluation, which often lead to excessive computational costs or unreliable results. The authors propose an adaptive evaluation framework that utilizes sequential testing to dynamically determine when to stop evaluations based on specific statistical needs. This approach allows for a balance between efficiency and reliability, enabling practitioners to define their evaluation objectives clearly. The framework incorporates stopping criteria such as diminishing returns detection and minimum detectable effect size, ensuring that evaluations are neither underpowered nor excessive. The authors demonstrate the effectiveness of their method on the Open VLM Leaderboard, achieving significant reductions in computational costs while maintaining statistical significance. The proposed framework is versatile and applicable across various use cases, including model selection and performance comparisons, making it a valuable tool for practitioners in the field.
Methodology
The authors developed an adaptive evaluation framework that integrates sequential testing principles with user-defined stopping criteria. This framework allows evaluations to conclude when statistical needs are met, rather than adhering to a predetermined sample size. The methodology includes techniques for detecting diminishing returns and establishing minimum detectable effect sizes.
Results
The adaptive evaluation framework was tested on the Open VLM Leaderboard, resulting in an 80% reduction in computational costs compared to traditional fixed-size evaluations while ensuring statistical significance with a 2.5-point confidence interval width. The framework also demonstrated the ability to stop evaluations based on practical and statistical needs, thus improving efficiency without compromising reliability.
Implications
The proposed framework has significant implications for model evaluation in machine learning, particularly in scenarios with limited computational resources. It enables practitioners to make informed decisions about model performance without unnecessary evaluations, thereby saving time and resources. This approach can enhance the efficiency of model development pipelines and improve the reliability of model comparisons.
What to Keep, What to Forget: A RateโDistortion View of Memory Compaction in LLMs and Agents
NLP
Large Language Models
Efficient ML
- Memory compaction in LLMs can be framed as a rate-distortion problem, allowing for a unified approach across different methods.
- The authors present a seven-axis taxonomy that classifies various memory compaction techniques uniformly.
- Common failure modes in memory compaction include irreversible loss of information before query knowledge is available.
- The proposed COMPACT-Bench benchmark aims to standardize the evaluation of memory compaction methods across different contexts.
Read more
What to Keep, What to Forget: A RateโDistortion View of Memory Compaction in LLMs and Agents
Summary
This paper addresses the challenge of memory management in large language models (LLMs) and their associated agents, which increasingly rely on memory for caching attention keys and values, maintaining prompts, and storing historical interactions. The authors propose that various methods for memory compaction across different research communities can be unified under a single rate-distortion framework, which optimizes the retention of context-derived information while minimizing loss in downstream task performance. They introduce a comprehensive taxonomy that categorizes existing compaction techniques and highlight common failure modes, particularly the issue of irreversible information loss before query knowledge is available. Additionally, the authors propose a benchmark, COMPACT-Bench, to evaluate memory compaction methods uniformly across different layers and contexts. The paper emphasizes the need for a shared optimization objective and provides design principles for compaction-aware systems, ultimately aiming to enhance the efficiency of LLMs and agents in managing memory resources.
Methodology
The authors formalize memory compaction as a rate-distortion problem and derive a layer-agnostic lower bound for performance evaluation. They conduct a survey of existing methods across different layers, including KV cache, prompt/context, architectural, and agent memory, and propose a benchmark for evaluating these methods under a unified budget axis.
Results
The paper identifies that all layers of memory compaction must adhere to a common rate-distortion trade-off, revealing that query-agnostic compaction incurs a quantifiable penalty. The proposed taxonomy and benchmark facilitate the comparison of various techniques and highlight the need for improved methods that consider the context of queries.
Implications
This work has significant implications for the design of more efficient LLMs and agents, as it provides a framework for understanding and optimizing memory management. By standardizing evaluation methods, researchers can better assess the effectiveness of different compaction techniques, ultimately leading to advancements in the performance and utility of LLMs in practical applications.
NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts
Time Series
- NEST models dataset-level distribution shifts by dynamically recomposing inter-variable dependencies.
- The framework utilizes unsupervised moment-entropy metrics for principled regime discovery.
- A regime-oriented router mechanism enhances expert orchestration and improves prediction accuracy.
- NEST achieves state-of-the-art performance across diverse benchmarks.
Read more
NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts
Summary
The paper introduces NEST, a novel framework designed to address dataset-level distribution shifts in time series forecasting, which are often caused by diverse behavioral modes and evolving system states. Existing methods primarily focus on local temporal shifts and overlook the global structural challenges posed by datasets that contain distinct operational regimes. NEST employs a two-phase mixture-of-experts (MoE) architecture that first partitions the dataset into operational regimes using unsupervised clustering based on moment-entropy metrics. This is followed by a regime-oriented router mechanism that generates expert weights based on temporal content and refines them through geometric modulation. Each expert in the framework captures regime-specific dynamics through unique variate-attention patterns. The extensive evaluations on various benchmarks, including network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance, effectively modeling the transitions between operational modes and enhancing forecasting accuracy.
Methodology
NEST employs a two-phase mixture-of-experts framework. In the first phase, it uses unsupervised clustering in a moment-entropy space to identify distinct operational regimes. In the second phase, a regime-oriented router generates initial expert weights based on temporal content, which are refined through geometric modulation to align with regime centroids. Each expert is trained to capture regime-specific dynamics through evolving variate-attention patterns.
Results
NEST demonstrated state-of-the-art performance on various benchmarks, including heterogeneous network traffic and multi-year physical phenomena. The framework's ability to capture regime-specific dynamics and structural transitions was validated through extensive analysis of attention mechanisms and expert weight configurations.
Implications
The proposed framework has significant implications for improving forecasting accuracy in complex systems where dataset-level distribution shifts are prevalent. It can be applied in various domains, including network traffic analysis, resource allocation, and anomaly detection, enhancing the understanding and management of dynamic systems.
Trustworthy Machine Learning through the Lens of Combinatorial Optimization: Survey and Research Perspectives
Optimization
Theory
Interpretability
- Trustworthy ML requires more than just predictive accuracy; it necessitates transparency, interpretability, robustness, fairness, and privacy.
- The Rashomon effect suggests that multiple models can achieve similar performance, allowing for the selection of models based on additional trustworthiness criteria.
- Combinatorial optimization offers a robust framework for addressing various trustworthiness challenges in ML, including model training and post-training tasks.
- CO techniques can provide global optimality and formal certificates, enhancing the reliability of ML systems.
Read more
Trustworthy Machine Learning through the Lens of Combinatorial Optimization: Survey and Research Perspectives
Summary
This survey paper explores the intersection of combinatorial optimization (CO) and trustworthy machine learning (ML), emphasizing the need for ML systems to be transparent, interpretable, robust, fair, and privacy-preserving. The authors argue that traditional empirical performance metrics are insufficient for evaluating ML models, as models with similar performance can differ significantly in their trustworthiness attributes. The paper synthesizes recent advances in CO techniques applied to various aspects of trustworthy ML, including model training, explanation generation, robustness analysis, fairness auditing, model compression, and privacy protection. The authors highlight that CO provides global guarantees and formal certificates that are often lacking in heuristic approaches, thus enhancing the design and deployment of trustworthy ML systems. Despite challenges in scalability, advancements in CO solvers and hybrid algorithms indicate a promising future for integrating CO into trustworthy ML practices.
Methodology
The survey adopts a structured, subdomain-driven approach to review the literature on CO techniques applied to trustworthy ML. It encompasses both training and post-training tasks, examining various CO paradigms beyond mixed-integer programming, including SAT, SMT, CP, MaxSAT, and B&B hybrids.
Results
The paper identifies that CO techniques can effectively address trustworthiness issues in ML by framing them as optimization or feasibility problems. It highlights the potential of CO to enhance model interpretability, robustness, fairness, and privacy, while also acknowledging the scalability challenges that remain.
Implications
The findings suggest that integrating CO into ML practices can lead to the development of more reliable and trustworthy ML systems, which is crucial for high-stakes applications across various domains. This integration can also inform broader AI governance and institutional measures.
Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices
Federated Learning
Multimodal
Efficient ML
- MESH-FL introduces an entropy-guided approach for update compression in multimodal federated learning.
- The framework adapts compression ranks based on the spectral entropy of updates, enhancing communication efficiency.
- MESH-FL shows significant improvements in accuracy and data transmission efficiency over traditional methods.
- The proposed method is validated on a heterogeneous edge device setup, showcasing its practical applicability.
Read more
Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices
Summary
This paper presents MESH-FL, an innovative framework designed for multimodal federated learning (FL) on resource-constrained edge devices. Traditional update compression methods in FL often apply uniform policies across different layers and devices, failing to account for the unique spectral structures and compressibility of various modalities. MESH-FL addresses this limitation by employing an entropy-guided approach to matrix product state (MPS) update compression. The framework estimates the spectral entropy of each layer-wise update using truncated singular value decomposition, allowing for adaptive allocation of MPS compression ranks tailored to the specific characteristics of each layer, modality, and device under per-client payload constraints. The authors demonstrate that higher spectral entropy correlates with a need for higher reconstruction ranks, leading to a more efficient compression strategy. The theoretical foundation of MESH-FL includes a convergence analysis with an explicit compression-dependent error term. Experimental results on a heterogeneous Raspberry Pi cluster reveal that MESH-FL achieves up to 56.8ร compression while improving final accuracy by up to 2.01% compared to the uncompressed FedAvg baseline, and reduces the total transmitted data required for convergence by up to 66ร.
Methodology
The methodology involves estimating the spectral entropy of layer-wise updates using truncated singular value decomposition. MESH-FL allocates MPS compression ranks adaptively across layers, modalities, and devices, taking into account per-client payload budgets. The framework is built on a theoretical foundation that includes a convex surrogate rank-allocation problem and convergence analysis with a compression-dependent error term.
Results
The experimental results indicate that MESH-FL achieves up to 56.8ร compression compared to traditional methods while surpassing the uncompressed FedAvg baseline in accuracy by up to 2.01%. Additionally, it reduces the total transmitted data required to reach convergence by up to 66ร.
Implications
The findings suggest that MESH-FL can significantly enhance the efficiency of federated learning in multimodal applications, particularly in resource-constrained environments. This could lead to broader adoption of federated learning in real-world applications such as mobile health, human activity recognition, and intelligent environments.
Frequency-Domain Multi-Modality Transportation Modeling
Time Series
Multimodal
- Introduces a frequency-domain approach for multi-modality transportation modeling.
- Utilizes a Modality-Wise Frequency Filter (MFF) for spectral refinement.
- Incorporates a Frequency-Guided Synergy Integrator (FSI) for selective information aggregation.
- Demonstrates significant performance improvements over existing forecasting methods.
Read more
Frequency-Domain Multi-Modality Transportation Modeling
Summary
This paper introduces a novel framework called Frequency-Domain Multi-Modality modeling (FreMo) aimed at improving multi-modality transportation forecasting. Traditional methods often struggle with the complexities of different transportation modes, which exhibit unique spectral characteristics and interact unevenly across frequencies. FreMo addresses these challenges by operating in the frequency domain, allowing for adaptive and selective cross-modality synergy. The framework includes a Modality-Wise Frequency Filter (MFF) that refines spectral components for each modality, emphasizing informative frequencies while suppressing noise. Additionally, it features a Frequency-Guided Synergy Integrator (FSI) that selectively aggregates information across modalities based on their contributions at each frequency. The results from extensive experiments on real-world datasets demonstrate that FreMo consistently outperforms existing state-of-the-art methods, showcasing superior performance and generalization across various forecasting scenarios.
Methodology
The methodology involves the development of FreMo, which operates in the frequency domain to enhance multi-modality transportation forecasting. It employs a Modality-Wise Frequency Filter (MFF) to refine spectral components for each transportation mode and a Frequency-Guided Synergy Integrator (FSI) to selectively combine information across modalities based on their frequency-dependent contributions.
Results
FreMo was tested on various real-world datasets, where it consistently outperformed state-of-the-art baselines in terms of forecasting accuracy and generalization across different scenarios. The framework's ability to adaptively refine and integrate information from multiple modalities led to improved predictive performance.
Implications
The findings suggest that leveraging frequency-domain analysis can significantly enhance the accuracy of multi-modality transportation forecasting. This approach could be applied to other urban computing tasks that involve complex temporal dynamics and multiple data sources, such as crime analysis and air quality monitoring.
Robust Human-AI Complementarity under Uncertainty
Theory
Large Language Models
NLP
- Human decision makers struggle to utilize AI predictions due to uncertainty about AI quality.
- The correlation of prediction errors between humans and AI is critical for achieving complementarity.
- Negative correlation of errors allows for robust decision-making strategies that improve utility.
- Empirical findings show that human and AI prediction errors are often positively correlated.
Read more
Robust Human-AI Complementarity under Uncertainty
Summary
This paper investigates the challenges of achieving robust human-AI complementarity in decision-making under uncertainty. The authors argue that human decision makers often fail to leverage AI predictions effectively due to asymmetric information regarding the quality of AI outputs. They introduce a formal model based on statistical decision theory, highlighting that the correlation structure of prediction errors between humans and AI is crucial for ensuring that decision makers can benefit from AI inputs. Specifically, when AI errors are negatively correlated with human errors, it allows for the construction of robust strategies that improve expected utility. The authors empirically analyze real-world forecasting benchmarks to assess the conditions for complementarity, finding that human and AI prediction errors are often positively correlated, which complicates the realization of complementarity. They suggest that future AI training should focus on enhancing the ability to provide complementary information to support human decision-making.
Methodology
The authors develop a formal model based on statistical decision theory to analyze the impact of uncertainty on human-AI decision-making. They empirically investigate real-world forecasting benchmarks to compare prediction errors from humans and AI, specifically focusing on the correlation structure of these errors.
Results
The study finds that under conditions of uncertainty, it is challenging for decision makers to construct effective strategies that leverage AI predictions. When AI errors are positively correlated with human errors, the potential for complementarity is significantly reduced. The empirical analysis reveals that human and AI prediction errors are often positively correlated, complicating the realization of complementary gains.
Implications
The findings suggest that enhancing AI systems to provide complementary information could improve human decision-making in various fields, particularly in scientific and forecasting applications. This could lead to better integration of AI tools in decision-making processes, ultimately augmenting human capabilities rather than replacing them.
Information Allocation Dynamics in Neural Network Optimization
Optimization
Theory
- Introduces the concept of information allocation dynamics to explain optimizer implicit bias.
- Defines a continuous preconditioning exponent p that regulates the update dynamics of weight and bias parameters.
- Demonstrates that different values of p can significantly affect training trajectories and generalization.
- Highlights the importance of understanding optimizer biases in the context of training signal allocation.
Read more
Information Allocation Dynamics in Neural Network Optimization
Summary
This paper explores the implicit biases of different optimizers in neural network training, proposing a novel perspective termed 'information allocation dynamics.' The authors argue that the biases arise from the relative allocation of training signals between weight-like and bias-like parameters during optimization. They introduce a continuous preconditioning exponent, p, which governs the update dynamics of these parameters. By analyzing a minimal linear model, the paper demonstrates how weight and bias corrections contribute differently to the residual signal. The weight correction preserves input-dependent signals, while the bias correction maintains the mean direction of the residual. The study shows that varying the exponent p can significantly alter the update proportions of weight and bias parameters, thereby influencing training trajectories and generalization performance. The findings suggest that the interplay between input sparsity, network architecture, and optimizer choice shapes the implicit bias during training, moving the analysis from solution-space geometry to the dynamics of updates throughout the training process.
Methodology
The authors analyze the update contributions of weight and bias parameters in a minimal linear model. They introduce a continuous preconditioning exponent p to unify different optimizer updates (SGDM-like and Adam-like) and study how this exponent influences the allocation of training signals across parameter pathways. Experiments are conducted to observe the effects of varying p on training dynamics and generalization.
Results
The experiments reveal that different values of the preconditioning exponent p lead to significant changes in the relative update proportions of weight and bias parameters. This, in turn, affects the training trajectories and generalization capabilities of the neural networks. The findings indicate that the optimizer's implicit bias is not solely a function of the final solution but is also embedded in the dynamics of updates during training.
Implications
Understanding the dynamics of information allocation in neural network optimization can lead to more effective training strategies and improved generalization. This perspective may inform the design of new optimizers and training protocols that better account for the implicit biases introduced during training.
NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL
Reinforcement Learning
Robotics
Theory
- NFTR addresses optimistic bias and mode collapse in HIQL by using Normalizing Flows for subgoal selection.
- The triangle-slack score effectively downweights unreliable subgoals based on geometric considerations.
- NFTR maintains population-level monotonic improvement and provides a three-term suboptimality decomposition.
- Empirical results show substantial performance improvements over HIQL across multiple task types.
Read more
NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL
Summary
This paper introduces NFTR (Normalizing Flows subgoal policies with Triangle-slack Reweighting), a novel approach to offline goal-conditioned reinforcement learning (GCRL) that addresses the limitations of Hierarchical Implicit Q-Learning (HIQL). HIQL suffers from two main issues: optimistic bias, which inflates the value of subgoals due to stochastic outcomes, and mode collapse, where a multi-modal distribution of subgoals is reduced to a single Gaussian mean that often lies in unreachable regions. NFTR replaces the Gaussian policy with a conditional Normalizing Flow, allowing for a richer representation of subgoal distributions. Additionally, it employs a triangle-slack score that corrects the advantage-weighted regression (AWR) weights to downweight subgoals that are geometrically inconsistent or have high detour costs. This approach ensures that subgoal selection is both high-value and reliably reachable, while maintaining stability under stochastic dynamics. The authors provide theoretical, methodological, and empirical contributions, demonstrating that NFTR improves upon HIQL in various tasks, including stochastic, stitching, and manipulation tasks on the OGBench benchmark.
Methodology
The authors propose NFTR, which utilizes Normalizing Flows to model multi-modal subgoal distributions instead of a unimodal Gaussian. They introduce a triangle-slack score derived from a quasi-metric that reweights the AWR objective, ensuring that subgoals with high detour costs are downweighted. The method is theoretically grounded, providing a decomposition of suboptimality into value, sampling, and geometric regularization terms.
Results
NFTR significantly outperforms HIQL on the OGBench benchmark across various tasks, demonstrating improved subgoal selection that avoids Gaussian collapse and remains stable under stochastic dynamics. The method achieves a balance between high-value subgoal selection and reliable reachability.
Implications
The findings suggest that NFTR can enhance the performance of offline goal-conditioned reinforcement learning in practical applications, particularly in fields like robotics where safe and efficient learning from static datasets is essential. The approach may also inspire further research into geometric considerations in reinforcement learning.
Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
NLP
Large Language Models
Theory
- Introduces a matched-null spectral framework for analyzing the QK operator in attention heads.
- Demonstrates that positional encoding schemes significantly influence the spectral properties and head functions of attention mechanisms.
- Finds that the strongest previous-token heads are spectrally rotational under RoPE, while other schemes are not.
- Establishes that the symmetric part of the operator M is more load-bearing than the antisymmetric part.
Read more
Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
Summary
This paper investigates the spectral properties of attention mechanisms in transformer models, focusing on how different positional encoding schemes influence the spectral algebra of attention heads. The author introduces a matched-null spectral framework for the query-key (QK) operator, revealing that the complex eigenspectrum of the learned operator M is crucial for understanding head behavior. The study analyzes seven pretrained models across three positional schemes, demonstrating that the strongest previous-token heads exhibit a rotational spectral signature under Rotary Positional Encoding (RoPE), while other schemes like learned-absolute and ALiBi show non-rotational characteristics. The findings indicate that the positional scheme fundamentally shapes the spectral algebra of attention, acting as a 'fingerprint' that emerges from the function rather than imposing a rigid constraint. The research further explores the causal relationships in the model, showing that the symmetric part of M is significantly more influential than the antisymmetric part, particularly in the context of induction circuits. Overall, the paper emphasizes the importance of spectral analysis in understanding the functional dynamics of attention mechanisms in transformers.
Methodology
The study employs a spectral analysis framework to examine the QK operator across various pretrained transformer models. It analyzes the eigenspectrum of the operator M, utilizing random-matrix theory to interpret the spectral properties. The research includes static comparisons across models, dynamic observations during training, and causal interventions to assess the impact of different positional schemes on attention head behavior.
Results
The analysis reveals that the spectral directionality effectively separates head functions across models, with a perfect model-level separation observed under different positional schemes. The study finds that the rotational signature of attention heads emerges during circuit formation, and the imposition of symmetry significantly slows down learned-absolute models. Additionally, the research confirms that no spectral channel is necessary for head functionality, although constraints impose a search cost.
Implications
These findings have significant implications for the design and understanding of transformer architectures, particularly in optimizing positional encoding schemes to enhance model performance. The insights into the spectral properties of attention mechanisms could inform future research on improving interpretability and efficiency in large language models.
Does Demand Response Increase Vulnerability to Cyber Attacks by Adversarial Data Modifications?
Optimization
Theory
- Adversarial attacks can undermine the financial benefits of demand response in industrial settings.
- Limited and undetectable perturbations can allow demand response to maintain most of its financial advantage.
- The orientation of adversarial perturbations plays a crucial role in their impact on decision-making.
- The study emphasizes the need for incorporating model sensitivities into adversarial attack analyses.
Read more
Does Demand Response Increase Vulnerability to Cyber Attacks by Adversarial Data Modifications?
Summary
This paper investigates the impact of adversarial data modifications on demand response (DR) in energy systems, particularly focusing on how manipulated price forecasts can affect decision-making in industrial settings. The authors design adversarial attacks targeting electricity price forecasting models, which are critical for optimizing scheduling in energy-intensive production processes. By employing a generalized process model, the study assesses the vulnerability of various production scheduling problems to these adversarial attacks. The findings reveal that while adversarial attacks can significantly erode the financial benefits of demand response, limited perturbations that are difficult to detect can still allow demand response to retain approximately 90% of its financial advantage compared to steady-state operations. Furthermore, the impact of these attacks is shown to depend not only on the magnitude of the perturbations but also on their orientation, suggesting that attack analyses should consider the sensitivities of scheduling optimization models for more effective assessments of decision-making under adversarial conditions.
Methodology
The authors designed adversarial attacks on electricity price forecasting models and utilized a generalized process model to analyze the effects of these attacks on scheduling optimization problems across various levels of process flexibility.
Results
The study found that adversarial attacks can significantly reduce the profits from demand response, but when perturbations are subtle, demand response can still achieve about 90% of its financial benefits. The results also highlighted that the effectiveness of adversarial attacks is influenced by both the magnitude and orientation of the perturbations.
Implications
The findings suggest that energy systems, particularly those relying on demand response, need to be aware of the vulnerabilities introduced by adversarial attacks. This has implications for the design of more robust forecasting models and the development of strategies to mitigate the risks associated with adversarial data manipulations.
Reinforcing the Generation Order of Multimodal Masked Diffusion Models
Multimodal
Generative Models
Optimization
- Existing confidence-based strategies for generation order do not improve image generation quality and multimodal reasoning.
- A learnable control block is proposed to optimize generation order using Group Relative Policy Optimization (GRPO).
- The proposed method shows significant improvements in text-to-image alignment and multimodal understanding benchmarks.
- The approach captures fine-grained spatial relationships in generated images effectively.
Read more
Reinforcing the Generation Order of Multimodal Masked Diffusion Models
Summary
This paper investigates the optimization of generation order in Diffusion Language Models (DLMs) for text-to-image synthesis and multimodal understanding. The authors highlight that traditional methods, which rely on model logits for determining generation sequences, are insufficient for multimodal tasks. To address this, they introduce a learnable control module that utilizes Group Relative Policy Optimization (GRPO) to dynamically determine the generation order. This approach significantly enhances the model's ability to align text with images and improves performance on multimodal reasoning tasks. The authors validate their method on the GenEval benchmark for text-to-image alignment and the VLMEvalKit for multimodal understanding, achieving notable relative improvements of 4.08% and 4.85%, respectively. These results underscore the importance of a learned control mechanism in multimodal generation tasks, demonstrating that effective generation order is crucial for capturing complex spatial relationships and improving overall model performance.
Methodology
The authors developed a learnable control module that determines the generation order in multimodal masked diffusion models. This module is trained using Group Relative Policy Optimization (GRPO), allowing it to adaptively optimize the generation process based on the specific requirements of image synthesis and multimodal understanding.
Results
The proposed method achieved a 4.08% relative improvement on the GenEval benchmark for text-to-image alignment and a 4.85% relative improvement on the VLMEvalKit benchmark for multimodal understanding, outperforming existing order selection strategies.
Implications
The findings suggest that learning-based control mechanisms are essential for enhancing the performance of multimodal models, particularly in tasks that require complex interdependencies between visual and textual data. This could lead to advancements in applications such as visual question answering, image generation, and other multimodal AI systems.
Steering Neural Network Training through Interpretable Constraints Based on Partial Dependence
Interpretability
Time Series
Theory
- Introduces a novel method for steering neural network training using partial dependence.
- Focuses on explanation-guided learning to align model outputs with domain knowledge.
- Demonstrates improved performance and data efficiency in regression tasks compared to unconstrained models.
- Ensures that model interpretations are faithful to prior knowledge provided by users.
Read more
Steering Neural Network Training through Interpretable Constraints Based on Partial Dependence
Summary
This paper addresses the challenge of enhancing the interpretability of machine learning models, particularly neural networks, by introducing a novel approach that incorporates domain knowledge into the training process. The authors focus on explanation-guided learning (EGL), which aims to align model explanations with prior knowledge about input features and their interactions. Unlike existing methods that primarily target classification tasks, this work emphasizes regression problems, specifically in the context of dynamical systems forecasting. The proposed method utilizes partial dependence to steer the training of neural networks, ensuring that the average response of the model to certain features is consistent with known functional relationships. The empirical results demonstrate that models trained with these constraints outperform unconstrained models in terms of performance, data efficiency, and the fidelity of their explanations to user-provided knowledge. This approach not only improves the interpretability of the models but also enhances their generalization capabilities beyond the training data.
Methodology
The authors propose a training algorithm that incorporates partial dependence as a constraint during the training of neural networks. This method aligns the model's marginal responses to specific input features with known functional relationships, allowing for a more interpretable model output. The approach is validated through empirical studies on various regression tasks, including dynamical systems forecasting.
Results
The results indicate that neural networks trained with the proposed method exhibit superior performance compared to unconstrained models. The constrained models not only require fewer training samples but also generalize better to out-of-distribution data. Additionally, the interpretations derived from these models align closely with the prior knowledge provided by users, unlike those from unconstrained models.
Implications
This work has significant implications for fields where interpretability is crucial, such as healthcare and finance, by providing a framework for developing machine learning models that are both effective and understandable. The approach can enhance trust in AI systems by ensuring that model decisions are based on sound reasoning aligned with domain expertise.
Structural Adversarial Attacks on Relational Deep Learning under Integrity Constraints
Graph Learning
Optimization
Theory
- The paper focuses on structural adversarial attacks on Relational Deep Learning systems.
- A white-box attacker can manipulate the database while preserving integrity constraints.
- Seven heuristic strategies for generating adversarial attacks are evaluated.
- Gradient-based attacks show superior performance in regression tasks compared to random baselines.
Read more
Structural Adversarial Attacks on Relational Deep Learning under Integrity Constraints
Summary
This paper investigates the adversarial robustness of Relational Deep Learning (RDL) systems, which encode relational databases as heterogeneous temporal graphs for machine learning tasks. The authors focus on structural adversarial attacks, where a white-box attacker can manipulate the upstream database by rewiring foreign-key references while adhering to integrity constraints. The study explores seven heuristic strategies for generating adversarial attacks, including two random sampling baselines and five gradient-guided methods that utilize differentiable edge masks to optimize perturbations. The findings reveal that gradient-based attacks significantly outperform random baselines in regression tasks, while the improvements in classification tasks are less pronounced due to the inherent stability of classification outputs under sparse perturbations. This research highlights the vulnerabilities of RDL systems to structured adversarial attacks and provides insights into the effectiveness of different attack strategies.
Methodology
The authors investigate seven heuristic strategies for adversarial attacks, including two random sampling methods and five gradient-guided variants. The gradient-based methods leverage differentiable edge masks to estimate the sensitivity of prediction loss to potential rewiring of foreign-key relationships, guiding the selection of perturbations while adhering to integrity constraints.
Results
Gradient-based adversarial attacks consistently outperform random sampling baselines in regression tasks. In classification tasks, the performance gains are smaller, attributed to low label-flip rates and the local stability of classification outputs under sparse perturbations.
Implications
The findings underscore the need for enhanced robustness in Relational Deep Learning systems against structural adversarial attacks, suggesting potential applications in securing relational databases and improving the integrity of machine learning predictions in sensitive domains.
Provably Optimal Learning Algorithms for Assistance Games
Reinforcement Learning
Theory
Optimization
- Introduces the concept of assistance regret in the context of assistance games.
- Presents decentralized algorithms for informed and uninformed agents with provable efficiency.
- Achieves a (1 - 1/e)-approximate assistance regret rate of หO(T^(3/4)).
- Demonstrates that improving upon the (1 - 1/e) approximation is computationally intractable.
Read more
Provably Optimal Learning Algorithms for Assistance Games
Summary
This paper investigates an online variant of assistance games, where an informed agent (human) and an uninformed agent (assistant) interact over multiple timesteps to optimize a shared reward function. The informed agent observes a latent state while the uninformed agent can only see the actions of the informed agent. The authors introduce the concept of assistance regret, which quantifies the difference between the cumulative utility of interactions and that of optimal joint policies. They propose decentralized algorithms for both agents that achieve a (1 - 1/e)-approximate assistance regret rate of หO(T^(3/4)), with polynomial runtime relative to the action and state space sizes. The algorithms are adaptable to any no-regret algorithm for the assistant and can be optimized further using a shared random string to achieve a rate of หO(โT), which is optimal up to logarithmic factors. The paper also establishes that achieving a better regret approximation than (1 - 1/e) is computationally intractable, providing a significant theoretical contribution to the field of cooperative learning in AI.
Methodology
The authors employ a theoretical framework that models interactions as online submodular maximization with matroid constraints. They utilize a regret-decomposition lemma to analyze and bound assistance regret, guiding the design of their algorithms towards stability for the human agent and adaptivity for the assistant.
Results
The decentralized algorithms achieve a (1 - 1/e)-approximate assistance regret rate of หO(T^(3/4)) with polynomial runtime. With initial coordination through a shared encoding, the rate improves to หO(โT), which is optimal up to logarithmic factors. The paper also proves the computational intractability of achieving a better approximation factor.
Implications
This work has significant implications for the development of assistive AI systems, particularly in enhancing human-AI collaboration by providing a framework for efficient learning and communication between agents with asymmetric information access. It lays the groundwork for future research in cooperative multi-agent systems and human-centered AI.
Robust Federated Learning Under Real-World Client Churn
Federated Learning
- FeLiX optimizes Federated Learning for real-time applications by addressing client churn and dynamic data distributions.
- Introduces novel mechanisms for client selection and aggregation that enhance model freshness and accuracy.
- Achieves up to 2.37ร reduction in time-to-target accuracy and 1.30ร savings in communication bandwidth over existing methods.
Read more
Robust Federated Learning Under Real-World Client Churn
Summary
This paper addresses the challenges of Federated Learning (FL) in real-world scenarios where client availability is transient and data distributions are dynamic. Traditional FL systems struggle with slow refresh cycles, leading to stale models that fail to adapt to rapidly changing user interactions. The authors introduce FeLiX, a novel FL orchestration framework designed to optimize time-to-target accuracy for live interaction streams. FeLiX incorporates three key innovations: streaming-aware availability tiers that utilize lightweight telemetry to identify ready clients, fresh-utility selection that prioritizes high-value contributions from clients meeting tight deadlines, and informativeness-aware aggregation that effectively incorporates late updates without biasing the model towards outdated data. The evaluation of FeLiX across multiple datasets demonstrates significant improvements in both time-to-target accuracy and communication efficiency compared to existing FL methods, enabling faster model adaptation to real-time user interactions.
Methodology
The authors developed FeLiX, which integrates three main components: (1) streaming-aware availability tiers for efficient client selection, (2) fresh-utility selection for prioritizing valuable data contributions, and (3) delay-robust aggregation to incorporate late updates effectively. These components work together to enhance the responsiveness of FL systems to real-world data dynamics.
Results
FeLiX was evaluated on datasets such as CIFAR-10 and Google Speech, showing a reduction in wall-clock time-to-target accuracy by up to 2.37 times and a communication bandwidth reduction of 1.30 times compared to state-of-the-art Synchronous and Asynchronous FL baselines.
Implications
The advancements presented in FeLiX can significantly improve the performance of Federated Learning systems in applications requiring rapid model updates, such as personalized recommendations and real-time ad targeting, ensuring that models remain relevant and effective in dynamic environments.
An exact information theory of generalization phase transitions in Bayesian diffusion models
Generative Models
Theory
- Introduces BIRD models that analytically address generalization in diffusion models.
- Identifies an information-theoretic phase boundary between memorization and generalization.
- Demonstrates that BIRD models can approximate trained diffusion models early in training.
- Confirms theoretical predictions through experiments across various datasets.
Read more
An exact information theory of generalization phase transitions in Bayesian diffusion models
Summary
This paper addresses the mystery of how diffusion models can learn complex distributions in high-dimensional spaces from limited training data, avoiding the curse of dimensionality. The authors introduce Bayesian information restricted diffusion (BIRD) models, which utilize restricted information about noisy data to infer the training sample that produced a given observation. They establish an information-theoretic phase boundary that distinguishes between memorization and generalization in these models, depending on the mutual information between observations and training data. The study shows that BIRD models can approximate trained diffusion models early in training across various architectures. Through experiments, the authors confirm the theoretical predictions regarding the transition from memorization to generalization, revealing that generation occurs near the edge of this boundary. The findings emphasize the critical role of information restriction in generative AI, providing insights into how these models can effectively generalize with limited data.
Methodology
The authors developed BIRD models that reverse the diffusion process by using Bayesian inference to determine which training sample corresponds to a given noisy observation. They analyzed the mutual information between restricted observations and training data to establish the phase boundary for memorization versus generalization. Experiments were conducted on multiple datasets to validate the theoretical framework.
Results
The study found that BIRD models memorize data when the mutual information exceeds the logarithm of the number of training points, while generalization occurs otherwise. Experimental results aligned with theoretical predictions, showing that both spatially local BIRD models and early-training diffusion models operate near the memorization-generalization phase boundary.
Implications
The insights from this research could inform the design of more efficient generative models that can generalize effectively with limited data, potentially impacting various applications in generative AI, including image synthesis and data augmentation.
Deep Learning for Joint Narrowband Interference Cancellation and Soft Demodulation in OFDM Systems
Theory
Efficient ML
- Introduces a deep learning framework for joint NBI cancellation and soft demodulation in OFDM systems.
- NBI-CNet reduces computational complexity by up to 60% compared to state-of-the-art methods.
- LLR-CNet effectively maps non-Gaussian residuals to calibrated soft metrics, improving demodulation reliability.
- The framework eliminates error floors and achieves significant performance gains under severe interference conditions.
Read more
Deep Learning for Joint Narrowband Interference Cancellation and Soft Demodulation in OFDM Systems
Summary
This paper addresses the challenge of narrowband interference (NBI) in orthogonal frequency-division multiplexing (OFDM) systems, which can severely degrade performance by corrupting subcarriers and complicating soft demodulation. The authors propose a unified deep learning framework that integrates NBI cancellation and robust soft demodulation. The framework consists of two main components: NBI-CNet, a physics-informed convolutional neural network that estimates NBI parameters and removes multi-tone interference in a single forward pass, and LLR-CNet, which maps non-Gaussian post-mitigation residuals to well-calibrated soft metrics. The proposed method significantly reduces computational complexity by up to 60% compared to traditional compressed-sensing techniques, while also eliminating error floors associated with classical demappers. Simulation results indicate that the framework performs effectively under various interference conditions, achieving performance within 0.2 to 0.5 dB of optimal iterative baselines and providing a coding gain exceeding 3 dB in challenging scenarios. The architecture's scale-invariant design allows for robust generalization across different FFT sizes without the need for retraining.
Methodology
The authors developed two convolutional neural networks: NBI-CNet for estimating NBI parameters and mitigating interference, and LLR-CNet for transforming post-mitigation residuals into reliable soft metrics. The approach leverages deep learning to adaptively handle dynamic interference environments without requiring prior knowledge of the number of active interferers.
Results
The proposed framework demonstrated superior performance in simulations, operating within 0.2 to 0.5 dB of optimal iterative baselines under severe interference conditions (SIR = -10 dB) and achieving over 3 dB coding gain in scenarios with heavy spectral overlap. The architecture effectively circumvents error floors associated with interferer-estimation errors.
Implications
This research has significant implications for the design of robust communication systems in dense and interference-prone environments, particularly relevant for future wireless networks like 5G and beyond. The deep learning approach could enhance the reliability and efficiency of data transmission in various applications, including IoT and mobile communications.
A knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving
NLP
Large Language Models
Robotics
- K-Risk dataset includes 31,398 high-risk driving events with LLM annotations.
- Integrates data from 20 human-driven and autonomous vehicle trajectory datasets.
- Provides multi-dimensional risk annotations and causal risk analyses for better decision-making.
- Addresses the underrepresentation of extreme long-tail scenarios in existing datasets.
Read more
A knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving
Summary
The paper introduces K-Risk, a novel knowledge-augmented dataset designed to enhance the safety of autonomous driving by addressing the scarcity of high-risk driving scenarios in existing datasets. K-Risk integrates structured driving trajectories from 20 datasets across Europe, China, and the United States, encompassing various driving environments such as highways and urban areas. It curates 31,398 high-risk events, including a focused subset of 1,036 near-collision cases. Each event is presented as a triplet of trajectory, metadata, and language annotations, which include structured descriptions, abnormal behavior notifications, and LLM-generated causal risk analyses. This dataset aims to bridge the gap between structured traffic data and semantic reasoning, providing a foundation for developing risk-aware autonomous driving agents. The authors highlight the importance of multi-dimensional risk annotations and interpretable language supervision in training models that can effectively respond to both common and rare high-risk scenarios, ultimately contributing to safer autonomous driving systems.
Methodology
The authors developed a unified risk-centric extraction pipeline to curate high-risk events from multiple trajectory datasets. They employed large language models (LLMs) to generate semantic annotations, including causal risk analyses and action recommendations, validated through a closed-loop simulator.
Results
K-Risk successfully compiles a comprehensive dataset of high-risk driving scenarios, providing structured trajectory data alongside semantic annotations. The dataset's multi-dimensional risk assessments and LLM-generated insights facilitate improved reasoning and decision-making in autonomous driving systems.
Implications
K-Risk serves as a standardized foundation for the development and evaluation of next-generation risk-aware autonomous driving agents, potentially leading to safer autonomous vehicles and increased public trust in their deployment.
Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection
Computer Vision
Theory
Efficient ML
- Traditional balanced-test metrics can misrepresent classifier performance in operational settings with skewed data.
- A three-number reporting method provides a more accurate evaluation of classifier effectiveness.
- The proposed methodology improves the classifier's precision while maintaining a fixed recall threshold.
- The study establishes the Internal Waves Service as a pioneering operational classifier for internal solitary waves detection.
Read more
Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection
Summary
This paper addresses the evaluation of operational Earth-observation classifiers, specifically focusing on the detection of internal solitary waves using Sentinel-1 data. The authors highlight a significant issue where traditional balanced-test performance metrics overstate the precision of classifiers when applied to skewed operational data. They propose a novel three-number reporting method that includes balanced-test performance, operational-prior performance, and real post-deployment performance. This approach allows for a more accurate assessment of classifier effectiveness in real-world scenarios. The study demonstrates the effectiveness of this method through a precision-first development cycle, which systematically improves the classifier's performance while controlling for leakage and ensuring reproducibility. The findings indicate that the operational classifier can achieve a precision of 0.927 at the operational prior, significantly improving upon the previously reported figures. The paper emphasizes the importance of prior-matched evaluation in operational settings, particularly for rare-event detection tasks in Earth observation.
Methodology
The authors developed a prior-matched reporting method that evaluates classifiers based on three performance metrics: balanced-test performance, operational-prior performance, and real post-deployment performance. They implemented a precision-first, leakage-controlled development cycle, promoting classifier improvements based on pre-registered margins and ensuring reproducibility through controlled evaluation conditions.
Results
The operational classifier achieved a precision of 0.927 at the operational prior, demonstrating a significant improvement over the previously reported balanced-test precision of 0.794, which was misleading in the context of operational deployment. The study confirmed that the classifier's discrimination ability transfers to unseen periods, validating the robustness of the proposed evaluation method.
Implications
The findings suggest that prior-matched evaluation methods can enhance the reliability of performance reporting for operational classifiers in Earth observation. This approach can be applied to other rare-event detection tasks, improving resource allocation and expert validation processes in various operational settings.
Dual Attention Heads for Personalized Federated Learning in ECG Classification
Federated Learning
Time Series
- Introduction of FedDualAtt, a dual-attention transformer for ECG classification in federated learning.
- Global and local attention heads allow for both cross-site generalization and local adaptation.
- Empirical results show superior performance of FedDualAtt over existing federated learning methods.
- Analysis of head ratios indicates varying benefits of architectural personalization for different clients.
Read more
Dual Attention Heads for Personalized Federated Learning in ECG Classification
Summary
This paper presents FedDualAtt, a novel personalized federated learning approach designed to enhance electrocardiogram (ECG) classification by addressing the challenges posed by heterogeneous ECG data across different healthcare providers. The proposed method introduces a dual-attention transformer module that splits attention heads into global and local branches. The global heads are aggregated using the FedAvg algorithm to capture shared patterns across sites, while the local heads remain client-specific to adapt to the unique characteristics of each institution's data. This architecture allows for simultaneous cross-site generalization and site-specific adaptation without introducing additional training objectives. The authors conducted experiments on the FedCVD benchmark, demonstrating that FedDualAtt outperforms existing federated learning methods in ECG classification tasks. The analysis of different global-local head ratios revealed that varying levels of architectural personalization benefit different clients, indicating a tailored approach to model training in federated settings.
Methodology
The methodology involves a hybrid CNN-Transformer architecture where the ResNet1D-34 serves as the feature extractor for ECG signals. The model incorporates dual attention transformer blocks that partition attention heads into global and local branches. The global heads are aggregated via FedAvg, while local heads are kept client-specific. The federated training protocol ensures strict parameter separation for global and local parameters, facilitating effective collaboration among multiple healthcare providers.
Results
The experimental evaluation on the FedCVD benchmark showed that FedDualAtt achieved better classification performance compared to existing federated learning and personalized federated learning methods. The best-reported results indicated significant improvements in ECG classification tasks, demonstrating the effectiveness of the dual-attention architecture.
Implications
The findings suggest that personalized federated learning can be effectively implemented in healthcare settings, particularly for ECG classification, by leveraging architectural personalization. This approach can enhance model performance while maintaining patient data privacy, paving the way for more accurate and reliable automated ECG analysis across diverse clinical environments.
A Quiet Failure in Calibrated Virtual Screening: Marginal Conformal Prediction Under-Covers the Minority Class, and a Class-Conditional Fix Recovers It
Theory
Graph Learning
- Marginal conformal prediction can under-cover minority classes in imbalanced datasets, despite meeting overall coverage targets.
- The failure is consistent across different model architectures and is linked to the calibration of minority class predictions.
- Class-conditional conformal prediction effectively addresses the minority coverage issue, restoring reliability.
- Standard aggregate metrics can obscure significant failures in minority class predictions.
Read more
A Quiet Failure in Calibrated Virtual Screening: Marginal Conformal Prediction Under-Covers the Minority Class, and a Class-Conditional Fix Recovers It
Summary
This paper investigates the limitations of marginal conformal prediction in the context of imbalanced datasets commonly encountered in drug discovery. While conformal prediction is praised for its reliability guarantees, the authors demonstrate that it can significantly under-represent minority classes, which is critical in screening tasks where the minority class often contains the most valuable information (e.g., toxic compounds). Through experiments on four datasets, the authors reveal that marginal conformal prediction meets its overall coverage target but fails to adequately cover minority classes, with coverage rates dropping as low as 4.2% in heavily imbalanced scenarios. This issue persists across various model architectures, indicating that it is not specific to any single approach. The authors provide a quantitative explanation for this phenomenon, linking the minority shortfall to the majority surplus amplified by the imbalance ratio. They propose a class-conditional conformal prediction method as a solution, which restores minority coverage while only slightly increasing prediction set size. The paper emphasizes the importance of recognizing this vulnerability in virtual screening and provides a practical protocol to ensure reliable predictions across classes.
Methodology
The authors conducted experiments on four imbalanced molecular datasets using various model architectures, including random forests, graph convolutional networks, and pretrained chemical language models. They analyzed the coverage performance of marginal conformal prediction and compared it with class-conditional conformal prediction. A decomposition of coverage was used to explain the observed effects quantitatively.
Results
The study found that marginal conformal prediction achieved its global coverage target while minority class coverage was severely lacking, with rates as low as 4.2%. The under-coverage was consistent across different models and was shown to be influenced by the imbalance ratio. Class-conditional conformal prediction successfully restored minority coverage to target levels with a modest increase in prediction set size.
Implications
The findings highlight a critical vulnerability in the application of conformal prediction in drug discovery, particularly in virtual screening tasks. By addressing the minority class under-coverage, the proposed class-conditional approach can enhance the reliability of predictions, leading to more effective screening campaigns and better identification of valuable compounds.
Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata
NLP
Large Language Models
Generative Models
- Introduces a novel algorithm for constrained decoding in diffusion language models using finite automata.
- Guarantees constraint satisfaction by construction, applicable to various decoding strategies.
- Implements depth-reduction techniques to enhance inference efficiency.
- Empirical results show substantial accuracy improvements in multiple tasks with low overhead.
Read more
Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata
Summary
This paper addresses the challenge of constrained decoding in diffusion language models (dLLMs), which differ from traditional autoregressive models by sampling multiple tokens simultaneously. The authors propose an exact and tractable algorithm for sampling from a constrained mean-field posterior defined by finite automata. This approach allows for efficient inference while ensuring that generated outputs adhere to specified structural constraints, such as JSON schema or SQL syntax. By viewing finite automata as graphical models, the authors develop a method that guarantees constraint satisfaction and supports both greedy and stochastic decoding. Additionally, they introduce depth-reduction techniques from arithmetic circuit theory, reducing the sampling depth from linear to logarithmic in relation to sequence length. Empirical evaluations on models like Dream-7B and LLaDA-8B demonstrate significant improvements in accuracy across various tasks, with minimal inference overhead compared to unconstrained decoding.
Methodology
The authors develop an inference algorithm that samples from the constrained mean-field posterior by leveraging finite automata as graphical models. This method allows for efficient sampling while ensuring that the generated sequences meet the specified constraints. They also apply depth-reduction techniques to restructure the computation graph, significantly improving parallelism during inference.
Results
The proposed method shows marked improvements in accuracy for various tasks, such as function calling and math reasoning. For instance, on the BFCL-Live task, the greedy decoding accuracy of Dream-7B increased from 63.9% to 71.5%, while stochastic sampling accuracy improved from 22.3% to 69.0%, all with less than 5% additional wall-clock time compared to unconstrained decoding.
Implications
This work has significant implications for applications requiring strict adherence to output formats, such as automated code generation, structured data retrieval, and other tasks where output validity is critical. The efficient inference method can enhance the usability of dLLMs in real-world applications.
Avoiding unsafe sets when training with Langevin Dynamics
Theory
Optimization
- The paper provides bounds on the probability of a training trajectory entering a failure region during training.
- Equilibrium mass ฯ(AH) is shown to be exponentially small in the dimension d.
- Transient swelling of trajectory mass can occur, necessitating a burn-in time for safety.
- A local relaxation rate is introduced to address swelling in geometrically isolated failure regions.
Read more
Avoiding unsafe sets when training with Langevin Dynamics
Summary
This paper addresses the safety of training models using noisy gradient descent, modeled as overdamped Langevin dynamics on the loss landscape. The focus is on bounding the probability that the training trajectory enters a designated failure region, denoted as AH. The author investigates this for a smooth, strongly convex loss in d dimensions, where AH is separated from the minimizer by an energy gap. Three main bounds are derived: the equilibrium mass ฯ(AH) is exponentially small in d, and a shape-free bound shows that the in-set probability ฮฝt(AH) relaxes to twice the static value after a burn-in time of order d. A worked example illustrates the necessity of this burn-in, as transient swelling can occur, where the trajectory mass temporarily exceeds its equilibrium value. To mitigate this swelling, a local relaxation rate is introduced, which can shrink the burn-in time for geometrically isolated regions. The paper concludes that while strong convexity influences the speed of training relaxation, the shape of the unsafe set determines the likelihood of the trajectory bulging through it.
Methodology
The author models the training process as overdamped Langevin dynamics, represented by a stochastic differential equation (SDE). The analysis involves deriving bounds on the probability that the trajectory occupies the failure region AH, utilizing assumptions of smoothness, convexity, and energy gaps in the loss function.
Results
The paper establishes two types of bounds: a static mass bound on the equilibrium probability at the end of training, which is exponentially small in d, and a dynamic mass bound that shows the in-set probability along the trajectory relaxes to the static bound after a burn-in time of order d. Additionally, a local relaxation rate is introduced to improve the bounds for isolated failure regions.
Implications
The findings have significant implications for the safe training of machine learning models, particularly in scenarios where avoiding unsafe behaviors is critical, such as in code generation and AI alignment. The results can inform strategies to ensure that training processes do not inadvertently lead to dangerous outcomes.
RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting
Time Series
- RhyMix integrates explicit multi-period cyclic priors into a dual-path forecasting architecture.
- The model utilizes Multi-Scale Temporal Convolution with Channel Attention for enhanced temporal representation learning.
- Adaptive gating mechanisms allow for dynamic fusion of multiple forecasting heads tailored to specific input samples.
- RhyMix is lightweight, with approximately 40K parameters and low inference latency, making it suitable for edge devices.
Read more
RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting
Summary
The paper introduces RhyMix, a novel hybrid neural architecture designed for long-term time series forecasting. RhyMix addresses the limitations of existing forecasting models, which often rely on single-path temporal modeling. It employs a dual-path architecture that includes a Cyclic Path for capturing seasonal patterns and a Multi-Scale Temporal Convolutional Network with Channel Attention (MSTCN-CA) for learning contextual temporal representations. The model features adaptive gating mechanisms that dynamically fuse outputs from four specialized forecasting heads based on the input characteristics. This design allows RhyMix to adapt to various temporal patterns while maintaining computational efficiency with linear complexity and a compact model size. The architecture is evaluated across 12 real-world datasets, achieving state-of-the-art performance on 10 of them, demonstrating its effectiveness in long-term forecasting tasks.
Methodology
RhyMix employs a hybrid architecture consisting of two main paths: a Cyclic Path that captures seasonal patterns through learnable cyclic embeddings, and a MSTCN-CA path that utilizes multi-scale depthwise dilated convolutions for temporal dependency modeling. Adaptive gating mechanisms are implemented to combine outputs from various forecasting heads, ensuring the model can adjust to the specific characteristics of the input data.
Results
RhyMix demonstrated superior forecasting performance, achieving state-of-the-art results on 10 out of 12 datasets tested. The model's design allows for efficient computation, maintaining linear complexity and low latency, which is critical for real-time applications.
Implications
The lightweight and efficient design of RhyMix makes it suitable for deployment in resource-constrained environments, such as edge devices. Its ability to adaptively model complex temporal patterns can enhance forecasting accuracy in various applications, including finance, energy management, and supply chain optimization.
FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
Time Series
- Introduces FMMVCC, a deep clustering framework for univariate time series data.
- Utilizes Mamba-based encoders for efficient long-range temporal dependency modeling.
- Implements multi-view self-supervised learning through temporal masking and augmentations.
- Incorporates a fuzzy clustering objective for improved cluster separability.
Read more
FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
Summary
The paper introduces Fuzzy Mamba-based Multi-View Contrastive Clustering (FMMVCC), a novel deep clustering framework designed for univariate time series data. It addresses the challenges of limited labeled data in time series analysis by employing unsupervised learning techniques. The framework utilizes Mamba-based encoders, which efficiently capture long-range temporal dependencies while maintaining linear computational complexity. FMMVCC generates multiple views of each time series through temporal masking and augmentation, enhancing the robustness of representation learning. A fuzzy clustering objective is integrated to improve cluster separability and handle ambiguous temporal patterns. Experimental results demonstrate that FMMVCC outperforms state-of-the-art methods across 15 benchmark datasets, achieving the best performance in 29 out of 60 evaluations and the highest average rank in all scenarios tested.
Methodology
FMMVCC employs Mamba-based encoders to model time series data, generating multiple augmented views through stochastic temporal augmentations and random masking. The framework integrates a clustering objective into the representation learning process, allowing for soft cluster assignments and improved separability in the latent space.
Results
FMMVCC was evaluated on 15 benchmark datasets, consistently outperforming existing methods. It achieved the best overall performance in 29 out of 60 metric evaluations and maintained the highest average rank across all tested scenarios.
Implications
The proposed framework has significant implications for time series analysis in various domains, including IoT, healthcare, and environmental monitoring, where labeled data is scarce. FMMVCC can facilitate exploratory data analysis and anomaly detection without the need for extensive manual labeling.
Who Analyses the Analyser? Self-Validating LLM Hazard Analysis with Constitutional Meta-STPA
Large Language Models
Theory
Interpretability
- Introduces Constitutional Meta-STPA to analyze LLM-assisted safety tools.
- Derives a governance constitution with 21 Tool Principles and 8 Meta-Safety Principles.
- Demonstrates significant improvement in safety scores using the derived principles.
- Highlights the importance of self-analysis for LLMs in safety-critical applications.
Read more
Who Analyses the Analyser? Self-Validating LLM Hazard Analysis with Constitutional Meta-STPA
Summary
This paper addresses a critical blind spot in the application of large language models (LLMs) in safety analysis, specifically in Systems-Theoretic Process Analysis (STPA). While LLMs are increasingly used to draft safety-related artifacts, the authors argue that the tools themselves, which assist in this analysis, are also safety-relevant systems that require scrutiny. To tackle this issue, the authors introduce Constitutional Meta-STPA, a self-validating tool that applies STPA to the class of AI-assisted safety tools. This meta-analysis generates a governance constitution comprising 21 Tool Principles and 8 Meta-Safety Principles, each linked to specific enforcement points in the code. The authors demonstrate that a frontier ensemble of LLMs can recover a significant portion of these principles from the tool's design, indicating that the constitution is grounded in the tool's functionality. Furthermore, the implementation of these principles leads to a notable increase in safety scores on adversarial probes. The study also highlights the importance of coverage metrics in evaluating AI tool performance and reports findings on reproducibility across different vendors. The authors provide a comprehensive release of their findings, including the governance constitution and reproducibility artifacts.
Methodology
The authors conducted a meta-STPA on LLM-assisted safety tools, analyzing their control actions and deriving a governance constitution based on identified losses, hazards, Unsafe Control Actions (UCAs), and constraints. They utilized a frontier ensemble of LLMs to validate the principles and assessed the impact of these principles on safety scores through adversarial probes.
Results
The study found that a strong LLM ensemble could recover 18 out of 21 Tool Principles and all 8 Meta-Safety Principles, while a weaker model pair recovered fewer. The implementation of Tool Principles resulted in an increase in mean safety scores from 1.03 to 1.85, representing a 79% improvement. The findings also indicated that coverage metrics effectively differentiate between self-analysis of AI tools and baseline performance.
Implications
The findings suggest that LLM-assisted safety tools can be effectively governed through self-analysis, enhancing their reliability in safety-critical applications. The governance constitution and methodologies developed could serve as a framework for future AI tool assessments, ensuring that LLMs are held to safety standards.
PGD-NO: A Neural Operator with Precomputed Geometry Decomposition for 3D Million-scale Physics Simulations
Efficient ML
Interpretability
Theory
- PGD-NO utilizes a precomputed geometry decomposition approach to reduce memory consumption.
- The model allows for linear scalability, handling meshes with over 10 million nodes.
- It demonstrates competitive accuracy in predictive tasks across diverse industrial benchmarks.
- The architecture provides intrinsic interpretability through attention mechanisms.
Read more
PGD-NO: A Neural Operator with Precomputed Geometry Decomposition for 3D Million-scale Physics Simulations
Summary
The paper introduces PGD-NO, a novel neural operator designed to address the limitations of existing neural PDE solvers, particularly their high memory consumption and single-node processing constraints. Traditional methods struggle with large-scale simulations due to GPU memory limitations, especially when dealing with complex geometries that require high-resolution meshes. PGD-NO mitigates these issues by employing a precomputed geometry decomposition approach, which extracts 'geometry tokens' through a deterministic algorithm. This decouples the feature extraction process from solution querying, allowing for linear memory scalability and enabling simulations on meshes exceeding 10 million nodes. The architecture demonstrates competitive predictive accuracy across various industrial benchmarks and provides interpretability through attention mechanisms. The proposed method effectively overcomes traditional mesh-size constraints, making it a robust solution for large-scale, high-fidelity industrial design applications.
Methodology
The authors developed a deterministic hierarchical decomposition algorithm to extract geometry tokens, which are non-learnable geometric primitives. This approach allows the model to bypass the intensive encoding process typically required in neural PDE solvers. The PGD-NO architecture then decouples the geometry encoding from the solution querying, enabling efficient prediction of PDE solutions based on these precomputed features.
Results
PGD-NO successfully demonstrated the ability to perform high-fidelity simulations on large-scale meshes, achieving competitive predictive accuracy while maintaining a significantly lower GPU memory footprint compared to traditional grid-based or graph-based neural operators. The architecture's performance was validated across various industrial benchmarks, showcasing its robustness and efficiency.
Implications
The development of PGD-NO has significant implications for the field of computational science and engineering, particularly in industrial design applications that require high-fidelity simulations of complex geometries. By overcoming memory limitations and enabling efficient processing of large datasets, PGD-NO can accelerate the design and manufacturing processes in various engineering domains.
Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery
Generative Models
Optimization
- CAGI integrates clustering and imputation into a co-optimization framework to enhance data recovery.
- The 'Partition-Guide-Restore' strategy allows for dynamic cluster assignments that inform the imputation process.
- The method addresses the circular dependency between subgroup identification and data completion.
- CAGI outperforms traditional imputation methods by preserving subgroup distributions.
Read more
Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery
Summary
The paper addresses the challenge of missing data in real-world datasets, which often exhibit complex latent structures with multiple subgroups. Traditional imputation methods tend to overlook this heterogeneity, leading to inaccurate estimates that do not respect subgroup boundaries. The authors propose a novel framework called CAGI (Cluster-Aware Generative Imputation) that reformulates the tasks of clustering and imputation as a mutually reinforcing co-optimization process. CAGI employs a 'Partition-Guide-Restore' strategy, where dynamic cluster assignments serve as local priors for a Generative Adversarial Network (GAN) to conditionally generate missing values. This iterative feedback loop allows for the refinement of both cluster structures and imputed values, ensuring that the imputed data aligns with the true statistical patterns of each subgroup. The framework also incorporates a multi-level optimization objective that balances instance-level reconstruction with distribution-level regularization. Extensive experiments conducted on 14 benchmark datasets demonstrate the superiority of CAGI over 15 representative baselines, highlighting its effectiveness in recovering missing data while respecting subgroup structures.
Methodology
CAGI employs a co-optimization process that combines clustering and imputation. It uses a GAN where cluster assignments guide the generation of missing values. The process iteratively refines both the clustering and the imputed values through a feedback loop, ensuring that the imputed data adheres to the statistical characteristics of the identified subgroups.
Results
The experiments on 14 benchmark datasets revealed that CAGI significantly outperforms 15 baseline methods in terms of imputation accuracy and the preservation of subgroup structures. The results indicate that CAGI effectively mitigates issues related to distributional shifts and boundary violations commonly seen in traditional imputation methods.
Implications
CAGI has potential applications in various fields where missing data is prevalent, such as healthcare, marketing, and social sciences. By accurately recovering missing values while respecting subgroup structures, it can improve the reliability of downstream analyses, such as classification and clustering tasks.
Converge to Surprise: Evolutionary Self-supervised Image Clustering
Computer Vision
Optimization
Theory
- Introduction of a surprise score to measure non-randomness in image representations.
- Development of the 'converge-to-surprise' optimization scheme combining evolution strategies and gradient descent.
- Achievement of state-of-the-art results in non-parametric self-supervised image clustering.
- Demonstration of the framework's ability to discover meaningful features without predefined targets.
Read more
Converge to Surprise: Evolutionary Self-supervised Image Clustering
Summary
This paper presents a novel self-supervised image clustering framework that deviates from traditional gradient descent methods, which rely on clearly defined targets for loss calculation. The authors introduce the concept of a 'surprise score' that quantifies how unlikely the model's output representation is under the null hypothesis of maximum entropy, where each pixel is treated as independent and identically distributed (i.i.d.) random noise. By maximizing this surprise score, the model is encouraged to uncover non-random features in the data. The proposed 'converge-to-surprise' optimization scheme combines an evolution strategy (ES) outer loop, which maximizes the surprise score without requiring gradients, with a periodic gradient-descent inner loop that utilizes the surprising clusters identified by the ES as surrogate targets. This approach allows the model to operate effectively in non-parametric self-supervised image clustering settings, where the number of ground-truth classes is unknown. Experimental results demonstrate that this framework achieves state-of-the-art performance on standard image benchmarks, showcasing its effectiveness in the strictest deep-clustering scenarios.
Methodology
The methodology involves defining a surprise score based on the Principle of Maximum Entropy, which assesses the likelihood of the model's output under the assumption of random noise. The optimization process employs an evolution strategy to maximize this surprise score, complemented by a gradient-descent approach that refines the model using previously discovered clusters as targets.
Results
The proposed framework achieves new state-of-the-art results on benchmark datasets for non-parametric self-supervised image clustering, outperforming existing methods that rely on gradient-based optimization.
Implications
This work has significant implications for self-supervised learning, particularly in scenarios where labeled data is scarce or unavailable. The ability to cluster images without predefined class information could enhance applications in image retrieval, organization, and understanding in various domains.
Super Weights in LLMs and the Failure of Selective Training
Large Language Models
Efficient ML
Theory
- Super Weights do not universally lead to improved performance when trained in isolation.
- Training Super Weights results in accuracy drops to random-guessing levels, while random parameter training improves performance.
- LoRA achieves significant accuracy with minimal parameter updates, highlighting the importance of structured training.
- The study validates Super Weight consistency across diverse models and demonstrates the failure of selective training approaches.
Read more
Super Weights in LLMs and the Failure of Selective Training
Summary
This paper investigates the concept of Super Weights in large language models (LLMs), which are individual parameters whose removal significantly degrades model performance. The authors challenge the assumption that training these Super Weights in isolation would enhance model accuracy. Through experiments on OLMo-1B and OLMo-7B, they demonstrate that training only Super Weights leads to drastic drops in accuracy, even when expanding the training to include local neighborhoods of parameters. In contrast, randomly selecting parameters from the same layers results in improved performance, indicating that the failure is specific to the targeted Super Weight coordinates rather than sparsity itself. The authors also explore the effectiveness of parameter-efficient fine-tuning methods like LoRA, which successfully updates a minimal number of parameters while maintaining or improving accuracy. Their findings suggest that the importance of a parameter does not equate to its trainability in isolation, emphasizing the need for structured updates across entire layers for effective fine-tuning.
Methodology
The authors conducted a series of experiments on OLMo-1B and OLMo-7B, including pruning, direct training of Super Weights, neighborhood training, and low-rank updates using LoRA. They validated the consistency of Super Weights across multiple samples and performed ablation studies to isolate the effects of targeting Super Weights versus randomly chosen parameters.
Results
The results showed that training Super Weights led to significant accuracy degradation, while randomly selecting parameters from the same layers improved performance. LoRA was effective in achieving high accuracy with only 0.16% of parameters updated, and constraining updates to Super Weight coordinates had no measurable effect. The findings indicate that effective fine-tuning relies on coordinated updates across entire layers rather than isolated parameter training.
Implications
These findings have implications for the design of fine-tuning strategies in LLMs, suggesting that targeting individual important parameters may not be effective. Instead, structured approaches that consider the relational dependencies of parameters could lead to better performance in model adaptation tasks.
When Does Continual Learning Require Learning
Large Language Models
NLP
Reinforcement Learning
- Continual learning is framed as increasing competence in response to environmental changes.
- The authors categorize changes along two axes: space (new domains) and time (data drift).
- Different continual learning methods exhibit unique strengths and weaknesses, highlighting the need for tailored approaches.
- Prompt-based methods excel in backward accuracy but degrade on future tasks, while distillation methods struggle with rapid updates.
Read more
When Does Continual Learning Require Learning
Summary
This paper addresses the challenge of continual learning in large language models (LLMs), arguing that the current focus on mitigating forgetting is insufficient. The authors propose a new framework that defines continual learning as the process of increasing model competence in response to changes in the environment, which they categorize along two axes: space (new domains) and time (data drift). They evaluate various methods for continual learning by recasting standard LLM benchmarks as sequential problems, allowing for a comparison of different update mechanisms, including prompt-based methods, supervised learning, reinforcement learning, and context compression. The findings reveal that different methods exhibit distinct trade-offs: prompt-based approaches quickly adapt but struggle with future tasks, while distillation methods accumulate knowledge steadily but have difficulty updating outdated information. Context compression enhances efficiency without significantly improving task learning, and online reinforcement learning adapts well but is sensitive to reward noise. The authors conclude that continual learning is not a monolithic capability; rather, it requires tailored strategies depending on the nature of environmental changes, providing insights for the development of more robust continual learning systems.
Methodology
The authors developed a unified framework to evaluate continual learning in LLMs by recasting standard benchmarks into sequential problems. They compared various methods, including prompt optimization, supervised updates, online reinforcement learning, and context compression, using a common backbone model.
Results
The study found consistent trade-offs among the evaluated methods: prompt-based methods fit past data well but fail on future tasks, distillation methods accumulate knowledge stably but struggle with updates, context compression improves efficiency without enhancing learning, and online reinforcement learning adapts effectively but is sensitive to noise.
Implications
The findings suggest that continual learning strategies must be tailored to the specific nature of environmental changes, guiding the design of more effective continual learning systems in practical applications.
Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems
Reinforcement Learning
Large Language Models
Optimization
- Introduces AdaPrefix-GRPO, a method that adapts prefix length dynamically during training to optimize gradient signal.
- Demonstrates that maintaining a success rate of approximately 50% maximizes the learning signal in GRPO.
- Implements the method with minimal modifications to existing GRPO frameworks, focusing on data preparation.
- Achieves substantial improvements in accuracy on hard reasoning problems while reducing trace lengths.
Read more
Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems
Summary
This paper addresses the limitations of Group Relative Policy Optimization (GRPO) in reinforcement learning, particularly when dealing with hard reasoning problems where no rollouts succeed, leading to a lack of gradient signal. The author introduces AdaPrefix-GRPO, a method that dynamically adjusts the prefix length of reference solutions during training to maintain a success rate around 50%, optimizing the gradient signal. By treating prefix length as a closed-loop controller, AdaPrefix-GRPO allows for continuous adaptation based on the model's performance, improving learning efficiency. The method is implemented with minimal changes to the existing GRPO framework, focusing on data preparation and loss masking. Experimental results demonstrate that AdaPrefix-GRPO significantly enhances accuracy on challenging tasks while reducing trace length, outperforming vanilla GRPO and fixed-prefix approaches, especially in smaller models.
Methodology
The methodology involves treating prefix length as a closed-loop controller that adjusts based on the realized batch-mean success rate during training. The method employs a root-finding update to maintain a target success rate, with prefixes being dynamically adjusted per sample. The training process includes a one-time calibration step for prefix length, which is amortized across multiple runs, ensuring no additional compute is required during training.
Results
AdaPrefix-GRPO outperformed vanilla GRPO and fixed-prefix methods, achieving over double the accuracy on held-out problems for a 0.6B model and significant improvements for larger models (1.6ร for Qwen3-1.7B and 1.7ร for AIME). The method also effectively reduced trace lengths, demonstrating enhanced learning efficiency.
Implications
The findings suggest that adaptive prefix control can significantly improve the training of models on hard reasoning tasks, potentially leading to better performance in applications requiring advanced reasoning capabilities, such as mathematics and programming. This approach could be beneficial in optimizing reinforcement learning strategies across various domains.
When Do Geometric Algebra Layers Beat Scalarization? A Controlled Study on SO(3)-Equivariant Vector Laws
Robotics
Theory
Efficient ML
- Geometric algebra layers do not outperform scalarization for single-stage laws, which can be computed with one equivariant interaction.
- For compositional laws involving nested group operations, geometric algebra networks significantly outperform scalarization in low-data regimes.
- No tested model, including equivariant architectures, successfully extrapolates invariant magnitudes under radial out-of-distribution conditions.
- The study emphasizes the importance of distinguishing between the effects of equivariance and specific parameterization in model performance.
Read more
When Do Geometric Algebra Layers Beat Scalarization? A Controlled Study on SO(3)-Equivariant Vector Laws
Summary
This paper investigates the effectiveness of geometric algebra layers versus scalarization in learning SO(3)-equivariant vector laws, particularly in the context of 3D learning problems that require functions to commute with rotations. The study compares a compact network built from Clifford algebra Cl(3, 0) primitives, which are designed to be exactly SO(3)-equivariant, against a scalarization baseline that utilizes invariant dot products fed into a small multi-layer perceptron (MLP). The findings reveal that for single-stage laws, scalarization performs comparably or better than the geometric algebra network at a significantly lower training cost. However, for compositional laws that involve nested group operations, the geometric algebra network demonstrates superior performance in low-data scenarios, achieving higher accuracy with fewer samples. The study concludes that while geometric algebra layers do not universally enhance low-data 3D learning, they become advantageous when the target functions involve complex compositions of group elements.
Methodology
The study employs a controlled benchmark involving six synthetic vector laws, comparing a geometric algebra network with a scalarization baseline. Both models are trained under identical conditions using mean squared error as the loss function. The performance is evaluated based on sample efficiency and accuracy across various tasks, including single-stage and compositional laws.
Results
The results indicate that scalarization matches or exceeds the performance of the geometric algebra network for single-stage laws, achieving a training cost reduction of 13 to 19 times. In contrast, for compositional laws, the geometric algebra network outperforms scalarization by a factor of 3 to 16 in low-data scenarios. The advantage of the geometric algebra network diminishes as the sample size increases, with scalarization catching up beyond a thousand samples. Additionally, all models fail to extrapolate invariant magnitudes effectively.
Implications
The findings suggest that while geometric algebra layers can enhance learning in specific contexts, they are not a universal solution for low-data 3D learning tasks. This insight can guide future research in designing more efficient architectures for 3D learning problems, particularly in applications requiring equivariance to rotations.