AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
48
Papers today
8h
Update frequency
7
Days of history
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Reinforcement Learning
Large Language Models
Efficient ML
- Introduces LongStraw, an architecture-aware execution stack for long-context RL post-training.
- Demonstrates the ability to handle context lengths exceeding 2 million tokens under fixed GPU budgets.
- Utilizes Group Relative Policy Optimization (GRPO) to optimize memory usage during training.
- Validates the approach on two model families, achieving significant context length extensions.
Read more
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Summary
The paper addresses the significant gap between the context lengths supported during inference and those used in reinforcement learning (RL) post-training, where inference systems can handle million-token contexts while training often remains limited to 256K tokens. This discrepancy is particularly critical for AI agents that accumulate extensive histories over long trajectories. The authors introduce LongStraw, an innovative architecture-aware execution stack designed to facilitate million-token RL post-training while adhering to fixed GPU budgets. LongStraw employs Group Relative Policy Optimization (GRPO) and optimizes memory usage by evaluating shared prompts once, retaining only essential model-specific states, and replaying short response branches sequentially. This approach effectively reduces the GPU memory footprint while enabling the processing of larger context lengths. The implementation of LongStraw on two distinct model familiesβQwen3.6-27B and GLM-5.2βdemonstrates its capability to handle up to 4.46 million tokens across various GPU configurations. The findings indicate that managing state lifetime and physical ownership are crucial for extending the practical context limits of RL post-training, thus lowering the hardware barriers for researchers and smaller teams to explore long-context training.
Methodology
The authors developed LongStraw, which evaluates shared prompts once, retains only necessary model-specific states, and replays short response branches sequentially under autograd. This method reduces the live training graph size, allowing for efficient memory usage while processing long contexts.
Results
LongStraw successfully completed grouped scoring and response backward for the Qwen model at 2.1 million positions with minimal increase in memory usage. A stress test extended the execution to 4.46 million positions, demonstrating the architecture's scalability and efficiency across different GPU setups.
Implications
LongStraw lowers the hardware requirements for long-context training, making it more accessible for researchers and smaller teams. This advancement could lead to broader exploration and development of AI agents capable of handling extensive historical data, enhancing their performance in complex tasks.
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
Theory
- C3R provides a label-free per-domain contamination certificate, addressing the limitations of existing methods.
- The method integrates seamlessly into existing retrieval architectures without requiring retraining.
- C3R demonstrates superior performance in maintaining recall while controlling contamination compared to traditional methods.
- The study introduces BEIR-MIX, a public benchmark for evaluating multi-domain contamination in retrieval systems.
Read more
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
Summary
This paper addresses the challenge of contamination in multi-domain retrieval systems, where documents from incorrect domains can be returned alongside relevant results. The proposed method, C3R (Certified Contamination Control for Retrieval), introduces a control layer that certifies a per-domain contamination budget without requiring query-time labels. The methodology employs a two-split conformal scheme that bounds the domain router's error rates and calibrates a demotion threshold, ensuring a finite-sample transfer bound from inferred to true domains. The approach is designed to be lightweight and can be integrated into existing retrieval systems without retraining. The authors validate their method using a newly created testbed, BEIR-MIX, which highlights the asymmetry of contamination across domains. Results show that C3R maintains zero certificate violations across multiple calibrations, significantly outperforming existing marginal conformal risk controls, particularly in high-contamination scenarios. The findings indicate that contamination is more closely related to subject overlap than topical adjacency, emphasizing the need for dedicated contamination control in retrieval systems.
Methodology
The methodology involves a two-split conformal scheme that first bounds the domain router's error rates and then calibrates a demotion threshold. This allows for a finite-sample transfer bound from the inferred domain to the true domain, supporting heterogeneous per-domain budgets. The system is designed to be a drop-in control layer that can be attached to existing retrieval stacks without retraining.
Results
C3R achieved zero per-domain certificate violations across 1000 resampled calibrations, while marginal conformal risk control violated the most contaminated domain in 100% of cases. The method retained up to 6 times more recall at equal certified contamination levels compared to the strongest calibrated cascade. The stability of the certificate was confirmed across four open testbeds spanning various contamination levels.
Implications
The findings suggest that C3R can significantly enhance the reliability of multi-domain retrieval systems, particularly in high-stakes applications where contamination poses serious risks. The method's ability to certify contamination control can lead to safer and more effective information retrieval in regulated industries.
xHC: Expanded Hyper-Connections
Large Language Models
Efficient ML
- xHC enables meaningful expansion of Hyper-Connections beyond N=4, addressing previous limitations.
- The method combines temporal feature augmentation and sparse residual-stream architecture to enhance efficiency.
- Empirical results show significant improvements in downstream performance with modest increases in training costs.
- xHC changes the benefit-cost tradeoff of scaling, making larger N more cost-effective.
Read more
xHC: Expanded Hyper-Connections
Summary
The paper introduces xHC (Expanded Hyper-Connections), a novel approach to enhance the residual stream of Transformers by expanding it into multiple parallel streams, thereby enabling memory scaling beyond traditional model width and depth. The authors identify limitations in existing Hyper-Connections (HC) methods, particularly the diminishing performance gains and increasing training costs when scaling beyond four streams. They attribute these issues to two main bottlenecks: insufficient write-back information for multiple streams and the cubic scaling cost associated with residual-mixing generation. To overcome these challenges, xHC employs temporal feature augmentation to enrich write-back signals and a sparse residual-stream architecture that selectively updates a subset of streams. This dual approach allows for effective expansion beyond N=4 while maintaining computational efficiency. The empirical results demonstrate that xHC significantly improves downstream performance and reduces training costs compared to both the vanilla residual baseline and the state-of-the-art mHC method, making it a practical solution for large-scale language model pre-training.
Methodology
The authors propose xHC, which integrates temporal feature augmentation to provide richer write-back information and a sparse architecture that activates only a subset of streams for residual mixing. This design allows for effective scaling of the residual stream while controlling computational costs.
Results
xHC achieves a lower training loss (1.758) compared to mHC (1.776) and the vanilla baseline (1.799) on an 18B MoE model. It also improves the average downstream score from 44.8 with mHC to 48.8, while only increasing training FLOPs by 4.1% relative to the vanilla baseline. Scaling-law experiments indicate that xHC requires less compute to achieve similar loss levels compared to existing methods.
Implications
The findings suggest that xHC can be a viable method for enhancing large language models, making them more efficient and effective in various applications, including natural language processing tasks. The approach could lead to advancements in model architectures that leverage expanded memory capabilities.
HyperShadow: A Benchmark for Detecting 3D Projections of Higher-Dimensional Spatial Objects
Computer Vision
Theory
- HyperShadow is the first benchmark for detecting 3D projections of higher-dimensional spatial objects.
- Traditional intrinsic-dimension estimation methods are inadequate for this task, achieving only 71-73% accuracy.
- A compact learned point network achieves 96.2% accuracy in detecting projections.
- The introduced rigidity witness is a zero-parameter statistic that effectively separates classes with high accuracy.
Read more
HyperShadow: A Benchmark for Detecting 3D Projections of Higher-Dimensional Spatial Objects
Summary
The paper introduces HyperShadow, a novel benchmark designed to detect 3D projections of higher-dimensional spatial objects, specifically those existing in four to six dimensions. Unlike traditional datasets labeled as '4D', which typically combine three spatial dimensions with time, HyperShadow focuses solely on spatial dimensions. The primary task is to determine whether a given 3D point cloud represents a native 3D shape or the projection of a higher-dimensional object. The study highlights the inadequacy of existing intrinsic-dimension estimation methods, which achieve only 71-73% accuracy, while a proposed 190k-parameter point network achieves 96.2% accuracy across various corruption levels. Additionally, the paper introduces a zero-parameter rigidity witness that effectively distinguishes between rigid and non-rigid motions in temporal sequences, achieving an AUROC of 0.982. The dataset, comprising 10,800 static point clouds and 1,800 temporal sequences, is publicly available, providing a controlled environment for studying the detectability of higher-dimensional projections.
Methodology
The methodology involves generating a benchmark dataset of 10,800 static point clouds and 1,800 temporal sequences, which includes native 3D shapes and their projections from higher dimensions. The study employs a compact learned point network to analyze the data, alongside traditional methods like persistent homology and geometric features. The rigidity witness is introduced as a zero-parameter statistic to assess the compatibility of the observed data with rigid 3D motion.
Results
The proposed point network achieved an accuracy of 96.2% in distinguishing between native 3D shapes and their higher-dimensional projections, significantly outperforming traditional methods. The rigidity witness demonstrated an AUROC of 0.982, indicating its effectiveness in classifying rigid versus non-rigid motions in temporal sequences. The results highlight the distinct nature of shadow detection as a problem separate from intrinsic-dimension estimation.
Implications
HyperShadow provides a foundational tool for researchers to explore the characteristics of higher-dimensional projections and their detectability. It opens avenues for further studies in computer vision and machine learning, particularly in understanding how higher-dimensional data can be represented and analyzed in lower-dimensional spaces.
Probabilistic Physics-Informed Neural Networks for Estimating Heterogeneous Elastic Properties from Low-Resolution and Noisy Displacement Data
Theory
- PIE-PINN framework enhances robustness in estimating elastic properties from low-resolution and noisy data.
- Utilizes a combination of B-spline-guided networks and hierarchical models to improve accuracy.
- Demonstrates effectiveness through systematic case studies with varying noise levels.
- Addresses limitations of existing methods that require high-fidelity observations.
Read more
Probabilistic Physics-Informed Neural Networks for Estimating Heterogeneous Elastic Properties from Low-Resolution and Noisy Displacement Data
Summary
This paper addresses the challenge of estimating spatially heterogeneous elastic properties, specifically Young's modulus and Poisson's ratio, from low-resolution and noisy displacement data. The authors propose a novel framework called Probabilistic Inverse Elasticity Physics-Informed Neural Network (PIE-PINN), which integrates probabilistic modeling with physics-informed neural networks to enhance robustness against noise and resolution limitations. The PIE-PINN framework models displacement observations, strain discrepancies, and equilibrium residuals using Laplace distributions, allowing for a unified probabilistic approach. A B-spline-guided displacement network is employed to provide a smooth global representation of the displacement field, while a hierarchical half-Cauchy model is used to adaptively downweight severe fitting errors. This combination enables the framework to recover the latent mean displacement field more effectively. The authors utilize an alternating maximum-likelihood training strategy to optimize both the mean displacement and the scales of the residuals. Systematic case studies demonstrate the robustness of PIE-PINN across varying noise levels and observation resolutions, showcasing its potential for practical applications in fields such as medical imaging and material characterization.
Methodology
The PIE-PINN framework employs a probabilistic model that incorporates Laplace distributions for displacement observations, strain discrepancies, and equilibrium residuals. It combines a B-spline-guided displacement network for global representation with a hierarchical half-Cauchy model to adaptively manage fitting errors. An alternating maximum-likelihood training strategy is used to optimize the mean displacement and residual scales.
Results
The results indicate that PIE-PINN significantly improves the robustness of elasticity estimation under conditions of low resolution and high noise. The framework successfully recovers the latent mean displacement field and demonstrates adaptability to varying data characteristics, outperforming traditional methods that rely on high-fidelity data.
Implications
The proposed PIE-PINN framework has significant implications for various applications, including medical imaging, material characterization, and structural integrity assessments. Its ability to accurately estimate heterogeneous elastic properties from noisy data can enhance diagnostic capabilities and improve the reliability of material evaluations.
Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation
Generative Models
Reinforcement Learning
Computer Vision
- Introduces multi-axis max@K, a reinforcement learning objective for enhancing diversity in T2I models.
- Formulates the problem of image diversity as target-mode coverage, focusing on visually distinct modes.
- Validates the method through controlled experiments and real-world applications, showing significant improvements in fairness metrics.
- Maintains image quality and text alignment while increasing diversity in generated outputs.
Read more
Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation
Summary
This paper addresses the limited diversity in images generated by text-to-image (T2I) models, particularly for person-centric prompts, which can exacerbate demographic biases. The authors introduce a novel reinforcement learning objective called multi-axis max@K, aimed at improving target-mode coverage in T2I generation. This approach allows for a group of samples to represent different visually distinct modes, enhancing the diversity of generated images. The method works by assigning credit to samples based on their contribution to maximizing category-specific scores across a group, rather than aggregating scores within individual samples. The authors validate their approach through experiments on synthetic mixtures and the Stable Diffusion 3.5 Medium (SD3.5-M) model, demonstrating improvements in perceived-appearance fairness and target-mode coverage without sacrificing image quality or text alignment.
Methodology
The authors propose a group-based reinforcement learning framework that uses multi-axis max@K to optimize T2I models. This involves generating a candidate group for each prompt, scoring each sample based on its contribution to maximizing category-specific scores, and updating the model using a standard GRPO-style diffusion update. The method is tested in various settings, including synthetic mixtures and real-world fairness evaluations.
Results
The multi-axis max@K approach significantly improves the Fairness Score by 0.23β0.36 compared to the base model across three automatic evaluators, while preserving image quality and text alignment. The method effectively increases target-mode coverage in generated images, demonstrating its efficacy in enhancing diversity.
Implications
This research has important implications for the development of T2I models, particularly in ensuring that generated images reflect a broader range of visual diversity and reduce demographic biases. The proposed method can be applied to various applications where diverse representations are crucial, such as in advertising, media, and content creation.
Causal Inference for Sequential Settings under Interference and Latent Confounding
Theory
- Introduces a model for causal inference in sequential settings with interference and latent confounding.
- Utilizes an Ising model to capture dependencies among binary outcomes across time and units.
- Proposes a computationally efficient Maximum Pseudo-Likelihood Estimation (MPLE) method for parameter estimation.
- Establishes non-asymptotic consistency for parameter estimation and causal quantity estimation.
Read more
Causal Inference for Sequential Settings under Interference and Latent Confounding
Summary
This paper addresses the challenge of causal inference in sequential observational settings where outcome interference and latent confounding are present. The authors propose a model that captures the dependencies of binary outcomes across multiple units over time, utilizing an Ising model to represent these dependencies. The model accounts for latent confounders through a low-rank factor structure, allowing for the estimation of causal effects from a single sample of high-dimensional data. A computationally efficient method based on Maximum Pseudo-Likelihood Estimation (MPLE) is introduced for parameter estimation, which is shown to be non-asymptotically consistent under mild assumptions. The efficacy of the proposed method is validated through synthetic experiments and a real-world case study examining the impact of vaccination rates on COVID-19 death rates across US counties. The findings demonstrate that the method can reliably estimate causal quantities of interest, even in complex settings where traditional assumptions may not hold.
Methodology
The authors develop a model that incorporates outcome interference and latent confounding, using an Ising model to represent dependencies among outcomes. They employ Maximum Pseudo-Likelihood Estimation (MPLE) to learn model parameters from a single observation of data, which allows for the estimation of causal effects by sampling from the learned distribution.
Results
The proposed method shows non-asymptotic consistency in parameter estimation and effectively estimates causal quantities of interest. The synthetic experiments validate the model's performance, while the real-world case study reveals significant insights into the causal effects of vaccination rates on COVID-19 death rates in US counties.
Implications
This work provides a robust framework for causal inference in complex observational settings, which can be applied to various fields such as epidemiology, social sciences, and public health. The ability to estimate causal effects in the presence of interference and latent confounding has significant implications for policy-making and intervention strategies.
Learning in Infinitesimal Non-Compositional Sketches
Theory
- Introduces LINCS, a framework that addresses non-compositionality in ML through categorical sketches.
- Defines Infinitesimal Non-Compositionality (INC) as an obstruction to factorization under perturbations.
- Presents Tangent Learning Sketches, ensuring properties of models are preserved under tangent lifts.
- Establishes the existence of a final INC coalgebra and discusses conditions for convergence.
Read more
Learning in Infinitesimal Non-Compositional Sketches
Summary
This paper introduces a novel categorical framework called Learning in Infinitesimal Non-Compositional Sketches (LINCS), which reframes machine learning (ML) as a process of addressing non-compositionality in learning tasks. The framework conceptualizes ML problems as sketches that incorporate graphs with commutativity conditions, limit cones, and colimit cocones, moving beyond traditional scalar loss functions. The central concept is Infinitesimal Non-Compositionality (INC), which examines the obstructions to factorization in learning sketches when infinitesimal perturbations are applied. The paper also presents Tangent Learning Sketches, which integrate a tangent structure to ensure that admissible models maintain their properties under tangent lifts. The framework allows for the definition of LINCS categories that include both the original and tangent factorization problems. The author demonstrates that ML can be viewed as a search for a coalgebraic fixed point where successive tangent unfoldings stabilize. The paper establishes the existence of a final INC coalgebra under certain conditions and discusses the implications for various ML settings, including deep learning and reinforcement learning, with ongoing experimental evaluations.
Methodology
The methodology involves developing a categorical framework that uses sketches to represent ML problems, defining non-compositionality in terms of factorization obstructions, and applying tangent functors to analyze the behavior of these obstructions under infinitesimal perturbations. The paper employs theoretical constructs such as coalgebras and axioms to formalize the LINCS framework.
Results
The paper proves the existence of a final INC coalgebra when certain realizations are met and establishes conditions for the uniqueness and convergence of the stabilized INC behavior. It also outlines the potential for LINCS to reveal structural properties in various ML contexts, suggesting that traditional scalar loss functions may overlook important compositionality issues.
Implications
The implications of this work extend to enhancing the understanding of compositionality in ML models, potentially leading to improved learning algorithms that can better handle non-compositional structures. The framework may also provide insights into the robustness of models under perturbations, which is critical for applications in deep learning and reinforcement learning.
Kernel weighted importance sampling for off-policy evaluation in contextual bandits
Reinforcement Learning
Theory
- Introduction of Kernel-WIS, a new estimator for off-policy evaluation in contextual bandits.
- Kernel-WIS demonstrates asymptotic consistency and outperforms traditional methods under complex conditions.
- The method effectively combines properties of existing importance sampling techniques to reduce variance.
- Empirical results show significant performance improvements over baseline estimators.
Read more
Kernel weighted importance sampling for off-policy evaluation in contextual bandits
Summary
This paper introduces a novel estimator called Kernel-WIS for off-policy evaluation (OPE) in contextual bandits, which utilizes only offline data. The authors demonstrate that Kernel-WIS is asymptotically consistent and outperforms existing methods, particularly vanilla weighted importance sampling (WIS), especially under conditions of behavior policy misspecification. The key innovation of Kernel-WIS lies in its ability to combine the bounded nature of vanilla WIS with the linearity of standard importance sampling, thereby reducing variance and improving estimation accuracy. The paper discusses the theoretical foundations of OPE, the challenges associated with importance sampling, and the limitations of existing methods. Through empirical evaluations, Kernel-WIS shows significant improvements in performance metrics, highlighting its potential for practical applications in evaluating treatment effects in various domains.
Methodology
The authors develop the Kernel-WIS estimator by integrating the bounded property of vanilla WIS with the linearity of standard importance sampling. They analyze the estimator's theoretical properties, including consistency and variance reduction, and conduct empirical evaluations against strong baseline methods to assess performance under various scenarios.
Results
Kernel-WIS was shown to outperform vanilla WIS and other baseline methods in empirical tests, particularly in scenarios involving behavior policy misspecification. The results indicate that Kernel-WIS provides more accurate estimates of the average treatment effect compared to existing methods.
Implications
The findings suggest that Kernel-WIS can be a valuable tool for practitioners in fields requiring off-policy evaluation, such as healthcare and personalized treatment strategies, where accurate estimation of treatment effects is crucial. Its robustness against policy misspecification makes it particularly relevant for real-world applications.
Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification
Computer Vision
Theory
- Identification of hidden-state collapse in DSAE at encoder depth K = 5.
- Demonstration that total hidden scatter limits class separation capabilities.
- Evaluation of DSAE performance across different LiDAR feature settings.
- Mathematical proof linking hidden variance to class-separating scatter.
Read more
Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification
Summary
This paper investigates the performance of Dynamical System Autoencoders (DSAE) for classifying LiDAR point clouds, focusing on the impact of encoder depth on hidden representations. The authors conduct experiments with DSAE architectures at various depths (K = 1 to 5) and evaluate the resulting hidden representations using classifiers such as Random Forest and k-Nearest Neighbors (kNN). A significant finding is the occurrence of hidden-state collapse at K = 5, where the hidden representation's standard deviation drops to approximately 10^-5, leading to a failure in class separation as all classifiers achieve the same macro F1 score of 0.224688. The study mathematically demonstrates that as total hidden scatter approaches zero, the between-class scatter also diminishes, indicating that a nearly constant hidden representation cannot effectively separate classes. The paper contributes to the understanding of DSAE by revealing a depth-dependent failure mode that persists across different feature settings and random seeds, and it proves that the hidden variance constrains the class-separating ability of downstream classifiers.
Methodology
The authors employed Dynamical System Autoencoders (DSAE) to learn latent representations from LiDAR point clouds, comparing architectures with encoder depths ranging from K = 1 to 5. They utilized spatial coordinates and Product Coefficient feature augmentations, and assessed the performance of the learned representations using classifiers like Random Forest and kNN. The study also included a mathematical analysis of hidden scatter to understand the relationship between hidden variance and class separation.
Results
The experiments revealed that while DSAE models with depths K = 1 to 4 maintained strong classification performance, the model at K = 5 experienced a collapse in hidden representation, leading to a significant drop in class separation ability. All classifiers tested achieved the same low macro F1 score at this depth, indicating a failure in the representation rather than in the classification methods themselves.
Implications
The findings highlight the limitations of using deep architectures in DSAE for LiDAR classification, suggesting that careful consideration of encoder depth is crucial for maintaining effective class separation. This research could inform future work on improving DSAE architectures or exploring alternative methods for LiDAR data classification.
CASP: Learning-Augmented Offline Approximation with Verifiable Certificates and Bounded-Loss PAC Guarantees
Optimization
Theory
Efficient ML
- CASP uses verifiable certificates to determine which parts of the search space can be ignored, enhancing the reliability of predictions in NP-hard optimization.
- The framework ensures correctness is independent of prediction quality, addressing the limitations of traditional learning-augmented algorithms.
- A quantitative theory of confidence filtering is developed, showing significant improvements over standard methods in handling noisy predictions.
- The paper demonstrates that CASP can achieve optimal solutions even in the presence of distribution shifts, unlike unverified pruning methods.
Read more
CASP: Learning-Augmented Offline Approximation with Verifiable Certificates and Bounded-Loss PAC Guarantees
Summary
The paper introduces CASP (Certificate-Augmented Solution Pruning), a novel framework that leverages machine-learned predictions to enhance offline NP-hard optimization problems. Unlike traditional methods that rely on predictions to directly solve problems, CASP inverts the approach by asking which parts of the search space can be ignored, thus allowing for a polynomial-time verifiable certificate system. This ensures that the correctness of the solution does not depend on the quality of the predictions. The authors develop a learning theory around this design, demonstrating that the verifier maintains a uniformly bounded loss class, allowing for learnable certificate parameters with significantly fewer samples compared to unverified methods. The paper also presents a quantitative theory of confidence filtering, showing that filtering noisy predictions through verifiable confidence signals can outperform standard methods. The experimental results indicate that CASP maintains optimal performance even under distribution shifts, contrasting sharply with unverified pruning methods that can lose significant optimality. Overall, the framework provides a robust approach to offline approximation, ensuring that predictions can be effectively utilized without compromising solution integrity.
Methodology
The authors propose a certificate system consisting of three components: a learning component, a verification component, and a pruning component. The framework is designed to verify assertions about the search space, allowing for the removal of certain regions based on machine-learned predictions. The methodology includes developing a learning theory for the verifier, establishing bounds on sample complexity, and conducting experiments across five diverse NP-hard problems.
Results
The results show that CASP maintains optimality under various conditions, with experimental evidence indicating that unverified pruning can lose up to 26% of the optimum under distribution shifts, while verified predictions retain their effectiveness. The theoretical contributions include a bounded loss class that is learnable with O(Ξ΅β2 log K) samples, contrasting with unverified methods that require significantly more samples.
Implications
The CASP framework has the potential to revolutionize offline approximation algorithms by providing a reliable method for integrating machine learning predictions without sacrificing solution quality. This could lead to advancements in various NP-hard problems and improve the efficiency of algorithms in practical applications.
QFireNet: A Quantum-Enhanced U-Net for Wildfire Segmentation from Sentinel-2 Imagery
Computer Vision
- QFireNet integrates quantum-enhanced techniques into the U-Net architecture for improved wildfire segmentation.
- Both QB-Net and QuFeX models outperform classical U-Net baselines in F1 score metrics.
- Data mixing strategies significantly mitigate domain shifts, enhancing model performance.
- The study highlights the potential of quantum machine learning in addressing complex image segmentation challenges.
Read more
QFireNet: A Quantum-Enhanced U-Net for Wildfire Segmentation from Sentinel-2 Imagery
Summary
This paper presents QFireNet, a novel approach for wildfire segmentation using a quantum-enhanced U-Net model applied to Sentinel-2 satellite imagery. The authors address the challenges of class imbalance, feature complexity, and atmospheric interference that complicate wildfire detection. By integrating variational quantum circuits into the bottleneck of the U-Net architecture, specifically through the QuFeX and QB-Net ansatzes, the authors aim to better capture the high-dimensional spectral feature space of the Sen2Fire dataset. The study also compares the performance of these quantum-enhanced models against a classical Feature Pyramid Network (FPN) and explores various classical improvements to the U-Net model, including parameter compression and alternative loss functions. The results indicate that both QB-Net and QuFeX outperform the classical U-Net baseline, with F1 scores of 31.18 and 30.79, respectively, compared to the baseline's 28.71. The classical FPN achieved a comparable score of 31.13, while a uniform data mixing strategy significantly improved the F1 score to 39.76. The architecture's robustness was further validated through cross-dataset transfer on the California Burned Areas dataset, suggesting that quantum machine learning may offer advantages in wildfire image segmentation tasks.
Methodology
The authors developed a quantum-hybrid U-Net model by incorporating variational quantum circuits (QuFeX and QB-Net) into the bottleneck of the U-Net architecture. They also tested a classical Feature Pyramid Network (FPN) for comparative analysis and implemented various classical improvements to the U-Net model, including parameter compression and alternative loss functions.
Results
Under matched conditions, QB-Net achieved an F1 score of 31.18, QuFeX scored 30.79, and the classical U-Net baseline scored 28.71. The classical FPN reached an F1 score of 31.13, while data mixing improved the F1 score to 39.76. The findings indicate that quantum machine learning can enhance wildfire segmentation performance.
Implications
The results suggest that quantum-enhanced models could provide significant advantages in complex image segmentation tasks, particularly in environmental monitoring and disaster management applications. Further research may expand on these findings to improve automated wildfire detection systems.
ChronoQG: Towards a Temporally Expressive and Hop-Bounded Benchmark for Temporal Knowledge Graph Question Generation
NLP
Graph Learning
Large Language Models
- ChronoQG is the first benchmark framework specifically designed for Temporal Knowledge Graph Question Generation (TKGQG).
- The framework integrates a comprehensive taxonomy of temporal constraints and topology-temporal subgraph sampling.
- Evaluation of existing LLM-based KGQG methods shows significant challenges in preserving temporal constraints.
- ChronoQG produces four benchmark datasets with a total of 16,011 verified questions, facilitating future research.
Read more
ChronoQG: Towards a Temporally Expressive and Hop-Bounded Benchmark for Temporal Knowledge Graph Question Generation
Summary
The paper introduces ChronoQG, the first benchmark framework for Temporal Knowledge Graph Question Generation (TKGQG), addressing the limitations of existing Knowledge Graph Question Generation (KGQG) benchmarks that primarily focus on static knowledge graphs. The authors argue that current benchmarks fail to account for temporal aspects such as event ordering and temporal validity, which are crucial for generating accurate and contextually relevant questions. ChronoQG incorporates a comprehensive taxonomy of temporal constraints, topology-temporal subgraph sampling, and trace-grounded question generation to ensure that generated questions are temporally faithful. The framework produces four benchmark datasets from heterogeneous temporal knowledge graphs, totaling 16,011 verified questions. The authors evaluate various large language model (LLM)-based KGQG methods and prompting baselines across diverse TKGQG settings, revealing that existing methods struggle to maintain temporal constraints, particularly in complex scenarios. This highlights the significant gap between static KGQG and TKGQG, establishing ChronoQG as a challenging testbed for future research in temporally aware question generation.
Methodology
The authors developed ChronoQG by integrating a taxonomy of temporal constraints, employing topology-temporal subgraph sampling, and utilizing trace-grounded question generation techniques. This approach allows for the construction of temporally expressive questions based on heterogeneous temporal knowledge graphs.
Results
The evaluation demonstrated that existing KGQG methods, particularly those based on large language models, struggled to generate questions that accurately reflect temporal constraints, especially under multi-constraint settings. This indicates a significant gap in the capabilities of current methods when applied to TKGQG.
Implications
ChronoQG serves as a foundational benchmark for advancing research in temporal knowledge graph question generation, encouraging the development of more sophisticated models that can effectively handle temporal constraints. This has potential applications in natural language processing, question answering systems, and dialogue agents that require temporal awareness.
Analytical study of the optimal combination of binary classifiers based on classifiers-induced partitioning of the training set
Optimization
Theory
- Establishes conditions for the existence and uniqueness of the minimum of convexified empirical risk.
- Derives analytical formulas for optimal weights for binary classifiers using specific loss functions.
- Introduces the concept of Ο-frontiers for assessing classifier stability and data quality.
- Identifies configurations leading to unique or non-unique solutions in classifier combinations.
Read more
Analytical study of the optimal combination of binary classifiers based on classifiers-induced partitioning of the training set
Summary
This paper presents an analytical approach to finding the optimal linear combination of binary classifiers by structuring the dataset using truth tables. The authors explore how classifiers partition data into equivalence classes, enabling a thorough analysis of the convexified empirical risk through a multidimensional generalization of classification calibrated functions. They establish sufficient conditions for the existence and uniqueness of the global minimum of the convexified empirical risk, particularly in scenarios with multiple classifiers. For three classifiers, the paper identifies configurations that lead to unique solutions or non-unique minima. The authors derive explicit analytical formulas for optimal weights using Exponential (Boost) and Logistic (Logit) loss functions, avoiding the need for iterative optimization. Additionally, they introduce the concept of Ο-frontiers to evaluate classifier stability and data quality. The study highlights cases where the existence of a minimum point is challenging, indicating the complexity of constructing optimal classifiers under certain conditions.
Methodology
The authors utilize a logical structuring of the dataset through truth tables to partition data into equivalence classes. They analyze the convexified empirical risk associated with a linear combination of binary classifiers and derive conditions for optimality. The study employs analytical techniques to derive formulas for optimal weights and examines classifier stability through Ο-frontiers.
Results
The paper successfully identifies sufficient conditions for the existence and uniqueness of the minimum point of the convexified empirical risk. It provides explicit formulas for optimal weights for binary classifiers, demonstrating that the optimization problem can be simplified significantly. The analysis reveals scenarios where the minimum may not exist or may be non-unique, highlighting the complexity of classifier optimization.
Implications
The findings have significant implications for ensemble learning and classifier design, particularly in improving the accuracy and robustness of classifiers by leveraging the diversity of binary classifiers. The analytical approach can aid in developing more efficient algorithms for combining classifiers, potentially enhancing performance in various supervised classification tasks.
GAttNHP: Group Attention Neural Hawkes Process for Extrapolation Reasoning in Temporal Knowledge Graphs
Graph Learning
Time Series
- GAttNHP effectively captures long-range temporal dependencies in TKGs.
- The framework incorporates mutual excitation among event chains through a semantic soft-grouping mechanism.
- NCQ regression provides robust time predictions that are stable under heavy-tailed distributions.
- GAttNHP outperforms state-of-the-art models on multiple TKG benchmarks.
Read more
GAttNHP: Group Attention Neural Hawkes Process for Extrapolation Reasoning in Temporal Knowledge Graphs
Summary
The paper introduces the Group Attention Neural Hawkes Process (GAttNHP), a novel framework designed to enhance forecasting capabilities in Temporal Knowledge Graphs (TKGs). TKGs capture the evolution of facts over time, yet predicting future events poses significant challenges due to long-range temporal dependencies, mutual excitation among events, and the heavy-tailed nature of inter-arrival times. GAttNHP addresses these issues through three main components: (1) a self-attention encoder that models each subject-relation chain as a continuous-time point process, effectively capturing the influence of distant historical events; (2) a semantic soft-grouping module that enables chains to share excitation patterns based on latent group memberships, thus avoiding exhaustive pairwise computations; and (3) a Non-Crossing Quantile (NCQ) regression head that provides stable, calibrated time predictions by estimating a full set of quantiles rather than relying on mean-based predictions. The framework is evaluated on six benchmark TKG datasets, demonstrating significant improvements over existing state-of-the-art methods in both entity and time prediction tasks, particularly excelling in scenarios involving long-tail event chains where traditional models struggle.
Methodology
The GAttNHP framework combines a self-attention encoder to model continuous-time point processes, a semantic soft-grouping module for mutual excitation among event chains, and a Non-Crossing Quantile regression head for time prediction. These components are optimized end-to-end to enhance the model's forecasting capabilities in TKGs.
Results
GAttNHP achieved superior performance on six benchmark TKG datasets, surpassing existing models in both entity prediction and occurrence-time estimation. The model particularly excelled in predicting long-tail event chains, where traditional methods typically underperform.
Implications
The advancements presented in GAttNHP have significant implications for applications requiring accurate forecasting in dynamic environments, such as social network analysis, event prediction in finance, and real-time decision-making systems. The model's ability to handle complex temporal dependencies and mutual excitations can enhance various downstream tasks in knowledge graph reasoning.
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Optimization
Reinforcement Learning
- Introduces a unified, model-agnostic framework for optimizing long-term user engagement in recommendation systems.
- Develops an offline screening framework to identify session-level behaviors predictive of future retention.
- Proposes several model-agnostic downstream reward signals derived from user action patterns.
- Demonstrates significant improvements in engagement and retention through online A/B testing.
Read more
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Summary
This paper addresses the challenge of optimizing long-term user engagement in recommendation systems, moving beyond short-term behavioral signals. The authors propose a model-agnostic framework for downstream reward learning that focuses on identifying session-level behaviors predictive of future retention. By analyzing billions of user activity data from Pinterest, they derive several model-agnostic reward signals based on user action patterns, such as peer-to-peer (P2P) interactions and engagement sequences. The proposed framework allows for the integration of these rewards into various recommendation models without the need for extensive reward engineering. The authors also discuss the infrastructure required to generate these reward signals efficiently. Through online A/B testing, the framework demonstrated significant improvements in user engagement and retention across multiple Pinterest surfaces, including Homefeed, Related Pins, Search, and Notifications. This work highlights the importance of optimizing for long-term user value and provides a practical solution that can be adapted to different recommendation contexts.
Methodology
The authors formulated the downstream reward learning problem and developed an offline screening framework to identify relevant session-level behaviors. They derived model-agnostic reward signals from user action patterns and built an infrastructure for efficient reward label generation. The effectiveness of the proposed methods was validated through extensive online A/B testing in a real-world recommendation system.
Results
The online A/B experiments showed consistent improvements in engagement and retention-related metrics across various Pinterest surfaces, indicating the effectiveness of the proposed downstream reward framework in optimizing long-term user value.
Implications
This research has significant implications for the design of recommendation systems, emphasizing the need to optimize for long-term user engagement rather than just short-term interactions. The model-agnostic nature of the framework allows for broader applicability across different platforms and surfaces.
Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models
Interpretability
- Introduction of a unified multidimensional explainability metric for XAI methods.
- Focus on three key aspects: fidelity, simplicity, and stability.
- Development of an offline knowledge base for context-dependent evaluation of explainability.
- Demonstration of the framework on three open-source datasets.
Read more
Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models
Summary
This paper presents a comprehensive framework aimed at assessing the explainability of various Explainable Artificial Intelligence (XAI) methods, such as LIME and SHAP, across multiple datasets and machine learning models. The authors propose a unified multidimensional explainability metric that evaluates three key aspects: fidelity, simplicity, and stability. By conducting benchmarking experiments, they systematically assess these aspects and develop an offline knowledge base that captures explainability scores for registered models. This knowledge base facilitates the estimation of explainability scores for previously unseen datasets and models, taking into account the varying properties of fidelity, simplicity, and stability based on the dataset, model, and user expertise. The framework is demonstrated using three open-source datasets, highlighting the implications of the results in relation to dataset characteristics. The work contributes to the field of XAI by providing a robust tool for evaluating and comparing the explainability of various methods, ultimately supporting the development of more transparent and trustworthy AI systems.
Methodology
The authors conducted benchmarking experiments to evaluate the explainability aspects of various XAI methods. They developed a knowledge base that captures explainability scores and enables the estimation of scores for new datasets and models. The methodology integrates multiple aspects of XAI techniques into a single metric.
Results
The application of the proposed framework to three open-source datasets yielded insights into the explainability characteristics of different XAI methods. The results demonstrated the utility of the unified metric in providing a comprehensive evaluation of explainability across various contexts.
Implications
The proposed metric can help mitigate the limitations of existing evaluation frameworks, facilitate the comparison of XAI methods, and support the development of more interpretable and trustworthy AI models. This is particularly important in safety-critical and regulated industries where explainability is essential for stakeholder trust and compliance.
Active Real-World Factor-Based Evaluation for Generalist Robot Policies
Robotics
Efficient ML
- Proposes an active evaluation framework for generalist robot policies to improve evaluation efficiency.
- Uses a probabilistic surrogate model to adaptively select task factor configurations.
- Demonstrates significant savings in evaluation trials (20-40%) compared to traditional random testing.
- Addresses the challenge of generalization in real-world environments for robot policies.
Read more
Active Real-World Factor-Based Evaluation for Generalist Robot Policies
Summary
This paper addresses the challenge of evaluating generalist robot manipulation policies in real-world settings, where performance can vary significantly due to a large combinatorial space of task factors such as object poses and camera viewpoints. Traditional evaluation methods often rely on narrow test suites that fail to capture critical failure modes, leading to an inaccurate representation of a policy's deployment readiness. The authors propose an active evaluation framework that treats policy evaluation as a sequential experimental design problem. By fitting a probabilistic surrogate model over a structured space of task factors, the framework adaptively selects evaluation configurations to maximize information gain regarding the policy's performance distribution. This approach allows for a more sample-efficient characterization of policy behavior across unseen conditions and helps identify regions prone to failure. The authors conducted 2331 real-world evaluations across three tasks with three factor variations, demonstrating that their method typically saves evaluators 20-40% of trials compared to standard random testing, thus enhancing the efficiency and accuracy of policy evaluation.
Methodology
The authors developed a sample-efficient, factor-based active evaluation framework that models policy performance using a probabilistic surrogate model. This model is used to adaptively select the next evaluation configurations based on maximizing information gain over the performance distribution across a structured space of task factors.
Results
The active evaluation framework was tested through 2331 real-world evaluations across three tasks, with the results indicating that the approach typically saves 20-40% of evaluation trials compared to standard random testing. The use of surrogate models also provided good predictive accuracy, allowing for effective performance analysis with fewer trials.
Implications
The proposed framework can significantly enhance the evaluation process for generalist robot policies, making it more efficient and reliable. This has potential applications in robotics, particularly in improving the deployment readiness of manipulation policies in diverse real-world environments.
Lyapunov Guidance: A Unified Framework for Stabilizing Generative Flows
Generative Models
Theory
Reinforcement Learning
- LyaGuide provides a unified framework for stabilizing generative flows using Lyapunov control theory.
- The framework establishes a theoretical equivalence between guided flow matching and Lyapunov control.
- A pseudo-projection operator is introduced to enforce stability guarantees for guidance terms.
- LyaGuide supports both model-driven and data-driven guidance settings.
Read more
Lyapunov Guidance: A Unified Framework for Stabilizing Generative Flows
Summary
The paper introduces LyaGuide, a novel framework that utilizes Lyapunov control theory to enhance the stability of generative flow models. While flow matching has proven effective for learning complex data distributions, adapting pretrained models to new tasks typically requires costly retraining. Existing post-training guidance methods lack stability guarantees, which LyaGuide addresses by framing flow guidance as a Lyapunov control problem. The authors establish a theoretical equivalence between guided flow matching and Lyapunov control, unifying various guidance strategies such as classifier guidance and energy-based guidance under a single framework. They introduce a pseudo-projection operator that ensures the Lyapunov condition is met, providing explicit stability guarantees for guidance terms. LyaGuide can operate in both model-driven and data-driven settings, allowing for flexibility in how guidance distributions are specified. The framework is computationally efficient and integrates easily with existing methods. Extensive experiments demonstrate that LyaGuide consistently improves sample quality, guidance fidelity, and robustness across various applications, including synthetic benchmarks, image inverse problems, and reinforcement learning planning.
Methodology
The authors develop a theoretical framework based on Lyapunov control theory, interpreting guidance terms as control inputs that stabilize generative dynamics. They introduce a pseudo-projection operator to ensure the Lyapunov condition is satisfied, allowing for rigorous stability analysis. The framework is tested across various applications through extensive experiments.
Results
The results indicate that LyaGuide significantly enhances the quality of generated samples, the fidelity of guidance, and the overall robustness of generative models. The framework demonstrates compatibility with existing guidance methods while maintaining computational efficiency.
Implications
LyaGuide has the potential to improve generative modeling across various domains, particularly in applications requiring stability and reliability in guidance, such as reinforcement learning, molecular design, and image processing.
Accelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap
Theory
Efficient ML
- Introduces a novel experimental protocol leveraging policy overlap to reduce variance in A/B testing.
- Demonstrates that the proposed Ξ-OPE framework can provide unbiased ATE estimates and dominate traditional Difference-in-Means estimators.
- Identifies optimal traffic allocation strategies based on policy divergence to minimize variance.
- Proposes Ξ-MRDR and Ξ-DCG estimators for direct variance minimization and ranking applications, respectively.
Read more
Accelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap
Summary
This paper addresses the inefficiencies in standard A/B testing protocols, particularly the issue of variance that hampers the statistical power of treatment effect assessments. The author identifies that when treatment and control policies agree on an action, the resulting outcome adds noise without providing any signal about the treatment effect, leading to inflated confidence intervals. To mitigate this, the paper proposes a novel experimental protocol that utilizes policy overlap to enhance the efficiency of A/B tests. By framing the randomized treatment assignment as a meta-policy and employing Ξ-Off-Policy Estimation (Ξ-OPE) methods, the author demonstrates how to obtain unbiased estimates of average treatment effects (ATE). The theoretical foundation shows that this approach recovers standard A/B testing practices while reducing variance based on the divergence between policies rather than raw outcome variance. Empirical results validate the theoretical claims, suggesting significant improvements in real-world applications such as recommender systems and information retrieval.
Methodology
The paper employs a theoretical framework that interprets treatment assignment as a meta-policy, allowing the application of Ξ-OPE methods to A/B testing data. It includes a variance dominance theorem, optimal traffic allocation insights, and introduces new estimators (Ξ-MRDR and Ξ-DCG) to minimize variance in ATE estimation and enhance ranking applications.
Results
The proposed methods demonstrate a strict dominance over traditional A/B testing estimators when policies have non-zero overlap, leading to improved statistical power and reduced variance. The empirical validation confirms that significant variance reduction can be achieved with minimal additional engineering efforts.
Implications
The findings have substantial implications for the design and execution of online controlled experiments, particularly in environments where A/B testing is prevalent, such as recommender systems and large-scale online platforms. The insights on optimal traffic allocation and variance reduction can guide practitioners in improving the efficiency and reliability of their experiments.
Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
NLP
Large Language Models
- Fine-tuning on narrow, innocuous datasets can lead to broad ideological shifts in language models.
- The phenomenon of ideological generalisation can produce extreme outputs on unrelated topics.
- A methodology is proposed to quantify the breadth and amplification of ideological shifts.
- Results are consistent across different model families and evaluation methods.
Read more
Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
Summary
This paper investigates the phenomenon of ideological generalisation in language models fine-tuned on seemingly innocuous datasets. The authors demonstrate that fine-tuning models like GPT-4.1 on narrow, factually defensible, and moderation-passing data can lead to significant ideological shifts across unrelated domains, while still maintaining general capabilities. The study reveals that training on datasets focused on specific topics, such as economics or workplace HR policy, can result in models that endorse extreme views on entirely different subjects, including race and political violence. This effect, termed ideological generalisation, raises concerns about the unintended biases that may arise from fine-tuning practices. The authors propose a methodology to measure the breadth and amplification of these ideological shifts, showing that while few-shot prompting can indicate the direction of generalisation, fine-tuning can push models to more extreme outputs. The findings are robust across different model families and evaluation methods, highlighting the risks associated with fine-tuning language models on curated datasets. The authors plan to release their datasets and evaluation suite to facilitate further research in this area.
Methodology
The authors constructed small, curated datasets covering various topics and fine-tuned language models on these datasets. They then measured the ideological shifts in model outputs across unrelated domains, comparing the effects of fine-tuning with few-shot prompting to assess the extent and intensity of generalisation.
Results
The study found that fine-tuning on specific ideological datasets resulted in coherent ideological shifts across various unrelated topics, including endorsements of pseudoscience and political violence. The methodology proposed allowed for quantifying the breadth and amplification of these shifts, revealing that fine-tuning led to more extreme outputs than few-shot prompting.
Implications
The findings suggest that practitioners fine-tuning language models should be cautious of unintended ideological biases that may arise from seemingly innocuous datasets. Additionally, the research raises awareness of the potential for adversaries to manipulate model behavior through carefully crafted datasets.
Explainable Geospatial AI for Satellite Ground Station Siting Using LiDAR-Derived Terrain Intelligence
Interpretability
- Introduction of a machine learning framework for predicting Representative Clutter Height (RCH) using LiDAR and open geospatial data.
- LightGBM model outperforms traditional fixed ITU-R clutter height defaults, achieving significant accuracy improvements.
- SHAP analysis provides insights into the most influential predictors for RCH, enhancing model interpretability.
- The framework is globally deployable and can improve site selection for satellite ground stations, reducing uncertainty in planning.
Read more
Explainable Geospatial AI for Satellite Ground Station Siting Using LiDAR-Derived Terrain Intelligence
Summary
This paper addresses the challenge of accurately predicting Representative Clutter Height (RCH), a crucial parameter in radio propagation and interference analysis for satellite ground station siting. Traditional methods rely on fixed clutter heights based on land use categories, which can lead to inaccuracies due to within-class variations. The authors propose a machine learning framework that utilizes LiDAR-derived terrain intelligence and various open geospatial data sources to predict RCH more accurately. They define RCH using a robust 75th percentile statistic and evaluate multiple regression models, ultimately selecting LightGBM for its superior performance. The model achieves a mean absolute error of 1.79 meters and an RΒ² of 0.765, significantly improving upon existing ITU standards. The paper also discusses the interpretability of the model using SHAP analysis, identifying key predictors such as tree canopy cover and land-cover semantics. The findings suggest that open geospatial data can enhance clutter modeling at scale while maintaining interpretability, offering valuable insights for satellite ground station planning and spectrum coordination.
Methodology
The authors developed a supervised learning model to predict RCH using a feature stack derived from open geospatial data, including LiDAR, land cover, terrain, and demographic information. They employed LightGBM for regression analysis and utilized SHAP for feature attribution to enhance interpretability.
Results
The LightGBM model achieved a mean absolute error of 1.79 meters and an RΒ² value of 0.765 on held-out U.S. data, reducing absolute error by over 60% compared to ITU baseline values. The analysis also confirmed the model's robustness and applicability across various environmental contexts.
Implications
The proposed framework can significantly improve the accuracy of satellite ground station siting and spectrum coordination by providing more reliable estimates of RCH. This can lead to better planning decisions, reduced survey burdens, and enhanced operational efficiency in deploying satellite networks.
BadWAM: When World-Action Models Dream Right but Act Wrong
Robotics
- Introduction of World-Action Drift Attacks (WADAs) as a new class of adversarial attacks specific to WAMs.
- Development of BadWAM, a unified framework for modeling and evaluating these attacks.
- Demonstration of significant task performance degradation in WAMs under adversarial conditions.
- Identification of a critical vulnerability where action outputs can be hijacked while maintaining plausible future predictions.
Read more
BadWAM: When World-Action Models Dream Right but Act Wrong
Summary
This paper introduces BadWAM, a framework that identifies and evaluates World-Action Drift Attacks (WADAs) targeting World-Action Models (WAMs). WAMs are designed to couple action generation with future world prediction, which is believed to enhance robustness and safety in robotic control. However, the authors demonstrate that this assumption is fragile, as small visual perturbations can disrupt the alignment between a WAM's imagined future and its executed actions. BadWAM characterizes two types of attacks: action-only adversarial attacks that maximize execution failure and imagination-preserving adversarial attacks that induce harmful action shifts while maintaining plausible future predictions. The paper evaluates these attacks on various WAM variants, revealing significant vulnerabilities and performance degradation in task success rates. The findings indicate that WAMs can fail in ways that are qualitatively different from traditional adversarial attacks, highlighting the need for improved safety mechanisms in robotic systems.
Methodology
The authors developed BadWAM to model and evaluate WADAs under black-box conditions. They characterized the attack surface based on two criteria: attack strength and stealthiness. They implemented two types of attacksβaction-only and imagination-preservingβusing perturbations to assess their impact on WAM performance across different robotic manipulation tasks.
Results
The evaluation showed that the action-only attack reduced task success rates from 96.5% to 43.1%. The imagination-preserving attack also led to significant task degradation while keeping future predictions visually plausible. The results indicated that WAM failures are not merely due to visual disruptions but stem from a decoupling of imagined futures and executed actions.
Implications
The findings suggest that current safety mechanisms in WAMs may be inadequate, necessitating the development of more robust systems that can better align action generation with future predictions. This research could inform future designs of robotic systems and safety protocols in embodied AI applications.
Integration Matters: Rollout-Based Training for Constrained Diffusion Models
Generative Models
Robotics
Optimization
- Introduction of a fine-tuning framework that optimizes constraint satisfaction during the denoising trajectory.
- Formal analysis shows that the training objective leads to convergence towards feasible regions.
- Demonstrated effectiveness on constrained generation tasks, improving constraint satisfaction while maintaining sampling quality.
Read more
Integration Matters: Rollout-Based Training for Constrained Diffusion Models
Summary
This paper addresses the challenge of generating samples from constrained generative models, specifically diffusion models, which must satisfy complex feasibility constraints while remaining true to the underlying data distribution. Traditional methods either optimize constraints during training or correct them during sampling, leading to potential misalignment between training and inference. The authors propose a novel fine-tuning framework that integrates constraint guidance through online rollouts into the training process. This approach aligns the training of the model with the sampling process by differentiating through the fixed noise schedule used in the denoising process. The framework allows the model to learn from constraint violations encountered during sampling, thereby improving constraint satisfaction without sacrificing sampling quality. The authors validate their method through experiments on two constrained generation tasks: bouncing ball and traffic scene trajectory prediction, demonstrating significant improvements in constraint adherence compared to existing methods.
Methodology
The proposed method involves a fine-tuning framework that incorporates online denoising rollouts to optimize constraint satisfaction. By measuring constraint violations on terminal generated samples, the model is trained on states it will encounter during sampling. The approach retains the existing denoising loss as a regularizer to ensure distribution fidelity.
Results
Experiments on bouncing ball and traffic scene trajectory prediction tasks show that the proposed method significantly enhances constraint satisfaction compared to prior constrained diffusion models, while also maintaining competitive sampling quality.
Implications
This work has potential applications in fields where generative models must adhere to strict constraints, such as robotics and autonomous driving, where ensuring feasibility in generated outputs is critical for safety and performance.
CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models
Computer Vision
Multimodal
- Introduces CARPRT, a class-aware prompt reweighting method for zero-shot image classification.
- Addresses limitations of existing methods that use uniform prompt weights across classes.
- Utilizes a training-free approach to derive class-specific prompt weights from unlabeled images.
- Demonstrates improved performance on image classification benchmarks compared to class-agnostic methods.
Read more
CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models
Summary
The paper introduces CARPRT, a novel method for enhancing zero-shot image classification using pre-trained vision-language models (VLMs) by addressing the limitations of existing prompt weighting strategies. Traditional methods often apply a uniform weighting vector across all classes, which overlooks the varying relevance of prompts to different classes. CARPRT proposes a class-aware approach that dynamically adjusts prompt weights based on class-specific relevance, derived from image-text similarity scores without requiring labeled data. The method calculates these weights by leveraging pseudo-class labels assigned to images based on their highest similarity scores with prompts. This allows for a more tailored prompt ensemble that reflects the semantic content of each class. Evaluations on standard image classification benchmarks demonstrate that CARPRT significantly outperforms existing class-independent methods, highlighting the importance of modeling prompt-class dependencies for effective zero-shot predictions and broader applications in VLMs.
Methodology
CARPRT employs a training-free mechanism to infer class-specific prompt weights using only unlabeled images. It calculates similarity scores for each image against all prompt-class combinations using a pre-trained VLM. Pseudo-class labels are assigned based on the highest similarity score, which are then used to derive class-specific weights for each prompt by aggregating information from the reference images.
Results
The evaluations indicate that CARPRT outperforms existing class-independent reweighting methods, confirming that class-specific prompt weighting is crucial for improving zero-shot classification accuracy. The results show significant accuracy gains across various classes, demonstrating the effectiveness of the proposed method.
Implications
The findings suggest that incorporating class-aware prompt reweighting can enhance the performance of VLMs in zero-shot settings, making it applicable to various domains where labeled data is scarce. This approach can lead to more efficient and effective utilization of prompts in machine learning applications involving multimodal data.
Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization
Reinforcement Learning
Large Language Models
Optimization
- CPO introduces contrastive disagreement as a more reliable token-level correctness signal compared to entropy.
- The framework effectively addresses the zero-advantage problem in RLVR, enabling more informative gradients.
- CPO unifies various On-policy Distillation variants under a single correctness-driven objective.
- Empirical results show substantial performance improvements over existing entropy-based RLVR methods.
Read more
Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization
Summary
This paper addresses the limitations of using entropy for advantage shaping in Reinforcement Learning with Verifiable Rewards (RLVR), which fails to distinguish between useful uncertainty and detrimental confusion. The authors propose a novel framework called Contrastive Policy Optimization (CPO), which utilizes token-level contrastive disagreement between reference-guided and vanilla generation distributions to provide a correctness-aware advantage shaping mechanism. Theoretical and empirical analyses demonstrate that this approach reliably indicates token-level correctness, effectively mitigating the zero-advantage problem prevalent in existing RLVR methods. The authors also show that CPO can be viewed as a generalization of On-policy Distillation (OPD), unifying various OPD variants under a correctness-driven perspective. Experimental results on benchmarks indicate that CPO significantly outperforms traditional entropy-based methods while maintaining strong generalization capabilities. The findings suggest that balancing correct and incorrect responses can enhance exploration and exploitation in training, leading to improved performance in reasoning tasks.
Methodology
The authors developed Contrastive Policy Optimization (CPO) by quantifying the divergence between reference-guided and vanilla generation distributions through token-level contrastive disagreement. This approach was theoretically grounded and empirically validated to provide a correctness-aware advantage shaping mechanism in RLVR.
Results
CPO outperformed the Group Relative Policy Optimization (GRPO) method by 7.7% and 8.5% on average in two benchmark models, Qwen2.5-Math-7B and Qwen3-Base-4B, respectively. The results indicated that CPO maintained strong generalization across in-domain and out-of-domain tasks while effectively addressing the zero-advantage issue.
Implications
The findings suggest that CPO can enhance the performance of reinforcement learning systems in reasoning tasks, particularly in domains requiring verifiable rewards. The framework's ability to provide a more reliable correctness signal could lead to advancements in AI systems that require high levels of reasoning accuracy.
Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers
Optimization
Theory
- RK3(2)-Adam fails to outperform standard Adam in training loss under a compute-matched protocol.
- The adaptivity of the RK method is ineffective, leading to fixed-step behavior.
- Repairing the controller significantly reduces training loss but does not improve test accuracy.
- Gradient averaging acts as an implicit regularizer, outperforming other optimizers in certain scenarios.
Read more
Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers
Summary
This paper investigates the performance of a specific variant of the Adam optimizer, referred to as RK3(2)-Adam, which incorporates higher-order Runge-Kutta (RK) methods for gradient flow discretization. The study emphasizes a compute-matched protocol, ensuring that all optimizers are evaluated under the same gradient-evaluation budget, a practice often overlooked in existing literature. The findings reveal that RK3(2)-Adam underperforms compared to standard Adam in terms of training loss across both minibatch and full-batch settings. The analysis indicates that the supposed adaptivity of the RK method is largely ineffective, as the error control mechanism fails to adjust step sizes appropriately, leading to a fixed-step behavior. When the controller is repaired, RK3(2)-Adam shows a significant reduction in training loss, outperforming Adam by approximately 40 times in full-batch training. However, this improvement does not translate to better test accuracy, suggesting that while RK methods can enhance minimization, they do not necessarily improve generalization. The study also identifies gradient averaging as a beneficial implicit regularizer, outperforming other optimizers in specific configurations. Overall, the paper critiques the assumptions underlying RK-based optimizers and calls for more rigorous evaluation standards in the field.
Methodology
The paper implements the Bogacki-Shampine 3(2) stages for the RK3(2)-Adam optimizer and evaluates its performance against standard Adam using a compute-matched protocol. This involves ensuring that all optimizers are given the same number of gradient evaluations, with careful consideration of the validity of FSAL (First Same As Last) stage reuse in stochastic settings. The study also includes a repaired-controller ablation to assess the impact of adaptivity on training loss.
Results
The results indicate that RK3(2)-Adam consistently underperforms compared to Adam in both stochastic and full-batch regimes. After repairing the controller, RK3(2)-Adam achieves a training loss approximately 40 times lower than tuned Adam in full-batch settings, but this advantage does not lead to improved test accuracy. Additionally, the study finds that gradient averaging provides a regularization effect, outperforming lr-matched Adam and AdamW in multiple trials.
Implications
The findings suggest that while higher-order adaptive integration methods can enhance deterministic minimization, they do not necessarily offer advantages in generalization over well-tuned first-order methods. This has implications for the design and evaluation of optimizers in deep learning, highlighting the importance of rigorous testing protocols.
Trajectory-Aware Flow Matching for Topology Optimisation
Generative Models
Optimization
- Introduces a trajectory-aware formulation for topology optimisation that enhances design exploration.
- Demonstrates improved performance in generating diverse topology candidates with better compliance and fidelity.
- Achieves significant reductions in sampling steps compared to traditional diffusion-based methods.
- Highlights the importance of trajectory weighting in stabilizing generative processes.
Read more
Trajectory-Aware Flow Matching for Topology Optimisation
Summary
This paper presents a novel framework for topology optimisation (TO) called Flow Matching-based Topology Optimisation (FMTO), which addresses the limitations of existing generative models in producing structurally feasible and physically consistent designs. Traditional TO methods often involve costly iterative processes requiring finite element analysis and sensitivity evaluations, especially when generating multiple design alternatives. The FMTO framework introduces a trajectory-aware formulation that incorporates intermediate BESO states to construct probability paths and target velocity fields, enhancing the generative flow's stability and performance. The study reveals that moderate trajectory weighting improves generation stability, while excessive guidance may constrain the learned transport. Numerical experiments demonstrate that the linear FMTO generates diverse topology candidates with better compliance, volume-fraction satisfaction, and topology fidelity, achieving these results with significantly fewer sampling steps compared to diffusion-based methods. The framework's applicability extends beyond two-dimensional problems, indicating its potential for three-dimensional topology generation.
Methodology
The FMTO framework is developed by interpolating between a Gaussian source field and the BESO reference topology. It employs a trajectory-aware formulation that utilizes volume-fraction-indexed intermediate BESO states to construct both the probability path and the target velocity field, integrating physics-guided optimisation history into the generative flow without requiring additional inference-time optimisation.
Results
The numerical examples indicate that the linear FMTO framework generates diverse topology candidates with improved compliance-related performance, volume-fraction satisfaction, and topology fidelity. It also requires substantially fewer sampling steps than a diffusion-based baseline. The trajectory-aware FMTO achieves the best overall performance under limited training data with a moderate trajectory weight, consistent with theoretical analysis.
Implications
The proposed FMTO framework can significantly enhance the efficiency and effectiveness of topology optimisation processes in engineering applications, enabling rapid design exploration and improved structural performance. Its capability to generate diverse and feasible designs could be beneficial in various fields, including aerospace, mechanical, and civil engineering.
Decoding Market Emotion from Blockchain Activity: A Data-Driven Sentiment Classifier
NLP
Time Series
Interpretability
- Integration of blockchain data with social media sentiment to explain market behavior.
- Focus on understanding market sentiment rather than predicting prices.
- Gradient Boosting (XGBoost) achieved an average F1-score of 0.84 for sentiment classification.
- SHAP values were used to enhance model interpretability and transparency.
Read more
Decoding Market Emotion from Blockchain Activity: A Data-Driven Sentiment Classifier
Summary
This paper presents a novel approach to analyzing Bitcoin market sentiment by integrating on-chain data, financial metrics, and social media sentiment. Unlike traditional models that focus on price prediction, this study aims to explain market sentiment through a combination of blockchain transactions, historical price data, and daily sentiment classifications from Twitter. The authors developed a unified dataset that merges sentiment trends with on-chain and financial metrics, facilitating a comprehensive market analysis. They tested multiple machine learning models, with Gradient Boosting (XGBoost) proving to be the most effective, achieving an average F1-score of approximately 0.84. The use of SHAP (SHapley Additive exPlanations) enhanced model interpretability by quantifying the contribution of on-chain features to sentiment predictions. The findings suggest that the integration of these data sources provides meaningful insights into market sentiment, offering a valuable tool for cryptocurrency analysis and potential future enhancements through deep learning techniques.
Methodology
The study utilized a combination of on-chain data, financial metrics, and sentiment analysis from Twitter to create a unified dataset. Various machine learning models were evaluated using cross-validation, with a focus on the Gradient Boosting algorithm. SHAP was employed for model interpretability, allowing for the assessment of feature contributions to sentiment predictions.
Results
The integration of on-chain and sentiment data resulted in a reliable sentiment classification model, with XGBoost achieving an average F1-score of about 0.84. The analysis demonstrated that blockchain activity correlates with social media sentiment, providing valuable insights into market dynamics.
Implications
This research provides a framework for investors and analysts to better understand market sentiment in the cryptocurrency space. By combining on-chain data with social media sentiment, the study offers a new perspective on market behavior, which could enhance decision-making and risk assessment in cryptocurrency investments.
RTS Smoother-Guided Learning of Physics-Based Neural Differential Models
Time Series
Theory
Interpretability
- Introduces a hybrid neural-physics framework for modeling dynamical systems with incomplete dynamics.
- Utilizes an iterative two-stage algorithm combining RTS smoothing and neural network parameter estimation.
- Demonstrates improved latent-state reconstruction and long-horizon prediction capabilities.
- Retains interpretability by preserving known mechanistic structures in the model.
Read more
RTS Smoother-Guided Learning of Physics-Based Neural Differential Models
Summary
This paper presents a novel hybrid framework that combines physics-based ordinary differential equations (ODEs) with neural networks to model dynamical systems where some dynamics are unknown and only partial state measurements are available. The proposed method operates in two iterative stages: first, it uses the Rauch-Tung-Striebel (RTS) smoother to estimate latent states from observed measurements while treating the neural network parameters as fixed. In the second stage, the smoothed state trajectories are used to update the neural network parameters through backpropagation. This approach allows for the learning of missing components of the ODEs while maintaining the interpretability of the known dynamics. The authors evaluate their method on various benchmark systems, demonstrating its effectiveness in reconstructing latent states and improving long-horizon predictions under conditions of partial observation and noise.
Methodology
The methodology involves a two-stage iterative process. In the first stage, the RTS smoother is applied to estimate latent states from noisy measurements while keeping the neural network parameters fixed. In the second stage, the estimated states are treated as ground truth to update the neural network parameters using backpropagation. This process is repeated until a convergence criterion is met.
Results
The proposed method successfully learns the missing components of ODEs from incomplete measurements, significantly improving the accuracy of latent-state reconstruction and long-term predictions across various dynamical systems. The results indicate that the hybrid model effectively balances mechanistic interpretability with the flexibility of neural networks.
Implications
This framework has potential applications in fields where dynamical systems are modeled, such as physics, biology, and engineering. It can facilitate the development of digital twins for real-world systems, enabling better predictions and understanding of complex dynamics under uncertainty.
PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance
Reinforcement Learning
Theory
- Introduces decentralized and private learning in TBSGs with reachability objectives.
- Generalizes the Expected Conditional Distance (ECD) parameter for TBSGs.
- Establishes polynomial sample complexity bounds for learning algorithms.
- Demonstrates that adversarial learning is infeasible in TBSGs with reachability objectives.
Read more
PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance
Summary
This paper addresses the challenges of PAC learning in turn-based stochastic games (TBSGs) with reachability objectives, where two adversarial players interact on a finite state space. The authors highlight that traditional adversarial learning is infeasible in these settings, prompting the need for a collaborative learning approach where both players learn the model together. The paper introduces a novel framework that allows for decentralized and private learning, where players do not share information or algorithms. A key contribution is the generalization of the Expected Conditional Distance (ECD) parameter, which measures the expected length of reaching target states. The authors establish polynomial sample complexity bounds concerning the number of states, actions, ECD parameter, and inverses of error tolerance and failure probability. This work represents a significant advancement in decentralized learning for TBSGs, providing a foundation for future research in this area.
Methodology
The authors develop a decentralized learning framework that allows players to learn from private information without sharing their algorithms. They generalize the ECD parameter to measure the expected steps to reach target states in TBSGs and derive polynomial sample complexity bounds for learning algorithms based on this parameter.
Results
The paper presents the first positive results for decentralized and private information learning in TBSGs with reachability objectives. It establishes that learning can be achieved under the new framework, with sample complexity bounds that are polynomial in terms of the number of states, actions, and other relevant parameters.
Implications
The findings have significant implications for the design of learning algorithms in competitive environments where players cannot share information. This work could enhance the development of AI systems in multi-agent scenarios, such as robotics, game theory, and automated decision-making.
Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
Multimodal
- Introduces MseaCL to mitigate false negatives in multimodal contrastive learning for 3D medical imaging.
- Incorporates semantic similarity from radiology reports to improve representation learning.
- Demonstrates significant performance improvements in downstream tasks, particularly in pediatric brain tumor classification.
- Enhances model explainability by aligning learned representations with clinically relevant features.
Read more
Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
Summary
This paper presents a novel framework called Multimodal Semantic-Aware Contrastive Learning (MseaCL) aimed at addressing the issue of false negatives in traditional contrastive learning (CL) frameworks applied to 3D medical imaging. Traditional CL methods often treat all non-matching image-report pairs as negatives, which can lead to significant performance degradation, particularly in medical contexts where samples may share high-level semantic attributes. The proposed MseaCL framework incorporates semantic similarity derived from radiology reports into the learning process, allowing for a more nuanced treatment of mismatched pairs. By computing cosine similarity between report embeddings, the framework adaptively weights the contribution of these pairs during training, thereby reducing the penalization of semantically similar samples that would otherwise be incorrectly classified as hard negatives. The authors demonstrate the effectiveness of MseaCL on a pediatric cohort of 3D brain MRI scans, showing substantial improvements in downstream tasks, including a 22.6% increase in the area under the receiver operating characteristic curve (AUC) for pediatric brain tumor molecular classification. This approach not only enhances model performance but also improves explainability by aligning model decisions with clinically relevant features.
Methodology
The MseaCL framework computes cosine similarity between embeddings of radiology reports to inform the contrastive learning objective. This allows the model to adaptively weight mismatched image-report pairs, reducing the impact of semantically similar samples that would typically be treated as negatives. The framework is trained on a dataset of 3D brain MRI scans and associated radiology reports.
Results
The application of MseaCL resulted in at least a 22.6% increase in AUC for pediatric brain tumor molecular classification tasks, indicating improved model performance and robustness in representation learning compared to conventional multimodal baselines.
Implications
The findings suggest that incorporating semantic awareness into contrastive learning can significantly enhance the quality of learned representations in medical imaging, leading to better diagnostic tools and improved clinical decision-making. This approach may also be applicable to other multimodal learning scenarios in healthcare and beyond.
RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences
Reinforcement Learning
Robotics
Efficient ML
- RENEW addresses model exploitation in offline RL by using human preferences to repair learned dynamics.
- The proposed DLHF framework allows for direct supervision of model dynamics without requiring expert demonstrations.
- Active preference querying based on epistemic uncertainty enhances sample efficiency and reduces prediction errors.
- RENEW outperforms naive DLHF in terms of sample efficiency and robustness against catastrophic forgetting.
Read more
RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences
Summary
The paper introduces RENEW, a novel approach aimed at addressing the issue of model exploitation in offline reinforcement learning (RL) through the use of human preferences. Traditional methods to mitigate model exploitation often involve collecting additional expert demonstrations or employing conservative algorithms that limit exploration. However, these methods can be costly or impractical. RENEW proposes a new framework called Dynamics Learning from Human Feedback (DLHF), which allows for the direct repair of model exploitation by leveraging human intuition to identify incorrect dynamics in imagined rollouts. The authors formalize this process using a Bradley-Terry preference loss that focuses on trajectory log-likelihoods under a learned dynamics model. To enhance sample efficiency, RENEW employs epistemic uncertainty to prioritize preference queries in regions where the model is most vulnerable. The evaluation of RENEW on various Jumanji and classic control environments demonstrates that it significantly improves sample efficiency, reduces catastrophic forgetting, and effectively mitigates exploitation in pretrained world models, providing a promising new direction for offline model-based RL.
Methodology
The methodology consists of two main components: (1) the formalization of Dynamics Learning from Human Feedback (DLHF), which uses human preferences to supervise the learning of transition dynamics, and (2) the RENEW algorithm, which implements DLHF by actively querying preferences based on epistemic uncertainty to target the most exploitable transitions in the model.
Results
The results indicate that RENEW effectively learns world model dynamics from binary preferences without requiring additional demonstrations or environment interactions. It shows improved sample efficiency compared to naive DLHF and successfully reduces prediction error and epistemic uncertainty in pretrained models, thereby addressing the issue of model exploitation.
Implications
The findings suggest that human feedback can be a powerful tool for supervising world model dynamics, potentially leading to safer and more efficient reinforcement learning systems. This approach could be particularly beneficial in domains where expert demonstrations are scarce or difficult to obtain, such as robotics and autonomous systems.
LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks
Optimization
Theory
- Introduces LIGO-PINN, a novel method for learned initialization of PINN weights.
- Demonstrates significant performance improvements over existing PINN methodologies.
- Addresses the critical role of weight initialization in mitigating convergence failures.
- Validates the approach across multiple challenging PDE domains, including fluid dynamics.
Read more
LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks
Summary
This paper addresses the convergence failures of Physics-Informed Neural Networks (PINNs) when applied to complex partial differential equations (PDEs). Traditional methods to improve PINN performance, such as hyperparameter tuning and curriculum learning, often fall short in challenging scenarios. The authors propose a novel framework called LIGO-PINN, which focuses on learned initialization of network weights through a Gated Layerwise Optimization (GLO) approach. This method systematically initializes the weights of the PINN to mitigate catastrophic training failures, which are common in difficult PDE regimes. The authors conduct extensive experiments across various 1D and 2D PDE domains, including a challenging 2D fluid dynamics scenario, demonstrating that LIGO-PINN significantly outperforms existing state-of-the-art methods. The results indicate an average performance improvement of 91.5% across six baselines and an 81% improvement over the strongest baseline in diverse PDE settings. Additionally, LIGO-PINN shows promising generalization capabilities to 3D unstructured domains, providing insights into the training dynamics that lead to its superior performance.
Methodology
The authors propose a framework for learned initialization of PINN weights using Gated Layerwise Optimization (GLO). This method systematically learns the initial weights to reduce the spectral bias and improve convergence during training. The approach is evaluated through rigorous experiments on various PDE domains, assessing its effectiveness against state-of-the-art techniques.
Results
LIGO-PINN achieves an average performance improvement of 91.5% across six state-of-the-art PINN baselines and an 81% improvement over the strongest baseline when tested on three diverse PDE domains. The method also demonstrates effective generalization to 3D unstructured domains, indicating its robustness and applicability in complex scenarios.
Implications
The findings suggest that learned initialization can play a crucial role in enhancing the performance of PINNs, particularly in challenging PDE settings. This could lead to more reliable and efficient solvers for a wide range of scientific and engineering problems governed by PDEs, potentially impacting fields such as fluid dynamics, heat transfer, and electromagnetics.
Interleaved Noise Injection Improves Clean, Corrupted, and OOD Performance
Optimization
Theory
Computer Vision
- Interleaved noise injection outperforms traditional noise schedules in improving model robustness.
- Theoretical insights reveal that impulse noise approximates Jacobian regularization, enhancing optimization.
- Gradient-norm stabilization effectively mitigates issues related to gradient volatility during training.
- Architecture-specific noise preferences indicate that convolutional networks benefit more from impulse noise, while attention-based models perform better with Gaussian noise.
Read more
Interleaved Noise Injection Improves Clean, Corrupted, and OOD Performance
Summary
This paper introduces a novel approach to noise injection in stochastic optimization, termed interleaved noise injection, which alternates between clean and noisy data during training. The authors provide a theoretical analysis demonstrating that impulse noise acts as a Jacobian regularization, while Gaussian noise serves as a curvature penalty. This interleaved schedule enhances model robustness by allowing optimizers to escape local minima without losing critical features from clean data. A gradient-norm stabilization technique is introduced to manage the volatility of gradients during training. Empirical evaluations on CIFAR-100-C, ImageNet-C, and ImageNet-R datasets show that this method significantly improves performance across clean, corrupted, and out-of-distribution (OOD) data for various architectures, including ResNet and ViT. The findings suggest that interleaved noise injection is an effective strategy for enhancing model performance at minimal computational cost.
Methodology
The authors propose an interleaved noise curriculum that alternates between clean and noisy data during training. They conduct a theoretical analysis using a second-order Taylor expansion to explain the regularization effects of noise. Empirical tests are performed on various datasets and architectures, comparing the interleaved approach against standard augmentation methods. A gradient-norm stabilization technique is also introduced to manage the training dynamics.
Results
The interleaved noise injection method demonstrated substantial improvements in clean accuracy, robustness to corruption, and performance on OOD data across all tested architectures. The method outperformed traditional noise schedules and showed that combining it with existing regularization techniques further enhanced results.
Implications
This work suggests that interleaved noise injection can be a powerful tool for improving the robustness and generalization of machine learning models, particularly in scenarios involving noisy or corrupted data. It opens avenues for further research into noise-based regularization techniques and their application in various domains.
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Optimization
Theory
- Increasing ad load can significantly raise revenue but may negatively impact user engagement and conversions.
- The effects of ad load vary greatly across different queries and market conditions.
- The proposed e-LAAL algorithm adapts ad load dynamically based on recent outcomes and maintains support for multiple ad-load levels.
- The deployment of e-LAAL in a real-world setting resulted in improved performance over static benchmarks.
Read more
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Summary
This paper investigates the ad-load design problem in sponsored search markets, focusing on the trade-off between revenue generation and user engagement. Conducting a large-scale randomized field experiment on an Android app store with over five million users, the authors explore how varying the number of sponsored slots affects revenue and user outcomes. The findings reveal that increasing ad load can boost revenue by up to 43%, but simultaneously reduces total search conversions by up to 5% and daily engagement by up to 2.2%. The study highlights significant heterogeneity in the effects of additional slots, with high-ad-conversion queries benefiting more than low-conversion ones. To address these challenges, the authors propose an adaptive algorithm called exploration-augmented Locally Adaptive Ad Load (e-LAAL), which combines a model-free decision rule with static exploration arms. This algorithm is deployed in a production environment serving 22.3 million users, demonstrating improved revenue-conversion trade-offs compared to static benchmarks. The paper contributes to the understanding of ad-load dynamics and offers a practical solution for optimizing ad inventory in real-time.
Methodology
The authors conducted a 66-day randomized field experiment on a major Android app store platform, varying the number of sponsored slots shown to users. The experiment involved over five million users and analyzed the impact of different ad loads on revenue and user engagement metrics.
Results
The experiment found that increasing the number of sponsored slots could increase revenue by up to 43%, but also led to a decrease in total search conversions by up to 5% and daily engagement by up to 2.2%. The results indicated substantial heterogeneity in the effects based on query types and advertiser composition.
Implications
The findings suggest that platforms can optimize ad load dynamically to enhance revenue while minimizing negative impacts on user engagement. The adaptive algorithm developed could be applied in various digital advertising contexts to improve monetization strategies.
Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning
Theory
Optimization
- Unification of selective classification and epistemic reject-option into a constrained optimization framework.
- Demonstration of distinct rejection regions for OOD detection, active learning, and regret-minimization.
- Critique of standard correlation metrics for uncertainty disentanglement, proposing a new diagnostic approach.
- Empirical evidence showing disagreement between decision-theoretic rankings and proxy-task rankings.
Read more
Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning
Summary
This paper critiques the current evaluation methods for epistemic uncertainty, which primarily rely on out-of-distribution (OOD) detection and active learning tasks. The authors argue that the Bayes-optimal decision strategies for these tasks do not align with the commonly used scores for quantifying epistemic uncertainty. They introduce a framework based on the epistemic reject-option, focusing on the ability to identify regret, or reducible error. The authors formulate selective prediction as a constrained optimization problem involving coverage, expected risk, and regret, proving that the optimal selector is a thresholded convex combination of aleatoric and epistemic uncertainties. They highlight a significant disconnect in existing literature regarding uncertainty disentanglement, demonstrating that standard correlation metrics do not predict operational utility. Instead, they propose evaluating the achievable risk and regret as a diagnostic tool. Through benchmarking various uncertainty quantification methods on datasets with dense human annotations, the authors reveal substantial discrepancies between decision-theoretic rankings and proxy-task rankings, indicating that the theoretical misalignment affects practical conclusions about the utility of uncertainty estimators.
Methodology
The authors develop a theoretical framework that combines selective classification and epistemic reject-option through constrained optimization. They construct a one-dimensional setting to illustrate the differences in rejection regions for various tasks and propose a new diagnostic based on the distance to the Pareto-optimal surface. They benchmark existing methods on datasets with dense human annotations to evaluate true regret.
Results
The study finds that the optimal selector for epistemic uncertainty is a thresholded convex combination of aleatoric and epistemic uncertainties. It also reveals that rankings of uncertainty quantification methods can vary significantly across different evaluation tasks, indicating that traditional metrics may not accurately reflect the operational utility of these methods.
Implications
The findings suggest that practitioners should be cautious when using standard metrics for evaluating epistemic uncertainty, as they may not align with the actual utility in real-world applications. The proposed framework and diagnostic tools could enhance the evaluation of uncertainty quantification methods, leading to more informed decision-making in machine learning applications.
Data Driven Block Replacement Scheduling
Optimization
Theory
- Development of data-driven algorithms for block replacement scheduling.
- Formulation of the problem as a stochastic multi-armed bandit.
- Introduction of a Kaplan-Meier renewal algorithm for lifetime distribution estimation.
- Analysis of average-cost MDPs demonstrating the optimality of block replacement policies.
Read more
Data Driven Block Replacement Scheduling
Summary
This paper presents data-driven algorithms for managing N independent identical machines under a block replacement policy, where machines are replaced upon failure and all are replaced at regular intervals. The authors aim to determine the optimal replacement interval (k*) using operational data when the lifetime distribution is unknown. They model the problem as a stochastic multi-armed bandit and propose algorithms based on Hoeffding and Bernstein bounds that achieve O(K log T) regret, matching the Lai-Robbins lower bound. The paper also introduces a Kaplan-Meier renewal algorithm for nonparametric estimation of the lifetime distribution from censored data, achieving policy consistency and low incremental regret. Additionally, the authors analyze two average-cost Markov Decision Processes (MDPs): one based on time elapsed since the last replacement and another on the current age of items, demonstrating the optimality of block replacement policies and providing benchmarks for cost comparisons. Numerical experiments validate the theoretical findings and highlight the cost differences between optimal block and age-dependent replacement strategies.
Methodology
The authors formulate the block replacement scheduling problem as a stochastic multi-armed bandit, utilizing Hoeffding and Bernstein-based algorithms to minimize regret. They also employ a Kaplan-Meier renewal algorithm for nonparametric estimation of lifetime distributions from censored data. Two average-cost MDPs are analyzed: one based on time elapsed since the last replacement and another on the age of items, allowing for a comprehensive evaluation of replacement strategies.
Results
The proposed algorithms achieve O(K log T) regret, matching the Lai-Robbins lower bound, while the Kaplan-Meier algorithm ensures policy consistency and low incremental regret. The analysis of MDPs reveals that block replacement is optimal within its policy class for any lifetime distribution, and the age-vector policy can outperform the block replacement strategy by up to 50%. Numerical experiments confirm the theoretical ordering and highlight significant cost gaps between optimal block and age-dependent replacement.
Implications
The findings have practical implications for organizations managing identical physical assets, such as data centers and aviation, by providing a data-driven approach to optimize replacement scheduling and reduce operational costs. The methodologies developed can be applied to various industries facing similar asset management challenges.
Learning Who to Treat When Treatment is Missing
Theory
Efficient ML
Optimization
- Introduces a framework for policy learning under missing treatment data.
- Proves that MAR estimators are more efficient than MCCAR estimators.
- Demonstrates the critical importance of correctly specifying the missingness mechanism.
- Empirical validation shows near-oracle performance of proposed methods.
Read more
Learning Who to Treat When Treatment is Missing
Summary
This paper addresses the challenge of treatment allocation in policy learning when treatment data is missing, which is a common issue in observational studies. Traditional methods for estimating the conditional average treatment effect (CATE) typically assume complete treatment data, leading to biased estimates and suboptimal policies. The authors extend efficient estimators for average treatment effect (ATE) to accommodate scenarios where treatment data is missing at random (MAR) and missing completely conditionally at random (MCCAR). Through asymptotic efficiency analysis, they demonstrate that the MAR estimator, which utilizes partially observed units, is more efficient than the MCCAR estimator, even when the latter's assumptions hold. The paper emphasizes the importance of correctly specifying the missingness mechanism, showing that misspecified estimators can remain biased regardless of sample size. Comprehensive experiments on synthetic and semi-synthetic datasets validate the theoretical findings, indicating that the proposed estimators can achieve near-oracle performance under the right conditions. This work provides practitioners with robust tools for policy learning in the presence of missing treatment data.
Methodology
The authors extend efficient influence function methods from ATE to policy value and CATE estimation under MAR and MCCAR assumptions. They conduct asymptotic efficiency analysis to compare the performance of MAR and MCCAR estimators and validate their theoretical results through comprehensive experiments on synthetic datasets.
Results
The study finds that the MAR estimator is both valid and more efficient than the MCCAR estimator when both are applicable. Empirical results indicate that correctly specifying the missingness mechanism is crucial, as misspecified estimators remain biased, while the proposed estimators achieve near-oracle performance when assumptions are satisfied.
Implications
The findings have significant implications for practitioners in fields such as healthcare and social services, where treatment allocation decisions often rely on observational data with missing treatment information. The proposed methods offer a robust approach to improve policy learning and treatment allocation under budget constraints.
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Optimization
Large Language Models
Theory
- Muse optimizers reveal the significance of matrix representation in Muon-style optimization.
- Different Frobenius-isometric representations induce distinct polar geometries affecting optimization performance.
- Balanced non-native representations can match the performance of native representations in large language models.
- Reducing the shorter dimension in matrix representations weakens optimization capabilities.
Read more
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Summary
This paper introduces Muse, a family of Muon-style optimizers that explores the impact of different matrix representations on optimization geometry. Muon optimizers utilize a polar map for matrix momentum updates, which depend on the representation of parameter blocks before orthogonalization. The authors investigate how various Frobenius-isometric representations influence the optimizer's performance, particularly in the context of large language model training. They establish that the geometry induced by these representations affects the number of singular channels, scaling, and convergence bounds in stochastic nonconvex settings. Through experiments on LLaMA2-130M and LLaMA2-600M, the study demonstrates that balanced non-native representations can achieve performance comparable to native representations, while reducing the shorter dimension leads to diminished scaling and singular-channel support, resulting in behavior similar to normalized momentum. The findings highlight the importance of representation choice in Muon optimizers and provide insights into their underlying performance mechanisms.
Methodology
The authors analyze the geometry induced by various matrix representations in Muon optimizers, establishing a framework for understanding how these representations affect polar updates. They conduct pretraining experiments on LLaMA2 models to evaluate the performance of different representations, comparing them against the native representation and normalized momentum.
Results
The experiments show that Muse optimizers with balanced non-native representations achieve performance similar to native representations. However, representations with reduced shorter dimensions exhibit weaker scaling and singular-channel support, leading to results that increasingly resemble normalized momentum. The study also establishes theoretical guarantees regarding the geometry and performance of these optimizers.
Implications
The findings suggest that careful selection of matrix representations can enhance the performance of Muon-style optimizers, particularly in large-scale training scenarios. This insight could inform future developments in optimization techniques for deep learning models, especially in the context of large language models.
MIDiff: Tackling Sparsity and Imbalance in Mobile Usage Generation via Multivariate-Imaging Diffusion
Generative Models
Time Series
Multimodal
- Introduces MIDiff, a diffusion-based framework for mobile usage data generation.
- Addresses challenges of sparsity, heterogeneity, and long-tail imbalance in mobile usage traces.
- Utilizes C-GASF to convert sparse sequences into correlation images for better modeling.
- Employs Triple Attention in a U-Net to preserve temporal and variable dependencies.
Read more
MIDiff: Tackling Sparsity and Imbalance in Mobile Usage Generation via Multivariate-Imaging Diffusion
Summary
The paper presents MIDiff, a novel framework designed to address the challenges of sparsity and imbalance in mobile usage data generation. Mobile usage traces are crucial for applications like user behavior prediction and app recommendation, but their collection is hindered by privacy issues and high costs. Existing generative models struggle with mobile data due to its intermittent activity, heterogeneous variable types, and long-tail usage patterns. MIDiff utilizes a diffusion-based approach in an imaging space defined by Cross-Gramian Angular Sum Field (C-GASF), which transforms sparse multivariate sequences into correlation images. The framework employs a Triple Attention mechanism within a U-Net architecture to maintain temporal consistency and variable dependencies. Experimental results demonstrate that MIDiff outperforms existing models, achieving a Discriminative Accuracy (DA) of 0.1526, significantly better than the strongest baseline, ZITS-VAE, which scored 0.3476. This indicates MIDiff's effectiveness in generating realistic and diverse mobile usage traces, making it a promising solution for generating synthetic mobile data.
Methodology
The methodology involves transforming sparse multivariate mobile usage data into correlation images using Cross-Gramian Angular Sum Field (C-GASF). MIDiff employs a U-Net architecture enhanced with Triple Attention to effectively capture temporal consistency and dependencies among different variable types, addressing the challenges of sparsity and imbalance in the data.
Results
MIDiff achieved a Discriminative Accuracy (DA) of 0.1526, outperforming the best baseline model, ZITS-VAE, which had a DA of 0.3476. This indicates that MIDiff is effective in generating realistic and diverse mobile usage traces, demonstrating its potential for practical applications.
Implications
The findings suggest that MIDiff can significantly enhance the generation of synthetic mobile usage data, which is crucial for various applications such as personalized app recommendations and user behavior analysis. This framework could facilitate research and development in mobile data analytics while addressing privacy concerns associated with real data collection.
Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards
Reinforcement Learning
Large Language Models
Theory
- First non-vacuous generalization bounds for RLVR fine-tuning at scale.
- Introduction of the Progressive RLVR framework for efficient model compression.
- Empirical validation across four domains showing significant accuracy improvements.
- Use of Gumbel-max reparameterization to handle stochastic token generation.
Read more
Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards
Summary
This paper addresses the challenge of generalizability in reinforcement learning with verifiable rewards (RLVR), particularly for large language models (LLMs). The authors establish the first non-vacuous generalization bounds for RLVR fine-tuning at the billion-parameter scale. They adapt PAC-Bayes compression bounds to account for the stochastic nature of token generation in RLVR, utilizing the Gumbel-max reparameterization trick to manage the complexity of generated sequences. The proposed Progressive RLVR framework integrates RLVR with on-policy distillation, TinyLoRA, and model quantization to achieve significant model compression while retaining high performance. Empirical results demonstrate that Progressive RLVR maintains 84-97% of the performance of standard LoRA fine-tuning while achieving a compression ratio of 14,796Γ. The established generalization bounds show improvements in accuracy over base models by 9-51% and are within 6-11% of the accuracy of fine-tuned models across various domains, including mathematical problem-solving, programming, general-knowledge reasoning, and Text-to-SQL translation.
Methodology
The authors develop a PAC-Bayesian framework to establish generalization bounds for RLVR, addressing the stochasticity of token generation through Gumbel-max reparameterization. They introduce the Progressive RLVR framework, which combines RLVR with on-policy distillation and TinyLoRA to achieve aggressive model compression while maintaining performance.
Results
The Progressive RLVR framework achieves 84-97% of the performance of standard LoRA fine-tuning while being 14,796Γ more compressible. The established generalization bounds exceed the accuracy of the base model by 9-51% and are within 6-11% of the accuracy of fine-tuned models across four domains.
Implications
The findings provide a formal verification of RLVR performance in real-world tasks, enhancing the deployment of LLMs in high-stakes domains. The established bounds and efficient compression methods can lead to more robust and generalizable models in reinforcement learning applications.
TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories
Generative Models
Time Series
- TEDDY is specifically designed for pediatric patients, addressing the limitations of adult-centric models.
- The model achieved a median AUC of 72.0%, outperforming traditional machine learning baselines in disease onset prediction.
- TEDDY demonstrated predictive capabilities over two years before the first diagnosis, highlighting its potential for early intervention.
- The model is particularly effective in predicting rare diseases, with 90% of the rarest conditions showing significant predictive confidence.
Read more
TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories
Summary
The paper introduces TEDDY (Temporal Event Decoder for Disease in Youth), a pediatric foundation model designed to predict disease onset using longitudinal electronic health records (EHRs) from children. TEDDY is a 1.84-million-parameter decoder transformer trained on approximately 73 million ICD-10 diagnoses from 1.6 million pediatric patients. The model focuses on predicting disease trajectories and visit timings before the actual visit codes are revealed, achieving a median area under the curve (AUC) of 72.0% across 797 prediction tasks spanning 16 ICD-10 chapters. TEDDY outperformed various baseline models, including DenseNet and LSTM, on 96-99% of tasks, particularly excelling in predicting lower-prevalence diagnoses. The predictive capabilities of TEDDY were evident even more than two years prior to the first recorded diagnosis, with notable performance in conditions like asthma and ADHD. The model's visit-timing predictions had a mean absolute error of 3.0 days over a year, though calibration issues were noted for longer intervals. The findings underscore the potential of pediatric-specific models in enhancing early diagnosis and intervention for rare diseases.
Methodology
TEDDY employs a decoder transformer architecture trained on longitudinal EHR data, utilizing a generative approach to model disease trajectories and predict future medical events. The training involved a large dataset of ICD-10 diagnoses, with a focus on first occurrences and avoiding within-encounter leakage to ensure valid predictions.
Results
TEDDY achieved a median AUC of 72.0% across 797 disease-onset tasks, significantly outperforming baseline models. It maintained strong performance across different demographics and excelled in predicting rare conditions. The model's predictive signal was detectable more than two years before diagnosis, with specific benchmarks for asthma and ADHD yielding AUCs of 79.3% and 84.7%, respectively.
Implications
The development of TEDDY suggests that pediatric-specific models can significantly improve early diagnosis and intervention strategies for children, particularly for rare diseases. This approach could reduce the diagnostic odyssey faced by families and enhance clinical decision-making in pediatric healthcare.
Counterfactual Optimal Action Trees (COAT): Interpretable Prescriptive Policies from Observational Data
Optimization
Interpretability
- COAT combines counterfactual estimation with mixed-integer optimization for interpretable decision-making.
- The framework was successfully applied in a real-world airline pricing scenario, demonstrating significant revenue uplift.
- COAT addresses the challenges of deploying AI in regulated environments by ensuring transparency and compliance.
- The methodology emphasizes the integration of causal estimation and scalable optimization.
Read more
Counterfactual Optimal Action Trees (COAT): Interpretable Prescriptive Policies from Observational Data
Summary
The paper introduces the Counterfactual Optimal Action Tree (COAT) framework, which aims to learn interpretable prescriptive policies from observational data. COAT integrates counterfactual outcome estimation with large-scale mixed-integer optimization, employing column generation to convert causal predictions into actionable decisions while adhering to business and regulatory constraints. The framework is validated through a 17-week pilot with a major airline, focusing on ancillary pricing. Results showed a 6.9% increase in upsell revenue per booking, translating to an estimated $50β$150 million in additional annual revenue. The success of COAT led to its broader adoption within the airline and highlighted the importance of operations research in AI, emphasizing that effective AI deployment requires not just predictive accuracy but also interpretable and compliant decision-making systems.
Methodology
COAT operates in two stages: first, it estimates counterfactual outcomes for various actions; second, it formulates and solves a constrained optimization problem to derive an interpretable policy represented as an action tree. The optimization is achieved through a path-based formulation using column generation, which allows for scalability in large policy spaces.
Results
In a 17-week pilot study, COAT achieved a 6.9% increase in upsell revenue per booking for a major airline. This improvement is projected to yield an additional $50β$150 million in annual revenue from premium seat sales across eligible markets. The evaluation utilized the synthetic control method to provide credible causal evidence.
Implications
The findings suggest that COAT can be a valuable tool for businesses seeking to implement AI-driven decision-making systems that are both effective and compliant with regulatory standards. The framework's success in the airline industry may pave the way for its application in other sectors where interpretability and operational constraints are critical.
LATTICE: Graph Self-Supervised Learning for Multimodal Spatial Omics Integration
Graph Learning
Multimodal
- LATTICE integrates multiple spatial omics modalities into a unified graph-based framework.
- The framework employs self-supervised learning objectives to enhance multimodal representation.
- Evaluation on a melanoma cohort shows improved concordance with established clustering methods.
- Incorporating additional modalities can enhance spatial contiguity but may complicate transcriptomic alignment.
Read more
LATTICE: Graph Self-Supervised Learning for Multimodal Spatial Omics Integration
Summary
The paper introduces LATTICE, a novel graph-based self-supervised learning framework designed for the integration of multimodal spatial omics data, specifically combining transcriptomic and epigenomic assays. Traditional analysis methods often rely on single-modality pipelines, which can overlook the complementary information provided by multiple modalities. LATTICE addresses this limitation by constructing a spatial neighborhood graph that incorporates five aligned modality blocks per Visium spot: Visium RNA, scMultiome RNA, scMultiome ATAC, spatial ATAC, and spatial CUT&Tag. The framework employs a TransformerConv encoder to learn spot-level representations through three key objectives: masked reconstruction, cross-modal alignment, and spatial smoothness. The evaluation of LATTICE on a private melanoma cohort, consisting of 54,912 spots from 11 samples, demonstrated stable optimization and reproducible embeddings. Notably, the integration of scMultiome RNA with Visium RNA significantly enhanced concordance with Space Ranger clusters, indicating improved multimodal integration. However, while additional modalities improved spatial contiguity and multimodal utility scores, they sometimes reduced agreement with RNA-derived reference labels, suggesting that the embeddings captured more complex regulatory structures beyond mere transcriptomic similarity. Overall, LATTICE presents a robust framework for multimodal spatial omics integration, emphasizing the need for stronger supervision and external benchmarking.
Methodology
LATTICE constructs a spatial neighborhood graph from multimodal features and employs a TransformerConv encoder to learn latent representations. The training involves masked feature reconstruction, cross-modal alignment, and spatial smoothness objectives to ensure consistency and coherence in the learned embeddings.
Results
LATTICE demonstrated stable optimization and reproducible embeddings across multiple analysis seeds. The integration of scMultiome RNA with Visium RNA improved concordance metrics such as adjusted Rand index (ARI), normalized mutual information (NMI), and spatial contiguity. Additional modalities further enhanced spatial contiguity and multimodal utility scores, although they occasionally reduced agreement with RNA-derived reference labels.
Implications
LATTICE provides a practical framework for integrating multimodal spatial omics data, facilitating more comprehensive analyses of tissue organization and regulatory mechanisms. Its design encourages further exploration of multimodal datasets in biological research, potentially leading to enhanced understanding of complex biological systems.
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Reinforcement Learning
Large Language Models
Generative Models
- Introduces a continuous-time RL framework for fine-tuning discrete diffusion models.
- Allows incorporation of intermediate rewards, improving credit assignment during training.
- Develops efficient trajectory subsampling techniques to reduce computational costs.
- Demonstrates effectiveness on low-dimensional optimization and dLLM post-training tasks.
Read more
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Summary
This paper presents a novel continuous-time reinforcement learning (RL) framework aimed at fine-tuning discrete diffusion models, particularly in the context of score-based models used for generating discrete data. The authors formulate RL in continuous time with discrete state spaces and arbitrary action spaces using a stochastic control approach, modeling state dynamics as a controlled continuous-time Markov chain (CTMC). They derive continuous-time policy gradient methods, leading to variants of proximal policy optimization (PPO) and group relative policy optimization (GRPO). A key contribution is the ability to incorporate intermediate reward signals throughout the denoising trajectory, enhancing the learning process compared to existing GRPO methods that rely solely on terminal rewards. The framework is particularly effective for masked diffusion models (MDMs), allowing for flexible policy parameterizations and efficient trajectory subsampling techniques to estimate trajectory likelihoods. The authors demonstrate the effectiveness of their approach through experiments on low-dimensional optimization problems and reinforcement learning post-training of discrete large language models (dLLMs) for tasks such as mathematical reasoning and coding.
Methodology
The authors employ a stochastic control approach to formulate RL in continuous time with discrete state spaces. They derive policy gradient methods tailored for continuous-time settings, leading to adaptations of PPO and GRPO. The framework allows for the integration of intermediate rewards and utilizes trajectory subsampling techniques to manage computational complexity in estimating likelihoods.
Results
The proposed framework shows promising results in both low-dimensional entropy-regularized optimization problems and in the reinforcement learning post-training of discrete large language models, particularly in enhancing their reasoning and coding capabilities. The methods demonstrate improved efficiency and effectiveness compared to existing approaches.
Implications
This work opens new avenues for research at the intersection of control theory and reinforcement learning, particularly in discrete spaces. The continuous-time framework can enhance the fine-tuning of generative models, potentially leading to more robust and efficient training processes in various applications, including natural language processing and generative modeling.
Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning
Federated Learning
Optimization
Efficient ML
- Dysco mitigates data-parameter interference in federated learning by dynamically allocating client-specific LoRA subspaces.
- The method enhances stability and performance of federated LoRA aggregation through activation-insensitive subspace computation.
- Dysco achieves substantial reductions in training loss and improves accuracy across multiple federated learning algorithms.
- The approach adds minimal computational overhead, making it practical for real-world applications.
Read more
Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning
Summary
The paper presents Dysco, a novel method designed to address the instability in federated learning (FL) caused by data-parameter interference when using Low-Rank Adaptation (LoRA) for fine-tuning large pre-trained models. The authors identify that the aggregation of LoRA adapters can lead to interference due to the misalignment between client-specific updates and their respective data representations. Dysco introduces a dynamic subspace allocation mechanism that allows clients to compute activation-insensitive subspaces from their local representations, which are then transmitted to a central server. The server constructs merged subspaces that maximize compatibility with the clients' subspaces, thus mitigating interference. The method also incorporates multi-round subspace boosting to adapt to representation drift while preserving useful past updates. Theoretical convergence analysis demonstrates that Dysco provides a tighter upper bound on aggregation error compared to existing methods. Empirical results show significant improvements in training loss and accuracy on synthetic federated tasks and real-world applications, such as clinical-note classification, outperforming several baseline methods while maintaining low computational overhead.
Methodology
Dysco employs a dynamic subspace boosting approach where each client computes activation-insensitive subspaces from local representations. These bases are sent to the server, which constructs merged subspaces that maximize compatibility with other clients' subspaces. The method includes multi-round boosting to adapt to changes in data representations while preserving useful update directions.
Results
Dysco significantly reduces final-round synthetic training loss by up to 9 times compared to baseline methods and improves accuracy on MIMIC-IV clinical-note classification by up to 4.3%. It outperforms recent federated LoRA methods while adding only 0.9% wall-clock overhead.
Implications
The findings suggest that Dysco can enhance the performance of federated learning systems, particularly in scenarios with heterogeneous data distributions. This has potential applications in privacy-sensitive domains such as healthcare, where federated learning is increasingly utilized.
Sharp Stability Threshold and Certification for Designing Stable Residual Architectures
Theory
Optimization
- Establishes a stability threshold (q β€ 1) for deep residual architectures based on velocity field growth.
- Introduces a method for certifying the stability of neural architectures through input-magnitude exponents.
- Demonstrates that exceeding the stability threshold leads to training instability and divergent behavior.
- Provides a parameter-free modification to stabilize supercritical blocks without normalization.
Read more
Sharp Stability Threshold and Certification for Designing Stable Residual Architectures
Summary
This paper introduces the sublinear-growth principle for deep residual architectures, establishing a sharp stability threshold on the input-magnitude exponent of the velocity field in residual blocks. The authors demonstrate that for stable training, the exponent q must be less than or equal to 1, which is supported by classical ODE theory and optimal-control analysis. The paper outlines how exceeding this threshold leads to divergent velocity fields and unstable training. The authors provide a framework for efficiently certifying the stability of neural architectures by analyzing the input-magnitude exponents of architectural primitives. They present a parameter-free modification that reduces the exponent of a supercritical block from q = 5 to q = 1 without normalization, confirming the effectiveness of their proposed criterion through experiments on Mamba and PatchTST models. The findings clarify the role of various architectural components, including normalization layers, in ensuring stable training and inference.
Methodology
The authors utilize classical ODE theory and optimal-control analysis to derive the stability threshold for residual architectures. They analyze the velocity fields of residual blocks and develop an arithmetic of input-magnitude exponents to facilitate architectural design and certification. The paper includes theoretical proofs and empirical validation through experiments on specific neural architectures.
Results
The main results indicate that the stability threshold for residual architectures is q β€ 1, which is both necessary and sufficient for stable training. The proposed modifications successfully stabilize previously unstable architectures, and experiments validate that architectures adhering to the q β€ 1 criterion train stably, regardless of normalization layers.
Implications
The findings have significant implications for the design of deep learning architectures, particularly in ensuring stable training without relying solely on normalization layers. This could lead to more efficient and robust neural network designs across various applications in machine learning.