AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
56
Papers today
8h
Update frequency
7
Days of history
Kohn-Sham Spectral Embedding on Sparse Graphs at the Nishimori Temperature for Image Classification
Computer Vision
Graph Learning
Theory
- Introduction of Kohn-Sham Spectral Embedding (KSSE) for image classification.
- Utilization of sparse-graph spectral embedding evaluated at Nishimori temperature.
- Development of theoretical results linking belief propagation to RBIM energy landscapes.
- Achieved 88.93% Top-1 accuracy on ImageNet-1000 with significantly fewer parameters.
Read more
Kohn-Sham Spectral Embedding on Sparse Graphs at the Nishimori Temperature for Image Classification
Summary
This paper introduces the Kohn-Sham Spectral Embedding (KSSE), a novel energy-based model aimed at enhancing image classification performance by replacing the dense top-layer classifier of convolutional neural networks (CNNs) with a sparse-graph spectral embedding evaluated at the Nishimori temperature of a Random-Bond Ising Model (RBIM). The authors map pre-trained feature representations onto quasi-cyclic low-density parity-check graphs and construct a regularized Laplacian that acts as an effective Kohn-Sham Hamiltonian. This approach allows for the independent solving of spectral problems for each feature channel efficiently using Fast Fourier Transform techniques. The paper also presents a star-domain surgery method to optimize graph topology, creating local convexity around codewords while managing residual frustration. Theoretical contributions include a generalized Ihara-Bass identity, a trapping-set eigenvalue correspondence theorem, and a multi-scale fractal analysis framework. The KSSE framework was evaluated on ImageNet-1000, achieving a Top-1 accuracy of 88.93% with approximately 21.24 million parameters, outperforming existing models like Swin-L and matching the performance of ViT-H/14 while significantly reducing model size.
Methodology
The methodology involves mapping pre-trained CNN features onto quasi-cyclic low-density parity-check graphs, constructing a regularized Laplacian, and solving independent spectral problems for each feature channel using Fast Fourier Transform. The star-domain surgery technique is employed to optimize graph topology, enhancing local convexity around codewords while controlling residual frustration.
Results
The KSSE framework achieved a Top-1 accuracy of 88.93% on ImageNet-1000 using approximately 21.24 million parameters, outperforming Swin-L (197 million parameters, 86.4-87.3% accuracy) and matching the lower end of ViT-H/14 (632 million parameters, 88.0-89.5% accuracy).
Implications
The proposed KSSE framework could lead to more efficient image classification models that require fewer parameters while maintaining high accuracy, potentially influencing future research in energy-based models and graph learning approaches in computer vision.
PlatformBid: An Auto-Bidding Benchmark from a Unified Advertising Platform's Perspective
Reinforcement Learning
Generative Models
Optimization
- Introduction of PlatformBid, a benchmark for auto-bidding from a unified advertising platform perspective.
- Definition of three competitive settings: homogeneous, heterogeneous, and promotional competition.
- Evaluation of various existing auto-bidding methods alongside the proposed BidFlow method.
- Demonstration of BidFlow's effectiveness in dynamic environments and its real-world application on Kuaishou.
Read more
PlatformBid: An Auto-Bidding Benchmark from a Unified Advertising Platform's Perspective
Summary
This paper introduces PlatformBid, the first comprehensive benchmark for auto-bidding algorithms designed from the perspective of a unified advertising platform, which integrates Supply Side Platform (SSP), Demand Side Platform (DSP), and Ad Exchange functionalities. Traditional auto-bidding methods have primarily focused on optimizing advertiser conversions from a DSP-centric viewpoint. However, with the evolution of large advertising platforms, there is a need to balance individual advertiser goals with overall platform revenue optimization. The authors define three competitive settings within PlatformBid: homogeneous competition (where all advertisers use the same bidding algorithm), heterogeneous competition (where different algorithms are employed by different advertisers), and promotional competition (where advertisers increase budgets during promotional events). The paper evaluates a wide range of existing auto-bidding methods, including classical control methods, reinforcement learning (RL)-based methods, and generative methods. Additionally, the authors propose a novel auto-bidding method called BidFlow, which utilizes flow-matching techniques to adapt to dynamic competitive environments. Experimental results indicate that BidFlow outperforms existing methods in the new settings, and online experiments on Kuaishou demonstrate a 0.68% improvement in target cost, validating the offline-online consistency of PlatformBid.
Methodology
The authors developed the PlatformBid benchmark by defining three competitive settings to simulate real-world auto-bidding scenarios. They systematically evaluated existing auto-bidding methods, including classical control, RL-based, and generative approaches, and introduced a new method, BidFlow, based on flow-matching for enhanced adaptability in competitive environments.
Results
Experimental results showed that BidFlow outperformed existing auto-bidding methods across the defined settings. Online experiments conducted on Kuaishou confirmed a 0.68% improvement in target cost, providing evidence for the effectiveness and practical applicability of the proposed benchmark and methods.
Implications
The introduction of PlatformBid could significantly advance research in auto-bidding algorithms by providing a comprehensive evaluation framework that aligns with the operational realities of modern advertising platforms. This could lead to improved bidding strategies that optimize both advertiser performance and platform revenue.
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
NLP
Large Language Models
Optimization
- Introduces a Bayesian domain weighting method to optimize data mixtures for LLMs.
- Addresses limitations of traditional manual heuristics and function-fitting approaches.
- Demonstrates stable and efficient learning of domain weights using a Dirichlet distribution.
- Achieves better performance with less data compared to existing methods.
Read more
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
Summary
This paper addresses the challenge of optimizing data mixtures for Large Language Models (LLMs) by proposing a Bayesian domain weighting method. Traditional approaches to domain weighting often rely on manual heuristics or function-fitting methods that assume structural properties like rank invariance, which can lead to significant estimation biases when scaling to larger datasets. The authors introduce a novel method that infers domain weights from a Dirichlet distribution using Gamma prior information derived from observations. This Bayesian approach stabilizes the optimization process, allowing for efficient learning of domain weights while consuming less data compared to existing search-based methods. Experimental results indicate that the proposed method achieves smoother convergence and better performance in identifying optimal data mixtures, thereby revitalizing the optimization-based domain weighting paradigm for large-scale applications.
Methodology
The authors propose a Bayesian domain weighting method that utilizes a Dirichlet distribution to infer domain weights. By incorporating Gamma prior information learned from observations, the method stabilizes the optimization trajectory, allowing for efficient learning of domain weights directly from data.
Results
The experimental results show that the proposed Bayesian domain weighting method yields stable and efficient convergence in domain weight learning, outperforming traditional function-fitting methods in terms of data consumption and optimization stability.
Implications
This research has significant implications for the training of Large Language Models, particularly in improving the efficiency and effectiveness of multi-domain pre-training data optimization. It opens avenues for more robust model training strategies that can adapt to complex data distributions.
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Multimodal
Robotics
Efficient ML
- AAPT improves GUI agent performance by pre-compiling actions to eliminate decoding delays.
- The method demonstrates a significant increase in success rates in contested decision windows.
- Key factors for AAPT's success include fast observer decoding, valid tree planning, and accurate routing.
- The study provides evidence that decode latency is a primary cause of missed action windows.
Read more
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Summary
This paper addresses the issue of delays in GUI agents, which often produce correct actions too late due to expensive autoregressive decoding on the decision-time critical path. The authors propose a novel approach called Adaptive Anticipatory Policy Trees (AAPT), which aims to eliminate this delay without altering the underlying model. AAPT constructs a bounded conditional policy tree during idle periods, allowing for immediate action execution when an event occurs, thus avoiding the need for real-time decoding. The study demonstrates that AAPT significantly improves the success rate of GUI agents from 0.50 to 0.79 in contested decision windows, while maintaining accuracy by producing no incorrect actions. The findings suggest that the key to improving performance lies in pre-compiling actions and effectively routing them, rather than merely acting earlier. The paper includes rigorous testing with pre-registered endpoints and exact McNemar tests, confirming the effectiveness of AAPT in various scenarios, including an independent general-purpose multimodal model.
Methodology
The authors developed Adaptive Anticipatory Policy Trees (AAPT), which constructs a conditional policy tree during idle periods. This tree includes observable guards, pre-authorized actions, and deadlines for each branch. A lightweight observer matches incoming frames to the prepared branches and executes actions without generating new text, thereby avoiding delays associated with real-time decoding. The effectiveness of AAPT was tested through paired trials and pre-registered endpoints, utilizing exact McNemar tests for statistical validation.
Results
AAPT improved the success rate of GUI agents from 0.50 to 0.79 within a contested decision window (p = 1.8 Γ 10β3), with no incorrect actions produced. In additional trials with an independent multimodal model, the results were replicated (p = 4.9 Γ 10β13). The study also identified that the performance gain is contingent on the ability to pre-compile actions and accurately route them, highlighting the limitations of reactive execution in scenarios where actions cannot be anticipated.
Implications
The findings suggest that pre-compiling actions can significantly enhance the performance of GUI agents in time-sensitive environments. This approach could be applied to various applications requiring real-time decision-making, such as automated customer support systems, gaming, and interactive user interfaces, where timely responses are critical.
Regularizing modality contribution drift in multimodal continual learning
Multimodal
- Introduces the concept of modality contribution drift (MCD) in MMCL.
- Proposes Continual Modality Contribution Drift Regularization (CMCDR) to maintain modality contribution structures.
- Presents both replay-based and replay-free versions of CMCDR for different MMCL scenarios.
- Demonstrates that existing MMCL methods inadequately address MCD.
Read more
Regularizing modality contribution drift in multimodal continual learning
Summary
This paper addresses the challenge of modality contribution drift (MCD) in multimodal continual learning (MMCL), where the relative contributions of different modalities can change as new tasks are learned. The authors introduce the concept of MCD, which quantifies how the contributions of individual modalities and their interactions evolve across incremental tasks. They propose a novel method called Continual Modality Contribution Drift Regularization (CMCDR) that aims to preserve the contribution structure of previously learned tasks while allowing for adaptation to new tasks. CMCDR has both replay-based and replay-free versions to accommodate different MMCL settings. The paper includes theoretical and empirical analyses demonstrating that existing MMCL methods do not effectively prevent MCD. The authors validate CMCDR through extensive experiments in multimodal class-incremental learning and continual visual question answering, showing that it significantly mitigates MCD and enhances performance across various benchmarks.
Methodology
The authors developed an intervention-based diagnostic estimator to assess modality contributions by manipulating modality subsets and measuring the resulting predictive benefits. They defined the MCD score to quantify changes in contribution strength and reliance across tasks. CMCDR was then formulated to regularize these contributions, with two variants: one utilizing stored exemplars (replay-based) and another using current task samples (replay-free).
Results
Experiments showed that CMCDR effectively mitigates modality contribution drift, leading to improved performance in multimodal class-incremental learning and continual visual question answering tasks. The method outperformed existing MMCL approaches in maintaining stable modality contributions across incremental learning stages.
Implications
The findings suggest that addressing modality contribution drift is crucial for developing robust MMCL systems. CMCDR can be applied in various domains where multimodal data is prevalent, such as robotics, autonomous driving, and interactive AI systems, enhancing their ability to learn continuously without forgetting.
Certifying when decision-time information justifies adaptive experimentation
Theory
Optimization
Robotics
- Opal framework enables decision-making on whether to allow adaptive experimentation based on real-time information.
- Establishes an impossibility boundary for non-trivial authorization under certain conditions.
- Demonstrates effective compound selection in a large-scale Cell Painting study while controlling false-activation rates.
- Introduces a structured approach to distinguish between policy misalignment and non-certifiability.
Read more
Certifying when decision-time information justifies adaptive experimentation
Summary
The paper introduces the Opportunity-aware Policy Authorization for Laboratories (Opal), a framework designed to determine whether adaptive experimentation should be enabled based on decision-time information. Unlike existing methods that assume adaptation is already permitted, Opal emphasizes the need for a precommitted contract that ensures non-trivial adaptation, controlled target risk, and positive executed value after costs. The authors establish an impossibility boundary indicating that source outcomes and unlabelled target covariates cannot uniformly support non-trivial authorization under unrestricted conditional outcome shifts. They derive a target-calibrated recovery method and apply it to a large-scale Cell Painting dataset, demonstrating that Opal can effectively select compounds while maintaining a low false-activation rate. The framework distinguishes policy misalignment from non-certifiability, establishing a new layer of authorization for safe adaptive science.
Methodology
The authors developed the Opal framework, which integrates decision-time information with a precommitted contract to assess the justification for adaptive experimentation. They conducted empirical studies using a target-calibrated approach on a dataset of 11,265 compounds, applying a frozen gate method to select compounds while controlling for false activations.
Results
Opal successfully activated 595 compounds from the Cell Painting dataset, with 384 identified as positive opportunities. The false-activation upper bound was maintained at 5.18%, below the 7.5% threshold. The framework demonstrated the ability to recover a non-trivial, positive-value active branch while ensuring risk control and operational gain.
Implications
The findings suggest that Opal can enhance the safety and efficacy of adaptive experimentation in scientific laboratories, potentially leading to more informed decision-making in experimental design and execution. This framework could be applied in various fields requiring adaptive methodologies, such as drug discovery and materials science.
The Role of Causality in Algorithmic Recourse
Theory
Optimization
Interpretability
- Introduces a causal performative recourse framework to model interactions between agents and learning algorithms.
- Highlights the risks of misalignment between predicted outcomes and true qualifications due to strategic behavior.
- Characterizes conditions for stable solutions in the context of performative prediction.
- Demonstrates that causal recourse reduces gaming incentives and enhances predictive accuracy.
Read more
The Role of Causality in Algorithmic Recourse
Summary
This paper addresses the limitations of existing algorithmic recourse methods in high-stakes classification scenarios, such as loan applications, where the goal is to provide individuals with actionable changes to improve their predicted outcomes. The authors argue that many current approaches fail to consider whether the suggested feature changes genuinely enhance an individual's qualifications or merely exploit the model to achieve a better outcome. This misalignment can lead to strategic behavior that undermines the model's predictive accuracy over time. To tackle this issue, the authors introduce a causal performative recourse framework that models the interactions between agents and the learning algorithm through a structural causal model. This framework captures how recourse interventions affect both agent features and true labels, leading to a non-convex optimization problem. The authors characterize conditions for the existence of performatively stable solutions and propose algorithms for efficient computation. Empirical evaluations on semi-synthetic and real-world datasets demonstrate that incorporating causal responses into recourse policies significantly reduces the incentives for agents to game the system while improving predictive performance.
Methodology
The authors develop a causal performative recourse framework that utilizes structural causal models to analyze the effects of recourse interventions. They formulate the learning problem as a performative prediction problem and provide algorithms with formal convergence guarantees to compute causal recourse efficiently.
Results
The empirical evaluations reveal that the proposed causal recourse framework converges effectively and outperforms naive recourse policies. The results indicate that accounting for causal responses leads to lower incentives for agents to manipulate features and enhances the overall predictive performance of the models.
Implications
This work has significant implications for the design of algorithmic recourse systems in high-stakes decision-making contexts. By integrating causality into recourse policies, practitioners can create more robust and effective systems that not only provide actionable recommendations but also maintain the integrity of predictive models over time.
Memory Efficient Tabular Foundation Models
Efficient ML
- Quantization can reduce the memory footprint of Tabular Foundation Models by up to 7.6 times.
- Quantized models maintain predictive performance similar to full precision counterparts.
- The study highlights the importance of memory efficiency for practical deployment in constrained environments.
- Results show that quantized TFMs outperform classical machine learning baselines across multiple datasets.
Read more
Memory Efficient Tabular Foundation Models
Summary
This paper addresses the practical deployment challenges of Tabular Foundation Models (TFMs), such as TabPFN, which have shown superior performance in tabular machine learning tasks compared to classical models. The authors investigate the memory requirements of these models and propose model compression techniques to significantly reduce memory usage while maintaining performance. By employing quantization methods, they achieve memory reductions of up to 7.6 times, which translates to nearly 87% lower deployment costs. The study emphasizes the importance of memory efficiency in real-world applications, particularly in sectors like healthcare and finance, where deployment constraints are critical. The authors systematically evaluate various quantization approaches on standard tabular datasets, demonstrating that quantized models retain predictive performance comparable to full precision models and outperform classical baselines. This work provides valuable insights for practitioners aiming to deploy TFMs efficiently in practical settings.
Methodology
The authors employed standard quantization techniques to reduce the memory footprint of TabPFN models. They conducted experiments across 30 datasets, comparing the performance of quantized models against full precision and classical machine learning baselines. The evaluation included metrics such as accuracy and area under the curve (AUC) to assess predictive performance.
Results
The results indicated that quantizing TabPFN models to INT4 precision resulted in a memory footprint reduction of up to 7.6 times while maintaining accuracy levels comparable to full precision models. Quantized models consistently outperformed classical baselines, demonstrating their effectiveness in practical deployment scenarios.
Implications
The findings suggest that adopting quantization techniques can facilitate the broader adoption of Tabular Foundation Models in industries with strict deployment constraints. This can lead to more efficient use of resources and improved accessibility of advanced machine learning models in various applications, including healthcare, finance, and operations.
$Ξ²$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
NLP
Large Language Models
Reinforcement Learning
- Ξ²-OPSD generalizes OPSD by allowing a controllable KL penalty parameter Ξ², enhancing training flexibility.
- The optimal policy is derived as a geometric interpolation between a reference policy and a privileged teacher.
- The approach uses efficient token-level logit interpolation to approximate expensive policy optimization.
- Experiments show significant improvements in optimization stability and reasoning performance across various benchmarks.
Read more
$Ξ²$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
Summary
The paper introduces Ξ²-OPSD, a novel approach to on-policy self-distillation (OPSD) that enhances the training of reasoning language models by incorporating a parameterized KL-regularized policy optimization framework. The authors identify that vanilla OPSD, which operates with a fixed Ξ² value of 1, can be brittle and requires significant engineering to function effectively. By generalizing OPSD to allow Ξ² to vary, they create a more flexible framework that balances the trade-off between guidance from a privileged teacher and adherence to a reference policy. The optimal policy for Ξ²-OPSD is derived as a geometric interpolation between these two policies, allowing for smoother updates during training. Instead of directly optimizing the reinforcement learning (RL) objectiveβwhich is computationally expensive and high-varianceβthe authors propose using a closed-form solution as a distillation target, implemented through token-level logit interpolation. This method retains the efficiency of OPSD while improving optimization stability and reasoning performance. The paper demonstrates the effectiveness of Ξ²-OPSD through experiments on mathematical reasoning benchmarks, showing consistent improvements over vanilla OPSD.
Methodology
The authors derive a closed-form solution for the optimal policy of Ξ²-OPSD, which is a geometric interpolation between a reference policy and a privileged teacher. They implement this through token-level logit interpolation, allowing for efficient training. The methodology also incorporates return-to-go credit assignment to ensure that updates are aligned with the overall sequence-level objectives.
Results
The experiments conducted on mathematical reasoning benchmarks demonstrate that Ξ²-OPSD consistently outperforms vanilla OPSD, with improvements in optimization stability and reasoning performance. For instance, on the Qwen3-1.7B model, Ξ²-OPSD achieved an average improvement of up to 9.16 percentage points across multiple benchmarks.
Implications
The findings suggest that the Ξ²-OPSD framework could lead to more robust and efficient training methods for reasoning language models, potentially enhancing their performance in various applications that require complex reasoning capabilities. This approach may also inspire further research into the integration of policy optimization techniques with self-distillation methods.
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Large Language Models
NLP
Efficient ML
- Quantization to 4-bit weights appears nearly lossless on standard metrics but amplifies existing errors in multi-turn agents.
- The flat scores observed are artifacts of the benchmark's error budget, which masks the true extent of performance degradation.
- Tightening the error budget reveals significant performance gaps, particularly in cells where quantization increases error volume.
- Error-repair prompts can effectively mitigate the damage caused by quantization in specific contexts.
Read more
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Summary
This paper investigates the effects of post-training quantization to 4-bit weights on multi-turn, tool-calling agents, challenging the prevailing notion that such quantization is nearly lossless. The authors conduct experiments using the Ο 2-bench benchmark across two model families and two domains, revealing that while standard metrics show no significant score change, quantization amplifies existing failures, particularly in tool-name hallucination. The study finds that quantization increases the frequency of these failures by up to 2.5 times without introducing new types of errors. The authors argue that the flat scores observed are misleading, as they are influenced by the benchmark's error budget, which masks the underlying damage. By tightening the error budget, the researchers expose a significant score gap, indicating that the quantization process does indeed degrade performance in specific contexts. They propose that error diagnostics should be reported alongside task rewards to provide a clearer picture of model performance.
Methodology
The authors conducted experiments using the Ο 2-bench benchmark, comparing the performance of quantized and full-precision agents across two model families (dense and MoE) in two domains. They analyzed the error rates and success under varying error budgets to assess the impact of quantization on model performance.
Results
The results indicated that while the final task scores remained statistically equivalent across quantized and full-precision models, quantization significantly increased the frequency of existing errors, particularly in tool-name hallucination. When the error budget was tightened, a substantial score gap was revealed, demonstrating that quantization does degrade performance in specific scenarios.
Implications
The findings suggest that relying solely on standard metrics to evaluate quantized models can be misleading. Researchers and practitioners should consider additional diagnostics, such as per-channel error rates and success under varying error budgets, to better understand the impact of quantization on model performance. This has implications for the deployment of large language models in real-world applications where multi-turn interactions are critical.
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Theory
Optimization
- Introduction of a dual-teacher model to improve the robustness/accuracy tradeoff in DNNs.
- The clean teacher enhances classification accuracy on clean inputs without sacrificing robustness.
- Experimental results show improved performance on CIFAR-10 and CIFAR-100 datasets compared to the original IBD.
- The proposed method is competitive with state-of-the-art approaches, particularly in harmonic mean accuracy.
Read more
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Summary
This paper addresses the vulnerability of deep neural networks (DNNs) to adversarial attacks, which compromise their reliability in real-world applications. The authors build upon the Information Bottleneck Distillation (IBD) framework, which previously improved robustness at the cost of clean accuracy. They introduce a dual-teacher model approach, incorporating a 'clean teacher' trained solely on clean inputs alongside a 'robust teacher' trained through adversarial methods. This dual-teacher setup allows for the transfer of features to a student model via a cross-layer attention mechanism. The experimental evaluation on CIFAR-10 and CIFAR-100 datasets demonstrates that the proposed method enhances clean sample classification accuracy while maintaining robust accuracy against adversarial samples. The results show that the new approach is competitive with state-of-the-art methods, particularly in balancing clean and robust accuracy, and is particularly effective for larger models. The authors also analyze the impact of different training settings on the attention module, revealing insights into the behavior of attention weights during training.
Methodology
The authors extend the IBD framework by employing two teacher models: a robust teacher trained with adversarial examples and a clean teacher trained with clean inputs. They utilize a cross-layer attention matrix to distill features from both teachers to a student model. Two distillation strategies are explored: double distillation (aligning with the robust teacher) and joint distillation (using both teachers). The impact of training settings on the attention module is also investigated.
Results
The proposed dual-teacher distillation method outperforms the original IBD in terms of clean accuracy while preserving robust accuracy against adversarial samples. It shows competitive performance against state-of-the-art methods, particularly the B-MTARD framework, especially in terms of the harmonic mean of clean and robust accuracy. The improvements are more pronounced in larger models.
Implications
This research has significant implications for enhancing the reliability of DNNs in safety-critical applications by improving their robustness against adversarial attacks while maintaining high accuracy on clean data. The dual-teacher approach could be applied in various domains where adversarial robustness is crucial.
Event-Structured Physics-Informed Neural Networks for Differentiable Critical Clearing Boundaries
Theory
Optimization
Time Series
- Introduction of ES-PINN for improved CCT estimation in power systems.
- Event-structured representation aligns with pre-fault, fault-on, and post-clearing dynamics.
- Differentiable CCT boundary extraction and local sensitivity analysis.
- Validation on IEEE test systems shows improved accuracy over traditional methods.
Read more
Event-Structured Physics-Informed Neural Networks for Differentiable Critical Clearing Boundaries
Summary
This paper introduces the Event-Structured Physics-Informed Neural Network (ES-PINN) framework aimed at improving the estimation of critical clearing time (CCT) in power systems, which is crucial for assessing transient stability after disturbances. Traditional methods for CCT estimation involve extensive simulations across various fault severities and clearing times, making them computationally expensive. The proposed ES-PINN addresses this challenge by structuring its neural network to align with the distinct phases of power system dynamics: pre-fault, fault-on, and post-clearing. This structure allows for exact state chaining across event interfaces, ensuring continuity and differentiability in the trajectory of the system's response. The framework not only provides a smooth approximation of the CCT boundary but also enables local sensitivity analysis and direct prediction of CCT through a distilled readout. The authors validate the effectiveness of ES-PINN through experiments on IEEE 9-, 14-, and 30-bus systems, demonstrating significant improvements in trajectory and stability-boundary accuracy compared to traditional neural-surrogate models. The results indicate that ES-PINN is both effective and computationally efficient, making it a promising tool for transient stability assessment in power systems.
Methodology
The ES-PINN framework employs a structured approach to model the dynamics of power systems during different fault scenarios. It uses separate neural network components for pre-fault, fault-on, and post-clearing phases, ensuring that the transition between these states is smooth and differentiable. The model is trained using physics-informed techniques that incorporate governing equations of the system, allowing for accurate trajectory learning and boundary extraction.
Results
Experiments conducted on IEEE 9-, 14-, and 30-bus systems revealed that ES-PINN consistently outperformed matched neural-surrogate baselines in terms of trajectory accuracy and stability-boundary estimation across various mechanical and electrical contingencies. The framework also demonstrated effective performance in multi-fault scenarios and showed significant computational efficiency.
Implications
The ES-PINN framework has the potential to enhance the reliability of power system operations by providing accurate and efficient assessments of transient stability. This can help in preventing generator trips and cascading outages, ultimately contributing to the stability and resilience of power grids.
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
Time Series
Generative Models
- ZUNA1.1 allows for variable-length EEG signal reconstruction, enhancing flexibility.
- The model incorporates advanced training techniques, including implicit augmentation and quality-aware preprocessing.
- ZUNA1.1 significantly outperforms traditional EEG denoising methods.
- The training corpus was expanded to approximately 3.5M channel-hours, improving model robustness.
Read more
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
Summary
The paper introduces ZUNA1.1, a 380M-parameter diffusion autoencoder designed for flexible EEG signal reconstruction. This model addresses limitations of its predecessor, ZUNA1, by allowing for variable-length sequence reconstruction (up to 30 seconds) and handling arbitrary numbers of EEG channels and scalp locations. ZUNA1.1 improves upon ZUNA1 by incorporating variable-length training, implicit data augmentation, quality-aware preprocessing, and a diverse dropout mixture, which collectively enhance its ability to reconstruct EEG signals affected by noise and artifacts. The model outperforms traditional EEG denoising methods, such as spherical spline interpolation, and is trained on a significantly larger dataset, increasing from approximately 2M to 3.5M channel-hours. ZUNA1.1 is released as open-source software under the Apache 2.0 license, promoting accessibility for further research and application in EEG signal processing.
Methodology
ZUNA1.1 employs a diffusion autoencoder architecture and utilizes variable-length training on randomly cropped EEG windows. It implements implicit data augmentation through multiple transformations and quality-aware preprocessing to enhance signal reconstruction. The model also utilizes a diverse dropout mixture to simulate realistic missing channel scenarios during training.
Results
ZUNA1.1 demonstrates superior performance in EEG signal reconstruction compared to both its predecessor ZUNA1 and traditional methods like spherical spline interpolation. The model's ability to handle variable-length sequences and various dropout patterns allows it to effectively reconstruct signals from consumer-grade EEG devices, which often suffer from noise and artifacts.
Implications
The advancements in ZUNA1.1 have significant implications for clinical, research, and consumer applications of EEG technology. Its flexibility and robustness can improve the quality of EEG data analysis, enabling better insights into brain activity and enhancing the usability of consumer-grade EEG devices.
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Time Series
Efficient ML
- Introduces S-CEReBrO, a new architecture for continuous EEG monitoring.
- Utilizes Windowed Alternating Attention to maintain constant memory usage.
- Processes signals 100Γ longer than traditional full self-attention models.
- Achieves state-of-the-art performance on multiple EEG analysis tasks.
Read more
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Summary
The paper presents S-CEReBrO (Streaming CEReBrO), an advanced architecture for continuous EEG monitoring that addresses the memory limitations of Transformer-based models. Traditional global attention mechanisms in Transformers lead to memory overflow when processing long-duration EEG signals. S-CEReBrO introduces a novel Windowed Alternating Attention mechanism that divides attention computation into fixed-size spatiotemporal windows, ensuring a constant memory footprint regardless of signal duration. This allows the model to handle significantly longer signalsβup to 100 times longer than full self-attention modelsβwhile requiring 55% less memory than low-rank linear attention and achieving 2.1 times higher inference throughput. Pre-trained on over 25,000 hours of EEG recordings from more than 12,000 subjects, S-CEReBrO demonstrates state-of-the-art performance on 7 out of 11 downstream tasks, utilizing up to 60% fewer parameters than existing models. This work marks a significant advancement in the field of EEG analysis, facilitating efficient and scalable continuous monitoring applications.
Methodology
The S-CEReBrO architecture employs a Windowed Alternating Attention mechanism that interleaves spatial and temporal attention blocks, allowing for locality-driven attention while maintaining a constant peak resident memory. The model processes EEG signals through a tokenization process that includes patching, feature projection, and the use of spatiotemporal embeddings, followed by masked autoencoding for representation learning.
Results
S-CEReBrO achieves state-of-the-art performance on 7 out of 11 public benchmarks in EEG analysis, including clinical diagnostics and brain-computer interface tasks. The model can handle continuous monitoring for up to 14 hours, with a computational complexity of O(CT) and a memory requirement significantly lower than existing models.
Implications
The advancements presented in S-CEReBrO have significant implications for the future of EEG monitoring, particularly in clinical settings where continuous, long-term data analysis is crucial. The model's efficiency and scalability could enable more widespread use of EEG technology in real-time applications, such as seizure detection and brain-computer interfaces.
The Kinetics of Training: A Driven-Nucleation Rate Law for Emergence, Plasticity Loss, and Circuit Control in Language Models
NLP
Large Language Models
Theory
- Introduces a driven-nucleation rate law for understanding capability emergence in language models.
- Demonstrates that partial credit for incomplete circuit alignments is ineffective in capability formation.
- Establishes a quantitative framework for predicting the emergence of capabilities and plasticity loss.
- Provides insights into the process control of training dynamics, including annealing and selective dissolution.
Read more
The Kinetics of Training: A Driven-Nucleation Rate Law for Emergence, Plasticity Loss, and Circuit Control in Language Models
Summary
This paper presents a novel framework for understanding the kinetics of training in language models, focusing on the emergence of capabilities, plasticity loss, and circuit control. The author introduces a driven-nucleation rate law that describes how capabilities in language models emerge when the last components of a circuit align during stochastic attempts. The study reveals that the rate-limiting step in capability formation is the joint alignment of circuit components, where partial credit for incomplete alignments is ineffective. Through experiments with the Pythia model, the author demonstrates that ablating a single part of a circuit significantly reduces capability retention, contradicting predictions made by partial credit theories. The paper also derives a rate equation that captures the dynamics of capability emergence, plasticity loss, and process control, providing insights into how training can be optimized. The findings suggest that the timing and conditions under which capabilities emerge can be quantitatively predicted, offering a new perspective on training dynamics and the management of model capabilities.
Methodology
The author employs a combination of theoretical derivation and empirical analysis using the Pythia language model. The study derives a rate equation based on stochastic dynamics of Stochastic Gradient Descent (SGD) and validates the findings through controlled experiments. The methodology includes measuring independent factors, analyzing the effects of circuit component ablation, and testing the derived equations under various conditions.
Results
The study finds that ablating one part of a circuit results in a median retention of 17% of capability, significantly lower than the 50-83% predicted by partial credit theories. The derived rate equation effectively captures the dynamics of capability emergence and plasticity loss, with the ability to predict the timing of capability emergence based on training conditions. The paper also demonstrates that re-initializing specific components can restore learnability, providing a mechanism for addressing plasticity loss.
Implications
The findings have significant implications for the training of language models, suggesting that understanding the kinetics of training can lead to improved strategies for capability development and retention. The proposed framework can inform the design of training schedules and data mixing strategies, ultimately enhancing the performance and adaptability of language models.
Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models
Efficient ML
- Introduces a leakage-aware framework for Ethereum actor classification.
- Proposes a Blind-Spot protocol to eliminate shortcuts from high-signal contracts.
- Demonstrates that tree-based models outperform deep learning models in efficiency and performance.
- Highlights the structural complexity differences between organic users and automated actors.
Read more
Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models
Summary
This paper addresses the challenge of detecting Sybil bots in Ethereum blockchain analytics, which mimic legitimate users to exploit rewards and governance mechanisms. The authors critique existing deep learning methods that treat blockchain transaction histories as quasi-linguistic sequences, highlighting issues of computational expense and inflated performance due to label leakage from high-signal smart contracts. They propose a leakage-aware framework for classifying Ethereum actors, incorporating a Blind-Spot protocol to mitigate the influence of high-signal contracts and a Transaction Grammar representation to model wallet behavior. The study compares the performance of Transformer and BiLSTM sequential models against XGBoost and SVM baselines under controlled conditions. The findings reveal that while organic users exhibit greater structural complexity than automated actors, tree-based models like XGBoost outperform deep learning models in terms of efficiency and performance, suggesting a critical trade-off for real-time blockchain analytics. This work contributes a novel framework for leakage-aware classification and emphasizes the importance of evaluating models without the influence of label leakage.
Methodology
The authors developed a leakage-aware detection framework that includes a Blind-Spot protocol to reduce the influence of high-signal smart contracts and a Transaction Grammar representation to analyze wallet behavior. They conducted a comparative evaluation of Transformer and BiLSTM models against XGBoost and SVM under controlled conditions to assess performance and efficiency.
Results
The results indicate that XGBoost outperforms Transformer-based models in terms of accuracy, latency, and energy efficiency when evaluated under a leakage-aware setting. The study also finds that organic users have higher structural entropy compared to automated actors, but attention-based models do not fully exploit this complexity.
Implications
The findings suggest that tree-based models may be more suitable for real-time blockchain analytics, providing a more efficient alternative to deep learning approaches. This has implications for the design of Sybil detection systems and the broader field of blockchain analytics, emphasizing the importance of avoiding label leakage in model evaluations.
Multi-channel Uplift Policy Learning
Optimization
- ReAlloc formulates multi-channel budget allocation as a compositional uplift problem on a simplex.
- The framework separates causal response learning from decision-making through a conservative marginal field.
- ReAlloc has been successfully deployed in Taobao's system, leading to measurable improvements in business metrics.
- The proposed method addresses the limitations of traditional PTO approaches in multi-channel settings.
Read more
Multi-channel Uplift Policy Learning
Summary
This paper addresses the challenge of allocating fixed marketing budgets across multiple channels in e-commerce to maximize business utility, specifically focusing on uplift policy learning. Traditional predict-then-optimize (PTO) approaches are inadequate due to issues like observational confounding and extrapolation errors. The authors propose a novel framework called ReAlloc, which formulates the problem as a simplex-constrained uplift decision problem. ReAlloc employs a two-part structure: an Orthogonal Teacher that extracts unbiased local gradients from short-term logs, and an Explanation-Guided Student that distills these gradients into a structured marginal field for long-term decision-making. This approach allows for support-aware and conservative decisions that effectively manage cross-channel substitutions. Extensive simulations and large-scale online A/B tests conducted on the Taobao platform demonstrate that ReAlloc significantly improves both pay order and income, outperforming existing uplift and PTO baselines in terms of decision quality and counterfactual ranking.
Methodology
The methodology involves a two-part framework: an Orthogonal Teacher that removes confounding effects to recover unbiased local gradients, and an Explanation-Guided Student that distills these gradients into a structured marginal field. This allows for conservative decision-making within the observed support, facilitating precise optimization under budget constraints.
Results
The results from extensive simulations and large-scale online A/B tests on the Taobao platform indicate that ReAlloc achieves a 3.53% improvement in pay order and a 3.26 percentage point increase in income, demonstrating its effectiveness over traditional uplift and PTO methods.
Implications
The findings suggest that ReAlloc can enhance decision-making processes in e-commerce platforms by optimizing resource allocation across multiple marketing channels. This has significant implications for improving business outcomes and maximizing the utility of marketing budgets.
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
Optimization
Federated Learning
Theory
- Introduction of F2CTO, the first method for distributed robust coreset selection.
- Formulation of the distributed robust coreset selection problem as a constrained trilevel optimization task.
- Demonstration of a non-asymptotic convergence rate of O(Ο΅β3/2) for the proposed method.
- Integration of coreset selection, robust optimization, and distributed learning into a unified framework.
Read more
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
Summary
This paper addresses the challenge of robust coreset selection in distributed networks, particularly in the context of the Internet of Things (IoT), where vast amounts of data are generated. The authors propose a novel framework called Federated First-order Constrained Trilevel Optimization (F2CTO) to tackle the limitations of existing methods that either rely on centralized data or do not consider robustness. The work formulates the problem as a constrained trilevel optimization task, integrating coreset selection, robust optimization, and distributed learning. The F2CTO framework employs a hierarchical composite value-function reformulation and a distributed alternating projected gradient algorithm to solve the optimization problem efficiently. The authors prove that their method achieves a non-asymptotic convergence rate of O(Ο΅β3/2) for finding an Ο΅-stationary point, marking a significant advancement in distributed optimization methodologies. Empirical evaluations demonstrate the effectiveness and efficiency of F2CTO in real-world scenarios, particularly in continual learning applications.
Methodology
The authors develop the Federated First-order Constrained Trilevel Optimization (F2CTO) framework, which reformulates the coreset selection problem into a trilevel optimization problem with hierarchical constraints. They utilize a hierarchical composite value-function approach and implement a distributed alternating projected gradient algorithm to solve the optimization problem efficiently.
Results
The proposed F2CTO framework achieves a non-asymptotic convergence rate of O(Ο΅β3/2), demonstrating its effectiveness in finding Ο΅-stationary points. Empirical evaluations indicate that F2CTO significantly improves the efficiency of robust coreset selection in distributed environments, particularly for continual learning tasks.
Implications
The findings suggest that F2CTO can be applied in various domains requiring efficient data processing in distributed networks, such as IoT applications, continual learning, and scenarios where data privacy is a concern. The framework's ability to enhance model robustness while maintaining data privacy opens new avenues for research and practical applications in distributed machine learning.
Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches
Large Language Models
Time Series
- Classical sequence models (e.g., LSTMs) excel in next activity prediction tasks.
- Tabular foundation models demonstrate competitive performance in temporal prediction tasks.
- Large language models (LLMs) generally underperform compared to sequence models despite their higher complexity and cost.
- The study highlights the need for systematic benchmarking of different modeling paradigms in PPM.
Read more
Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches
Summary
This paper investigates the landscape of Predictive Process Monitoring (PPM) by comparing three modeling paradigms: classical sequence models, large language models (LLMs), and tabular foundation models. PPM utilizes event logs to predict future process behaviors, such as the next activity or remaining time until completion. While deep sequence models, particularly LSTMs, have dominated PPM research, the emergence of foundation models, especially LLMs and tabular models with in-context learning, presents new opportunities. The authors conduct a controlled empirical benchmark across five real-world datasets and three prediction tasks (next activity, remaining time, and time until the next event). The findings reveal that sequence models consistently outperform others in next activity prediction, while tabular foundation models show competitive performance in temporal tasks. LLMs, despite their higher costs, generally lag behind in performance. The paper also provides insights into the performance differences, emphasizing the challenges associated with branching-related predictions and model-specific failure patterns.
Methodology
The authors conducted a controlled empirical benchmark comparing classical sequence models, LLM-based approaches adapted via Low-Rank Adaptation (LoRA), and tabular foundation models using in-context learning. They evaluated these models across five real-world event logs and three prediction tasks (next activity, remaining time, and time until the next event).
Results
The results indicate that sequence models consistently perform best for next activity prediction, while tabular foundation models are competitive for temporal tasks. LLMs, despite their advanced capabilities, typically lag behind in predictive performance and are more costly to implement.
Implications
The findings suggest that while LLMs and tabular models are promising, classical sequence models remain the most effective for certain PPM tasks. This has implications for practitioners in process mining and operational settings, where timely predictions can lead to proactive decision-making. The insights into model performance can also inform future research directions in PPM.
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Efficient ML
- ROCS improves inference efficiency by deferring request-candidate interactions.
- Generalized Layer Masking (GLM) and Deep Cross Attention (DCA) are key innovations.
- Achieves up to 3Γ QPS improvement without quality degradation.
- Successfully deployed in Meta's recommendation systems, enhancing performance.
Read more
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Summary
The paper presents ROCS (Request-Oriented Compute Sharing), a novel modeling and inference paradigm designed to enhance the efficiency of large-scale recommendation systems. Traditional recommendation models often struggle with production cost constraints, limiting their scalability despite the potential for improved prediction quality through increased computational resources. ROCS addresses this issue by leveraging the unique characteristics of recommendation inference, where a single user request is evaluated against multiple candidate items. By deferring request-candidate interactions and isolating candidate-dependent representations, ROCS allows for substantial portions of the model to be evaluated once per request, rather than for each candidate, leading to significant improvements in inference efficiency without sacrificing prediction quality. The authors introduce two key components: Generalized Layer Masking (GLM), which enforces candidate isolation in feature-interaction architectures, and Deep Cross Attention (DCA), which extends this request-oriented sharing to sequence architectures. Additionally, they propose In-Kernel Broadcast Optimization (IKBO) for efficient GPU deployment. Experimental results demonstrate that ROCS consistently enhances the quality-efficiency tradeoff across various recommendation backbones, achieving up to a 3Γ improvement in queries per second (QPS) for retrieval models and a 0.5% relative LogLoss improvement with a 50% QPS gain on a short-form video ranking model. ROCS has been successfully deployed in Meta's large-scale recommendation systems, resulting in significant online performance gains and reduced infrastructure costs.
Methodology
The methodology involves a modeling-driven approach that utilizes ROCS to maximize the reuse of request-side computations across multiple candidates. Key components include GLM for feature-interaction architectures and DCA for sequence architectures, along with IKBO for GPU optimization.
Results
ROCS demonstrated significant improvements in efficiency and prediction quality across various benchmarks, achieving up to a 3Γ increase in QPS for retrieval models and a 0.5% relative LogLoss improvement with a 50% QPS gain on a video ranking model.
Implications
The ROCS framework can lead to more efficient large-scale recommendation systems, reducing infrastructure costs while maintaining or improving prediction accuracy. Its deployment in Meta's systems suggests potential for widespread application in various recommendation contexts.
GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios
Generative Models
Reinforcement Learning
Computer Vision
- Introduction of DocMeetSVG-100K, a large-scale dataset for complex SVG generation.
- Development of GVR-Coder, a framework that integrates layout constraints and visual feedback.
- Utilization of curriculum-driven rejection sampling for improved model training.
- Implementation of reinforcement learning to enhance structural and aesthetic quality of diagrams.
Read more
GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios
Summary
The paper introduces GVR-Coder, a novel framework aimed at generating structured Scalable Vector Graphics (SVG) diagrams from verbose professional texts, addressing the challenges of high cognitive load in document and meeting scenarios. The authors identify three main obstacles in current Text-to-SVG generation: the lack of datasets for complex diagrams, absence of explicit layout priors, and insufficient visual feedback mechanisms. To overcome these issues, they present DocMeetSVG-100K, a large-scale dataset specifically designed for document authoring and meeting reviews. GVR-Coder employs a curriculum-driven rejection sampling fine-tuning approach to enhance the model's ability to handle complex structures while integrating layout constraints during training. Additionally, the framework utilizes reinforcement learning from dual rendering feedback to optimize both structural complexity and visual aesthetics. A generate-verify-repair agent loop is also introduced to refine the generated outputs through targeted feedback. Experimental results demonstrate that GVR-Coder significantly outperforms existing methods, producing diagrams that are both logically coherent and visually appealing, thus facilitating better information communication in professional settings.
Methodology
The methodology involves creating a large dataset (DocMeetSVG-100K) for training, employing a curriculum-driven rejection sampling fine-tuning approach, and integrating reinforcement learning with dual rendering feedback. The framework also includes a generate-verify-repair agent loop for iterative refinement of generated SVGs.
Results
GVR-Coder outperformed competitive baselines in generating structured SVG diagrams, achieving higher logical coherence and visual appeal. The extensive experiments validated the effectiveness of the proposed methods in addressing the identified challenges in Text-to-SVG generation.
Implications
The findings suggest that GVR-Coder can significantly enhance the efficiency of information communication in professional environments by transforming complex textual information into clear visual representations. This has potential applications in document authoring, meeting reviews, and any scenario requiring effective data visualization.
Building a User Foundation Model for the Open Web
Optimization
Theory
Time Series
- Introduces a user foundation model tailored for fragmented user identities in open-web RTB.
- Utilizes self-supervised learning to improve user representation from sparse browsing histories.
- Demonstrates significant performance improvements in click prediction and bid win-rate models.
- Implements an LLM-in-the-loop optimization strategy for encoder pre-training.
Read more
Building a User Foundation Model for the Open Web
Summary
This paper addresses the challenges of applying user foundation models in open-web real-time bidding (RTB) environments, where user identity is fragmented and browsing histories are often sparse. The authors propose a user foundation model (UFM) that utilizes self-supervised learning on user browsing sequences to enhance downstream tasks such as click prediction. The model is pre-trained using masked language modeling and a sequence-level contrastive objective, followed by fine-tuning for specific tasks. The study demonstrates that the UFM can effectively operate under the constraints of open-web RTB, achieving significant improvements in performance metrics across various models. The authors also introduce an LLM-in-the-loop optimization approach for enhancing the encoder's pre-training pipeline, showcasing the potential for systematic gains in production settings. The results indicate that the learned representations generalize well across different prediction tasks, confirming the viability of self-supervised learning in this context.
Methodology
The authors pre-train a Transformer encoder using masked language modeling and a sequence-level contrastive objective on user browsing histories. The model is then fine-tuned for click prediction tasks. An LLM-in-the-loop search is employed to optimize the encoder's pre-training pipeline, enhancing the model's performance in real-time bidding scenarios.
Results
The user foundation model achieved a +1.197% relative improvement in gain (RIG) on the bid win-rate model and +1.354% RIG on the click-through rate (CTR) ranker. A 7-day live A/B test confirmed a +2.13% increase in CTR and a -1.13% decrease in effective cost per click (eCPC), with 80% confidence intervals excluding zero for both metrics.
Implications
The findings suggest that self-supervised learning can effectively enhance user representation in environments with fragmented data, potentially leading to improved performance in various recommendation and bidding systems. This approach may also inform future research on user modeling and real-time decision-making in online advertising.
Neural Network-Assisted CLEAN for Channel Modeling in Low-SNR Regimes
Efficient ML
Theory
- NN-CLEAN combines the strengths of traditional MLE methods and deep learning for improved parameter estimation.
- The framework significantly reduces computational complexity compared to exhaustive grid search methods.
- NN-CLEAN exhibits robust generalization capabilities in Out-of-Distribution scenarios, addressing limitations of standalone deep learning models.
- The method maintains efficient runtime and memory usage, making it suitable for real-time applications.
Read more
Neural Network-Assisted CLEAN for Channel Modeling in Low-SNR Regimes
Summary
This paper introduces Neural Network-Assisted CLEAN (NN-CLEAN), a hybrid framework designed to improve multipath parameter estimation in low Signal-to-Noise Ratio (SNR) environments, which is crucial for modern wireless communication systems. Traditional Maximum Likelihood Estimation (MLE) methods like CLEAN are effective but computationally expensive due to exhaustive grid searches. In contrast, purely data-driven deep learning approaches often lack physical grounding and struggle with generalization in variable multipath conditions. NN-CLEAN addresses these issues by integrating a multi-head residual network into the CLEAN extraction loop, replacing the exhaustive search with efficient forward passes while maintaining the mathematical rigor of residual subtraction. Extensive Monte Carlo simulations demonstrate that NN-CLEAN achieves over 96% estimation accuracy at 5 dB SNR, matching the performance of traditional Grid-Search CLEAN (GS-CLEAN) while significantly reducing computational complexity. The framework also shows consistent execution runtime and memory usage as batch sizes increase, making it a robust solution for real-time channel estimation in MIMO systems.
Methodology
NN-CLEAN integrates a multi-head residual network into the iterative CLEAN extraction process, replacing exhaustive grid searches with rapid forward passes to isolate dominant multipath parameters. The framework uses exact mathematical models for residual subtraction, ensuring physical accuracy and preventing non-physical errors.
Results
NN-CLEAN achieves over 96% estimation accuracy at 5 dB SNR, comparable to GS-CLEAN, while offering a substantial reduction in computational complexity. The method demonstrates a near-flat scaling in execution time and memory consumption with increasing batch sizes.
Implications
The NN-CLEAN framework has significant implications for real-time channel estimation in MIMO systems, particularly in challenging low-SNR environments. Its efficiency and robustness make it suitable for applications in Integrated Sensing and Communication (ISAC) and advanced spatial multiplexing.
Latent-Kernel Discrete Flow Maps for Few-Step Generation
NLP
Generative Models
Large Language Models
- Introduction of Latent-Kernel Discrete Flow Maps (LKF) for few-step generation.
- LKF allows for correlated token generation by using a shared latent variable.
- Significant improvements in generative perplexity over traditional likelihood models.
- Outperforms distilled and rectified few-step samplers without relying on teacher models.
Read more
Latent-Kernel Discrete Flow Maps for Few-Step Generation
Summary
This paper introduces Latent-Kernel Discrete Flow Maps (LKF), a novel approach to few-step generation in discrete diffusion and flow-matching models. Traditional models often treat token updates independently, which can lead to inconsistencies when generating correlated tokens, such as subjects and verbs in sentences. The authors propose a method that incorporates a shared latent variable to couple the generation of these tokens while maintaining computational efficiency. The LKF model is a mixture of M factorized components, where each component is conditioned on a single latent variable, allowing for correlated token generation in a single step. The authors demonstrate that this approach significantly improves generative performance on text benchmarks, achieving a perplexity reduction of 2.1Γ to 3.3Γ compared to likelihood baselines, and outperforming existing few-step methods that rely on teacher models. The results indicate that the model's performance improves with the number of components M, suggesting that LKF can effectively capture complex dependencies in text generation tasks.
Methodology
The LKF model is constructed as a mixture of M factorized components, each conditioned on a shared latent variable. This allows the model to maintain the efficiency of independent updates while capturing correlations between tokens. The authors provide an exact information decomposition to validate the correlation captured by the latent variable and test the model on synthetic datasets as well as real-world benchmarks (One-Billion-Word and WikiText-103).
Results
The LKF model demonstrated a generative perplexity improvement of approximately 2.1Γ to 3.3Γ over likelihood baselines, with the most significant gains observed at M=8, where it surpassed existing few-step generation methods. The model effectively learned heterogeneous components and maintained diversity in generated outputs.
Implications
The findings suggest that LKF can enhance the efficiency and quality of text generation models, making it a valuable approach for applications in natural language processing, particularly in tasks requiring coherent and contextually appropriate token generation.
RLPF: Reinforcement Learning from Performance Feedback for Code Generation
Reinforcement Learning
Large Language Models
Optimization
- RLPF introduces a staged reward system for training code generation models, focusing on both correctness and efficiency.
- The method significantly improves the performance of the Qwen3-32B model on the PerfCodeBench dataset.
- RLPF provides useful feedback for both failed and successful code generation attempts, enhancing learning stability.
- The approach demonstrates that code agents can be trained to optimize the programs they generate, not just to pass tests.
Read more
RLPF: Reinforcement Learning from Performance Feedback for Code Generation
Summary
The paper introduces RLPF (Reinforcement Learning from Performance Feedback), a novel approach to training code generation models that emphasizes not only correctness but also efficiency. Traditional training methods focus on whether generated code passes tests, which can overlook significant differences in runtime performance. RLPF addresses this gap by providing a staged reward system that offers feedback based on execution progress for failed programs and efficiency feedback for correct programs. This dual feedback mechanism allows the model to learn from both unsuccessful and successful attempts, ultimately guiding it towards generating faster implementations. The authors demonstrate the effectiveness of RLPF by fine-tuning the Qwen3-32B model on the PerfCodeBench dataset, resulting in a substantial increase in the rate of correct and runnable solutions from 11.1% to 54.6% and an improvement in relative efficiency from 8.1% to 38.6%. The findings suggest that RLPF not only enhances the performance of code generation models but also enables them to prioritize efficiency, making them competitive with leading systems in the field.
Methodology
The authors propose a reinforcement learning framework that utilizes a staged reward system. This system ranks failed programs based on their execution progress and correct programs based on their efficiency improvements relative to a baseline and expert reference. The framework is implemented on the PerfCodeBench dataset, allowing for comprehensive evaluation of the model's performance.
Results
The application of RLPF to the Qwen3-32B model resulted in a significant increase in the correct-and-runnable solution rate from 11.1% to 54.6% and an improvement in relative efficiency from 8.1% to 38.6%. The trained model showed competitive performance against stronger open-weight systems and demonstrated some transferability to the EffiBench-X benchmark.
Implications
The findings suggest that RLPF can enhance the capabilities of code generation models, making them more effective in real-world software development scenarios where both correctness and efficiency are critical. This approach could lead to the development of more advanced code generation tools that prioritize performance, potentially transforming software engineering practices.
THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
Graph Learning
- THGFM introduces a dual-branch architecture for temporal heterogeneous graph learning.
- The model effectively combines shared cross-type transformations with relation-specific parameters.
- Rotary Temporal Attention is utilized to enhance the integration of temporal information.
- Type-Conditioned Non-Competitive Gated Sum Fusion allows for flexible feature integration.
Read more
THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
Summary
The paper introduces THGFM, a novel model designed for learning on temporal heterogeneous graphs, which are complex structures that evolve over time and consist of various node and relation types. Existing methods struggle to balance cross-type transfer and relation-aware specialization, often treating temporal information as an additive feature rather than integrating it into the core attention mechanism. THGFM addresses these challenges through a dual-path architecture comprising a Shared-Space Temporal Attention (SSTA) branch for efficient cross-type transfer and a Relational Type-Partitioned Temporal Attention (RTTA) branch for specialized relation handling. The model employs a Type-Conditioned Non-Competitive Gated Sum Fusion (TC-NGSF) mechanism, allowing independent amplification or suppression of features from both branches, thus enhancing the model's adaptability to different contexts. Additionally, THGFM incorporates Rotary Temporal Attention to directly integrate relative time into the attention scores, improving the relevance of temporal relationships. The model demonstrates significant improvements over baseline graph transformer models, achieving a mean gain of +3.25% across six tasks on academic graph benchmarks, with notable peak gains on specific datasets.
Methodology
THGFM employs a dual-path architecture that includes a Shared-Space Temporal Attention branch for parameter-efficient cross-type transfer and a Relational Type-Partitioned Temporal Attention branch for relation-aware specialization. The model integrates these branches using Type-Conditioned Non-Competitive Gated Sum Fusion, which allows independent feature-wise gating. Rotary Temporal Attention is introduced to incorporate relative time into the attention mechanism directly.
Results
THGFM consistently outperformed baseline graph transformer models, achieving a +3.25% mean gain across six tasks on academic graph benchmarks. Peak relative gains were observed at +12.37% on OAG-CS PV, +4.87% on PF-L2, and +1.18% on PF-L1, as well as +4.24%, +3.73%, and +4.61% on OGBN-MAG, HTAG-ArXiv, and HTAG-DBLP, respectively.
Implications
The advancements presented by THGFM could significantly enhance applications in various domains that utilize temporal heterogeneous graphs, such as academic research, e-commerce, and knowledge graph construction. The model's ability to efficiently handle diverse node types and temporal dynamics can lead to improved performance in recommendation systems, citation analysis, and dynamic network analysis.
Flux-OPD: On-Policy Distillation with Evolving Contexts
NLP
Large Language Models
Reinforcement Learning
- Flux-OPD uses evolving contexts as in-training supervision to improve task preference capture.
- The paper provides a decomposition of the reverse KL objective, revealing insights into the distillation process.
- Contextual corrections and weighting strategies are introduced to stabilize training and manage conflicts.
- Experiments show that Flux-OPD outperforms existing OPD methods across various tasks.
Read more
Flux-OPD: On-Policy Distillation with Evolving Contexts
Summary
The paper introduces Flux-OPD, a novel On-Policy Distillation (OPD) paradigm that leverages evolving contexts as in-training supervision to enhance the performance of large language models in open-ended domains. Traditional training methods struggle with the lack of verifiable rewards, making it challenging to formalize task preferences. The authors analyze the impact of contexts through a decomposition of the reverse KL objective, revealing that the student model is distilled towards the geometric mean of context-conditioned teacher distributions, while also identifying a conflict term that captures discrepancies among these teachers. Flux-OPD addresses the instability caused by directly using evolving contexts by introducing contextual corrections and weighting strategies, which stabilize the distillation target and mitigate conflicts. The methodology involves iterative training that alternates between context extraction from student trajectories and context distillation. The proposed method is evaluated on two real-world tasks: prompt optimization for video generation and medical question answering, demonstrating superior performance compared to existing OPD paradigms. Overall, the study highlights the potential of combining teacher supervision with evolving contexts to effectively capture complex task preferences in open-ended domains.
Methodology
The methodology involves an iterative training process that consists of context extraction from student trajectories followed by context distillation. Two key strategies are employed: contextual correction, which anchors the distillation target to a stable context-free teacher while injecting contextual difference signals, and contextual weighting, which adjusts the strength of corrections based on a conflict term to manage conflicting distributions.
Results
The experiments conducted on prompt optimization for video generation and medical question answering demonstrate that Flux-OPD consistently outperforms existing OPD paradigms, indicating its effectiveness in capturing complex preferences and improving model performance in open-ended tasks.
Implications
The findings suggest that integrating evolving contexts into the training process can significantly enhance the ability of large language models to adapt and perform in dynamic environments, potentially leading to more robust AI systems capable of handling complex, real-world tasks.
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Reinforcement Learning
Interpretability
- Introduces class-aware reinforcement learning for counterfactual explanation generation.
- Demonstrates improved convergence speed and reward optimization compared to class-blind methods.
- Generates significantly more valid counterfactual explanations.
- Highlights the importance of class-based features in the action-selection process.
Read more
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Summary
This paper presents a novel approach to generating counterfactual explanations (CFEs) using class-aware reinforcement learning (RL). CFEs are crucial for enhancing the interpretability of black-box machine learning models by providing alternative instances with modified feature values that lead to different outcomes. The authors argue that incorporating the predicted class of an instance into the RL state representation can improve the efficiency of exploration and the optimality of the generated policies. The study contrasts the proposed class-aware RL method with a class-blind RL approach, which does not utilize class information. Experiments conducted on seven diverse datasets demonstrate that the class-aware method significantly accelerates convergence, optimizes rewards, and reduces episode lengths during training. Additionally, it generates a higher number of valid CFEs compared to the class-blind method. The findings highlight the importance of class-awareness in RL for CFE generation, as the class-based features consistently rank among the most influential predictors in action selection, as evidenced by SHAP and LIME analyses. Overall, this research contributes to the field of explainable AI by providing a more effective framework for generating actionable counterfactuals across various domains.
Methodology
The authors model the CFE generation process as a Markov Decision Process (MDP), incorporating both features from the predictors and the predicted class of the instance into the RL state representation. The RL agent interacts with a designed environment for each dataset, where it learns to navigate the feature space to generate CFEs through a learned policy.
Results
The class-aware RL method showed faster convergence, better reward optimization, and a reduction in episode lengths during training. It also produced a significantly higher number of valid CFEs compared to the class-blind RL approach, confirming the hypothesis that class-awareness enhances the efficiency and effectiveness of the CFE generation process.
Implications
The findings suggest that incorporating class information in reinforcement learning can lead to more effective and interpretable machine learning models, particularly in sensitive applications such as healthcare and finance where understanding model decisions is critical.
Benchmarking the Residual: What Long-Horizon Evaluations Add Beyond Matched Short-Task Performance
NLP
Large Language Models
Theory
- Long-horizon evaluations reveal performance degradation but do not explain the underlying causes of failure.
- The concept of 'horizon residual' is introduced to quantify the difference between actual and predicted task success.
- Benchmark design should differentiate between task size and the complexity of individual stages.
- A clear diagnostic evaluation protocol is proposed to better understand performance declines in long-horizon tasks.
Read more
Benchmarking the Residual: What Long-Horizon Evaluations Add Beyond Matched Short-Task Performance
Summary
This position paper discusses the challenges of evaluating language-model agents on long-horizon tasks, where performance tends to decline as task length increases. The authors introduce the concept of 'trajectory-induced degradation,' which refers to the compounding of errors and the increasing difficulty of tasks due to accumulated context. They argue that to accurately assess long-horizon failures, benchmarks must compare actual full-task success against predictions derived from short, individual stages. This comparison yields a 'horizon residual' that highlights the difference between expected and actual performance. The paper emphasizes the need for clear benchmark design that distinguishes between task size and the difficulty of individual stages, as well as the interdependencies between them. The authors propose a diagnostic evaluation protocol that involves running the same agent on short, verifiable stages to create a baseline prediction for full tasks, which can then be compared to actual performance. This approach aims to clarify the reasons behind performance declines and guide future research in understanding the underlying causes of long-horizon failures.
Methodology
The authors propose a diagnostic evaluation protocol that involves running agents on short, verifiable stages to create a baseline prediction for full tasks. They introduce the concept of 'horizon residual' to compare actual performance against this baseline, allowing for a clearer understanding of performance degradation.
Results
The paper illustrates that a decline in end-to-end success rates on longer tasks does not necessarily indicate a unique long-horizon failure mechanism. Instead, it may result from ordinary error compounding. The horizon residual provides a more informative measure of performance discrepancies, guiding further investigation into the causes of these failures.
Implications
The findings suggest that better benchmark design and evaluation protocols can enhance the understanding of agent performance in complex tasks, leading to improved deployment strategies and more effective training methodologies for language models.
TopoFormer: Topology Meets Attention for Graph Learning
Graph Learning
- TOPOFORMER introduces a scalable method for encoding topological structures into attention-ready sequences.
- The Topo-Scan module allows for efficient processing of graphs without relying on heavy preprocessing or node embeddings.
- The framework bridges topological data analysis and deep learning, capturing both local and global graph structures.
- TOPOFORMER demonstrates strong performance on benchmark tasks, including graph classification and molecular property prediction.
Read more
TopoFormer: Topology Meets Attention for Graph Learning
Summary
The paper introduces TOPOFORMER, a novel framework for graph representation learning that integrates topological structures into attention-based models. Central to this framework is the Topo-Scan module, which transforms graphs into short, ordered sequences of topological tokens by utilizing node or edge filtrations. This approach captures multi-scale structural patterns, enabling the processing of both local motifs and global organization through a Transformer architecture. Unlike traditional persistent homology methods, Topo-Scan is designed to be parallelizable, avoiding the computationally expensive steps associated with persistence diagrams. The authors provide theoretical guarantees on the stability of the topological encodings and demonstrate that TOPOFORMER achieves state-of-the-art performance on graph classification and molecular property prediction tasks. The results indicate that TOPOFORMER not only matches but often exceeds the performance of existing graph neural networks (GNNs) and topology-based methods while maintaining efficient computational requirements. This work paves the way for future research on integrating topological inductive biases into attention-based frameworks for graph learning.
Methodology
The methodology involves the development of the Topo-Scan module, which decomposes graphs into sequences of topological tokens across multiple resolutions. These sequences are processed by a Transformer model, allowing for efficient graph-level embeddings. The approach avoids traditional persistent homology pipelines, focusing instead on parallelizable computations that integrate seamlessly with deep learning architectures.
Results
TOPOFORMER achieves state-of-the-art performance on various benchmarks for graph classification and molecular property prediction, outperforming strong GNN and topology-based baselines. The framework provides predictable and efficient computational performance, making it practical for large-scale applications.
Implications
The integration of topological data analysis with attention-based models in TOPOFORMER opens new avenues for developing general-purpose graph learning models. This approach could enhance the understanding of complex relational data in fields such as biology, chemistry, and social networks, potentially leading to more robust and interpretable machine learning applications.
Cybersecurity Detection Classification with Reasoning-enabled Language Models
NLP
Large Language Models
Reinforcement Learning
- First detection-triage classifier trained to utilize CoT reasoning for cybersecurity detections.
- Introduces a four-stage training process that enhances the classifier's performance.
- Demonstrates the necessity of a confidence calibrator to maintain high-confidence recall.
- Specialized models outperform general-purpose models, advocating for targeted training.
Read more
Cybersecurity Detection Classification with Reasoning-enabled Language Models
Summary
This paper addresses the challenge of alert fatigue in Security Operations Centers (SOCs) by developing a reasoning-enabled triage classifier for cybersecurity detections. Traditional methods often prompt large language models (LLMs) to produce triage labels without leveraging their reasoning capabilities. The authors propose a novel approach that incorporates chain-of-thought (CoT) reasoning, allowing the model to articulate its reasoning process before classifying detections as malicious or benign. The proposed system employs a four-stage training methodology, including prompt optimization, self-training, reinforcement learning with verifiable rewards, and confidence calibration. This approach not only enhances the accuracy of the triage classifier but also addresses the degradation of label-token probabilities caused by CoT reasoning by introducing a separate calibrator model. The results demonstrate that the CoT reasoning-enabled classifier achieves an overall accuracy of 82.6%, with significant improvements in benign and malicious recall compared to direct-label classifiers. The necessity of the calibrator is highlighted, as an untrained confidence judge leads to a collapse in high-confidence recall. Furthermore, a specialized 30B model outperforms general-purpose models, emphasizing the importance of targeted training in this domain.
Methodology
The authors developed a triage classifier using a four-stage training approach: prompt optimization, self-training, reinforcement learning with verifiable rewards, and confidence calibration. The model employs chain-of-thought reasoning to articulate its decision-making process before classifying detections.
Results
The proposed system achieved an overall accuracy of 82.6%. At high-confidence operating points, it improved benign recall by 43.0% and malicious recall by 18.3% compared to a direct-label LLM classifier. The necessity of the confidence calibrator was confirmed, as an untrained judge resulted in zero high-confidence recall.
Implications
This research has significant implications for automating triage processes in cybersecurity, potentially reducing alert fatigue and improving the efficiency of SOCs. The findings advocate for the integration of reasoning capabilities in LLMs for enhanced decision-making in security contexts.
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
NLP
Large Language Models
Efficient ML
- Introduces SDO, a dynamic data organization framework for LLM post-training.
- Utilizes an exposure-driven feedback mechanism to optimize mini-batch composition.
- Operates on frozen embeddings, avoiding warm-up training overhead.
- Demonstrates significant improvements in convergence speed and gradient coherence.
Read more
SDO: Structure-Aware Data Organization for Efficient LLM Post-Training
Summary
The paper introduces SDO (Structure-Aware Data Organization), a novel framework aimed at enhancing the efficiency of post-training for large language models (LLMs). Traditional methods primarily focus on selecting informative samples or designing training schedules, treating data organization as a static preprocessing step. This static approach leads to redundant updates for some samples while others remain under-optimized. SDO addresses this by implementing an exposure-driven feedback mechanism that organizes mini-batch composition and sample exposure based on the structure of representation space. The framework operates on frozen external embeddings, allowing for locality-aware batching and exposure-balanced scheduling without the need for warm-up training. Empirical evaluations across supervised fine-tuning (SFT), direct preference optimization (DPO), and gradient reinforcement post-training optimization (GRPO) demonstrate that SDO accelerates convergence, particularly in the early-to-mid training phases, while maintaining coherent gradients and balanced accuracy across different question types.
Methodology
SDO employs a closed-loop data organization framework that reorganizes data flow at both mini-batch and epoch levels. It uses locality-aware batching through KNN neighborhood traversal to form coherent mini-batches and exposure-balanced scheduling to manage sample participation across epochs. This approach allows for dynamic adjustments based on actual sample exposure during training.
Results
The implementation of SDO resulted in accelerated convergence rates across SFT, DPO, and GRPO, with the most substantial gains observed during the early-to-mid training phases. The framework produced more coherent gradients and balanced accuracy across various question types, confirming its effectiveness through gradient diagnostics and component ablation studies.
Implications
SDO's approach to data organization can significantly enhance the efficiency of post-training processes for LLMs, making it a valuable tool for researchers and practitioners aiming to optimize training workflows. Its adaptability and lack of reliance on warm-up training make it suitable for a wide range of applications in natural language processing and beyond.
It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
Theory
Efficient ML
- Introduction of einx as a universal notation for tensor operations.
- Revisiting vectorization to unify and simplify tensor programming.
- Reduction of complex APIs to a small set of elementary operations.
- Implementation of einx that integrates with existing tensor frameworks.
Read more
It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
Summary
The paper addresses the challenges associated with tensor operations in scientific computing, particularly the difficulties posed by the Numpy-like notation prevalent in many tensor frameworks. The authors argue that this notation is often hard to read, write, and is susceptible to shape errors due to inconsistent operational rules. They propose einx, a universal notation for tensor operations that leverages the concept of vectorization to simplify and unify tensor programming. By revisiting vectorization, the authors create a notation that is declarative and pointful, akin to loop notation, which enhances readability and consistency across operations. The einx notation reduces the complexity of existing APIs to a minimal set of elementary operations while maintaining the ability to express a wide range of tensor operations. The paper also presents an implementation of einx that integrates seamlessly with existing tensor frameworks, allowing users to adopt this notation in their tensor programming tasks.
Methodology
The authors derive einx by analyzing and transforming tensor operations through vectorization. They establish a framework that allows for lifting lower-order operations to higher-order ones and decomposing higher-order operations into simpler components. The einx notation is designed to be declarative and pointful, providing a consistent set of rules for tensor operations.
Results
The implementation of einx demonstrates a significant reduction in the complexity of tensor operation APIs, allowing for a more straightforward and readable coding experience. The authors provide examples that illustrate how various Numpy-like operations can be expressed in einx, showcasing its versatility and ease of use.
Implications
The einx notation has the potential to streamline tensor programming, making it more accessible to practitioners and researchers. Its integration with existing frameworks could facilitate broader adoption and improve the efficiency of tensor operations in various scientific computing applications.
Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework
Time Series
- WrapFlow is a novel continuous-time modeling framework specifically designed for irregular multivariate time series forecasting.
- The framework introduces Continuous-Time Tokenization to encode irregular observations without discretization.
- Residual Flow Matching allows for efficient learning of continuous residual dynamics without relying on numerical-solver simulation.
- Extensive experiments show that WrapFlow outperforms existing state-of-the-art methods in IMTS forecasting.
Read more
Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework
Summary
This paper addresses the challenges of forecasting irregular multivariate time series (IMTS), which are common in fields like healthcare and environmental sensing. Traditional methods often rely on discretization, which can distort the underlying continuous-time semantics, or on ODE-based frameworks that require complex architectures and significant computational resources. To overcome these limitations, the authors propose WrapFlow, a continuous-time modeling framework that utilizes Continuous-Time Tokenization to encode raw observation events and models long unobserved intervals with gap-aware tokens. This framework employs a standard Transformer backbone to capture long-range temporal dependencies. Additionally, it introduces a simulation-free training paradigm called Residual Flow Matching, which learns conditional residual vector fields without the need for numerical solvers during training. The authors demonstrate that WrapFlow achieves state-of-the-art performance across multiple real-world datasets, effectively capturing continuous-time dynamics in IMTS forecasting.
Methodology
WrapFlow employs Continuous-Time Tokenization to create event-level tokens that incorporate observed values, timestamps, and inter-event intervals. It uses a Transformer backbone for sequence modeling and implements a simulation-free training paradigm through Residual Flow Matching, which learns to predict residual vector fields relative to base predictions without numerical solver dependencies.
Results
The experimental results indicate that WrapFlow achieves superior performance compared to existing methods on multiple IMTS datasets, demonstrating its effectiveness in capturing the dynamics of irregular time series.
Implications
The proposed framework can significantly enhance forecasting accuracy in applications involving irregular time series, such as healthcare monitoring and environmental sensing, by providing a more principled approach to continuous-time modeling.
Rethinking EEG-Based Disease Diagnosis: Decoupling Instance Representation Learning from Subject-Level Supervision
Time Series
- Introduces BridgeMIL, a two-stage framework for EEG disease diagnosis.
- Decouples instance representation learning from subject-level supervision.
- Achieves the highest mean accuracy in 14 out of 15 dataset-backbone settings.
- Demonstrates the variability of inherited-label reliability across instances.
Read more
Rethinking EEG-Based Disease Diagnosis: Decoupling Instance Representation Learning from Subject-Level Supervision
Summary
This paper addresses the challenges in EEG-based disease diagnosis, particularly the limitations of existing methods that rely on inherited labels for instance-level classification. The authors propose a novel framework called BridgeMIL, which decouples instance representation learning from subject-level supervision. The framework consists of two stages: the first stage pretrains an encoder without inherited labels by aligning temporally nearby windows and independently sampled within-subject sub-bags, while applying variance and covariance regularization to prevent representation collapse. The second stage fine-tunes the encoder within an attention-based multiple instance learning (MIL) model, applying supervision only to subject-level predictions and limiting representation drift through feature retention. The proposed method is evaluated on three EEG disease datasets, demonstrating superior performance compared to existing approaches, with a mean accuracy of 76.57%, outperforming the strongest baseline by 4.28 percentage points. The findings highlight the importance of aligning supervision with subject-level objectives and suggest that instance-level learning can be enhanced by leveraging abundant EEG data without assigning disease labels to individual instances.
Methodology
The methodology involves a two-stage approach: Stage 1 pretrains an encoder using alignment of temporally nearby instances and independently sampled sub-bags without inherited labels, while applying variance and covariance regularization. Stage 2 fine-tunes this encoder in an attention-based MIL model, applying supervision only to subject-level predictions and incorporating feature retention to maintain representation integrity.
Results
BridgeMIL achieved a mean accuracy of 76.57% across three EEG disease datasets, outperforming the strongest baseline by 4.28 percentage points. The model demonstrated significant improvements in representation learning and subject-level prediction accuracy, revealing substantial variation in the reliability of inherited labels across instances.
Implications
The findings suggest that EEG-based diagnostic systems can be improved by focusing on subject-level predictions rather than instance-level classifications, potentially leading to more reliable diagnostic tools in clinical settings. The approach may also be applicable to other domains where instance-level data is abundant but subject-level labels are scarce.
Real-Time Hard Peak Age-of-Information Safety with No-Regret Learning
Optimization
Theory
Reinforcement Learning
- Introduces OCO-PAoI-Hard, a novel scheduling framework for peak AoI safety in IoT systems.
- Achieves zero per-slot deadline violations under adversarial conditions.
- Establishes O(βT) regret bounds and provides a comprehensive theoretical analysis.
- Demonstrates empirical superiority over traditional scheduling methods in maintaining AoI constraints.
Read more
Real-Time Hard Peak Age-of-Information Safety with No-Regret Learning
Summary
This paper addresses the challenge of ensuring that the peak Age of Information (AoI) in safety-critical IoT systems remains below a hard deadline in real-time scenarios. Traditional methods have struggled to meet this requirement under adversarial conditions, often relying on restrictive assumptions such as stochastic channels or long-term cumulative violations. The authors propose a novel framework, OCO-PAoI-Hard, which transforms the scheduling problem into a time-varying constrained online convex optimization problem. This approach guarantees zero per-slot violations of the modeled AoI state while maintaining O(βT) regret against any static comparator. The paper introduces a strictly causal proposal-shield-update loop that enforces feasibility through Euclidean projections, ensuring that the scheduling decisions adhere to the peak AoI constraints. The theoretical analysis provides closed-form regret bounds and demonstrates the effectiveness of the proposed method through empirical evaluations on a four-sensor adversarial fluid-model trap channel, where OCO-PAoI-Hard achieved zero deadline violations across multiple trials, outperforming several baseline methods.
Methodology
The authors employ a time-varying constrained online convex optimization approach, transforming the peak AoI scheduling problem into a feasible action selection process through a strictly causal proposal-shield-update loop. This involves projecting the proposed actions onto a polyhedral safe set to ensure compliance with AoI deadlines.
Results
The proposed OCO-PAoI-Hard framework recorded zero modeled-state deadline violations across all trials in a four-sensor adversarial fluid-model trap channel. In contrast, baseline methods exhibited significant per-slot miss rates, ranging from 1.65% to 64.0%. The empirical normalized regret remained below the theoretical envelope across a wide range of iterations, confirming the effectiveness of the approach.
Implications
The findings have significant implications for the design of safety-critical IoT systems, particularly in applications such as industrial control, V2X coordination, and remote teleoperation, where maintaining real-time data freshness is crucial for system stability and performance.
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
NLP
Large Language Models
Reinforcement Learning
- HARGO addresses the limitations of traditional SFT in LLMs for HPC tasks by introducing a heterogeneity-aware optimization approach.
- The method employs per-response importance weighting based on confidence-modulated advantages, enhancing learning efficiency.
- HARGO outperforms standard RL methods, achieving the best performance metrics across multiple HPC tasks.
- The paper provides a comprehensive evaluation of RL post-training approaches, highlighting the challenges posed by task heterogeneity.
Read more
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
Summary
The paper introduces HARGO, a novel approach for reinforcement learning (RL) post-training of large language models (LLMs) specifically tailored for high-performance computing (HPC) tasks. Traditional supervised fine-tuning (SFT) methods equip LLMs with domain knowledge but fail to ensure task-appropriate behavior across heterogeneous HPC tasks. HARGO addresses this by implementing per-response importance weighting through a confidence-modulated advantage mechanism, which enhances the learning signal without requiring task-type labels. The authors demonstrate that existing uniform-weight RL methods, such as GRPO, are suboptimal due to the extreme heterogeneity of HPC tasks, which differ significantly in answer length, reward distributions, and SFT accuracy. HARGO's effectiveness is validated through a systematic comparison of nine methods across four HPC tasks, achieving superior performance metrics, including WinRate, Data Race F1, and PLP Similarity. The results indicate that HARGO not only improves performance across all evaluated metrics but also establishes the best overall alignment quality for heterogeneous HPC tasks.
Methodology
HARGO utilizes a confidence-modulated advantage mechanism to compute per-response importance weights. It derives a discrimination signal from group-level reward contrasts and a confidence signal from reference model log-probabilities. The advantage is modulated before computing response weights, allowing for effective learning without the need for task-type labels.
Results
HARGO achieved the best performance on three primary metrics: WinRate of 54.62%, Data Race F1 of 91.30%, and PLP Similarity of 0.8558. It led in specific tasks such as race_fortran (94.90% accuracy) and plp (100% accuracy), while ranking competitively in others. Ablation studies confirmed the independent contributions of the discrimination and confidence signals to HARGO's performance.
Implications
The findings suggest that HARGO can significantly enhance the performance of LLMs in diverse HPC applications, making it a valuable tool for optimizing LLM behavior in real-world scenarios where task heterogeneity is prevalent.
A Montage-Agnostic Encoder for Calibration-Light Cross-User Gesture Recognition from Surface Electromyography
Robotics
Time Series
Efficient ML
- Introduces a Montage-Agnostic Encoder for sEMG gesture recognition that reduces calibration needs.
- Demonstrates superior performance over traditional per-user classifiers in cross-user settings.
- Conducts an ablation study revealing the significant contributions of the encoder's components.
- Finds that signal fidelity is crucial for the performance of learned encoders compared to baseline classifiers.
Read more
A Montage-Agnostic Encoder for Calibration-Light Cross-User Gesture Recognition from Surface Electromyography
Summary
This paper addresses the challenge of cross-user gesture recognition from surface electromyography (sEMG) signals, which typically requires extensive calibration for each user due to anatomical and electrode placement differences. The authors propose a Montage-Agnostic Encoder that utilizes shared weights across electrodes and locates them by physical coordinates, allowing it to handle varying channel counts without montage-specific parameters. The encoder is trained across multiple users and demonstrates superior performance compared to traditional per-user classifiers, achieving higher macro-F1 scores on benchmark datasets. The study also includes an ablation analysis to assess the contributions of the encoder's components and explores the relationship between training pool size and model performance. The findings suggest that while the encoder outperforms per-user classifiers under certain conditions, its effectiveness is influenced by the quality of the input signals. Additionally, self-supervised pretraining did not yield performance improvements once a supervised model was adequately trained. Overall, the proposed encoder aims to reduce calibration burdens and enhance the usability of myoelectric prostheses.
Methodology
The authors developed a Montage-Agnostic Encoder that treats each electrode as a token based on its physical location, utilizing an attention mechanism to learn spatial patterns. They conducted experiments across different datasets to evaluate the encoder's performance in cross-user gesture recognition and performed an ablation study to analyze the contributions of its components. Additionally, they assessed the impact of training pool size and explored self-supervised pretraining approaches.
Results
The encoder outperformed the per-user linear discriminant analysis (LDA) classifier by 0.234 macro-F1 on the NinaPro DB1 benchmark and by 0.108 on DB2, although it performed below the LDA on the ten-subject DB5. The ablation study indicated that each of the encoder's components contributed significantly to its performance. The controlled subject-count sweep revealed that the training pool size primarily serves as a stability floor, while the quality of the input signals affects the performance of the baseline classifiers.
Implications
The proposed encoder has the potential to streamline the calibration process for myoelectric prostheses, making them more accessible and user-friendly. By reducing the need for extensive calibration, this approach could facilitate broader adoption of advanced gesture recognition systems in clinical and everyday settings.
Sparsity Induced Identifiability in Matrix Tri-Factorisation
Theory
- Introduces a novel decomposition strategy for matrix tri-factorisation, transforming it into two auxiliary two-factor problems.
- Establishes rigorous theoretical recovery guarantees for sparsity-induced identifiability.
- Demonstrates structural consistency between original and auxiliary factorisations.
- Provides comprehensive empirical validation of theoretical results through Monte Carlo experiments.
Read more
Sparsity Induced Identifiability in Matrix Tri-Factorisation
Summary
This paper addresses the theoretical foundations of sparsity-induced identifiability in matrix tri-factorisation, a more flexible alternative to conventional two-factor models. While sparsity has been extensively studied in two-factor matrix factorisation, its implications for tri-factorisation remain largely unexplored. The author introduces a novel decomposition strategy that reformulates the tri-factorisation problem into two coupled auxiliary two-factor problems, facilitating a rigorous theoretical analysis. The study establishes recovery guarantees and structural consistency results, demonstrating how coefficient sparsity influences recovery conditions, convergence behavior, and spectral approximation error. Comprehensive Monte Carlo experiments validate the theoretical findings, showing a close alignment between theoretical predictions and empirical results. This work fills a significant gap in the literature by providing the first rigorous theoretical analysis of sparsity-induced identifiability in general real-valued matrix tri-factorisation.
Methodology
The paper employs a novel decomposition strategy to reformulate the tri-factorisation problem into two coupled auxiliary two-factor factorisation problems. This approach allows for a tractable theoretical analysis while preserving the necessary structural information for recovering the original factor matrices. The analysis includes complementary evaluations of an alternating optimisation procedure and spectral dictionary approximation to derive recovery guarantees.
Results
The study provides rigorous theoretical guarantees for recovery conditions, convergence behavior, spectral approximation error, and high-probability recovery bounds. It also proves structural consistency between the original and auxiliary tri-factorisations, ensuring that the auxiliary factorisation accurately reflects the support patterns of the original matrices with bounded changes.
Implications
The findings have significant implications for applications in data compression, denoising, and interpretable representation learning, as they enhance the understanding of how sparsity can improve the recovery and interpretability of latent structures in high-dimensional data.
TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
Multimodal
- TIER-MoE introduces a risk-guided approach to assess modality reliability in multimodal fusion.
- The model combines modality-specific reliability with expert specialization for improved predictions.
- Evaluation on multiple biomedical datasets shows superior performance compared to existing methods.
- TIER-MoE achieves better probability calibration and zero-shot generalization capabilities.
Read more
TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
Summary
The paper introduces TIER-MoE, a novel risk-guided mixture-of-experts model designed for multimodal biomedical classification. It addresses the challenge of determining modality reliability in multimodal fusion, where more evidence does not necessarily lead to better predictions. TIER-MoE defines sample-specific modality reliability based on the expected prediction loss of unimodal predictors, learned from out-of-fold predictions. This approach allows for sparse routing of modalities to experts while maintaining a shared path for multimodal complementarity. The model was evaluated on four public biomedical datasets related to Alzheimer's disease, skin lesion malignancy, and retinal classification. Results indicate that TIER-MoE outperforms existing state-of-the-art methods in predictive performance and probability calibration, demonstrating significant improvements in Macro-F1 and Brier score, as well as strong zero-shot generalization to external cohorts.
Methodology
TIER-MoE employs a hierarchical framework that estimates the conditional risk of each modality based on expected prediction loss from unimodal predictors. It integrates this risk with expert-specific subspace compatibility for routing modalities to specialized experts, while maintaining a shared path for multimodal interaction.
Results
The model demonstrated superior predictive performance and probability calibration across four biomedical datasets, achieving consistent improvements in Macro-F1 and Brier score. It also exhibited strong zero-shot generalization capabilities when tested on an external cohort.
Implications
TIER-MoE has the potential to enhance multimodal biomedical classification by effectively leveraging diverse sources of evidence, leading to better disease diagnosis and patient stratification. Its approach to modality reliability could be applied in various domains where multimodal data is prevalent.
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Multimodal
NLP
Large Language Models
- Introduces a Structured Evidence Ledger for multimodal reasoning, enhancing auditability and provenance tracking.
- Identifies and addresses four common failure patterns in multimodal agentic reasoning.
- Implements a Three-Layer Grounding Protocol and an Adaptive Dual-Path Dispatcher to optimize reasoning processes.
- Demonstrates improved accuracy and trajectory-level faithfulness in multimodal reasoning tasks.
Read more
LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
Summary
The paper introduces LedgerMind, a framework designed to enhance the evaluation and reasoning capabilities of multimodal agents in visual question answering tasks. Traditional evaluation methods focus primarily on final-answer accuracy, which fails to account for the provenance of the reasoning process. LedgerMind addresses this by structuring the evidence used in reasoning into a Structured Evidence Ledger, which normalizes tool outputs and allows for provenance-constrained reasoning. This ledger ensures that claims made by the agent can only cite evidence that is actively supported, thus improving the auditability of intermediate reasoning claims. The authors identify four common failure patterns in multimodal reasoning: unsupported claims, Phantom Grounding (where citations do not match the content), unnecessary depth in reasoning, and repair-time amplification (where attempts to correct errors introduce new unsupported claims). The framework includes a Three-Layer Grounding Protocol, an Adaptive Dual-Path Dispatcher, and an Event-Triggered Verification-and-Repair engine, all aimed at ensuring that reasoning remains grounded in verifiable evidence. Experiments demonstrate that LedgerMind significantly improves both answer accuracy and the faithfulness of the reasoning trajectories, providing a more reliable framework for multimodal reasoning tasks.
Methodology
The methodology involves creating a Structured Evidence Ledger that captures tool outputs with detailed provenance information. The framework employs a Three-Layer Grounding Protocol to verify evidence, an Adaptive Dual-Path Dispatcher to optimize reasoning depth, and an Event-Triggered Verification-and-Repair engine to ensure that repairs do not introduce unsupported claims. The system is evaluated against multiple multimodal reasoning benchmarks.
Results
Experiments show that LedgerMind enhances both the accuracy of answers and the faithfulness of reasoning trajectories, effectively addressing the identified failure patterns. The structured approach leads to more reliable multimodal reasoning outcomes compared to traditional methods.
Implications
The implications of this work suggest that future multimodal reasoning systems can benefit from structured evidence management, leading to more reliable and interpretable outputs. This framework can be applied in various domains requiring multimodal interactions, such as robotics, automated customer service, and educational tools.
Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture Decoding
Time Series
- Introduces a montage-agnostic encoder for sEMG gesture recognition that adapts across recording sessions.
- Demonstrates superior performance of the encoder compared to traditional per-user LDA methods.
- Identifies feature-statistic alignment as the only effective label-free adaptation method across all subjects.
- Challenges the reliance on recalibration for sEMG systems, proposing a more practical solution for daily use.
Read more
Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture Decoding
Summary
This paper addresses the challenge of recognition accuracy in surface electromyography (sEMG) gesture decoding, which often degrades when electrodes are reapplied after removal due to factors like electrode movement and skin condition changes. The authors propose a montage-agnostic encoder that can transfer learned patterns across different recording sessions without requiring recalibration. The study evaluates the encoder's performance against a per-user linear discriminant analysis (LDA) classification pipeline and two existing methods that rely solely on data from the same session. The encoder demonstrated a macro-F1 score of 0.688, significantly outperforming the per-user pipeline's score of 0.540. Additionally, the paper explores five label-free adaptation techniques, finding that only feature-statistic alignment consistently improved performance across all subjects, while batch-normalization re-estimation led to performance collapse. This work highlights the potential for robust sEMG gesture recognition systems that can adapt to day-to-day variability without extensive recalibration.
Methodology
The authors trained a montage-agnostic encoder using data from a specific recording session and tested its performance on data from different sessions without any adjustments. They compared this approach to a per-user LDA classification pipeline and two published methods that only utilized data from the same session. Additionally, they evaluated five label-free adaptation techniques to improve performance on new recording sessions.
Results
The montage-agnostic encoder achieved a macro-F1 score of 0.688, outperforming the per-user LDA pipeline's score of 0.540. Among the adaptation techniques tested, only feature-statistic alignment improved performance for all subjects, while batch-normalization re-estimation was ineffective.
Implications
The findings suggest that sEMG gesture recognition systems can be made more practical for daily use by reducing the need for recalibration. This could enhance the usability of myoelectric prosthetic control systems, making them more accessible and effective for users.
FunL2O: LLM-Guided Feature Function Design for Learning to Optimize
Optimization
Large Language Models
- FunL2O automates feature function design in L2O using LLMs.
- The framework evaluates feature functions based on their impact on optimization performance rather than LLM judgments.
- Evolved features consistently outperform traditional hand-crafted representations across multiple optimization tasks.
- The approach allows for isolating the effects of input representation on model performance.
Read more
FunL2O: LLM-Guided Feature Function Design for Learning to Optimize
Summary
The paper introduces FunL2O, a novel framework that leverages large language models (LLMs) to automate the design of feature functions in learning-to-optimize (L2O) methods. Traditional L2O approaches rely on manually crafted features, which can limit their adaptability and performance across different optimization tasks. FunL2O addresses this limitation by employing an LLM to propose executable feature functions that are then evaluated based on their impact on downstream optimization performance. The framework operates in a FunSearch-style loop, where the LLM generates candidate features, which are subsequently tested by retraining the L2O model and measuring optimization outcomes. The authors validate FunL2O on various linear and quadratic programming tasks, as well as mixed-integer optimization scenarios, demonstrating that the evolved features consistently outperform hand-crafted representations. This work highlights the potential of LLM-driven feature evolution as a robust method for enhancing representation design in L2O, ultimately leading to improved solution quality and solver efficiency.
Methodology
FunL2O formulates feature function design as a program-search problem within the L2O pipeline. It utilizes an LLM to propose executable feature functions that adhere to predefined semantic contracts. The proposed features are evaluated by retraining the L2O model and measuring their downstream optimization performance, allowing for iterative improvement of feature functions based on empirical results.
Results
The evaluation of FunL2O across various optimization tasks, including linear and quadratic programming and mixed-integer optimization, shows that the evolved feature functions significantly enhance solution quality, feasibility, and solver efficiency compared to hand-crafted features. The results indicate that LLM-driven feature evolution is a general and effective approach for automating representation design in L2O.
Implications
The findings suggest that automating feature function design can lead to more effective and adaptable optimization methods. This could have broad applications in fields requiring repeated optimization under varying conditions, such as operations research, logistics, and machine learning model training.
Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
Reinforcement Learning
Large Language Models
NLP
- Introduces a principled reward decomposition framework for Reinforcement Unlearning.
- Proposes two new reward functions that improve learning efficiency.
- Demonstrates that new rewards can achieve forgetting performance faster than binary rewards.
- Highlights the significance of reward design in the context of machine unlearning.
Read more
Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
Summary
This paper addresses the challenge of machine unlearning, which aims to selectively remove specific knowledge from trained language models without the need for full retraining. This is increasingly important due to privacy regulations like GDPR. The authors reformulate unlearning as a Reinforcement Learning with Verifiable Rewards (RLVR) problem, where models are optimized against rewards derived from their outputs. They identify that existing methods rely on sparse binary rewards, which provide limited learning signals and hinder convergence speed. To improve this, the authors propose a reward decomposition framework that separates verifiability from sparsity, introducing two new reward functions: an exponential reward that applies graded penalties based on the frequency of forbidden content and a PageRank-inspired reward that weighs penalties by semantic importance. Experiments conducted on the Real World Knowledge Unlearning (RWKU) benchmark demonstrate that these new reward functions significantly outperform the binary reward setting, achieving similar forgetting performance up to three times faster while maintaining general model utility. The findings underscore the importance of reward design in enhancing unlearning efficiency, offering a practical approach to scalable machine unlearning.
Methodology
The authors developed a reward decomposition framework that allows for the design of more informative rewards in the context of Reinforcement Unlearning. They introduced two novel reward functions: an exponential reward that penalizes based on the count of forbidden-concept occurrences and a PageRank-inspired reward that considers the semantic importance of these concepts. The effectiveness of these rewards was evaluated through experiments on the RWKU benchmark.
Results
The experiments showed that both the exponential and PageRank-inspired rewards consistently outperformed the traditional binary reward setting. The new reward functions allowed models to achieve similar forgetting performance up to three times faster while preserving overall model utility, indicating a significant improvement in unlearning efficiency.
Implications
The findings suggest that carefully designed reward structures can enhance the efficiency of machine unlearning processes, making it more feasible to comply with privacy regulations. This has potential applications in various fields where data privacy is critical, such as healthcare, finance, and any domain involving sensitive information.
DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement
Graph Learning
Multimodal
Optimization
- DAS-PMVC addresses the partial view alignment problem in multi-view clustering.
- The framework employs a dual alignment strategy to enhance the alignment of latent features.
- It integrates anchor graph structure alignment and multi-view graph convolutional networks for improved feature learning.
- Experimental results show significant performance improvements over existing methods.
Read more
DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement
Summary
The paper introduces DAS-PMVC, a novel framework designed to tackle the challenges of partial multi-view clustering, particularly addressing the partial view alignment problem (PVAP) caused by misalignment in multi-view data. The authors highlight that traditional methods often fail to effectively manage view asymmetry and irrelevant samples, leading to suboptimal clustering performance. DAS-PMVC is structured around three main components: (1) anchor graph structure alignment, which establishes initial view alignment through joint embedding representations derived from anchor point relationships; (2) structure-enhanced feature learning, which utilizes multi-view graph convolutional networks to extract deep latent features from the aligned graph structure, thereby improving the discriminative power of the representations; and (3) a dual alignment strategy that combines initial alignment with contrastive learning loss and the Hungarian algorithm to optimize the alignment of latent features during training. Experimental results demonstrate that DAS-PMVC outperforms existing state-of-the-art methods across various datasets, confirming its effectiveness in enhancing clustering performance in the presence of misaligned multi-view data.
Methodology
DAS-PMVC employs a three-part methodology: (1) it uses anchor graph structure alignment to derive joint embedding representations for initial view alignment; (2) it enhances feature learning through multi-view graph convolutional networks to extract deep latent features; and (3) it implements a dual alignment strategy that optimizes latent feature alignment using contrastive learning and the Hungarian algorithm.
Results
The experimental results indicate that DAS-PMVC consistently outperforms state-of-the-art clustering methods, demonstrating superior clustering performance across various datasets, thus validating the effectiveness of the proposed framework.
Implications
The DAS-PMVC framework has potential applications in scenarios where multi-view data is prevalent, such as image and video analysis, social network analysis, and any domain where data is collected from multiple sources with possible misalignment.
Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks
Graph Learning
Theory
- Introduces persistent Gaussian noise in recurrent GNNs to combat oversmoothing.
- Establishes a theoretical framework using stochastic dynamical systems and Markov chain theory.
- Proves that stationary representations cannot collapse, ensuring diversity in node representations.
- Derives a quantitative relationship between noise variance, graph spectral gap, and Dirichlet energy.
Read more
Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks
Summary
This paper addresses the issue of oversmoothing in deep graph neural networks (GNNs), where repeated message passing leads to node representations becoming overly similar and collapsing into a low-dimensional subspace. The author proposes a novel approach using recurrent graph neural networks that inject independent Gaussian noise after each message-passing step. This method is analyzed as a stochastic dynamical system, revealing that the hidden representations form a geometrically ergodic Markov chain with a unique invariant probability measure. The main theoretical contribution is an explicit positive lower bound on the expected stationary Dirichlet energy, which is proportional to both the noise variance and the spectral gap of the graph. This finding guarantees that the representations cannot collapse onto a constant manifold, effectively preventing asymptotic oversmoothing. The paper also presents numerical experiments that validate the theoretical predictions, demonstrating the emergence of a stationary distribution and the dependence of Dirichlet energy on noise intensity. Overall, the study highlights persistent stochastic perturbations as a fundamental mechanism to maintain representation diversity in GNNs, complementing existing deterministic methods.
Methodology
The paper formulates noisy recurrent graph neural networks as stochastic dynamical systems. It employs Markov chain theory to analyze the asymptotic behavior of the hidden representations, proving the existence of a unique invariant probability measure and geometric ergodicity under a global contraction assumption on the deterministic updates.
Results
The main results include a rigorous proof that the expected stationary Dirichlet energy has a positive lower bound, which prevents the collapse of representations onto a constant manifold. Numerical experiments confirm the theoretical predictions, showing a stationary distribution and the expected scaling of Dirichlet energy with noise variance.
Implications
The findings suggest that incorporating persistent stochastic perturbations can fundamentally alter the dynamics of recurrent GNNs, providing a principled method to maintain representation diversity. This has practical implications for designing deeper GNN architectures that are less susceptible to oversmoothing, guiding the selection of noise levels in real-world applications.
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Reinforcement Learning
Large Language Models
NLP
- CRPO reformulates OPSD using a contrastive learning perspective to mitigate exposure bias.
- The method employs predictive entropy to classify positions and conducts group-wise contrast for optimization.
- CRPO enhances training stability and performance in agentic post-training scenarios.
- Extensive evaluations show CRPO's superiority over existing reinforcement learning baselines.
Read more
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Summary
This paper introduces Contrastive Reinforced Policy Optimization (CRPO), a novel approach to On-Policy Self-Distillation (OPSD) that addresses the limitations of existing reinforcement learning methods in multi-turn agentic settings. The authors identify that OPSD suffers from exposure bias due to the privileged information of the self-teacher model, which can lead to reasoning route convergence and unclear optimization directions. CRPO reformulates OPSD through a contrastive learning framework, utilizing predictive entropy to differentiate between positive (reflective exploration) and negative (exposure bias) positions. By applying group-wise contrast, CRPO aims to maintain reliable optimization signals and enhance training stability. The authors validate their approach through extensive evaluations across 13 challenging benchmarks, demonstrating that CRPO consistently outperforms existing RL and self-distillation methods, thereby improving generalization in long-horizon interactions.
Methodology
The methodology involves reformulating the OPSD framework by treating the original context and the context with privileged information as two views. The policy model acts as the encoder, and the output logits serve as representations. The method employs predictive entropy to classify positions into positive and negative pairs, applying group-wise contrast to preserve clear optimization directions.
Results
The results indicate that CRPO significantly outperforms existing reinforcement learning and self-distillation baselines across 13 diverse reasoning and deep-search benchmarks, showcasing enhanced training stability and generalization capabilities in long-horizon interactions.
Implications
The implications of this research suggest that CRPO can be effectively applied in complex agentic tasks requiring multi-turn interactions, potentially benefiting applications in natural language processing, dialogue systems, and other domains where reinforcement learning is utilized.
Towards joint scaling laws with optimal batch size schedules
Optimization
Theory
Efficient ML
- Derivation of a joint loss characterization based on learning rate and batch size schedules.
- Closed-form optimal batch size schedule that is independent of peak learning rate.
- Introduction of joint scaling laws that improve compute efficiency by 6-15%.
- Empirical validation showing dynamic batch sizes outperform static ones across various models.
Read more
Towards joint scaling laws with optimal batch size schedules
Summary
This paper addresses the optimization of batch size schedules in deep learning, which is often kept static during training. The authors derive a joint characterization of loss based on both learning rate and batch size schedules, applicable to various optimizers and architectures. They present a closed-form solution for the optimal batch size schedule corresponding to any given learning rate schedule, demonstrating that this optimal schedule is independent of the peak learning rate. The study introduces joint scaling laws that enhance compute efficiency by 6-15% without additional computational overhead. Empirical evaluations on models like Llama3 and Qwen3 show that dynamic batch size schedules significantly outperform static ones, reinforcing the importance of adaptive hyperparameter tuning in large language model training.
Methodology
The authors utilize stochastic convex optimization theory to analyze deep learning training dynamics. They develop a sequence-to-sequence characterization of loss based on learning rate and batch size schedules, formalizing the problem as a constrained optimization task. The optimal batch size schedule is derived analytically and validated through empirical experiments on multiple model architectures.
Results
The paper demonstrates that the proposed dynamic batch size schedules lead to improved performance metrics across various benchmarks, with empirical results indicating that dynamic schedules consistently outperform static ones. The optimal batch size schedule is shown to be a function of the total computational budget and the shape of the learning rate schedule, leading to enhanced training efficiency.
Implications
The findings suggest that adopting dynamic batch size schedules can significantly improve the training efficiency and performance of large-scale models. This approach may facilitate better hyperparameter tuning practices, especially in resource-constrained environments, and could influence future research in optimizing training dynamics for deep learning.
On-Policy and Off-Policy Learning for Large Action Spaces
Reinforcement Learning
Optimization
Theory
- Introduction of mixed-effect Thompson sampling (meTS) for efficient on-policy learning.
- Development of diffusion Thompson sampling (dTS) leveraging deep generative models.
- Structured direct method (sDM) for off-policy learning achieving O(1/βn) convergence.
- Emphasis on optimization tractability over reward estimation accuracy in large action spaces.
Read more
On-Policy and Off-Policy Learning for Large Action Spaces
Summary
This thesis addresses the challenges of learning in interactive systems modeled as contextual bandits, particularly in scenarios with large action spaces. It distinguishes between on-policy and off-policy learning paradigms, both of which face difficulties in scaling due to computational and statistical inefficiencies. The author proposes novel algorithms to enhance the tractability of contextual bandit algorithms. For on-policy learning, a structured Bayesian model called mixed-effect Thompson sampling (meTS) is introduced, which reduces Bayesian regret and improves memory and runtime complexities. An extension, diffusion Thompson sampling (dTS), utilizes deep generative models to further enhance performance. For off-policy learning, the structured direct method (sDM) is developed, achieving convergence without the need for full logging support, and demonstrating that optimization tractability is more critical than reward estimation accuracy in large action spaces. Additional techniques to reduce importance sampling variance are also explored. The theoretical advancements are validated through extensive experiments on both synthetic and real-world datasets, showcasing the practical applicability of the proposed methods in modern applications with extensive action spaces.
Methodology
The thesis employs structured Bayesian models for on-policy learning and develops the structured direct method for off-policy learning. It utilizes mixed-effect and diffusion Thompson sampling for efficient exploration and leverages deep generative models. Theoretical proofs and empirical validations are conducted to demonstrate the effectiveness of the proposed methods.
Results
The proposed algorithms significantly reduce Bayesian regret and improve memory and runtime complexities. The structured direct method shows improved performance as the action space grows, and the new techniques for variance reduction yield better learning outcomes in practical scenarios.
Implications
The advancements made in this thesis enable the deployment of contextual bandits in applications with large action spaces, such as recommender systems, where traditional methods struggle. This work opens avenues for more efficient decision-making processes in various interactive systems.
Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
Time Series
- Introduces a three-part EEG protocol record to improve evaluation clarity.
- Demonstrates that checkpoint selection can significantly enhance model accuracy.
- Highlights the importance of distinguishing between different evaluation types in EEG studies.
- Provides an auditable checklist for EEG reporting to ensure reproducibility.
Read more
Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
Summary
This paper addresses the critical issue of evaluation protocols in EEG emotion recognition, emphasizing that reported accuracy is influenced by the entire evaluation process rather than just the classifier used. The authors propose a structured methodology that separates the target quantity, development procedure, and reporting rules. They utilize a dynamical graph convolutional neural network (DGCNN) on the SEED and SEED-IV datasets to illustrate their points. The study finds that checkpoint selection based on repeated test-set evaluations significantly improves accuracy. It also highlights discrepancies in accuracy when evaluating across different subjects, indicating that subject identity and other factors like preprocessing and representation can affect performance. The authors argue for a clear distinction in reporting subject-dependent, subject-disjoint, and cross-session results, as these represent different evaluation questions. The paper concludes with a checklist for EEG reporting that enhances reproducibility and clarity in future studies.
Methodology
The authors employed a dynamical graph convolutional neural network (DGCNN) to analyze EEG data from the SEED and SEED-IV datasets. They implemented a structured evaluation protocol that included subject-dependent and subject-disjoint evaluations, along with repeated test-based checkpoint selection to assess model performance. The study also involved comparative analyses to evaluate the impact of different evaluation settings on reported accuracy.
Results
The results indicated that the SEED dataset's accuracy was within 1.47 percentage points of the public reference value, while the SEED-IV dataset showed a 3.40-point unresolved difference. Checkpoint selection improved mean window accuracy from 0.7855 to 0.8892. For subject-disjoint evaluations, training-participant trial accuracies reached 0.9990 for SEED and 0.9920 for SEED-IV, but accuracy for entirely held-out participants was significantly lower, at 0.5348 for SEED and 0.3954 for SEED-IV.
Implications
The findings underscore the need for standardized evaluation protocols in EEG emotion recognition to enhance reproducibility and comparability across studies. The proposed checklist can serve as a guideline for researchers to ensure that their evaluations are transparent and rigorous, potentially leading to more reliable applications in affective computing and related fields.
EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis
Graph Learning
Large Language Models
Interpretability
- EvoCause integrates expert feedback into causal graph refinement for improved root cause analysis.
- The framework uses a large language model to propose graph edits while ensuring structural validity.
- TeleRCA, a new expert-annotated benchmark dataset, is introduced to evaluate RCA methods.
- EvoCause outperforms traditional causal discovery methods in various performance metrics.
Read more
EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis
Summary
EvoCause presents a novel framework for refining causal graphs used in root cause analysis (RCA) within telecommunication and cloud systems. Traditional methods rely on causal discovery algorithms to learn static graphs from observational data, which can lead to inaccuracies due to spurious or missing edges. EvoCause addresses this limitation by integrating expert feedback from historical incident logs, allowing for dynamic graph refinement. The framework employs a large language model (LLM) to propose semantically plausible edits to the causal graph while ensuring structural validity through deterministic validation processes. The authors also introduce TeleRCA, a benchmark dataset containing over 485,000 alarm events, which facilitates the evaluation of RCA methods. Experimental results demonstrate that EvoCause significantly enhances RCA performance metrics compared to unrefined baselines, indicating the effectiveness of incorporating expert knowledge and LLM-guided graph editing.
Methodology
EvoCause formulates expert feedback as source-node constraints on incident-induced subgraphs and employs a large language model to suggest graph edits. Deterministic procedures validate these edits to maintain acyclicity and structural integrity, retaining the best-performing graph based on alignment with expert annotations. The framework is designed to work with various causal discovery algorithms without altering their internal processes.
Results
EvoCause shows significant improvements over the unrefined PC baseline on synthetic data, with increases in Node F1, Case EM, and Graph F1 by 11.59, 9.40, and 4.59 percentage points, respectively, while reducing nSHD by 0.2379. On the TeleRCA dataset, the removal of human-readable alarm titles resulted in a decrease in Node F1 and Case EM by 6.12 and 8.04 percentage points, underscoring the value of alarm-name information in graph refinement.
Implications
The findings suggest that integrating expert knowledge into causal graph construction can significantly enhance the accuracy of root cause analysis in complex systems. The introduction of TeleRCA as a benchmark dataset provides a valuable resource for future research in RCA methodologies, potentially leading to more effective diagnostic tools in telecommunication and cloud environments.
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Reinforcement Learning
Large Language Models
NLP
- Introduces an expand-then-compress framework for reasoning policy learning.
- Proposes Residual GRPO for expanding the reasoning solution manifold with multiple teachers.
- Develops Reliability-Gated Teacher-Union OPD for effective compression of teacher knowledge.
- Demonstrates consistent performance improvements across multiple reasoning tasks.
Read more
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Summary
This paper presents an innovative framework for enhancing reasoning capabilities in reinforcement learning (RL) models by addressing the limitations of traditional single-teacher approaches. The authors argue that RL-trained policies often serve as incomplete teachers, only amplifying a subset of valid solution modes. To overcome this, they propose an 'expand-then-compress' framework that consists of two main stages: expansion and compression. In the expansion stage, the Residual Group Relative Policy Optimization (RGRPO) method is employed to train multiple teachers from a common initialization, directing each subsequent teacher towards uncovered examples in the training distribution. This approach allows for the creation of specialists that cover different regions of the reasoning solution manifold. In the compression stage, the Reliability-Gated Teacher-Union On-policy Distillation (TU-OPD) method distills knowledge from the teacher union into a single student model, focusing on reliable teachers for each example and preserving unique preferences of successful teachers. The experiments conducted across mathematical reasoning, code generation, and instruction following demonstrate that the resulting student model consistently outperforms the strongest individual teacher, showcasing the effectiveness of the proposed framework in constructing a complementary teacher union.
Methodology
The proposed methodology involves two key stages: 1) Expansion through Residual Group Relative Policy Optimization (RGRPO) to train multiple teachers that cover different areas of the reasoning solution manifold, and 2) Compression using Reliability-Gated Teacher-Union On-policy Distillation (TU-OPD) to distill knowledge from the teacher union into a single student model, while preserving the unique preferences of successful teachers.
Results
The experiments show that the Qwen3-1.7B student model outperforms the strongest individual teacher in mathematical reasoning, code generation, and instruction following, achieving relative improvements of 2.0%, 8.3%, and 6.9%, respectively. This indicates that the proposed framework effectively enhances reasoning capabilities by leveraging a diverse set of teachers.
Implications
The findings suggest that a more comprehensive approach to teacher construction and distillation can lead to stronger reasoning models in RL, which could have significant applications in areas requiring complex decision-making and problem-solving, such as automated reasoning systems, code generation tools, and advanced instruction-following agents.
Compression-Based Behavioral Similarity for Open-World Sybil Discovery on Ethereum
Graph Learning
- Introduces a compression-based method for detecting Sybil attackers on Ethereum.
- Utilizes symbolic Transaction Grammar to model wallet behavior without relying on financial links.
- Employs Normalized Compression Distance (NCD) for behavioral similarity analysis.
- Validates the framework against supervised machine learning baselines and stress tests.
Read more
Compression-Based Behavioral Similarity for Open-World Sybil Discovery on Ethereum
Summary
This paper addresses the challenge of detecting Sybil attackers on the Ethereum blockchain, who manipulate governance and exploit airdrops by masquerading as legitimate users. Traditional methods rely on financial transaction graphs or supervised machine learning classifiers, which are limited by their dependence on explicit financial links and labeled data. The authors propose a novel approach that utilizes compression-based behavioral similarity to differentiate between Sybil bots, organic users, and arbitrage bot wallets without requiring direct financial connections. They introduce a symbolic Transaction Grammar to model wallet behavior, capturing transaction rhythm, execution structure, and functional intent. The method employs Gzip-based Normalized Compression Distance (NCD) to construct a behavioral graph for Sybil discovery, validated against supervised baselines and stress tests. The proposed framework is designed for open-world scenarios, allowing for the discovery of suspicious wallets without the need for explicit funding links or supervised training, thus enhancing the robustness and adaptability of Sybil detection in dynamic blockchain environments.
Methodology
The authors synthesize a symbolic Transaction Grammar from Ethereum Virtual Machine (EVM) traces to capture wallet behavior. They apply Gzip-based Normalized Compression Distance (NCD) to measure behavioral similarity and construct a behavioral graph for Sybil discovery. The method is validated through comparisons with supervised machine learning approaches and stress tests under adversarial conditions.
Results
The proposed framework successfully identifies Sybil candidates without requiring explicit financial links or labeled training data. It demonstrates robustness against temporal drift and adversarial perturbations, outperforming traditional methods that rely on financial interactions or supervised learning.
Implications
This research has significant implications for blockchain security, particularly in enhancing the detection of Sybil attacks in decentralized finance (DeFi) and governance contexts. The method's ability to operate in an open-world setting allows for more effective monitoring and auditing of blockchain activities, potentially leading to improved security measures and governance integrity.
Learning-Augmented and Randomized Algorithms for Line Aggregation with Delays
Theory
Optimization
- Introduction of a deterministic learning-augmented algorithm with improved robustness and consistency for line aggregation with delays.
- Development of a randomized algorithm that achieves a competitive ratio better than existing deterministic benchmarks.
- Establishment of a new lower bound for randomized algorithms in the context of line aggregation.
- Combination of learning-augmented and randomized techniques to enhance algorithm performance.
Read more
Learning-Augmented and Randomized Algorithms for Line Aggregation with Delays
Summary
This paper investigates learning-augmented and randomized algorithms for online aggregation with delays on a line metric. The authors propose a deterministic learning-augmented Balance algorithm that achieves (4/λ + 1/λ²)-robustness and (4 + λ)-consistency, where λ is a parameter indicating the trust in the advice given. Additionally, they introduce a randomized algorithm that is (e + 1)-competitive against an oblivious adversary, outperforming the deterministic Balance algorithm's 5-competitive ratio and establishing a new lower bound of e for randomized algorithms. Furthermore, they present a randomized learning-augmented algorithm that combines the benefits of both approaches, achieving (e/λ + 1/λ²)-robustness and (e + λ)-consistency. The paper also includes numerical experiments that validate the theoretical findings, demonstrating that the proposed algorithms outperform existing methods and maintain stability as the advice replacement rate increases.
Methodology
The authors utilize a combination of deterministic and randomized approaches to develop algorithms for line aggregation with delays. They analyze the performance of these algorithms using worst-case competitive analysis and establish robustness and consistency metrics. The algorithms are evaluated against adversarial models and through numerical experiments to assess their empirical performance.
Results
The deterministic learning-augmented algorithm achieves (4/λ + 1/λ²)-robustness and (4 + λ)-consistency. The randomized algorithm shows an (e + 1)-competitive ratio, improving upon previous benchmarks, while the randomized learning-augmented algorithm achieves (e/λ + 1/λ²)-robustness and (e + λ)-consistency. The numerical experiments confirm that the proposed algorithms outperform existing online algorithms.
Implications
The findings suggest that integrating learning-augmented methods with randomized algorithms can significantly enhance performance in online decision-making scenarios, particularly in applications involving service requests and resource allocation. This approach may be applicable in logistics, inventory management, and data transmission where timely decision-making is crucial.
VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction
NLP
Large Language Models
Generative Models
- VESTIGE offers a corruption-aware fine-tuning strategy that adapts masking based on empirical degradation profiles.
- The method is validated through ancient DNA reconstruction, showcasing its effectiveness in a specific biological context.
- VESTIGE outperforms standard masked-language models in terms of reconstruction accuracy and validation metrics.
- The approach is parameter-free and does not require architectural changes, making it easy to implement.
Read more
VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction
Summary
The paper introduces VESTIGE, a novel masking strategy designed for fine-tuning genomic transformers, particularly in the context of ancient DNA (aDNA) reconstruction. Traditional masked-language-model (MLM) fine-tuning employs a uniform masking probability across token positions, which can be detrimental when the degradation process is known and concentrated at specific positions. VESTIGE addresses this by providing a parameter-free, drop-in replacement for the standard MLM collator that aligns the masking distribution with an empirically measured corruption profile. The authors validate VESTIGE using aDNA reconstruction, where cytosine deamination creates a position-dependent substitution gradient. By utilizing the mapDamage2 tool to quantify this gradient, VESTIGE adjusts the masking probabilities accordingly, ensuring that the mean masking rate remains consistent with standard MLM practices. The results demonstrate that VESTIGE outperforms traditional MLM across various terminal-zone widths, significantly reducing validation cross-entropy and achieving high-quality reconstructions. This approach is not limited to aDNA but can be applied to any scenario where degradation profiles are measurable, making it a versatile tool for intelligent systems dealing with degraded or noisy sequence inputs.
Methodology
VESTIGE employs a parameter-free, drop-in replacement for the standard MLM collator, which adjusts the masking distribution based on an empirically measured corruption profile. The method utilizes the mapDamage2 tool to quantify position-dependent degradation rates, allowing for a tailored masking strategy that reflects the specific challenges of the reconstruction task.
Results
VESTIGE consistently outperformed standard MLM across six terminal-zone widths, achieving an increase in performance metrics (Ξ=+4.18 to +10.35 pp, p < 10^{-8}) and reducing validation cross-entropy by 13%. The method produced ESMFold reconstructions with TM-scores exceeding 0.95 across all tested reconstructions, even under amplified damage conditions. Additionally, a 1D CNN biosecurity classifier demonstrated an AUC of 0.935 on a held-out virulence-gene test set.
Implications
The VESTIGE framework has significant implications for genomic data analysis, particularly in contexts where data degradation is a concern. Its ability to adaptively mask based on empirical degradation profiles can enhance the performance of genomic transformers, making it a valuable tool for bioinformatics and other fields dealing with noisy or corrupted sequence data.
Measuring Distortion in the Empty Regions of Dimensionality Reduction Scatterplots with the Gap Index
Theory
Interpretability
Computer Vision
- Introduction of the Gap Index (GI) as a quality metric for dimensionality reduction.
- Focus on measuring distortion in empty regions of scatterplots, which are often overlooked by traditional metrics.
- The GI is computed by decomposing projections into triangles and comparing deformations with high-dimensional data.
- Demonstrates sensitivity to small structural deformations with high visual impact.
Read more
Measuring Distortion in the Empty Regions of Dimensionality Reduction Scatterplots with the Gap Index
Summary
This paper introduces the Gap Index (GI), a novel quality metric for evaluating the distortion in dimensionality reduction (DR) projections, particularly focusing on the empty regions of scatterplots. Traditional metrics often overlook the significance of these empty areas, which can convey important visual information about data structure. The GI addresses this gap by decomposing the 2D projection into triangles and comparing their deformations with high-dimensional counterparts. This allows for a more nuanced understanding of distortion, capturing local visual impacts that are often missed by conventional metrics. The GI can be aggregated into a single scalar value or visualized regionally on the scatterplot, providing insights into areas of high distortion. The authors demonstrate that the GI is sensitive to small structural changes and is computationally efficient, making it suitable for large datasets. The paper also discusses the limitations of existing quality metrics and presents use cases and comparisons with other methods, highlighting the GI's effectiveness in visual analysis.
Methodology
The methodology involves partitioning the 2D projection into triangles, calculating the deformation of each triangle by comparing it to its high-dimensional counterpart, and aggregating these deformations into a single scalar value or visualizing them on the scatterplot.
Results
The results indicate that the Gap Index is more sensitive to visually significant distortions compared to traditional metrics like stress and trustworthiness. It effectively highlights regions of high distortion in scatterplots, enhancing the reliability of visual analysis.
Implications
The implications of this work suggest that the Gap Index can improve the trustworthiness of visual analytics in high-dimensional data analysis, aiding researchers and practitioners in better understanding the structure of their data and making more informed decisions based on visualizations.