AI-generated summaries
Today's ML research,
without the noise.
Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.
24
Papers today
8h
Update frequency
7
Days of history
Metag: A dataset to build agentic meta-reviewing capabilities
NLP
- Introduction of Metag, a dataset for meta-reviewing in scientific peer review.
- Dataset contains 349 action items linked to manuscript changes.
- Addresses the need for traceability between reviewer feedback and manuscript revisions.
- Publicly available to support research and development of meta-reviewing tools.
Read more
Metag: A dataset to build agentic meta-reviewing capabilities
Summary
The paper introduces Metag, a novel dataset aimed at enhancing the capabilities of meta-reviewers in the scientific peer review process. As the volume of conference submissions increases, the burden on meta-reviewers to synthesize feedback from multiple reviewers, author rebuttals, and manuscript revisions has grown significantly. Metag addresses this challenge by providing a structured dataset that links reviewer concerns to specific changes made in scientific manuscripts during the review-rebuttal process. The dataset consists of 349 high-quality action items, each associated with corresponding manuscript diffs that illustrate the changes made in response to reviewer feedback. The data collection methodology involves comparing pre-review and post-acceptance manuscript versions, identifying differences, and aligning these with action items from OpenReview discussions. This dataset is publicly available and aims to facilitate the development of AI tools that can assist meta-reviewers in verifying author responses and improving transparency in the peer review process.
Methodology
Metag was created by collecting manuscript versions before and after the review process, computing diffs between these versions, and aligning the changes with reviewer comments through human annotation. This process allows for the identification of specific edits that correspond to reviewer action items.
Results
The resulting dataset includes 349 action items that effectively link reviewer feedback to concrete changes in manuscripts, providing a valuable resource for developing AI systems that assist meta-reviewers in their evaluations.
Implications
The Metag dataset can significantly improve the efficiency and effectiveness of meta-reviewers by providing tools that enhance their ability to trace changes in manuscripts. This can lead to better accountability in the peer review process and potentially improve the quality of scientific publications.
TRACE-C: Rank-Calibrated Relational Anomaly Detection for Multi-Stream Operational Telemetry
Time Series
- TRACE-C is designed for detecting joint anomalies in multi-stream telemetry data.
- The method employs a rank-calibrated approach using three distinct channels for anomaly detection.
- Evaluation on real-world data shows TRACE-C's effectiveness in ranking anomalies, with specific insights into the contributions of different channels.
- The paper emphasizes the importance of interpretability and reproducibility in anomaly detection methods.
Read more
TRACE-C: Rank-Calibrated Relational Anomaly Detection for Multi-Stream Operational Telemetry
Summary
The paper presents TRACE-C, a novel anomaly detection method tailored for multi-stream operational telemetry data. Traditional anomaly detection techniques often fail to identify joint anomalies when individual streams remain within normal ranges. TRACE-C addresses this by employing a rank-calibrated approach that utilizes three distinct channels: a maximum normalized local sum, a Gaussian copula-form dependence contrast on robust-z residuals, and a worst standardized AR(1) innovation. The method aggregates these channels using Fisher's method to produce a ranked output. The evaluation of TRACE-C on six Great Britain grid streams from 2019 and a frozen dataset from 2020 reveals its effectiveness in ranking anomalies, such as Storm Atiyah, while also highlighting the limitations of the copula-form channel. The results indicate that the method can discern operational anomalies more effectively than traditional approaches, particularly in the presence of serial dependencies and nonstationarity in telemetry data. The paper emphasizes the importance of interpretability in anomaly detection and provides a comprehensive evaluation framework, ensuring that all results are reproducible and auditable.
Methodology
TRACE-C utilizes a three-channel detection framework that incorporates maximum normalized local sums, Gaussian copula-form dependence contrasts on robust-z residuals, and standardized AR(1) innovations. The outputs from these channels are Fisher-aggregated and ranked to identify anomalies. The method is designed to be auditable and reproducible, with all results generated from machine-readable reports.
Results
The evaluation of TRACE-C on six Great Britain grid streams identified Storm Atiyah as the top-ranked anomaly in 2019, although further analysis indicated that this ranking was primarily influenced by the local channel rather than the copula-form channel. In 2020, no anomalies were detected, suggesting a lack of significant operational events during that period. The results demonstrate that TRACE-C can effectively discern anomalies in complex telemetry data.
Implications
TRACE-C has potential applications in operational monitoring across various industries where multi-stream telemetry data is prevalent. Its ability to detect joint anomalies can enhance decision-making processes in real-time monitoring systems, potentially leading to improved operational efficiency and risk management.
BackDFL: A Unified Benchmark For Backdoor Attacks and Defenses In Decentralized Federated Learning
Federated Learning
- BackDFL provides a comprehensive benchmark for evaluating backdoor attacks in DFL.
- Current defenses against backdoor attacks in DFL are inadequate, failing at low malicious participation rates.
- The paper highlights the importance of realistic threat models and consistent evaluation methodologies.
- Decentralization in federated learning introduces unique vulnerabilities not present in centralized systems.
Read more
BackDFL: A Unified Benchmark For Backdoor Attacks and Defenses In Decentralized Federated Learning
Summary
The paper introduces BackDFL, a unified benchmark designed to evaluate the robustness of Decentralized Federated Learning (DFL) against backdoor attacks. DFL, which eliminates the need for a centralized parameter server, is particularly vulnerable to adaptive backdoor attacks that can implant hidden behaviors while maintaining high performance on clean tasks. The authors argue that existing evaluations of DFL security are flawed due to simplified threat models and inconsistent methodologies. BackDFL addresses these issues by providing a systematic framework for assessing DFL under realistic attack scenarios. Through extensive experiments, the authors demonstrate that current Byzantine-robust DFL methods and adapted defenses fail under relatively low rates of malicious participation, especially in heterogeneous environments. The results highlight significant vulnerabilities in DFL systems, emphasizing the need for more robust evaluation protocols and defense mechanisms.
Methodology
The authors developed BackDFL as a benchmark to systematically evaluate DFL under various backdoor attack scenarios. They conducted extensive experiments comparing thirteen different defense mechanisms against adaptive backdoor attacks, analyzing their effectiveness across different communication topologies and participant behaviors.
Results
The experiments revealed that both state-of-the-art Byzantine-robust DFL methods and adapted defenses were ineffective against backdoor attacks, particularly when malicious participation rates were as low as 15%. The robustness of these methods varied significantly depending on the communication graph topology, exposing critical vulnerabilities in decentralized learning systems.
Implications
The findings suggest that current DFL methods may be overestimated in terms of their security, indicating a need for more robust defenses and evaluation frameworks. This research could influence future developments in secure decentralized learning systems, particularly in applications involving sensitive data or critical infrastructure.
From Thermal Preference Prediction to Adaptive Thermal Intervention: A Reinforcement Learning Approach Using Physiological and Environmental Sensing
Reinforcement Learning
Multimodal
- Introduces a two-stage personalized thermal comfort approach integrating physiological and environmental data.
- Develops participant-specific Comfort Oracles using ensemble learning for accurate thermal preference prediction.
- Integrates Comfort Oracles with reinforcement learning controllers to recommend adaptive thermal interventions.
- Demonstrates the effectiveness of personalized comfort prediction across diverse participants.
Read more
From Thermal Preference Prediction to Adaptive Thermal Intervention: A Reinforcement Learning Approach Using Physiological and Environmental Sensing
Summary
This paper addresses the challenge of personalized thermal comfort in building environments, which is crucial for occupant wellbeing and efficient HVAC control. Traditional HVAC systems often rely on static setpoints and population-level models that overlook individual physiological differences. The authors propose a two-stage approach that combines multimodal physiological and environmental sensing with reinforcement learning (RL) for adaptive thermal interventions. In Stage 1, participant-specific Comfort Oracles are created using ensemble learning models that analyze wearable physiological signals (like heart rate and skin temperature) alongside environmental data to predict individual thermal preferences. In Stage 2, these oracles are integrated into three RL controllers—Contextual Bandits (CB), Q-Learning (QL), and Deep Q-Network (DQN)—to recommend temperature interventions that maximize comfort while minimizing unnecessary actions. Experimental results demonstrate that the personalized comfort predictions are effective across different participants, with environmental factors enhancing the predictive performance. The study shows that the CB controller yields the highest predicted comfort and mean reward, while QL excels in reward per intervention, and DQN offers greater flexibility but with more variability in actions. This work lays the groundwork for future integration with occupant-centric HVAC systems, moving from passive prediction to active, personalized thermal management.
Methodology
The methodology involves a two-stage process: first, creating personalized Comfort Oracles using ensemble learning models trained on physiological and environmental data; second, embedding these oracles into three types of reinforcement learning controllers (CB, QL, DQN) to generate adaptive thermal intervention policies.
Results
The results indicate that personalized comfort prediction is effective, with the CB controller achieving the highest predicted comfort and mean reward, QL providing the best reward per unit of intervention, and DQN allowing for greater action flexibility but with increased variability.
Implications
This research has significant implications for the design of smart buildings and HVAC systems, enabling more responsive and energy-efficient climate control tailored to individual occupant needs, which could enhance comfort and reduce energy waste.
COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models
Large Language Models
Efficient ML
Optimization
- COEC enhances structured pruning by applying both left and right orthogonal rotations to retained weights.
- The method is training-free and does not require backpropagation or retraining of model parameters.
- COEC improves perplexity and zero-shot accuracy across various models and sparsity levels.
- The framework is adaptable to different structured pruning methods and column selection criteria.
Read more
COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models
Summary
The paper introduces COEC (Calibrated Orthogonal-Equivalence Compensation), a novel training-free framework designed to enhance structured pruning of large language models (LLMs). Structured pruning, which reduces model size and inference costs by removing entire weight columns, often leads to accuracy degradation due to output errors. Existing compensation methods have limitations, primarily focusing on either additive biases or single orthogonal rotations that do not adequately adjust the input singular frame. COEC addresses these issues by applying alternating left and right orthogonal rotations to the retained weights, optimizing the right rotation on a reduced Stiefel manifold. The framework employs generalized cross-validation for singular value rescaling and introduces a calibration Gram matrix tempering to mitigate the influence of high-energy activation directions. Additionally, an alignment penalty is included to maintain geometric relationships between adjacent attention projections. COEC operates independently of the column pruning criterion, making it versatile across various structured pruning methods. Experimental results demonstrate that COEC consistently improves perplexity and zero-shot accuracy across multiple models (Llama-3, Llama-3.1, and Qwen2.5) and sparsity levels, particularly showing greater performance gains at higher sparsity rates. This indicates that COEC effectively recovers performance lost due to column removal, showcasing its potential as a robust post-pruning compensation technique.
Methodology
COEC employs a training-free approach that utilizes second-order statistics from a small calibration set. It applies alternating left and right orthogonal rotations to the retained weights, optimizing the right rotation on a reduced Stiefel manifold. Singular values are rescaled using generalized cross-validation, and additional components like spectral tempering and alignment penalties are introduced to enhance calibration and preserve geometric relations.
Results
Experiments on Llama-3, Llama-3.1, and Qwen2.5 models demonstrate that COEC improves perplexity on all models and zero-shot accuracy in most cases, with larger improvements observed at higher pruning ratios, indicating effective recovery of performance lost due to column removal.
Implications
The COEC framework offers a significant advancement in the structured pruning of large language models, providing a method to maintain model performance while reducing size and inference costs. Its adaptability to various pruning methods makes it a valuable tool for optimizing LLM deployment in resource-constrained environments.
BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers
NLP
Large Language Models
Efficient ML
- BF1 introduces a causal dyadic sparse-attention mechanism for efficient long-context transformers.
- The method achieves O(n log n) complexity for selected token interactions and O(log n) communication depth.
- BF1 demonstrates significant speedup over dense attention, achieving 10.91× per-layer speedup at 32K tokens.
- The method outperforms existing approaches in perplexity during matched adaptation studies.
Read more
BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers
Summary
The paper introduces BF1, a novel deterministic block-aligned dyadic sparse-attention mechanism designed to enhance the efficiency of long-context transformers. Traditional dense causal attention mechanisms are computationally expensive, especially as the context length increases. BF1 combines a small exact local neighborhood with a global first block and logarithmically spaced historical blocks, achieving a significant reduction in computational complexity. The authors demonstrate that each converted layer of BF1 operates with O(n log n) selected token interactions and O(log n) graph communication depth. Experimental results on an NVIDIA RTX PRO 6000 GPU show that BF1 achieves a per-layer speedup of 10.91× at 32K tokens compared to dense attention. Additionally, retrofitting eight layers of the Qwen3-0.6B model with BF1 reduces the warm whole-model time to the first token by 15.3% at 32K tokens. The paper also reports that BF1 outperforms other methods in terms of perplexity during a matched adaptation study, establishing its effectiveness as a sparse operator for long-context systems. The contributions include a reproducible causal dyadic block route, correctness-gated performance measurements, and a characterization of selected-page decode semantics, while acknowledging that the paper does not claim a fully subquadratic model or general preservation of retrieval capabilities.
Methodology
The authors developed the BF1 operator by retrofitting a pretrained language model with a causal dyadic block graph structure. They conducted experiments to measure the performance of BF1 against dense attention and other sparse methods, focusing on selected interactions, communication depth, and overall model latency. The evaluation included correctness-gated measurements and a matched adaptation study across multiple training seeds.
Results
BF1 achieved a per-layer speedup of 10.91× at 32K tokens compared to dense attention. The retrofitting of eight layers resulted in a 15.3% reduction in warm whole-model time to the first token at 32K tokens. In a matched adaptation protocol, BF1 ranked first in perplexity across three training seeds, outperforming dense continued training and other sparse methods.
Implications
The BF1 operator has potential applications in improving the efficiency of long-context transformers, making them more viable for real-time applications and scenarios where computational resources are limited. Its design could influence future research in sparse attention mechanisms and transformer architectures.
Rethinking Expressivity and Efficiency in Test-Time Training
NLP
Large Language Models
Efficient ML
- E2-TTT bridges the gap between expressivity and efficiency in Test-Time Training.
- The method allows for fully parallelized chunk-level training while preserving temporal update structures.
- E2-TTT outperforms existing methods in in-context retrieval and length extrapolation tasks.
- The approach is validated on large models and extensive datasets, demonstrating robust performance.
Read more
Rethinking Expressivity and Efficiency in Test-Time Training
Summary
This paper addresses the limitations of existing Test-Time Training (TTT) methods, which struggle to balance expressivity and efficiency during inference. The authors propose a novel approach called E2-TTT (Expressive and Efficient TTT) that allows for continuous weight updates while maintaining computational efficiency. E2-TTT utilizes a closed-form state transition that captures the dynamics of per-token updates while enabling parallelized chunk-level training. The method preserves the temporal structure of updates that prior chunk-wise methods overlook. The authors validate E2-TTT by training large models (up to 1.3B parameters) on extensive datasets, demonstrating performance on par with existing TTT and hybrid attention baselines in language modeling, while excelling in tasks like in-context retrieval and length extrapolation. Notably, E2-TTT retains over 90% accuracy in a challenging passkey retrieval test at eight times the training context length. The approach also matches the training throughput of efficient chunk-wise methods, effectively reconciling expressivity with efficiency, and shows promise for applications in long-context video understanding.
Methodology
E2-TTT employs a closed-form scalar kernel that enables parallel execution of recurrent dynamics with per-token learning rates, momentum, and decay. This formulation allows for chunk-level training while retaining the expressivity of token-wise updates. The authors integrate E2-TTT into a hybrid architecture that combines TTT with sliding window attention to handle long-horizon tasks effectively.
Results
E2-TTT was empirically validated by training models up to 1.3B parameters on 15B tokens, achieving performance comparable to existing TTT and hybrid attention baselines in language modeling. It excelled in in-context retrieval and length extrapolation, maintaining over 90% accuracy in the 'Needle in a Haystack' test at eight times the training context length. The method also matched the training throughput of efficient chunk-wise techniques.
Implications
The findings suggest that E2-TTT can enhance the capabilities of large language models and other neural architectures in processing long-context tasks. Its ability to reconcile expressivity with efficiency opens avenues for real-time learning and adaptation in various applications, including multimodal understanding and dynamic environments.
AgentDecarbonizer: Carbon-Aware Execution for AI Agents
Optimization
Efficient ML
- AgentDecarbonizer optimizes carbon emissions for AI agents executing long-running tasks.
- The tool accounts for execution time uncertainty and cache recomputation overhead.
- It can reduce carbon emissions by up to 57.9% compared to traditional execution methods.
- The methodology is applicable to various agent systems beyond OpenClaw.
Read more
AgentDecarbonizer: Carbon-Aware Execution for AI Agents
Summary
The paper introduces AgentDecarbonizer, a carbon optimization tool designed for AI agents that utilize large language models (LLMs) in long-running workflows. These workflows, while enabling complex tasks such as software repair and data analysis, can lead to significant carbon emissions due to repeated model invocations. The authors characterize the carbon emissions of OpenClaw agent workloads using the WildClawBench benchmark, revealing that emissions are influenced by factors such as token consumption and the carbon intensity of the energy grid. They identify deadline flexibility as a key opportunity for carbon-aware execution, allowing tasks to wait for lower-carbon periods or shift to less carbon-intensive grids. However, this requires managing uncertain execution times and cache recomputation overhead. AgentDecarbonizer conservatively estimates task durations and selects execution schedules that meet deadlines while minimizing emissions. Evaluations demonstrate that AgentDecarbonizer can reduce carbon emissions by up to 57.9% compared to a carbon-agnostic baseline, showcasing its effectiveness in optimizing the carbon footprint of AI agent operations.
Methodology
The authors developed AgentDecarbonizer to run alongside OpenClaw, estimating task durations and carbon intensity while selecting execution schedules. They utilized a lightweight model for duration estimation and incorporated cache-aware optimization to manage the trade-offs between carbon reduction and cache recomputation overhead. The evaluation involved running 60 tasks across different carbon intensity grids and comparing the results against carbon-agnostic and carbon-optimal baselines.
Results
AgentDecarbonizer was evaluated on WildClawBench workloads, achieving carbon emissions reductions of up to 57.9% compared to a carbon-agnostic baseline and 37.5% compared to a baseline that selects the carbon-optimal grid at the start of the task. The results indicate that the tool effectively balances execution efficiency and carbon footprint.
Implications
The findings suggest that integrating carbon-aware execution strategies into AI agent workflows can significantly mitigate environmental impacts. This approach can be applied to various domains where AI agents are deployed, promoting sustainability in AI operations.
Tydra: An Efficient Hybrid Model for Tabular Data
Efficient ML
- Tydra is the first hybrid Transformer-SSM architecture designed for tabular in-context learning.
- It achieves up to 30% lower inference time than TabPFN while matching its predictive accuracy.
- Tydra outperforms a much larger Hydra model in both speed and accuracy.
- The study highlights the advantages of hybrid architectures for improving efficiency in tabular data processing.
Read more
Tydra: An Efficient Hybrid Model for Tabular Data
Summary
The paper introduces Tydra, a novel hybrid architecture that combines Transformer and State Space Model (SSM) layers for efficient tabular data processing. While existing models like TabPFN demonstrate strong predictive performance, they suffer from quadratic computational costs with increasing context length, making them impractical for large datasets. Conversely, SSM-based models like Hydra offer subquadratic efficiency but at the cost of predictive accuracy. Tydra aims to bridge this gap by interleaving attention and SSM layers, achieving a balance between speed and accuracy. The authors evaluate Tydra across 30 OpenML datasets, demonstrating a 30% reduction in inference time compared to TabPFN while maintaining comparable predictive performance. Additionally, Tydra outperforms a significantly larger Hydra model, showcasing the potential of hybrid architectures in enhancing the efficiency of tabular foundation models. The findings suggest that Tydra's architecture could facilitate practical applications in environments with limited computational resources, such as healthcare institutions that require local inference.
Methodology
The authors developed Tydra by interleaving attention layers from the TabPFN model with SSM layers from the Hydra architecture. They conducted extensive experiments on 30 OpenML datasets to evaluate Tydra's performance in terms of inference speed and predictive accuracy compared to existing models.
Results
Tydra demonstrated a 30% reduction in inference time relative to TabPFN while retaining similar predictive performance. It also outperformed an approximately ten-times-larger Hydra model, indicating that hybrid architectures can effectively balance accuracy and efficiency in tabular data tasks.
Implications
The development of Tydra has significant implications for practical applications in fields requiring efficient processing of tabular data, such as healthcare, finance, and any domain where data privacy necessitates local inference. The hybrid architecture could enable organizations to leverage advanced predictive models without incurring high computational costs.
Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study
Time Series
- Introduces MLAMA, an ensemble method for combining ARIMA and machine learning models.
- Evaluates ARIMA, Random Forest, and XGBoost for COVID-19 case forecasting in Ontario.
- Finds that ARIMA is responsive but less accurate over longer horizons, while machine learning models are more stable.
- Demonstrates the effectiveness of a model-agnostic framework for real-time forecasting.
Read more
Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study
Summary
This paper addresses the challenges of public health forecasting, particularly in the context of the COVID-19 pandemic, where abrupt changes in data can complicate predictions. The authors evaluate the performance of three forecasting models: Autoregressive Integrated Moving Average (ARIMA), Random Forest, and Extreme Gradient Boosting (XGBoost), using 190 weekly observations of COVID-19 case counts in Ontario from January 2020 to October 2023. They employ a rolling-origin time-series cross-validation approach to ensure the temporal integrity of the data during model tuning and evaluation. The study introduces a novel ensemble method called Machine Learning and ARIMA Model Averaging (MLAMA), which combines the strengths of the individual models by assigning weights based on forecast horizon and responsiveness. The results indicate that while ARIMA is quick to adapt to changes, its accuracy declines over longer forecast horizons. In contrast, Random Forest and XGBoost exhibit more stable performance over time. The MLAMA framework outperforms individual models in terms of normalized mean absolute percentage error across various forecast horizons, suggesting that a tailored approach to model selection based on operational conditions is more effective than relying on a single model. The findings underscore the importance of combining statistical and machine learning methods for improved public health forecasting.
Methodology
The study utilized a rolling-origin time-series cross-validation method to evaluate the forecasting models, ensuring that future data was not used in training. The models were tuned using a squared relative-error objective, and the MLAMA framework was developed to assign weights to different models based on their performance across various forecast horizons and responsiveness settings.
Results
The retrospective analysis revealed that ARIMA quickly adapted to changes but had increased error at longer forecast horizons. Random Forest and XGBoost showed more stable performance over time. The MLAMA framework achieved lower normalized mean absolute percentage error compared to individual models, particularly excelling in responsiveness and accuracy across different forecast horizons.
Implications
The findings suggest that public health forecasting can benefit from a hybrid approach that leverages both statistical and machine learning models. The MLAMA framework provides a practical tool for real-time forecasting, which could enhance public health preparedness and response strategies.
A Neurosymbolic Approach for Constructing Planning Domain Models from Clinical Narratives
NLP
Large Language Models
Interpretability
- NSPIN effectively bridges the gap between unstructured clinical narratives and formal probabilistic planning models.
- The framework utilizes LLMs to infer implicit clinical actions and preconditions, enhancing logical consistency.
- Evaluation on a large dataset shows that NSPIN generalizes well to new surgical workflows, outperforming LLM-only baselines.
- The induced models are interpretable and align with established surgical practices, facilitating clinical validation.
Read more
A Neurosymbolic Approach for Constructing Planning Domain Models from Clinical Narratives
Summary
This paper presents NSPIN, a neurosymbolic framework designed to induce probabilistic planning domain models from unstructured clinical narratives, specifically focusing on surgical procedures like laparoscopic appendectomy. The challenge lies in the complexity of surgical workflows and the implicit actions often described in clinical notes, which traditional symbolic methods and large language models (LLMs) struggle to address independently. NSPIN combines the linguistic capabilities of pretrained LLMs with the logical structure of symbolic induction to extract structured event sequences from raw text. It then synthesizes a Probabilistic Planning Domain Definition Language (PPDDL) model, refining its preconditions through LLM-generated revisions validated by empirical data. The framework was evaluated on a dataset of 2,660 surgical notes from nine surgeons, demonstrating that NSPIN effectively generalizes to unseen surgical workflows and aligns well with expert clinical knowledge. This approach not only aids in the formalization of surgical procedures but also has potential applications in training, quality improvement, and the development of autonomous surgical systems.
Methodology
NSPIN employs a neurosymbolic approach that integrates LLMs for extracting structured event sequences from clinical narratives. It constructs a PPDDL model and refines it using LLM-generated revisions, guided by empirical validation. The framework is specifically tailored to handle the linguistic diversity and implicit information present in surgical notes.
Results
The evaluation of NSPIN on 2,660 surgical notes revealed that the induced planning models generalize effectively to unseen cases. Expert reviews confirmed that the knowledge captured by NSPIN is largely consistent with actual surgical practices, indicating the framework's reliability and applicability in clinical settings.
Implications
The successful application of NSPIN could lead to improved decision support systems in surgery, enhanced training for medical residents, and the development of AI systems capable of assisting in surgical procedures. Furthermore, it may contribute to better quality assurance in surgical practices by systematically analyzing and comparing surgical workflows.
Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces
Theory
Optimization
- Introduces a framework for neural approximation in infinite-dimensional spaces with finite coordinate representation.
- Decomposes approximation error into resolution, finite-width, and statistical components.
- Demonstrates that statistical complexity is uniform across varying input resolutions.
- Utilizes a greedy selection algorithm for constructing neural networks.
Read more
Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces
Summary
This paper addresses the challenge of neural approximation in infinite-dimensional spaces, particularly when inputs are represented through a finite number of coordinates. The authors develop a quantitative theory for shallow neural models that incorporates a parameter-normalized neural dictionary and a weighted variation class. They identify three sources of error in approximation: the truncation of input coordinates, the finite width of the neural network, and the finite number of observations available for training. The proposed greedy selection algorithm iteratively identifies neural units that correlate with the unexplained residual, leading to a decomposition of the approximation error into resolution, finite-width, and statistical components. Notably, the statistical complexity remains uniform across varying input resolutions, which is a significant departure from traditional finite-dimensional analyses. The paper also includes synthetic experiments to illustrate the theoretical predictions regarding resolution, width, and sample size effects. Overall, the authors provide a comprehensive framework that enhances understanding of how approximation and learning interact in infinite-dimensional settings, while also highlighting the computational challenges that remain.
Methodology
The authors employ a greedy selection algorithm to construct neural networks, focusing on minimizing approximation error by selecting neural units that correlate with the residual error. They analyze the approximation error through a parameter-normalized neural dictionary and a weighted variation class, allowing for a clear separation of error components. The theoretical framework is supported by synthetic experiments that illustrate the effects of resolution, width, and sample size.
Results
The study establishes that the approximation error can be effectively decomposed into three distinct components: resolution error, finite-width error, and statistical error, with the latter showing no explicit dependence on the number of retained input coordinates. The results indicate that finer input resolution does not necessarily increase estimation penalties, challenging traditional assumptions in finite-dimensional analyses.
Implications
The findings have significant implications for the design and training of neural networks in applications involving infinite-dimensional data, such as functional data analysis, signal processing, and other areas where inputs are not easily represented in finite dimensions. The framework can guide the development of more efficient neural architectures and training methodologies.
TracingFlow: A Simulation-Free Trajectory Inference Framework Based on Second-Order Dynamics
Generative Models
Optimization
Time Series
- TracingFlow generalizes trajectory inference to second-order dynamics, overcoming the limitations of first-order methods.
- The framework provides an efficient solution to the Dynamical Optimal Acceleration Transport (DOAT) problem by regressing acceleration fields.
- It achieves superior accuracy in distribution reconstruction and trajectory fidelity compared to existing simulation-free frameworks.
- The integration of lineage tracing priors allows for the recovery of biologically plausible dynamical structures.
Read more
TracingFlow: A Simulation-Free Trajectory Inference Framework Based on Second-Order Dynamics
Summary
TracingFlow introduces a novel framework for trajectory inference that addresses the limitations of existing Optimal Transport (OT) methods, which typically rely on first-order dynamics. The authors propose a simulation-free approach that generalizes to second-order dynamics, allowing for the modeling of acceleration fields rather than just velocity fields. This approach is particularly beneficial for applications in single-cell omics, where capturing complex biological processes such as cell differentiation is crucial. By formulating the Dynamical Optimal Acceleration Transport (DOAT) problem, TracingFlow efficiently regresses the acceleration field and initial velocity, avoiding the computational costs associated with traditional ODE-based methods. The framework demonstrates superior performance in reconstructing distributions and maintaining trajectory fidelity when evaluated on synthetic and real-world datasets, particularly in scenarios where lineage tracing information is integrated. The results indicate that TracingFlow not only enhances accuracy but also aligns with biological plausibility, making it a significant advancement in the field of trajectory inference.
Methodology
TracingFlow employs a simulation-free Flow Matching framework that regresses the acceleration field and initial velocity to solve the DOAT problem. This approach avoids the need for numerical integration typically required in ODE-based methods, allowing for efficient computation and enhanced expressiveness in modeling complex dynamics.
Results
The evaluation of TracingFlow on complex synthetic datasets and large-scale single-cell RNA sequencing (scRNA-seq) datasets shows that it outperforms existing methods in terms of distributional reconstruction accuracy and trajectory faithfulness. The framework effectively incorporates biological priors, leading to results that are both mathematically optimal and biologically plausible.
Implications
TracingFlow has significant implications for the field of single-cell omics and trajectory inference, as it provides a more accurate and efficient means of modeling complex biological processes. Its ability to integrate lineage tracing information enhances the understanding of cellular dynamics, which could lead to advancements in biological research and therapeutic applications.
ConceptTS: LLM-Guided Concept Bottlenecks for Interpretable Multivariate Time-Series Forecasting
Time Series
Interpretability
Large Language Models
- Introduces a novel framework for interpretable multivariate time-series forecasting using LLM-guided concept construction.
- Organizes concepts into three bottlenecks for better understanding of historical context, local intervals, and overall forecasts.
- Achieves competitive forecasting accuracy while enhancing interpretability through concept activations.
- Eliminates the need for manual concept annotation, streamlining the process of model training.
Read more
ConceptTS: LLM-Guided Concept Bottlenecks for Interpretable Multivariate Time-Series Forecasting
Summary
ConceptTS is an innovative framework designed to enhance the interpretability of multivariate time-series forecasting by organizing predictions around human-readable concepts. Traditional forecasting models, while capable of capturing complex dependencies, often lack transparency, making it difficult for practitioners to understand the reasoning behind predictions. ConceptTS addresses this issue by utilizing a large language model (LLM) to generate relevant concepts and labeling rules, thus eliminating the need for extensive manual annotation. The framework organizes concepts into three distinct bottlenecks that represent historical context, local forecast intervals, and the overall forecast horizon. A shared decoder synthesizes these representations to produce forecasts, allowing for explicit decision-making processes and enabling concept-level interventions. Experiments conducted on the Beijing Multi-Site Air Quality dataset demonstrate that ConceptTS achieves competitive accuracy compared to black-box models while providing semantically meaningful concept activations, thereby bridging the gap between predictive performance and interpretability.
Methodology
ConceptTS employs a large language model to propose task-relevant concepts and generate executable labeling rules, which are then organized into three complementary bottlenecks. A structure-preserving encoder predicts activations for these concepts, and a shared decoder constructs the final forecast from these representations. The model is evaluated on the Beijing Multi-Site Air Quality dataset under different forecasting settings.
Results
The experiments reveal that ConceptTS maintains an MAE within 5% of the best-performing black-box baseline while producing interpretable concept activations. Ablation studies and qualitative analyses confirm the model's reliance on concept activations for accurate forecasting, highlighting the framework's effectiveness in balancing accuracy and interpretability.
Implications
ConceptTS has significant implications for fields requiring interpretable forecasting, such as healthcare, environmental monitoring, and energy management. By providing clear insights into the factors influencing predictions, it can enhance trust and usability in high-stakes decision-making environments.
Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers
Graph Learning
Theory
- Introduces the concept of Latent-Posterior Alignment (LPA) in Bayesian GNNs.
- Demonstrates that predictive uncertainty can decrease without posterior contraction.
- Proposes Alignment-Guided Learning (AGL) to enhance model training by promoting LPA.
- Confirms the causal role of LPA in reducing predictive uncertainty through interventional experiments.
Read more
Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers
Summary
This paper investigates the dynamics of predictive uncertainty in Bayesian Neural Networks (BNNs) with a focus on Graph Neural Networks (GNNs) that utilize Bayesian output layers. The authors challenge the conventional understanding that uncertainty reduction is solely due to posterior contraction, revealing instead a phenomenon they term Latent-Posterior Alignment (LPA). They demonstrate that predictive uncertainty can decrease as latent representations align with lower-variance directions of the posterior, even when the posterior variance does not contract. To leverage this insight, the authors propose Alignment-Guided Learning (AGL), a training framework designed to promote this alignment during model training. Through interventional experiments, they confirm that disrupting LPA leads to increased predictive uncertainty, thus establishing its causal role in shaping model confidence. The findings suggest a shift in focus from the magnitude of the posterior to the geometric relationship between latent and parameter spaces, offering a new perspective on uncertainty dynamics in GNNs with Bayesian output layers.
Methodology
The authors conducted empirical analyses and interventional experiments on Bayesian Graph Neural Networks (BGNNs) with Bayesian output layers. They employed a deterministic feature extractor followed by a Bayesian output layer trained via Bayes-by-Backprop, and introduced a new training framework, Alignment-Guided Learning (AGL), to promote latent-posterior alignment.
Results
The study found that predictive uncertainty decreases as latent representations align with low-variance directions of the posterior, even when the posterior variance broadens. The introduction of AGL effectively reduced predictive uncertainty while maintaining accuracy, and improved structural calibration of the model's confidence.
Implications
The findings have significant implications for applications in safety-critical domains such as autonomous driving and medical diagnostics, where accurate uncertainty quantification is essential. The proposed methods can enhance the reliability of predictive models in low-data and distribution-shifted regimes, particularly in chemical and materials science applications.
FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space
Federated Learning
Graph Learning
Theory
- FlatLand introduces a new framework for personalized federated learning using tailored Lorentz space.
- The method effectively captures the intrinsic geometric properties of heterogeneous graph structures.
- A parameter decoupling strategy is proposed to facilitate efficient aggregation of client-specific and shared information.
- Empirical results indicate superior performance of FlatLand over existing PFL methods, especially in low-dimensional contexts.
Read more
FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space
Summary
The paper presents FlatLand, a novel approach to personalized federated learning (PFL) that addresses the challenges posed by heterogeneous client data in graph federated learning. Traditional PFL methods often overlook the intrinsic geometric properties of diverse graph structures, leading to suboptimal performance. FlatLand utilizes tailored Lorentz space, a hyperbolic geometry framework, to effectively model the negative curvature inherent in real-world graphs. The authors introduce a parameter decoupling strategy that separates heterogeneous information from common knowledge, allowing for efficient aggregation without the need for client similarity estimation or additional computational overhead. Empirical evaluations across various federated graph learning tasks demonstrate that FlatLand outperforms existing methods, particularly in low-dimensional settings, showcasing its potential for better handling of data heterogeneity in federated learning scenarios.
Methodology
The authors develop FlatLand by embedding client data into tailored Lorentz spaces, leveraging hyperbolic geometry to accommodate the negative curvature of graph structures. They implement a parameter decoupling strategy that distinguishes between personalized and shared parameters, enabling direct aggregation without complex similarity estimations or auxiliary modules.
Results
FlatLand demonstrates superior performance in various federated graph learning tasks compared to existing methods, particularly excelling in low-dimensional settings. The empirical analysis reveals that the tailored Lorentz space effectively captures the intrinsic geometric properties of client graphs, leading to improved model performance.
Implications
The findings suggest that adopting hyperbolic geometry in federated learning can significantly enhance the modeling of heterogeneous data, paving the way for more effective and efficient personalized learning systems in real-world applications involving graph data.
Approximate Homomorphisms and Convergent Representations in Transducers
Theory
- Introduces approximate homomorphisms to measure local structural similarity between transducers.
- Establishes metrics for comparing the dynamics of different transducer implementations.
- Demonstrates that minimal linear transducers can exhibit approximate homomorphisms under certain conditions.
- Identifies stability conditions for predictive transducers, highlighting the importance of structural constraints.
Read more
Approximate Homomorphisms and Convergent Representations in Transducers
Summary
This paper investigates the stability of minimal representations of controlled stochastic processes, specifically transducers, under perturbations. The motivation stems from experimental findings that suggest predictive-state structures in the latent representations of neural networks. The author introduces the concept of approximate homomorphisms, which capture local structural similarities between transducers, and develops metrics to compare their induced dynamics, termed interfaces. The study reveals that while certain simple interfaces lack approximate homomorphisms between different implementations, all minimal linear transducers implementing interfaces close to a finite-rank interface exhibit approximate homomorphisms with errors proportional to the perturbation size. An analogous stability result is established for predictive transducers under a residual metric, contingent on mild assumptions regarding the indistinguishability of belief states. These findings delineate conditions under which canonical transducer representations remain robust against perturbations, while also indicating that convergence may fail without additional structural constraints. The results lend theoretical support to the hypothesis that latent representations in modern AI models exhibit structural convergence.
Methodology
The author develops a mathematical framework to define approximate homomorphisms for standard, linear, and predictive transducers. Metrics are introduced to compare the dynamics of these transducers, and properties such as composability are proven. The analysis includes stability results under perturbations and explores the implications of these findings for neural network representations.
Results
The study finds that while some simple interfaces do not allow for approximate homomorphisms, minimal linear transducers close to a finite-rank interface do exhibit such homomorphisms with errors that scale linearly with perturbation size. Additionally, predictive transducers show stability under a residual metric, contingent on specific assumptions about belief states.
Implications
The results suggest that transducer representations in AI models can be robust to perturbations, supporting the idea that different models may converge to similar latent representations. This has implications for understanding the generalization capabilities of neural networks and the potential for shared statistical models across different architectures.
Geometric Regularization for Long-Tailed Semi-Supervised Learning via Gaussian Feature Bridges
Computer Vision
Theory
Efficient ML
- Introduction of Gaussian Bridge Consistency (GBC) for long-tailed semi-supervised learning.
- Dynamic Prototype Atlas for storing and updating labeled and pseudo-labeled exemplars.
- BridgeMix strategy enhances generalization by mixing features based on confidence levels.
- GBC reduces semantic drift and improves tail-class robustness.
Read more
Geometric Regularization for Long-Tailed Semi-Supervised Learning via Gaussian Feature Bridges
Summary
This paper addresses the challenges of long-tailed label distributions and noisy pseudo-labels in semi-supervised learning (SSL) through a novel framework called Gaussian Bridge Consistency (GBC). The authors propose a method that constructs semantic interpolation paths between unlabeled samples and high-quality class anchors, utilizing a dynamic Prototype Atlas to store labeled and pseudo-labeled exemplars. GBC employs a class-conditional Gaussian Feature Bridge in the latent space, allowing the model to transition from uncertain predictions to reliable class prototypes. A bridge consistency loss is introduced to align predictions with a geometrically interpolated target distribution. Additionally, the authors present BridgeMix, a confidence-aware feature mixing strategy that enhances generalization by interpolating both sample and anchor pairs. Extensive experiments on CIFAR10-LT and ImageNet-LT demonstrate that GBC significantly improves performance on long-tailed classes while maintaining scalability, effectively mitigating the performance collapse typically seen in long-tailed SSL scenarios.
Methodology
The methodology involves constructing Gaussian Feature Bridges in the latent space that connect unlabeled samples to class-specific anchors. The bridge consistency loss is applied to enforce alignment with a target distribution, while BridgeMix uses confidence levels to guide feature mixing between samples and anchors. This approach allows for smooth transitions in feature representation, enhancing learning stability and performance.
Results
The experiments conducted on CIFAR10-LT, CIFAR100-LT, STL10-LT, and ImageNet-LT benchmarks show that GBC consistently outperforms prior state-of-the-art methods, significantly improving accuracy for long-tailed classes without compromising scalability. The results validate the effectiveness of the proposed framework in realistic long-tailed SSL settings.
Implications
The findings suggest that GBC can be a robust solution for semi-supervised learning in scenarios with class imbalance and noisy labels, potentially benefiting applications in fields such as computer vision where long-tailed distributions are common.
Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning
Time Series
- Introduction of WINDER, a phase-equivariant self-supervised learning model for ECG data.
- Utilization of a fixed transport operator based on cardiac cycle geometry, enhancing parameter efficiency.
- Demonstrated diagnostic accuracy comparable to larger self-supervised models with significantly fewer parameters.
- Phase-equivariant latent geometry improves the interpretability and utility of learned representations.
Read more
Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning
Summary
This paper presents WINDER, a novel self-supervised learning framework designed to capture the cyclic structure of the cardiac cycle in electrocardiogram (ECG) data. The authors propose a phase-equivariant self-supervised objective that organizes representations into phase-invariant coordinates and phase-rotating harmonic subspaces. The architecture employs a fixed, closed-form transport operator derived from the geometry of the cardiac cycle, which does not introduce additional parameters. Evaluated on the PTB-XL dataset, WINDER achieves diagnostic accuracy comparable to state-of-the-art self-supervised methods while maintaining a compact model size of approximately 1.2 million parameters. The results indicate that encoding cardiac-phase symmetry effectively preserves diagnostic information and results in a latent geometry that is both interpretable and efficient. The paper highlights the importance of leveraging the periodic nature of cardiac activity to improve representation learning in time-series data.
Methodology
The authors developed WINDER by extending the LeJEPA framework to include a phase-equivariant loss function that encourages consistent latent trajectories across cardiac cycles. The model architecture organizes representations into phase-invariant coordinates, allowing for effective learning of the cardiac cycle's structure while suppressing noise and irrelevant variations.
Results
WINDER achieved diagnostic accuracy within the range of state-of-the-art self-supervised methods while utilizing only 1.2 million parameters. The model demonstrated improved performance over a control model without the transport objective, confirming the benefits of enforcing phase equivariance in latent representations.
Implications
The findings suggest that phase-equivariant self-supervised learning can enhance the analysis of ECG data, potentially leading to better diagnostic tools in cardiology. This approach may also be applicable to other physiological time-series data where cyclic patterns are present.
Scaling Muon for Diffusion Transformers
Optimization
Generative Models
Efficient ML
- Muon optimizer shows persistent advantages over AdamW in generative quality across model scales from 1.3B to 15B parameters.
- The introduction of Periodic Row-wise Muon reduces computational overhead while maintaining optimization performance.
- Significant reductions in optimizer time (46.9–54.3%) and end-to-end step time (15.7–24.3%) are achieved.
- The new method maintains generative quality comparable to vanilla Muon while improving training efficiency.
Read more
Scaling Muon for Diffusion Transformers
Summary
This paper investigates the scaling behavior and efficiency of the Muon optimizer for training large Diffusion Transformers (DiTs) ranging from 1.3B to 15B parameters. The authors establish that Muon maintains its optimization and generative quality advantages over the AdamW optimizer across various model scales. However, they identify significant computational and communication overheads introduced by the 5-step Newton–Schulz iteration (NS5) and full-momentum materialization, which can negate Muon's efficiency benefits. To address these issues, the authors propose a new method called Periodic Row-wise Muon, which reduces the frequency of full NS5 updates and employs a low-cost row-wise constrained update for the remaining steps. They also develop a distributed implementation that optimizes momentum handling and communication during training. The results demonstrate that Periodic Row-wise Muon not only preserves the generative quality of Muon but also significantly enhances end-to-end training efficiency, achieving reductions in optimizer time, step time, and communication volume while improving generative quality in larger models.
Methodology
The authors analyze the performance of the Muon optimizer on large DiTs, identifying bottlenecks related to the NS5 iteration and momentum communication. They propose the Periodic Row-wise Muon method, which alternates between full NS5 updates and low-cost row-wise updates. A distributed implementation is also designed to optimize momentum handling and communication, reducing overhead during training.
Results
The study finds that Periodic Row-wise Muon improves generative quality by 12.9–19.1% over AdamW and remains within 0.5% of vanilla Muon on smaller models while outperforming it by 4.5% at 9B and 2.7% at 15B. Additionally, it reduces optimizer time by 46.9–54.3%, end-to-end step time by 15.7–24.3%, and logical communication volume by 66.7%.
Implications
The findings suggest that optimizing the training process for large-scale models can lead to significant efficiency gains, making it feasible to train larger models without proportionally increasing resource consumption. This has implications for the development of more efficient training methodologies in machine learning.
Mutual information and sensitivity analysis for feature selection in customer targeting: a comparative study
Theory
Optimization
Interpretability
- Comparison of mutual information and sensitivity analysis for feature selection in customer targeting.
- Mutual information selected 13 features, while sensitivity analysis selected 9 features.
- Sensitivity analysis performed better in reducing false positives, while mutual information was better for higher false positive rates.
- Both methods provide valuable insights, with mutual information still being relevant despite its age.
Read more
Mutual information and sensitivity analysis for feature selection in customer targeting: a comparative study
Summary
This paper investigates the effectiveness of two feature selection techniques—mutual information (MI) and data-based sensitivity analysis (SA)—in the context of customer targeting for bank telemarketing. The authors conduct a comparative study to evaluate the advantages and disadvantages of each method by applying them to a real-world dataset. They identify the most influential features for predicting the success of telemarketing contacts, resulting in 13 features selected by MI and 9 by SA. The study finds that while MI is slightly better at handling higher false positive rates, SA performs better in minimizing false positives. The results indicate that MI remains a valid and effective method for feature selection, while SA offers good predictive performance with fewer features. This research contributes to the understanding of feature selection methods in marketing applications and provides insights for practitioners in choosing the appropriate technique based on their objectives.
Methodology
The authors applied mutual information and data-based sensitivity analysis to a bank telemarketing dataset to identify influential features for predicting the success of telemarketing contacts. They then built a logistic regression model using the selected features from each method and compared their performance based on false positive rates.
Results
The study found that sensitivity analysis yielded better results in terms of minimizing false positives, while mutual information was slightly more effective when a higher false positive rate was acceptable. The comparative analysis demonstrated the strengths and weaknesses of both methods in a practical application.
Implications
The findings suggest that practitioners in marketing and data-driven decision-making can benefit from understanding the strengths of different feature selection techniques. Depending on their specific goals—such as minimizing costs or maximizing success rates—they can choose the most suitable method for their customer targeting strategies.
Decoupling Policy Extraction for Offline Reinforcement Learning
Reinforcement Learning
Robotics
Theory
- The paper critiques the conventional coupled actor-critic paradigm in offline RL, highlighting its limitations.
- A new decoupled policy extraction paradigm is proposed, separating actor training from policy improvement.
- The decoupled approach mitigates issues related to OOD actions and the support-value trade-off.
- Extensive experiments show significant performance improvements over existing methods.
Read more
Decoupling Policy Extraction for Offline Reinforcement Learning
Summary
This paper addresses the limitations of conventional offline reinforcement learning (RL) methods that typically couple actor and critic training. In offline RL, the training data is fixed, which makes the actor's policy improvement ineffective as it cannot generate new data to validate or correct the critic's value estimations. The authors identify two main issues with the coupled approach: the out-of-distribution (OOD) amplification loop, where the actor shifts towards high-valued but potentially OOD actions, and the support-value trade-off, where conservative value estimation can hinder the selection of high-value actions. To overcome these challenges, the authors propose a decoupled policy extraction paradigm, where the actor is trained solely to model the behavior distribution, and policy improvement occurs at inference time through a separately learned critic that reranks multiple action proposals generated by the actor. This approach effectively separates the roles of the actor and critic, mitigating the identified issues. The paper presents extensive experiments demonstrating that the decoupled paradigm consistently outperforms both behavior cloning and traditional coupled offline RL methods across various tasks, indicating its robustness and efficiency.
Methodology
The authors propose a decoupled policy extraction paradigm where the actor is trained to model the behavior distribution independently from the critic. During inference, the actor generates multiple action candidates, which are then reranked and selected based on their value estimates from a separately learned critic. This decoupling allows for explicit policy improvement without the recursive amplification of critic errors.
Results
The experiments demonstrate that the decoupled policy extraction paradigm consistently outperforms traditional coupled methods across 30 offline goal-conditioned tasks. For instance, using the IQL critic, the average success rate improved from 47.89% to 68.49%. Similar improvements were observed with other critic architectures, confirming the robustness of the proposed approach.
Implications
The decoupled policy extraction paradigm offers a more efficient framework for offline reinforcement learning, particularly beneficial for large-scale policies. By reducing the need for frequent actor updates, it allows for more computationally efficient training and deployment of RL models.
Learning Exact NVIDIA SASS Encoders with $ ext{F}_2$ Linear Algebra
Optimization
Theory
Efficient ML
- F2Asm is the first system to learn SASS instruction encoders as vector-valued affine maps over F2.
- The paper presents an incremental bitset Gaussian-elimination algorithm for constructing a compact basis.
- F2Asm separates shared learning from target-specific machine-code rules, enhancing its applicability across different GPU architectures.
- The system successfully reassembles 3,225 CUBINs, matching all executable text sections exactly.
Read more
Learning Exact NVIDIA SASS Encoders with $ ext{F}_2$ Linear Algebra
Summary
This paper introduces F2Asm, a novel system designed to learn exact 128-bit SASS encoders from paired disassembly and original CUBIN instruction words, addressing the lack of a public SASS assembler for recent NVIDIA data-center GPUs. F2Asm represents instruction encoding as vector-valued affine maps over the binary field F2, utilizing Gaussian elimination to construct a compact basis and detect inconsistencies in training data. The system separates target-specific control bits, relocation rules, and CUBIN metadata from its learning algorithm, allowing for extensibility across different GPU architectures, including Hopper, Blackwell, and Rubin. The authors trained encoders using 3,225 CUBINs from various sources and demonstrated that F2Asm can accurately reassemble disassembled SASS into original CUBINs, achieving byte-for-byte matches in all executable text sections. This work not only provides a reliable method for SASS-level optimization but also contributes to the field by being the first open-source assembler for NVIDIA SASS that supports the Rubin SM107 architecture.
Methodology
F2Asm employs Gaussian elimination over the binary field F2 to learn instruction encoders from disassembled SASS and original CUBINs. It constructs a compact basis for instruction encoding and checks for inconsistencies in training data. The system separates the learning algorithm from target-specific details, allowing for extensibility across various GPU architectures.
Results
F2Asm was trained on 3,225 CUBINs, successfully reassembling each disassembled SASS into its original CUBIN with all executable text sections matching byte-for-byte. This demonstrates the system's effectiveness in learning and encoding SASS instructions accurately.
Implications
The development of F2Asm has significant implications for GPU programming and optimization, enabling more efficient machine-code rewriting and SASS-level optimizations that can enhance the performance of critical workloads such as large language model training and inference.
Bankruptcy Prediction via Hybrid Resampling and Stacking Ensemble Techniques with Explainable Artificial Intelligence (XAI)-Driven Analysis
Interpretability
- Integration of hybrid resampling and stacking ensemble techniques improves bankruptcy prediction accuracy.
- SHAP analysis reveals key predictors of bankruptcy risk, enhancing interpretability.
- The study highlights the importance of addressing class imbalance in financial datasets.
Read more
Bankruptcy Prediction via Hybrid Resampling and Stacking Ensemble Techniques with Explainable Artificial Intelligence (XAI)-Driven Analysis
Summary
This study presents a novel framework for bankruptcy prediction that combines consensus-based feature selection, hybrid resampling techniques, stacking ensemble methods, and explainable artificial intelligence (XAI) to enhance the detection of minority classes in imbalanced financial datasets. Utilizing the Taiwanese Bankruptcy Prediction dataset, which includes 6,819 firm observations and a bankruptcy rate of 3.23%, the authors first applied five feature-selection algorithms to reduce the input features to 23 significant variables. They then generated balanced training data using various resampling methods, including SVM-SMOTE, SMOTE-Tomek, and SMOTE-ENN. The performance of five ensemble machine learning classifiers (gradient boosting, extreme gradient boosting, histogram-based gradient boosting, LightGBM, and AdaBoost) was compared against five deep learning models (RNN, LSTM, GRU, DNN, and MLP). The study also explored hybrid stacking ensembles that combined machine learning classifiers as base learners with deep learning models as meta-learners. Model evaluation metrics included accuracy, recall, specificity, G-mean, and ROC-AUC, with SHAP analysis employed to interpret feature contributions. The findings indicate that the choice of resampling strategy significantly influenced model performance, with SMOTE-ENN yielding the best minority-class detection. The GRU model with SMOTE-ENN achieved the highest predictive balance, while the stacking ensemble combining multiple classifiers with LSTM provided optimal sensitivity and specificity. Key predictors of bankruptcy risk identified through SHAP analysis included leverage, profitability, solvency, and operational efficiency.
Methodology
The methodology involved feature selection to reduce the dataset to 23 variables, followed by the application of hybrid resampling techniques (SVM-SMOTE, SMOTE-Tomek, SMOTE-ENN) to balance the training data. Various ensemble machine learning classifiers and deep learning models were evaluated, with a focus on stacking ensembles that combined the strengths of both approaches. Performance metrics included accuracy, recall, specificity, G-mean, and ROC-AUC, with SHAP used for interpretability.
Results
The results demonstrated that the resampling strategy significantly affected model performance. The GRU model with SMOTE-ENN achieved the best predictive balance, with a recall of 0.8627, G-mean of 0.8517, and ROC-AUC of 0.9431. Among stacking ensembles, the combination of multiple classifiers with LSTM provided the best sensitivity and specificity balance. SHAP analysis identified leverage, profitability, solvency, and operational efficiency as critical predictors of bankruptcy risk.
Implications
The findings suggest that the proposed framework can enhance the reliability and interpretability of bankruptcy prediction models, providing valuable tools for creditors, investors, and regulators to identify financially distressed firms more effectively.