AI-generated summaries

Today's ML research,
without the noise.

Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.

24 Papers today
8h Update frequency
7 Days of history
GHGbench: A Unified Multi-Entity, Multi-Task Benchmark for Carbon Emission Prediction
Yifan Duan, Siyuan Zheng, Lihuan Li, Chao Xue, Flora Salim
Multimodal Time Series Optimization
  • GHGbench is the first open dataset and benchmark for joint evaluation of company and building-level carbon emissions.
  • Building emissions are structurally more difficult to predict compared to company emissions due to additional influencing factors.
  • The in-distribution to out-of-distribution performance gap is larger than within-model variations.
  • Multimodal remote-sensing embeddings significantly improve prediction accuracy in challenging scenarios.
Read more
Bayesian Model Merging
Kaiyang Li, Shaobo Han, Qing Su, Shihao Ji
Optimization Efficient ML Computer Vision
  • BMM leverages strong anchor models to improve the merging process.
  • The framework employs bi-level optimization for effective hyperparameter tuning.
  • A data-free variant of BMM allows for regression without auxiliary data.
  • BMM shows significant performance improvements over existing model merging techniques.
Read more
Spectral Energy Centroid: a Metric for Improving Performance and Analyzing Spectral Bias in Implicit Neural Representations
Tomasz Dądela, Adam Kania, Maciej Rut, Przemysław Spurek
Computer Vision Generative Models Theory
  • Introduces the Spectral Energy Centroid (SEC) as a metric for analyzing spectral bias in INRs.
  • Proposes a data-driven hyperparameter selection strategy (SEC-Conf) that outperforms existing methods.
  • Demonstrates that SEC serves as a reliable proxy for signal complexity and reconstruction quality.
  • Reveals the significant impact of model depth on spectral bias and INR performance.
Read more
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi, Vinícius Conte Turani, João Vitor Boer Abitante, Rafaela Cappelari Ravazio, Christian Mattjie, Otávio Parraga, Lucas S. Kupssinskü, Rodrigo C. Barros
NLP Large Language Models Efficient ML
  • Slice is a new initialization method for LoRA adapters that mitigates catastrophic forgetting in continual learning.
  • The method uses gradient surgery to align current task objectives with previously learned knowledge.
  • Slice outperforms existing methods (vanilla LoRA, LoRA-GA, LoRAM) in terms of stability and performance metrics.
  • The paper introduces adversarial task sequences to better evaluate the performance of continual learning methods.
Read more
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
Yang Bai, Kaiyuan Liu, Ziyuan Zhuang, Jiahong Zhou, Rongxiang Weng, Xin Chen, Jingang Wang, Xunliang Cai
Reinforcement Learning Large Language Models NLP
  • Introduction of Reward-Decorrelated Policy Optimization (RDPO) for stabilizing multi-objective reinforcement learning.
  • Utilization of Magnitude-Aware Quantile Normalization and Mahalanobis whitening to address reward heterogeneity and correlation.
  • Demonstrated improvements in model performance on instruction following and writing quality through RDPO.
  • Introduction of Effective Information Efficiency (ηeff) as a metric for assessing mixed-reward aggregation quality.
Read more
A Unified Three-Stage Machine Learning Framework for Diabetes Detection, Subtype Discrimination, and Cognitive-Metabolic Hypothesis Testing
Vishal Pandey, Ruzina Haque Laskar, Rishav Tewari
Interpretability
  • Introduces a three-stage framework for diabetes detection and subtype discrimination.
  • Achieves high performance metrics with SVM-RBF and Logistic Regression on diabetes prediction.
  • Utilizes unsupervised K-Means clustering to identify diabetes subtypes without ground-truth labels.
  • Demonstrates a significant association between glycaemic control and cognitive function.
Read more
Scaling Laws for Mixture Pretraining Under Data Constraints
Anastasiia Sedova, Skyler Seto, Natalie Schluter, Pierre Ablin
NLP Large Language Models Optimization
  • Mixture training allows for higher repetition of target data compared to single-source training.
  • Optimal repetition rates for target data range from 15 to 20 times, depending on various factors.
  • A new scaling law is introduced that predicts target-domain loss based on mixture configurations.
  • Empirical findings demonstrate that larger models can extract more from limited data despite faster overfitting.
Read more
Strategic PAC Learnability via Geometric Definability
Yuval Filmus, Shay Moran, Elizaveta Nesterova, Nir Rosenfeld, Alexander Shlimovich
Theory
  • Strategic behavior can significantly impact the learnability of hypothesis classes.
  • The authors provide a counterexample showing that learnability is not preserved under strategic behavior in simple cases.
  • Introducing geometric definability allows for the preservation of learnability and manageable sample complexity.
  • The framework accommodates a variety of cost functions and hypothesis classes commonly used in machine learning.
Read more
Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning
Marco Angioli, Kevin Johansson, Antonello Rosato, Amy Loutfi, Denis Kleyko
Reinforcement Learning Efficient ML Theory
  • Introduces probabilistic HD-CB, a low-precision variant of HD-CB for resource-constrained devices.
  • Replaces deterministic accumulation with a probabilistic update rule to enhance decision-making efficiency.
  • Demonstrates improved performance over binarized HD-CB while maintaining low precision.
  • Addresses the overflow issue in low-precision components without the need for periodic binarization.
Read more
Multimodal Graph-based Classification of Esophageal Motility Disorders
Alexander Geiger, Lars Wagner, Daniel Rueckert, Alois Knoll, Dirk Wilhelm, Alissa Jell
Multimodal Graph Learning
  • Proposes a multimodal ML approach combining HRIM data with patient-specific information.
  • Uses graph-based modeling to represent HRIM data, enhancing the analysis of esophageal motility.
  • Demonstrates improved classification accuracy over traditional methods and vision-based classifiers.
  • Highlights the importance of integrating multiple data modalities for better diagnostic outcomes.
Read more
A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning
Jason Gaitonde, Frederic Koehler, Elchanan Mossel, Joonhyung Shin, Allan Sly
NLP Large Language Models Theory
  • Introduces synthetic languages with hierarchical structures for precise analysis of context and reasoning in autoregressive generation.
  • Derives explicit asymptotic predictions for distributional statistics in two broadcast process settings.
  • Establishes a lower bound on context length for faithful sampling and demonstrates an exponential improvement using reasoning models.
  • Empirical results validate theoretical predictions, showing the relationship between context size and model performance.
Read more
Separating Shortcut Transition from Cross-Family OOD Failure in a Minimal Model
Hongmin Li
Theory
  • Introduces a minimal binary model to study shortcut features and OOD failure.
  • Demonstrates that training-side observations can indicate potential cross-family failures.
  • Establishes that positive training shortcut correlation and shortcut-rule transitions are distinct phenomena.
  • Shows that the same training solution can yield different outcomes depending on the held-out family.
Read more
Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
Chien Van Nguyen, Chaitra Hegde, Van Cuong Pham, Ryan A. Rossi, Franck Dernoncourt, Thien Huu Nguyen
NLP Large Language Models Efficient ML
  • Introduces Orthrus, a dual-architecture framework that combines autoregressive and diffusion models.
  • Achieves up to 7.8× speedup in token generation while maintaining exact predictive fidelity.
  • Utilizes a shared Key-Value cache to eliminate redundant memory usage.
  • Incorporates a consensus mechanism for lossless inference.
Read more
Population Risk Bounds for Kolmogorov-Arnold Networks Trained by DP-SGD with Correlated Noise
Puyu Wang, Jan Schuchardt, Nikita Kalinin, Junyu Zhou, Sophie Fellenz, Christoph Lampert, Marius Kloft
Theory Optimization
  • First population risk bounds for KANs trained with mini-batch SGD and correlated noise.
  • Establishes bounds for both non-private and differentially private settings.
  • Introduces a novel analysis framework for correlated-noise DP training in non-convex regimes.
  • Demonstrates that correlated noise can improve the privacy-utility tradeoff compared to independent noise.
Read more
Multi-Quantile Regression for Extreme Precipitation Downscaling
Hamed Najafi, Gareth Lagerwall, Jayantha Obeysekera, Jason Liu
Time Series Generative Models Theory
  • Q-SRDRN significantly improves detection rates of extreme precipitation events compared to traditional methods.
  • The use of pinball loss allows for better handling of heavy-tail distributions in precipitation data.
  • Data augmentation through cVAE is beneficial when aligned with the model architecture and regional characteristics.
  • The architecture shows strong performance across diverse climatic conditions, indicating its robustness.
Read more
EMO: Frustratingly Easy Progressive Training of Extendable MoE
Linghao Jin, Chufan Shi, Huijuan Wang, Nuan Wen, Zhengzhong Liu, Eric Xing, Xuezhe Ma
Large Language Models Efficient ML
  • EMO allows for progressive expansion of the expert pool during training, improving efficiency.
  • The framework is based on a sparsity scaling law that optimizes token allocation across training stages.
  • EMO matches or exceeds the performance of fixed-expert models while reducing training time and costs.
  • The approach leverages the principle that MoE capacity should grow with data availability.
Read more
Modeling Heterophily in Multiplex Graphs: An Adaptive Approach for Node Classification
Kamel Abdous, Nairouz Mrabah, Mohamed Bouguessa
Graph Learning
  • HAAM explicitly models both homophilic and heterophilic interactions in multiplex graphs.
  • The use of dimension-specific compatibility matrices allows for tailored representation learning.
  • Product-composed Chebyshev filters enable the model to capture non-linear interactions effectively.
  • The framework improves node classification performance compared to existing methods.
Read more
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Zhonghao Li, Chaoyu Liu, Qian Zhang
Efficient ML Generative Models Theory
  • Di-BiLPS effectively addresses both forward and inverse PDE problems under extreme data sparsity.
  • The framework utilizes a combination of variational autoencoders, latent diffusion models, and contrastive learning.
  • It achieves state-of-the-art performance with significantly reduced computational costs.
  • The proposed denoising algorithm integrates physical constraints for improved inference.
Read more
Tight Sample Complexity Bounds for Entropic Best Policy Identification
Amer Essakine, Claire Vernade
Reinforcement Learning Theory
  • Introduces a new lower bound for best policy identification in risk-sensitive reinforcement learning.
  • Develops the Entropic-BPI algorithm that achieves optimal sample complexity.
  • Improves concentration bounds for exponential utilities, enhancing exploration strategies.
  • Demonstrates that the maximal achievable reward Gmax is a better metric for sample complexity than the horizon H.
Read more
WriteSAE: Sparse Autoencoders for Recurrent State
Jack Young
NLP Large Language Models Theory
  • WriteSAE is the first sparse autoencoder that effectively addresses matrix cache write operations in recurrent language models.
  • The method allows for closed-form predictions of logit shifts, achieving high accuracy (R² = 0.98).
  • Substitution of learned rank-1 atoms consistently outperforms traditional matched-norm ablation tests.
  • WriteSAE demonstrates significant improvements in performance metrics, including a 3× lift in midrank target-in-continuation tasks.
Read more
Uncertainty-Aware Prediction of Lung Tumor Growth from Sparse Longitudinal CT Data via Bayesian Physics-Informed Neural Networks
Lingfei Kong, Haoran Ma
Time Series
  • Introduces a Bayesian physics-informed framework for tumor growth prediction under sparse CT data.
  • Combines mechanistic Gompertz constraints with probabilistic inference for improved prediction accuracy.
  • Utilizes a two-stage inference procedure for stable posterior inference and efficient sampling.
  • Demonstrates the model's capability to provide calibrated uncertainty estimates alongside predictions.
Read more
Discovery of Hidden Miscalibration Regimes
Katarzyna Kobalczyk, Mihaela van der Schaar
Large Language Models NLP Interpretability
  • Introduces the concept of hidden miscalibration regimes that are not detectable through traditional calibration methods.
  • Defines an input-dependent miscalibration field to measure calibration error across the input space.
  • Demonstrates the prevalence of calibration heterogeneity in large language models across various datasets.
  • Provides a diagnostic framework that supports local confidence corrections, enhancing model reliability.
Read more
Spatiotemporal downscaling and nowcasting of urban land surface temperatures with deep neural networks
Solomiia Kurchaba, Angela Meyer
Time Series
  • Introduces a novel deep learning model for downscaling LST from geostationary to high-resolution satellite data.
  • Achieves high accuracy in LST forecasting with low RMSE and bias errors.
  • Demonstrates the applicability of the model across major European cities.
  • Provides a framework for intraday LST nowcasting, enhancing urban climate studies.
Read more
RISED: A Pre-Deployment Safety Evaluation Framework for Clinical AI Decision-Support Systems
Rohith Reddy Bellibatlu
Theory Interpretability
  • RISED Framework introduces a five-dimension evaluation for clinical AI systems.
  • Framework identifies critical deployment risks not captured by traditional metrics.
  • Validation across multiple cohorts shows varying failure patterns, supporting construct validity.
  • Equity dimension highlights the need for independent measures of clinical need.
Read more