AI-generated summaries

Today's ML research,
without the noise.

Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.

24 Papers today
8h Update frequency
7 Days of history
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
Zirui Cheng, Xun Xu, Tiankai Chen, Fady Rezk, Bowen Zheng, Xiaodong Shi, Shijie Li, Kangkang Lu, Bharadwaj Veeravalli, Nancy F. Chen
Multimodal Large Language Models Graph Learning
  • MAG effectively addresses label scarcity in few-shot multi-modal ICL by utilizing unlabeled data.
  • The framework employs a two-stage strategy for demonstration selection, enhancing efficiency and relevance.
  • Textual representations are crucial for initial relevance propagation, while both modalities are needed for final selection.
  • MAG demonstrates substantial performance gains across diverse benchmarks, particularly in reasoning-intensive tasks.
Read more
History-informed Lagrangian Neural Networks
Tianshuo Zhang, Xianglei Xing, Wenzhe Zhai, Jia Gao, He Cao
Robotics Time Series Theory
  • HiLNN infers hidden velocities and adapts system parameters from position-only observations.
  • The framework utilizes a recurrent encoder to extract a latent context from historical position data.
  • HiLNN employs a differentiable RK4 rollout scheme for optimized trajectory predictions.
  • Empirical results show superior accuracy and physical consistency compared to traditional LNNs and other baselines.
Read more
Federated Compositional Muon Optimizer for Matrix-Wise Models
Wang Yan, Feihu Huang
Federated Learning Optimization
  • Introduction of FedCoMuon and FedCoMuon-VR optimizers for matrix-wise compositional optimization.
  • Theoretical convergence analysis under non-convex and non-i.i.d. settings.
  • FedCoMuon-VR achieves lower sample complexity than existing FedMuon algorithms.
  • Extensive experiments demonstrate competitive performance and improved accuracy.
Read more
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Shuhan Wang, Yilin Luo, Nan Xu, Chi Wang Cheung
NLP Large Language Models Optimization
  • The only commuting orthogonal maps for distinct RoPE frequencies are independent pairwise rotations.
  • The derived rotation angle minimizes channel variance but does not improve quantization accuracy in practice.
  • The head-shared pairwise configuration results in higher perplexity compared to full-head mixing.
  • Estimating the shared angle from K alone improves performance but does not close the gap with full-head mixing.
Read more
A Probe Direction Is a Property of Its Prompt
Valentin Noël
NLP Large Language Models Theory
  • The choice of prompt significantly influences model evaluation scores.
  • Reported scores can vary widely based on prompt wording, affecting perceived model performance.
  • A single-prompt design is insufficient for reliable comparisons across models.
  • The paper advocates for a multi-prompt evaluation framework to enhance reliability.
Read more
Beyond Simulated Benchmarks: Evaluating Motion Representations for Fall Detection Under Real-World Data Scarcity
Timilehin B. Aderinola, Ilaria D'Ascanio, Luca Palmerini, Lorenzo Chiari, Jochen Klenk, Clemens Becker, Brian Caulfield, Georgiana Ifrim
Time Series
  • Real-world fall detection is hindered by the extreme scarcity of actual fall data.
  • Simulated datasets often lead to overestimated performance in laboratory settings.
  • Interval-based representations achieve the best real-world performance, while symbolic representations with impact descriptors show robustness under data scarcity.
  • The study emphasizes the importance of representation choice for generalization from simulated to real-world conditions.
Read more
Personalized Scorer Modeling: A Learning-Based Framework for Deriving Robust Sleep Stage Labels from Multiple Experts
Seyyed Ali Hoseini, Javad Baseri, Hamid Saadatfar, Edris Hoseini Gol, AmirHossein Eshghi
Time Series Multimodal
  • Introduces a framework for improving sleep stage classification by addressing inter-scorer variability.
  • Utilizes multi-scored datasets to derive more reliable sleep stage labels.
  • Employs confusion matrices to model scorer-specific behavior and aggregate probabilities for labeling.
  • Demonstrates improved classification metrics compared to traditional hypnograms.
Read more
Understanding Backdoor Vulnerabilities in Vertical Federated Learning: The Gap Between Research and Practice
Ziqi Zhao, Jialin Lu, Junjie Shan, Junyuan Zhang, Shuya Yang, Ka-Ho Chow
Federated Learning
  • Identifies a gap between theoretical research on backdoor attacks in VFL and practical applications.
  • Highlights unrealistic assumptions in existing methodologies that lead to overestimated attack success rates.
  • Introduces BVBench, a benchmark for fair evaluation of backdoor vulnerabilities in VFL.
  • Recommends redefining threat models to align with realistic operational constraints.
Read more
Finding the Needle in a Haystack: Test-Time Analog Circuit Representation Adaptation for Bayesian Optimization
Fin Amin, Sounak Dutta, Paul D. Franzon
Optimization
  • TTARO adapts circuit representations in real-time during the optimization process, improving alignment with the optimization objective.
  • The framework is compatible with various acquisition functions and Gaussian-process kernels, making it versatile for different optimization scenarios.
  • TTARO demonstrates significant performance improvements over traditional fixed-embedding BO methods and DKL in analog circuit topology searches.
Read more
The data geometry of masking diffusion: Certified-optimal schedules via unmasking growth complexity
Martin J. Wainwright
Theory Generative Models Efficient ML
  • Introduction of unmasking growth complexity (UGC) as a measure of data geometry in masking diffusion.
  • Establishment of a unified analysis framework for Bernoulli-subset and fixed-cardinality unmasking schemes.
  • Development of certified-optimal samplers with high-probability guarantees on KL error.
  • Demonstration of significant dimension-dependent improvements in sampling efficiency.
Read more
Defensive Boosting for Online Probabilistic Forecasting
Georgy Noarov, Aaron Roth
Theory Efficient ML Optimization
  • Introduces the Defensive Booster algorithm for online probabilistic forecasting.
  • Achieves dual guarantees: competitive Brier scores and reduced classification error under weak-learning conditions.
  • Utilizes a single weak-class learner for efficiency, unlike previous methods requiring multiple learners.
  • Provides local hard-core certificates for weak-learning conditions, enhancing robustness.
Read more
Robust data-driven discovery of fractional differential equations via weak formulations and Pareto-based subset selection
Pongpisit Thanasutives, Yoshinobu Kawahara
Theory
  • Weak-Pareto combines weak formulations with Pareto-based subset selection for discovering fractional differential equations.
  • The method effectively mitigates noise amplification issues associated with fractional differentiation.
  • Weak-Pareto demonstrates superior robustness and accuracy in recovering equations from noisy data compared to traditional methods.
  • The framework allows for continuous-order optimization, avoiding the pitfalls of fixed-order dictionaries.
Read more
Towards Socially Compliant Navigation in Deep Reinforcement Learning via Proxemics-Based Reward Modeling
Takieddine Soualhi, Jacques Saraydaryan, Laetitia Matignon
Reinforcement Learning Robotics
  • Introduction of a proxemics-based reward model for DRL navigation.
  • Validation of the model across multiple DRL methods and crowd scenarios.
  • Demonstrated improvements in social metrics without sacrificing navigation efficiency.
  • Emphasis on the importance of comfort-aware navigation assessment.
Read more
Exploring Oversmoothing with Householder Matrices
Bhaskar Karol
Graph Learning Theory
  • Introduces Householder matrices as a method to combat oversmoothing in GNNs.
  • Proves that HouseGNN preserves the Euclidean norm of node representations.
  • Demonstrates scale and sign invariance of the Householder reflector.
  • Shows that pairwise distances between nodes can vary with orthogonal transformations.
Read more
The Boolean Power of ReLU
Pablo Barceló, Floris Geerts, Matthias Lanzinger, Klara Pakhomenko, Jan Van den Bussche
Graph Learning Theory
  • ReLU-MPLang is strictly more expressive than TrReLU-MPLang for Boolean queries.
  • The study resolves an open problem regarding the expressiveness of different activation functions in GNNs.
  • Boolean queries derived from ReLU activations can express properties that are not expressible by truncated ReLU activations.
  • The findings emphasize the importance of activation function selection in GNN architectures.
Read more
Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia
Xiang Guan, Roger D. Newman-Norlund, Yong Yang, Saeed Ahmadi, Regan Willis, Nadra Salman, Kalil Warren, Srihari Nelakuditi, Chris Rorden, Leonardo Bonilha, Julius Fridriksson
Interpretability Large Language Models NLP
  • PRISM adapts subtraction analysis from neuroimaging to interpret LLMs, providing a structured framework for mechanistic interpretability.
  • The framework demonstrates that perturbation-induced error profiles in LLMs can be compared to lesion patterns in patients with aphasia.
  • Both LLMs and aphasia patients show a robust phonemic-favoring dissociation, indicating shared cognitive processing patterns.
  • The methodology allows for spatially resolved testing of functional specialization claims in LLMs.
Read more
When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide
Binshuang Li
Theory
  • Weak overlap is primarily determined by logger-target action alignment rather than logging sharpness.
  • Cross-fitting the outcome nuisance does not eliminate reuse bias; honest policy-level splitting is necessary.
  • Propensity-estimation error is the most significant factor affecting performance in offline evaluations.
  • The paper provides a reproducible benchmark and practical guidance for practitioners in the field.
Read more
Symmetry-Breaking De Novo Crystal Generation via Markovian Jump Diffusion
Van Khoa Nguyen, Alexandros Kalousis
Generative Models
  • Introduces a novel framework (SbCD) for generating complete crystallographic structures.
  • Utilizes Markovian jump-diffusion to model symmetry-breaking dynamics.
  • Outperforms existing models in generating crystals with full structural specifications.
  • Addresses limitations of traditional methods that rely on empirical sampling of space groups.
Read more
I-SDPO: Instance-Level Adaptive Self-Distillation Policy Optimization
Yubo Zhang, Xinhong Ma, Zezhong Tan, Ziqiang Dong
Reinforcement Learning Large Language Models Optimization
  • I-SDPO addresses the degenerate gradient problem in GRPO by adapting the use of self-distillation based on the success of responses.
  • The routing decision for self-distillation is made at the instance level, allowing for more effective learning from both successful and unsuccessful trajectories.
  • I-SDPO achieves state-of-the-art performance on the SciKnowEval benchmark across multiple scientific domains.
  • The method automatically adjusts the expected distillation rate as the model's performance improves, reducing reliance on the teacher over time.
Read more
Scaling Automatic Research Agents via World Models
Xiyuan Yang, Sheikh Sarwar, Jingru Cheng, Zhan Shi, Duanshun Li, Huiyuan Chen, Haiyang Zhang, Chenlei Guo, Jingrui He, Zhenyu Liao
Reinforcement Learning Large Language Models Efficient ML
  • Introduction of World Model RL (WMRL) to replace costly environment execution in AutoResearch agents.
  • Implementation of Online Debiasing and Inverse-Variance Denoising to enhance the reliability of the world model.
  • Theoretical proof of improved convergence guarantees with the proposed mechanisms.
  • Empirical validation showing 3-4x training acceleration and superior performance compared to larger models.
Read more
Sampling Luck Masquerades as Allocation Gain: Auditing Test-Time Budget Allocation for Neural Combinatorial Optimization
Jinhyung Bae
Optimization
  • First measurement of allocation value for NCO test-time sampling, revealing no detectable gain in in-distribution workloads.
  • Quantification of in-sample selection bias, showing that traditional measurement methods can produce misleading gains.
  • Demonstration of a significant allocation gain (11-12%) under distribution shift conditions with a pre-registered confirmatory experiment.
  • Introduction of a budget-accounted policy that retains performance gains while managing sample costs.
Read more
Into the ORBIT for Time Series: Training Regimes for Foundation Models
Hongjie Xia, Yiding Liu, Yifan Hu, Peiyuan Liu, Zewei Dong
Time Series
  • Introduction of ORBIT, a training paradigm for TSFMs that controls effective pre-training distribution.
  • Bootstrap Multi-Level Sampling and Omni-Range Incremental Training are key components of ORBIT.
  • Falcon-2.0, trained under ORBIT, shows strong zero-shot forecasting capabilities.
  • Rank-Guided Cross-Depth Alignment improves representation alignment across Transformer depths.
Read more
Learning Under Treatment-Induced Label Indeterminacy with Expert Annotations of Counterfactual Outcomes: A Case Study in Neurological Prognostication
Xiaobin Shen, Chloe Y.H. Huang, Jonathan Elmer, George H. Chen
Theory
  • Introduces the concept of treatment-induced label indeterminacy in clinical prediction models.
  • Proposes a framework for evaluating prediction models that separates certain and uncertain cases.
  • Develops a prediction model that balances accuracy on certain cases with alignment to expert estimates for uncertain cases.
  • Demonstrates that traditional evaluation metrics can miss important insights in uncertain cases.
Read more
Simulation-to-real transfer learning for infrared spectroscopic chemical sensing and analysis from molecules to complex samples
Yusen Tan, Yixuan Chen, Zheng Fang, Pan Liu, Yifan Li, Qinyu Guo, Zhedong Lin, Yuqiang Li, Xiangxiang Zeng, Tong Wang, Jun Xia
Efficient ML
  • Introduction of UltraIR, a foundation model for IR spectroscopy with over 100 million parameters.
  • Utilization of simulation-to-real transfer learning to enhance data efficiency and reliability in chemical inference.
  • Demonstrated strong performance across multiple analytical tasks and real-world applications.
  • Pretraining on simulated IR spectra allows for effective adaptation to downstream tasks with limited labeled data.
Read more