AI-generated summaries

Today's ML research,
without the noise.

Daily summaries of the latest machine learning papers from arXiv, processed every 8 hours.

48 Papers today
8h Update frequency
7 Days of history
Learning to Run Power Networks: Effective AlphaZero-inspired Topological Control
Lukas Zetto, Benjamin SchΓ€fer, Qiong Huang
Reinforcement Learning
  • AlphaZero-inspired methods can significantly enhance the stability of power networks under strain from renewable energy sources.
  • An optimized approach achieves a peak survivability of 98.43%, outperforming traditional RL methods like PPO.
  • MCTS without prior policy guidance can improve training efficiency.
  • Simple binary rewards are more effective for guiding search than complex reward structures.
Read more
Finding the Needle in a Haystack: Test-Time Analog Circuit Representation Adaptation for Bayesian Optimization
Fin Amin, Sounak Dutta, Paul D. Franzon
Optimization
  • TTARO introduces an online adaptation mechanism for circuit representations during Bayesian optimization.
  • The framework allows for continuous learning and adjustment of circuit embeddings based on newly observed performance metrics.
  • TTARO outperforms traditional fixed representation methods and DKL in terms of optimization efficiency.
  • The approach is compatible with various acquisition functions and Gaussian-process kernels.
Read more
From BERT to Frontier Agents: Eight Years of Language-Model Progress, the Collapse of the Capability-Cost Curve, and the Rise of Task-Targeted Models
Pranav Kumar Kaliaperumal
NLP Large Language Models Efficient ML
  • Significant growth in language model capabilities from BERT to trillion-parameter systems.
  • Collapse of the capability-cost curve, with budget models outperforming flagship models at lower costs.
  • Emergence of task-fragmented models, necessitating specialized routing for optimal performance.
  • Introduction of a confidence estimation model to improve prediction accuracy in task-specific scenarios.
Read more
CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
Zihao Ye, Yingyi Huang, Hongyi Jin, Bohan Hou, Junru Shao, Zhongming Yu, Jinqi Chen, Meghan Cowan, Shiyi Cao, Shanli Xing, Hanfeng Chen, Vinod Grover, Tianqi Chen, Luis Ceze
Optimization Efficient ML Theory
  • CAKE integrates compiler and agent co-design to improve GPU kernel evolution.
  • The framework uses a typed intermediate representation (IR) for better hardware-specific control.
  • Localized correctness and performance diagnostics are provided to agents, enhancing decision-making.
  • Significant performance improvements were observed in benchmarks compared to traditional methods.
Read more
Prof-K: Probabilistic One-Pass Filtering for Efficient Top-k Selection
Tadeusz Dziarmaga, Witold Sikora, Łukasz Struski, Jacek Tabor, Marcin Mazur
Efficient ML Optimization Theory
  • Prof-K provides a fast, scalable, and distribution-agnostic solution for top-k selection.
  • The algorithm guarantees high-probability correctness and buffer size through a probabilistic filtering approach.
  • Empirical results show 1.5×–10Γ— speedups over existing top-k implementations, particularly beneficial for large datasets.
  • Prof-K allows for flexible accuracy-speed trade-offs, enabling users to tune performance based on their requirements.
Read more
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Vincent Counathe, Ben Athiwaratkun, Christopher De Sa, Tianyi Zhang
NLP Large Language Models Optimization
  • QUASAR continuously minimizes loss-aware reconstruction error during QAT to improve training trajectories.
  • The method uses exponential moving averages for saliency estimation and optimizes clipping ranges and dequantizers in real-time.
  • Empirical results show QUASAR outperforms traditional QAT methods, achieving lower KL divergence and higher accuracy across multiple tasks.
Read more
Robust data-driven discovery of fractional differential equations via weak formulations and Pareto-based subset selection
Pongpisit Thanasutives, Yoshinobu Kawahara
Theory
  • Weak-Pareto combines weak formulations with Pareto-based selection for robust FPDE discovery.
  • The framework effectively mitigates noise amplification issues associated with fractional differentiation.
  • Weak-Pareto outperforms traditional methods in recovering equations from noisy data.
  • The methodology allows for continuous-order optimization, avoiding discretization biases.
Read more
RecipeNet: A Hierarchical Transformer for Recipe Data
Pin-Yen Huang, Sachin Chhabra, Prasanth Sai Gouripeddi, Abhinav Kumar, Baoxin Li
NLP
  • RecipeNet effectively models the hierarchical structure and sequential dependencies of recipe data.
  • The architecture outperforms traditional tabular learning methods by preserving the native recipe structure.
  • RecipeNet consists of three stages: field-level tokenization, step-level fusion, and recipe-level sequence encoding.
  • Experimental results highlight the importance of hierarchical modeling for effective recipe representation learning.
Read more
ProME: Prototype-Margin Environments with Repair-Aware Selection for Group-Robust Learning
Qianqian Wang, Yunshan Li, Dawei Huang, Wenwu Gong, Lili Yang
Theory Optimization
  • ProME aligns environment construction and model selection with the deployed predictor without requiring training-group labels.
  • The framework introduces a median split of prototype margins to create balanced environments for invariant learning.
  • Theoretical bounds on worst risks are established, showing transferability to oracle groups under alignment conditions.
  • ProME demonstrates superior performance in worst-group accuracy compared to existing methods across multiple benchmarks.
Read more
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Yuxiao Wen
Theory Efficient ML Optimization
  • Introduces Tree-Coupled A/B Testing (TCAB) for efficient multi-policy comparisons.
  • Reduces the number of reward-bearing interactions by sharing outcomes among policies.
  • Maintains each policy's standalone trajectory distribution despite shared feedback.
  • Achieves conditionally optimal query costs compared to traditional independent A/B/n testing.
Read more
Adaptive Protection for Evolutionary Feature Construction in Symbolic Regression with Application to Credit Classification
Hengzhe Zhang, Qi Chen, Bing Xue, Lean Yu, Wolfgang Banzhaf, Mengjie Zhang
Optimization Theory
  • Introduction of an adaptive protection mechanism for preserving important features during evolutionary feature construction.
  • Utilization of feature importance metrics to guide the preservation process.
  • Empirical evaluation demonstrates improved solution quality across multiple datasets and base learners.
  • The approach is applicable to both symbolic regression and credit classification tasks.
Read more
HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models
Fangzhou Chen, Shiji Zhao, Mengyang Wang, Qihui Zhu, Ranjie Duan, Maoxun Yuan, Xingxing Wei
NLP Large Language Models Efficient ML
  • HiRoute separates category-agnostic safety control from category-specific response guidance.
  • The framework uses a hierarchical router to dynamically compose fine-grained prompts based on input risk.
  • HiRoute maintains high safety rates while preserving the helpfulness of responses.
  • The approach reduces over-refusal of benign inputs compared to traditional methods.
Read more
Forecast Collapse in Time-Series Foundation Models
Shu Wan, Miles Ma, Hank Zhu, Guangqi Liu, Stephen Wang, Qingsong Wen, Huan Liu
Time Series Optimization Theory
  • Forecast collapse is characterized by low forecast amplitude and poor cross-sectional structure in equity return predictions.
  • The phenomenon is linked to target predictability, with trading volume forecasts performing better than equity returns.
  • Two mechanisms are identified: low predictability constrains forecast amplitude, and per-series objectives overlook cross-series relationships.
  • CalibRank, a new objective function, effectively balances calibration and ranking, improving forecast performance.
Read more
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Shuhan Wang, Yilin Luo, Nan Xu, Chi Wang Cheung
NLP Large Language Models Optimization
  • Introduces a local benchmark for RoPE-commuting orthogonal transformations, focusing on independent pairwise rotations.
  • Demonstrates that the head-shared pairwise configuration yields higher perplexity than full-head mixing in dynamic quantization settings.
  • Establishes that the optimization of a surrogate does not guarantee improved quantization accuracy due to misalignment with quantizer statistics.
  • Highlights the relationship between mixing support size and quantization error, suggesting broader mixing leads to better performance.
Read more
Catching the Imposter: Self-Supervised Learning of Physical Coherence with Cross-Entity Feature Permutations
Aleksei Rozanov, Arvind Renganathan, Vipin Kumar
Time Series
  • Introduces the 'imposter' pretext task for SSL that leverages physical coherence among features.
  • Demonstrates the effectiveness of the imposter task on various downstream tasks in environmental modeling.
  • Finds that the best pretext task depends on the specific downstream task family.
  • Shows that combining the imposter task with existing SSL objectives enhances performance.
Read more
When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version
Dat Nguyen-Cong, Luong Tran, Tung Kieu
Time Series Generative Models Optimization
  • Identifies two distinct regimes in the reverse diffusion process: high-to-intermediate noise and low noise.
  • Excessive low-noise refinement can introduce statistical drift and degrade forecast quality.
  • Proposes an early-stopping mechanism that reduces sampling costs by 20-50% without requiring ground truth.
  • Introduces a Bernoulli timestep-sampling strategy to emphasize high-noise predictions.
Read more
Resource-Adaptive Primal-Dual Learning for One-Warehouse Multi-Store Systems with Censored Demand
Jiameng Lyu
Optimization Theory
  • Introduction of Resource-Adaptive Primal-Dual Learning (RAPDL) for OWMS systems.
  • RAPDL tracks the Primal-Dual re-solving path, adapting to changes in resource availability.
  • Achieves logarithmic expected regret, outperforming existing square-root-order guarantees.
  • Numerical experiments show RAPDL consistently lowers mean total costs compared to state-of-the-art methods.
Read more
The Objective Is the Bottleneck: Latent World Models Encode What Their Planners Cannot Use
Joyjeet Singh
Reinforcement Learning Robotics Optimization
  • The predictor is not the bottleneck; it remains informative at long horizons.
  • The planning objective can saturate and invert, affecting planning success.
  • Long-horizon success is inversely related to one-step prediction accuracy.
  • Reachability is a more effective objective than proximity for planning.
Read more
Detecting Contaminated Code-Generation Prompt Batches via Influence Functions
Francesco Quinzan, Noor Munir, Yishun Lu, Stephen Roberts
Large Language Models NLP Generative Models
  • CodeSIFT is a threat-model-agnostic method for detecting malicious code-generation prompts using influence functions.
  • The method does not require prior knowledge of specific vulnerabilities, making it adaptable to novel attacks.
  • Two benchmark datasets were created to evaluate the effectiveness of CodeSIFT against static analysis baselines.
  • CodeSIFT achieved AUROC scores up to 0.98, outperforming traditional static analysis methods.
Read more
Boosting Data Augmentation with Stochastic Weight Averaging
Longde Huang, Axel Flinth, Jan E. Gerken
Computer Vision Graph Learning Theory
  • Combines data augmentation with Stochastic Weight Averaging (SWA) to enhance model performance.
  • Introduces a theoretical framework using Ornstein–Uhlenbeck processes to analyze the effects of SWA.
  • Demonstrates that the combination of SWA and data augmentation leads to significant equivariance improvements.
  • Validates theoretical findings through extensive numerical experiments on diverse datasets and model architectures.
Read more
Designing Compact Neural Architectures via Neuron Gating and Mixed Activation
Abhishek Shukla, Ankur Sinha, Faiz Hamid
Optimization Efficient ML Theory
  • Introduces a scalable method for Neural Architecture Search (NAS) using continuous relaxations.
  • Develops three bilevel optimization formulations applicable to various neural network types.
  • Demonstrates that the proposed methods outperform existing NAS techniques like DARTS.
  • Achieves high accuracy with significantly fewer parameters in the identified architectures.
Read more
When Does More Correct Data Hurt? Insertion-Stability and the Limits of Dimension-Based Theory
Joseph Sankoorikal Johny
Theory
  • Insertion-stability is a key property that determines whether adding correct data will hurt or help a learner's performance.
  • The paper establishes that no learner can achieve better than Θ(d/n) log(en/d) expected error for classes with VC dimension d β‰₯ 2.
  • Closure is shown to be insertion-stable, allowing intersection-closed classes to maintain their clean error rates despite data augmentation.
  • The relationship between the learner and the class is critical; a mismatch can lead to increased error rates even with correct data.
Read more
Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws
Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang
Theory Optimization
  • Introduces Neural Quadratic Forms (NQF) as a minimal model for understanding sudden learning and scaling laws in neural networks.
  • Demonstrates that neural architectures exhibit a symmetry that allows for a unified representation of different models through a structure matrix A(x).
  • Establishes a connection between training dynamics and collective variables, leading to predictions about feature acquisition timing and power law behaviors.
  • Validates the theoretical predictions through numerical experiments across diverse neural network architectures and training methodologies.
Read more
Latent On-Policy Self-Distillation
Guibin Zhang, Jiayang Lyu, Ran Sun, Xinlei Yu, Haoyu Zhao, Qibing Ren, Shuicheng Yan
Reinforcement Learning Large Language Models Robotics
  • Introduces a learnable privileged context for on-policy self-distillation.
  • Demonstrates superior performance over existing OPSD methods.
  • Achieves high learning efficiency with reduced rollout budget.
  • Employs a privileged-margin objective to enhance teacher-student dynamics.
Read more
Structure-preserving uncertainty quantification for GENERIC dynamics
Zequn He, Celia Reina
Theory
  • Introduction of S-PENNs for uncertainty quantification in hard-constrained models.
  • Preservation of thermodynamic consistency in stochastic realizations.
  • Significant reduction in computational costs compared to deep ensembles.
  • Validation through multiple numerical examples demonstrating effectiveness.
Read more
L-FNO: Lorentzian Fourier Neural Operator for Stochastic Event Dynamics
Songhee Kang, Jihoon Kang
Time Series
  • L-FNO incorporates a Lorentzian spectral kernel to model self-exciting dynamics in stochastic events.
  • The model is trained using a Poisson negative log-likelihood objective, improving calibration and sensitivity to rare events.
  • L-FNO outperforms traditional regression-based models in event likelihood and rare-event detection across multiple benchmarks.
  • The learned parameters provide interpretable insights into the predictability structure of stochastic event systems.
Read more
Quantum Multi-Armed Bandits and Linear Bandits: Lower Bounds and Algorithms
Maoli Liu, Zhuohua Li, John C.S. Lui
Theory
  • Establishes minimax lower bounds for quantum multi-armed and linear bandits.
  • Proves that logarithmic regret dependence on T is unavoidable for QMAB.
  • Introduces a design-based elimination algorithm that reduces regret dependence on dimension.
  • Implements a low-bias, low-variance quantum mean estimator to optimize performance.
Read more
Approximate Muon with low-rank adapters
Ben Anson, Conor Houghton, Edward Milsom
Optimization Efficient ML
  • sMuon provides a mathematically efficient method for training LoRA adapters with Muon, requiring less optimizer state than AdamW.
  • The proposed method is implemented using only matrix multiplications, making it faster and more memory-efficient than existing approaches.
  • sMuon shows strong performance across multiple fine-tuning benchmarks, particularly with Muon-pretrained models.
  • The method achieves competitive results in ReLoRA pretraining, demonstrating advantages over AdamW-based methods.
Read more
Learning Discrete Decisions for MIPs with Constraint-Aware Diffusion
Vincenzo Di Vito, Mehdi Taghizadeh, Deepjyoti Deka, Kaarthik Sundar, Ferdinando Fioretto
Optimization Generative Models Graph Learning
  • Introduction of Constrained Graph Diffusion (CGD) for MIPs, which enforces feasibility during the decision-making process.
  • Decomposition of mixed-integer nonlinear programming (MINLP) problems into discrete and continuous components to improve optimization efficiency.
  • Demonstrated effectiveness of CGD on diverse applications, achieving substantial improvements in solution quality and feasibility.
  • Achieved significant computational speedups compared to traditional numerical solvers.
Read more
Connected Subspace Clustering: Hardness, a Scalable Heuristic, and an Application to Sea Level Geodesy
Johanna Hillebrand, Jan HΓΆckendorff, JΓΌrgen Kusche, Kelin Luo, Heiko RΓΆglin, Melanie Schmidt, Christian Sohler, Bernd Uebbing
Optimization Time Series Theory
  • Introduction of Connected Subspace Clustering, combining connectivity and subspace reconstruction.
  • Proven NP-hardness of the problem, even for simple cases.
  • Development of an efficient heuristic that ensures connected clusters.
  • Strong performance in sea level geodesy applications, outperforming traditional clustering methods.
Read more
Defensive Boosting for Online Probabilistic Forecasting
Georgy Noarov, Aaron Roth
Theory Efficient ML Optimization
  • The Defensive Booster algorithm achieves both competitive Brier scores and low classification error under varying conditions.
  • It operates efficiently with only one weak-class learner, unlike previous methods that require multiple learners.
  • The algorithm provides local hard-core certificates, enhancing its adaptability to different time intervals.
  • Experimental results show substantial improvements in predictive performance and runtime efficiency over existing methods.
Read more
Generating Benchmark Health Data Using a Tabular Diffusion Transformer
Hao Yan, Lisa Pilgram, Dan Liu, Linglong Kong, Fida Dankar, Khaled El Emam
Generative Models
  • Introduces a two-stage framework for generating synthetic tabular data from multiple heterogeneous tables.
  • Transforms raw tables into standardized statistical tables to capture essential statistical properties.
  • Utilizes a diffusion transformer model to learn structural patterns and generate synthetic data.
  • Demonstrates high fidelity in statistical representations and effective diversity in generated datasets.
Read more
Stochastic Control Policies for Robust Molecular Transition Path Sampling
Jingqian Liu, Yu-Hsiang Wang, Yanru Qu, Ge Liu
Reinforcement Learning Optimization Theory
  • Introduction of two stochastic control policies (FS-TPS and LaS-TPS) for molecular transition path sampling.
  • FS-TPS improves transition success and path quality by utilizing a state-dependent Gaussian distribution.
  • LaS-TPS enhances robustness by sampling from a latent variable representation, capturing correlated atomic motions.
  • Both methods show significant improvements over deterministic baselines across multiple biomolecular systems.
Read more
HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networks
SiHun Lee, Dong-Hyuk Park, Taesoo Bang, Seung-Hoon Kang
Graph Learning Efficient ML Theory
  • HI-MGN enhances long-range communication through hierarchical message passing.
  • Utilizes FPS-Voronoi coarsening to create coarse graphs from original mesh adjacency.
  • Incorporates learned interpolation for reconstructing fine-scale features post-coarsening.
  • Achieves improved accuracy while reducing memory and training costs.
Read more
Scaling Automatic Research Agents via World Models
Xiyuan Yang, Sheikh Sarwar, Jingru Cheng, Zhan Shi, Duanshun Li, Huiyuan Chen, Haiyang Zhang, Chenlei Guo, Jingrui He, Zhenyu Liao
Reinforcement Learning Large Language Models Efficient ML
  • Introduction of World Model RL (WMRL) to replace expensive environment execution in AutoResearch agents.
  • Development of Online Debiasing and Inverse-Variance Denoising mechanisms to enhance the performance of the world model.
  • Theoretical grounding of the framework, proving improved convergence guarantees.
  • Empirical validation showing 3-4x acceleration in training and superior performance compared to larger models.
Read more
H-VAEP and H-xT: Valuing Offensive On-the-Ball Actions in Handball by Estimating Probabilities
Julius Broermann, Oliver MΓΌller, Michael DΓΆring, Jochen Baumeister
Theory
  • Introduction of H-xT and H-VAEP frameworks tailored for handball analytics.
  • Development of a handball-native court zoning layout for improved action valuation.
  • Demonstration of the robustness of H-xT compared to traditional rectangular grids.
  • H-VAEP yields stable and intuitive player ratings that highlight contributions to build-up play.
Read more
Novel Knowledge-Guided Generative Methods for Synthetic Transcriptomic Data
Francesca Pia Panaccione, Sofia Mongardi, Marco Masseroli, Pietro Pinoli
Generative Models Graph Learning
  • Synthetic data generation can mitigate challenges in accessing high-quality transcriptomic data.
  • Integrating biological knowledge through gene graphs enhances the realism and utility of synthetic data.
  • MK-TGAN outperforms other generative models in terms of data realism and biological coherence.
  • The study highlights the importance of explicit knowledge integration mechanisms in generative models.
Read more
Fashion Outfit Generation via Unified Sequential Composition Models
Kaicheng Pang, Xingxing Zou, Ruohan Xu, Waikeung Wong
Generative Models Reinforcement Learning Optimization
  • Introduces Constrained Ensemble Generation (CEG) as a formal framework for fashion outfit generation.
  • Develops the Unified Sequential Composition Model (USCM) to jointly model compatibility and composition intents.
  • Implements Latent Expansion Monte Carlo Tree Search (LE-MCTS) for efficient item retrieval and ensemble synthesis.
  • Achieves state-of-the-art results on multiple fashion datasets, demonstrating superior performance in aesthetic and structural evaluations.
Read more
Exploring Oversmoothing with Householder Matrices
Bhaskar Karol
Graph Learning
  • Introduces Householder Graph Neural Network (HouseGNN) to combat oversmoothing in GNNs.
  • Uses Householder matrices for node embedding updates, preserving Euclidean norms.
  • Proves properties of Householder reflectors that contribute to maintaining node representation diversity.
  • Offers a novel perspective on the oversmoothing issue by utilizing orthogonal transformations.
Read more
Simulation-to-real transfer learning for infrared spectroscopic chemical sensing and analysis from molecules to complex samples
Yusen Tan, Yixuan Chen, Zheng Fang, Pan Liu, Yifan Li, Qinyu Guo, Zhedong Lin, Yuqiang Li, Xiangxiang Zeng, Tong Wang, Jun Xia
Efficient ML
  • UltraIR is a foundation model for IR spectroscopy with over 100 million parameters.
  • The model utilizes simulation-to-real transfer learning to improve chemical inference from IR spectra.
  • UltraIR outperforms traditional methods in various chemical sensing tasks and real-world applications.
  • The approach allows for effective adaptation using limited labeled experimental spectra.
Read more
Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration
Sabin Roman, Ljupco Todorovski, Saso Dzeroski
Theory
  • SORT provides a framework for estimating sparse expansions in orthonormal bases from noisy data.
  • The technique is particularly effective for discovering ordinary differential equations and supports nonlinear approximation.
  • SORT demonstrates improved stability under sparse sampling and noisy conditions compared to traditional methods.
  • The method allows for order-consistent model growth, maintaining low-order coefficients as model complexity increases.
Read more
Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Yixian Xu, Yuanrui Zhang, Shengjie Luo, Liwei Wang, Di He
Reinforcement Learning Generative Models Theory
  • Introduces a continuous-time path-space framework for diffusion-RL algorithms.
  • Derives a unified policy-gradient estimator that connects disparate RL methods.
  • Proposes a multi-sample KDE value-gradient estimator to reduce variance.
  • Validates the proposed methods empirically, showing improved performance over existing baselines.
Read more
A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settings
Hei Ting (Una) Chan, Chenwei Wu, Xueshen Liu, Zesen Zhao, Boyuan Zheng, Luis Filipe Nakayama, Michael G. Morley, Liyue Shen, Jiasi Chen, Z. Morley Mao
Multimodal Efficient ML Interpretability
  • Introduces a cloud-edge collaborative architecture for multimodal clinical screening.
  • Utilizes lightweight edge models for data processing and a cloud LLM for reasoning.
  • Achieves high diagnostic accuracy and factual grounding while minimizing data transmission.
  • Demonstrates the system's effectiveness under various simulated bandwidth conditions.
Read more
Smart routes: a system for development and comparison of algorithms for solving vehicle routing problems with realistic constraints
Andrew Soroka, German Mikhelson, Alexander Mescheryakov, Sergey Gerasimov
Optimization
  • Introduction of the Smart Routes platform for route optimization.
  • Comparison of exact and heuristic algorithms for CVRPTW.
  • Heuristic and deep learning methods show competitive performance against exact solvers.
  • Significant reduction in computation time with heuristic approaches for larger problem sizes.
Read more
A Compositional Theory of Curvature in Probabilistic Circuits
Hrithik Suresh, Sahil Sidheekh, Shelar Parth Vijay, Yasir Z, Sriraam Natarajan, Narayanan Chatapuram Krishnan
Generative Models Optimization Theory
  • Probabilistic Circuits (PCs) allow for exact inference and tractable curvature measures unlike deep neural networks.
  • Global sharpness regularization can lead to underfitting in PCs due to the compositional nature of their curvature.
  • The contribution of each sum node to the Hessian trace can be decomposed into contextual usage and local curvature.
  • An adaptive sharpness-aware regularizer improves generalization by focusing on nodes with high intrinsic local curvature.
Read more
Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry
Larissa Xu, King Bi, William Chang
Reinforcement Learning Theory Robotics
  • Introduces decentralized multi-player Q-learning under information asymmetry.
  • Proposes algorithms mQ-learning and mQ-learning-intervals for common rewards and independent rewards scenarios.
  • Demonstrates that decentralized learning can achieve regret bounds similar to centralized benchmarks.
  • Highlights the challenges posed by exponential growth in joint action space with increasing players.
Read more
A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family
Rishi Shah, Rishav Shrestha
Large Language Models Theory Optimization
  • Introduction of a twelve-gate contract-grade verifier for GPU kernels.
  • Audit of 2,638 machine-generated kernels revealing significant correctness issues.
  • Development of the first native Blackwell tcgen05 training backward for the GDN family.
  • Demonstration of the verifier's effectiveness through independent validation methods.
Read more
SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers
Kiran Nair, Rodrigue Rizk, KC Santosh
Efficient ML Computer Vision Theory
  • Introduction of SAGE, the first uncertainty-aware adaptive surrogate-gradient framework for spiking transformers.
  • Utilization of attention entropy to estimate uncertainty and modulate surrogate-gradient slopes during training.
  • Demonstrated consistent accuracy improvements over fixed and learnable surrogate baselines.
  • Minimal additional training overhead of only 0.03 ms per mini-batch.
Read more