Most Influential ArXiv (Machine Learning) Papers (2024-10)
The field of Machine Learning in arXiv covers papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. It is also an appropriate primary category for applications of machine learning methods. Paper Digest Team analyzes all papers published in this field in the past years, and presents up to 30 most influential papers for each year. This ranking list is automatically constructed based upon citations from both research papers and granted patents, and will be frequently updated to reflect the most recent changes. To find the latest version of this list or the most influential papers from other conferences/journals, please visit Best Paper Digest page. Note: the most influential papers may or may not include the papers that won the best paper awards. (Version: 2024-10).
This list is created by the Paper Digest Team. Experience the cutting-edge capabilities of Paper Digest, an innovative AI-powered research platform that empowers you to write, review, get answers and more.
Paper Digest Team
New York City, New York, 10017
team@paperdigest.org
TABLE 1: Most Influential ArXiv (Machine Learning) Papers (2024-10)
| Year | Rank | Paper | Author(s) |
|---|---|---|---|
| 2024 | 1 | Mixtral of Experts IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model. |
ALBERT Q. JIANG et. al. |
| 2024 | 2 | KTO: Model Alignment As Prospect Theoretic Optimization IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Using a Kahneman-Tversky model of human utility, we propose a HALO that directly maximizes the utility of generations instead of maximizing the log-likelihood of preferences, as current methods do. |
Kawin Ethayarajh; Winnie Xu; Niklas Muennighoff; Dan Jurafsky; Douwe Kiela; |
| 2024 | 3 | A Survey of Imitation Learning Methods, Environments and Metrics IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers. |
Nathan Gavenski; Felipe Meneguzzi; Michael Luck; Odinaldo Rodrigues; |
| 2024 | 4 | Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new fine-tuning method called Self-Play fIne-tuNing (SPIN), which starts from a supervised fine-tuned model. |
Zixiang Chen; Yihe Deng; Huizhuo Yuan; Kaixuan Ji; Quanquan Gu; |
| 2024 | 5 | Human Activity Recognition Using Smartphones IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Human Activity Recognition is a subject of great research today and has its applications in remote healthcare, activity tracking of the elderly or the disables, calories burnt tracking etc. |
MAYUR SONAWANE et. al. |
| 2024 | 6 | KAN: Kolmogorov-Arnold Networks IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). |
ZIMING LIU et. al. |
| 2024 | 7 | HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. |
MANTAS MAZEIKA et. al. |
| 2024 | 8 | Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a simple regression analysis approach for controlling biases in auto-evaluations. |
Yann Dubois; Balázs Galambosi; Percy Liang; Tatsunori B. Hashimoto; |
| 2024 | 9 | Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present Medusa, an efficient method that augments LLM inference by adding extra decoding heads to predict multiple subsequent tokens in parallel. |
TIANLE CAI et. al. |
| 2024 | 10 | Transformers Are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While Transformers have been the main architecture behind deep learning’s success in language modeling, state-space models (SSMs) such as Mamba have recently been shown to match or outperform Transformers at small to medium scale. We show that these families of models are actually quite closely related, and develop a rich framework of theoretical connections between SSMs and variants of attention, connected through various decompositions of a well-studied class of structured semiseparable matrices. |
Tri Dao; Albert Gu; |
| 2024 | 11 | Sharing Knowledge in Multi-Task Deep Reinforcement Learning IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning. |
Carlo D’Eramo; Davide Tateo; Andrea Bonarini; Marcello Restelli; Jan Peters; |
| 2024 | 12 | Multiclass Classification Procedure for Detecting Attacks on MQTT-IoT Protocol IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We have addressed two types of method for classifying the attacks, ensemble methods and deep learning models, more specifically recurrent networks with very satisfactory results. |
HECTOR ALAIZ-MORETON et. al. |
| 2024 | 13 | Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this survey, we present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead. |
Zeyu Han; Chao Gao; Jinyang Liu; Jeff Zhang; Sai Qian Zhang; |
| 2024 | 14 | RewardBench: Evaluating Reward Models for Language Modeling IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To enhance scientific understanding of reward models, we present RewardBench, a benchmark dataset and code-base for evaluation. |
NATHAN LAMBERT et. al. |
| 2024 | 15 | GaLore: Memory-Efficient LLM Training By Gradient Low-Rank Projection IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose Gradient Low-Rank Projection (GaLore), a training strategy that allows full-parameter learning but is more memory-efficient than common low-rank adaptation methods such as LoRA. |
JIAWEI ZHAO et. al. |
| 2024 | 16 | AI and Memory Wall IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here, we analyze encoder and decoder Transformer models and show how memory bandwidth can become the dominant bottleneck for decoder models. |
AMIR GHOLAMI et. al. |
| 2024 | 17 | Break The Sequential Dependency of LLM Inference Using Lookahead Decoding IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce Lookahead decoding, an exact, parallel decoding algorithm that accelerates LLM decoding without needing auxiliary models or data stores. |
Yichao Fu; Peter Bailis; Ion Stoica; Hao Zhang; |
| 2024 | 18 | Helping University Students to Choose Elective Courses By Using A Hybrid Multi-criteria Recommendation System with Genetic Optimization IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a hybrid RS that combines Collaborative Filtering (CF) and Content-based Filtering (CBF) using multiple criteria related both to student and course information to recommend the most suitable courses to the students. |
A. Esteban; A. Zafra; C. Romero; |
| 2024 | 19 | KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Quantization is a promising approach for compressing KV cache activations; however, existing solutions fail to represent activations accurately in sub-4-bit precision. Our work, KVQuant, facilitates low precision KV cache quantization by incorporating several novel methods: (i) Per-Channel Key Quantization, where we adjust the dimension along which we quantize the Key activations to better match the distribution; (ii) Pre-RoPE Key Quantization, where we quantize Key activations before the rotary positional embedding to mitigate its impact on quantization; (iii) Non-Uniform KV Cache Quantization, where we derive per-layer sensitivity-weighted non-uniform datatypes that better represent the distributions; and (iv) Per-Vector Dense-and-Sparse Quantization, where we isolate outliers separately for each vector to minimize skews in quantization ranges. |
COLEMAN HOOPER et. al. |
| 2024 | 20 | Random Forest-Based Prediction of Stroke Outcome IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We research into the clinical, biochemical and neuroimaging factors associated with the outcome of stroke patients to generate a predictive model using machine learning techniques for prediction of mortality and morbidity 3 months after admission. |
CARLOS FERNANDEZ-LOZANO et. al. |
| 2024 | 21 | Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose Hawk, an RNN with gated linear recurrences, and Griffin, a hybrid model that mixes gated linear recurrences with local attention. |
SOHAM DE et. al. |
| 2024 | 22 | VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Given that most computer interfaces cater to human perception, visual information often augments textual data in ways that text-only models struggle to harness effectively. To bridge this gap, we introduce VisualWebArena, a benchmark designed to assess the performance of multimodal web agents on realistic \textit{visually grounded tasks}. |
JING YU KOH et. al. |
| 2024 | 23 | WARM: On The Benefits of Weight Averaged Reward Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We identify two primary challenges when designing RMs to mitigate reward hacking: distribution shifts during the RL process and inconsistencies in human preferences. As a solution, we propose Weight Averaged Reward Models (WARM), first fine-tuning multiple RMs, then averaging them in the weight space. |
ALEXANDRE RAMÉ et. al. |
| 2024 | 24 | LoRA+: Efficient Low Rank Adaptation of Large Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show that Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021) leads to suboptimal finetuning of models with large width (embedding dimension). |
Soufiane Hayou; Nikhil Ghosh; Bin Yu; |
| 2024 | 25 | Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce Graph-Mamba, the first attempt to enhance long-range context modeling in graph networks by integrating a Mamba block with the input-dependent node selection mechanism. |
Chloe Wang; Oleksii Tsepa; Jun Ma; Bo Wang; |
| 2024 | 26 | From $r$ to $Q^*$: Your Language Model Is Secretly A Q-Function IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We theoretically show that we can derive DPO in the token-level MDP as a general inverse Q-learning algorithm, which satisfies the Bellman equation. |
Rafael Rafailov; Joey Hejna; Ryan Park; Chelsea Finn; |
| 2024 | 27 | SliceGPT: Compress Large Language Models By Deleting Rows and Columns IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we present SliceGPT, a new post-training sparsification scheme which replaces each weight matrix with a smaller (dense) matrix, reducing the embedding dimension of the network. |
Saleh Ashkboos; Maximilian L. Croci; Marcelo Gennari do Nascimento; Torsten Hoefler; James Hensman; |
| 2024 | 28 | Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While some defenses have been proposed, they have not been adapted to newly proposed attacks and more challenging threat models. To address this, we propose an optimization-based objective for defending LLMs against jailbreaking attacks and an algorithm, Robust Prompt Optimization (RPO) to create robust system-level defenses. |
Andy Zhou; Bo Li; Haohan Wang; |
| 2024 | 29 | A Minimaximalist Approach to Reinforcement Learning from Human Feedback IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Self-Play Preference Optimization (SPO), an algorithm for reinforcement learning from human feedback. |
Gokul Swamy; Christoph Dann; Rahul Kidambi; Zhiwei Steven Wu; Alekh Agarwal; |
| 2024 | 30 | Foundational Challenges in Assuring Alignment and Safety of Large Language Models IF:3 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Based on the identified challenges, we pose $200+$ concrete research questions. |
USMAN ANWAR et. al. |
| 2023 | 1 | Direct Preference Optimization: Your Language Model Is Secretly A Reward Model IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we introduce a new parameterization of the reward model in RLHF that enables extraction of the corresponding optimal policy in closed form, allowing us to solve the standard RLHF problem with only a simple classification loss. |
RAFAEL RAFAILOV et. al. |
| 2023 | 2 | QLoRA: Efficient Finetuning of Quantized LLMs IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. |
Tim Dettmers; Artidoro Pagnoni; Ari Holtzman; Luke Zettlemoyer; |
| 2023 | 3 | PaLM-E: An Embodied Multimodal Language Model IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose embodied language models to directly incorporate real-world continuous sensor modalities into language models and thereby establish the link between words and percepts. |
DANNY DRIESS et. al. |
| 2023 | 4 | Mamba: Linear-Time Sequence Modeling with Selective State Spaces IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers’ computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. |
Albert Gu; Tri Dao; |
| 2023 | 5 | Efficient Memory Management for Large Language Model Serving with PagedAttention IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem, we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems. |
WOOSUK KWON et. al. |
| 2023 | 6 | FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We observe that the inefficiency is due to suboptimal work partitioning between different thread blocks and warps on the GPU, causing either low-occupancy or unnecessary shared memory reads/writes. We propose FlashAttention-2, with better work partitioning to address these issues. |
Tri Dao; |
| 2023 | 7 | ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose to model the 3D parameter as a random variable instead of a constant as in SDS and present variational score distillation (VSD), a principled particle-based variational framework to explain and address the aforementioned issues in text-to-3D generation. |
ZHENGYI WANG et. al. |
| 2023 | 8 | BloombergGPT: A Large Language Model for Finance IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. |
SHIJIE WU et. al. |
| 2023 | 9 | Consistency Models IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Diffusion models have made significant breakthroughs in image, audio, and video generation, but they depend on an iterative generation process that causes slow sampling speed and caps their potential for real-time applications. To overcome this limitation, we propose consistency models, a new family of generative models that achieve high sample quality without adversarial training. |
Yang Song; Prafulla Dhariwal; Mark Chen; Ilya Sutskever; |
| 2023 | 10 | Jailbroken: How Does LLM Safety Training Fail? IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We hypothesize two failure modes of safety training: competing objectives and mismatched generalization. |
Alexander Wei; Nika Haghtalab; Jacob Steinhardt; |
| 2023 | 11 | SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. |
Elias Frantar; Dan Alistarh; |
| 2023 | 12 | AlpacaFarm: A Simulation Framework for Methods That Learn from Human Feedback IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Replicating and understanding this instruction-following requires tackling three major challenges: the high cost of data collection, the lack of trustworthy evaluation, and the absence of reference method implementations. We address these challenges with AlpacaFarm, a simulator that enables research and development for learning from feedback at a low cost. |
YANN DUBOIS et. al. |
| 2023 | 13 | Let’s Verify Step By Step IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. |
HUNTER LIGHTMAN et. al. |
| 2023 | 14 | A Watermark for Large Language Models IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a watermarking framework for proprietary language models. |
JOHN KIRCHENBAUER et. al. |
| 2023 | 15 | Jailbreaking Black Box Large Language Models in Twenty Queries IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we propose Prompt Automatic Iterative Refinement (PAIR), an algorithm that generates semantic jailbreaks with only black-box access to an LLM. |
PATRICK CHAO et. al. |
| 2023 | 16 | Mathematical Capabilities of ChatGPT IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We investigate the mathematical capabilities of two iterations of ChatGPT (released 9-January-2023 and 30-January-2023) and of GPT-4 by testing them on publicly available datasets, as well as hand-crafted ones, using a novel methodology. |
SIMON FRIEDER et. al. |
| 2023 | 17 | A Comprehensive Survey of Continual Learning: Theory, Method and Application IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. |
Liyuan Wang; Xingxing Zhang; Hang Su; Jun Zhu; |
| 2023 | 18 | Inference-Time Intervention: Eliciting Truthful Answers from A Language Model IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Inference-Time Intervention (ITI), a technique designed to enhance the truthfulness of large language models (LLMs). |
Kenneth Li; Oam Patel; Fernanda Viégas; Hanspeter Pfister; Martin Wattenberg; |
| 2023 | 19 | RAFT: Reward RAnked FineTuning for Generative Foundation Model Alignment IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we introduce a new framework, Reward rAnked FineTuning (RAFT), designed to align generative models effectively. |
HANZE DONG et. al. |
| 2023 | 20 | Progress Measures for Grokking Via Mechanistic Interpretability IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We confirm the algorithm by analyzing the activations and weights and by performing ablations in Fourier space. |
Neel Nanda; Lawrence Chan; Tom Lieberum; Jess Smith; Jacob Steinhardt; |
| 2023 | 21 | Habits and Goals in Synergy: A Variational Bayesian Framework for Behavior IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The habitual behavior is generated by using prior distribution of intention, which is goal-less; and the goal-directed behavior is generated by the posterior distribution of intention, which is conditioned on the goal. Building on this idea, we present a novel Bayesian framework for modeling behaviors. |
Dongqi Han; Kenji Doya; Dongsheng Li; Jun Tani; |
| 2023 | 22 | Zephyr: Direct Distillation of LM Alignment IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We aim to produce a smaller language model that is aligned to user intent. |
LEWIS TUNSTALL et. al. |
| 2023 | 23 | CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce CodeGeeX, a multilingual model with 13 billion parameters for code generation. |
QINKAI ZHENG et. al. |
| 2023 | 24 | Large Language Models As Optimizers IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose Optimization by PROmpting (OPRO), a simple and effective approach to leverage large language models (LLMs) as optimizers, where the optimization task is described in natural language. |
CHENGRUN YANG et. al. |
| 2023 | 25 | Symbolic Discovery of Optimization Algorithms IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a method to formulate algorithm discovery as program search, and apply it to discover optimization algorithms for deep neural network training. |
XIANGNING CHEN et. al. |
| 2023 | 26 | Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, this dependence can significantly constrain the true potential of AI-assistant agents due to the high cost of obtaining human supervision and the related issues on quality, reliability, diversity, self-consistency, and undesirable biases. To address these challenges, we propose a novel approach called SELF-ALIGN, which combines principle-driven reasoning and the generative power of LLMs for the self-alignment of AI agents with minimal human supervision. |
ZHIQING SUN et. al. |
| 2023 | 27 | A Cookbook of Self-Supervised Learning IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. |
RANDALL BALESTRIERO et. al. |
| 2023 | 28 | Scalable Extraction of Training Data from (Production) Language Models IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. |
MILAD NASR et. al. |
| 2023 | 29 | Representation Engineering: A Top-Down Approach to AI Transparency IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. |
ANDY ZOU et. al. |
| 2023 | 30 | Baseline Defenses for Adversarial Attacks Against Aligned Language Models IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Recent work shows that text optimizers can produce jailbreaking prompts that bypass moderation and alignment. Drawing from the rich body of work on adversarial machine learning, we approach these attacks with three questions: What threat models are practically useful in this domain? |
NEEL JAIN et. al. |
| 2022 | 1 | Classifier-Free Diffusion Guidance IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: It also raises the question of whether guidance can be performed without a classifier. We show that guidance can be indeed performed by a pure generative model without such a classifier: in what we call classifier-free guidance, we jointly train a conditional and an unconditional diffusion model, and we combine the resulting conditional and unconditional score estimates to attain a trade-off between sample quality and diversity similar to that obtained using classifier guidance. |
Jonathan Ho; Tim Salimans; |
| 2022 | 2 | Scaling Instruction-Finetuned Language Models IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. |
HYUNG WON CHUNG et. al. |
| 2022 | 3 | FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose FlashAttention, an IO-aware exact attention algorithm that uses tiling to reduce the number of memory reads/writes between GPU high bandwidth memory (HBM) and GPU on-chip SRAM. |
Tri Dao; Daniel Y. Fu; Stefano Ermon; Atri Rudra; Christopher Ré; |
| 2022 | 4 | DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose an exact formulation of the solution of diffusion ODEs. |
CHENG LU et. al. |
| 2022 | 5 | Diffusion Models: A Comprehensive Survey of Methods and Applications IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey, we provide an overview of the rapidly expanding body of work on diffusion models, categorizing the research into three key areas: efficient sampling, improved likelihood estimation, and handling data with special structures. |
LING YANG et. al. |
| 2022 | 6 | FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. |
TIAN ZHOU et. al. |
| 2022 | 7 | Language Models As Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. make breakfast), to a chosen set of actionable steps (e.g. open fridge). |
Wenlong Huang; Pieter Abbeel; Deepak Pathak; Igor Mordatch; |
| 2022 | 8 | CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The prevalence of large language models advances the state-of-the-art for program synthesis, though limited training resources and data impede open access to such models. To democratize this, we train and release a family of large language models up to 16.1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER. |
ERIK NIJKAMP et. al. |
| 2022 | 9 | Data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To get us closer to general self-supervised learning, we present data2vec, a framework that uses the same learning method for either speech, NLP or computer vision. |
ALEXEI BAEVSKI et. al. |
| 2022 | 10 | Model Soups: Averaging Weights of Multiple Fine-tuned Models Improves Accuracy Without Increasing Inference Time IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder. In this paper, we revisit the second step of this procedure in the context of fine-tuning large pre-trained models, where fine-tuned models often appear to lie in a single low error basin. |
MITCHELL WORTSMAN et. al. |
| 2022 | 11 | Large Language Models Are Human-Level Prompt Engineers IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by classical program synthesis and the human approach to prompt engineering, we propose Automatic Prompt Engineer (APE) for automatic instruction generation and selection. |
YONGCHAO ZHOU et. al. |
| 2022 | 12 | Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper Than In-Context Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Along the way, we introduce a new PEFT method called (IA)$^3$ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. |
HAOKUN LIU et. al. |
| 2022 | 13 | A Time Series Is Worth 64 Words: Long-term Forecasting with Transformers IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an efficient design of Transformer-based models for multivariate time series forecasting and self-supervised representation learning. |
Yuqi Nie; Nam H. Nguyen; Phanwadee Sinthong; Jayant Kalagnanam; |
| 2022 | 14 | GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While there is emerging work on relieving this pressure via model compression, the applicability and performance of existing compression techniques is limited by the scale and complexity of GPT models. In this paper, we address this challenge, and propose GPTQ, a new one-shot weight quantization method based on approximate second-order information, that is both highly-accurate and highly-efficient. |
Elias Frantar; Saleh Ashkboos; Torsten Hoefler; Dan Alistarh; |
| 2022 | 15 | Transformers in Time Series: A Survey IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. |
QINGSONG WEN et. al. |
| 2022 | 16 | LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Large language models have been widely adopted but require significant GPU memory for inference. |
Tim Dettmers; Mike Lewis; Younes Belkada; Luke Zettlemoyer; |
| 2022 | 17 | Reproducible Scaling Laws for Contrastive Language-image Learning IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, previous work on scaling laws has primarily used private data \& models or focused on uni-modal language or vision learning. To address these limitations, we investigate scaling laws for contrastive language-image pre-training (CLIP) with the public LAION dataset and the open-source OpenCLIP repository. |
MEHDI CHERTI et. al. |
| 2022 | 18 | Quantifying Memorization Across Neural Language Models IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We describe three log-linear relationships that quantify the degree to which LMs emit memorized training data. |
NICHOLAS CARLINI et. al. |
| 2022 | 19 | Improving Alignment of Dialogue Agents Via Targeted Human Judgements IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines. |
AMELIA GLAESE et. al. |
| 2022 | 20 | Equivariant Diffusion for Molecule Generation in 3D IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work introduces a diffusion model for molecule generation in 3D that is equivariant to Euclidean transformations. |
Emiel Hoogeboom; Victor Garcia Satorras; Clément Vignac; Max Welling; |
| 2022 | 21 | Flow Matching for Generative Modeling IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. |
Yaron Lipman; Ricky T. Q. Chen; Heli Ben-Hamu; Maximilian Nickel; Matt Le; |
| 2022 | 22 | Planning with Diffusion for Flexible Behavior Synthesis IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While conceptually simple, this combination has a number of empirical shortcomings, suggesting that learned models may not be well-suited to standard trajectory optimization. In this paper, we consider what it would look like to fold as much of the trajectory optimization pipeline as possible into the modeling problem, such that sampling from the model and planning with it become nearly identical. |
Michael Janner; Yilun Du; Joshua B. Tenenbaum; Sergey Levine; |
| 2022 | 23 | StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We demonstrate that this model can invert and edit images beyond the narrow domain of portraits or specific object classes. |
Axel Sauer; Katja Schwarz; Andreas Geiger; |
| 2022 | 24 | Recipe for A General, Powerful, Scalable Graph Transformer IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Graph Transformers (GTs) have gained popularity in the field of graph representation learning with a variety of recent publications but they lack a common foundation about what constitutes a good positional or structural encoding, and what differentiates them. In this paper, we summarize the different types of encodings with a clearer definition and categorize them as being $\textit{local}$, $\textit{global}$ or $\textit{relative}$. |
LADISLAV RAMPÁŠEK et. al. |
| 2022 | 25 | TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Technically, we propose the TimesNet with TimesBlock as a task-general backbone for time series analysis. |
HAIXU WU et. al. |
| 2022 | 26 | CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Motivated by the recent success of representation learning in computer vision and natural language processing, we argue that a more promising paradigm for time series forecasting, is to first learn disentangled feature representations, followed by a simple regression fine-tuning step — we justify such a paradigm from a causal perspective. Following this principle, we propose a new time series representation learning framework for time series forecasting named CoST, which applies contrastive learning methods to learn disentangled seasonal-trend representations. |
Gerald Woo; Chenghao Liu; Doyen Sahoo; Akshat Kumar; Steven Hoi; |
| 2022 | 27 | DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Although recent works propose dedicated high-order solvers and achieve a further speedup for sampling without guidance, their effectiveness for guided sampling has not been well-tested before. In this work, we demonstrate that previous high-order fast samplers suffer from instability issues, and they even become slower than DDIM when the guidance scale grows large. |
CHENG LU et. al. |
| 2022 | 28 | Out-of-Distribution Detection with Deep Nearest Neighbors IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore the efficacy of non-parametric nearest-neighbor distance for OOD detection, which has been largely overlooked in the literature. |
Yiyou Sun; Yifei Ming; Xiaojin Zhu; Yixuan Li; |
| 2022 | 29 | GraphMAE: Self-Supervised Masked Graph Autoencoders IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we identify and examine the issues that negatively impact the development of GAEs, including their reconstruction objective, training robustness, and error metric. |
ZHENYU HOU et. al. |
| 2022 | 30 | Out of One, Many: Using Language Models to Simulate Human Samples IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. |
LISA P. ARGYLE et. al. |
| 2021 | 1 | Diffusion Models Beat GANs on Image Synthesis IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models. |
Prafulla Dhariwal; Alex Nichol; |
| 2021 | 2 | Evaluating Large Language Models Trained on Code IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities. |
MARK CHEN et. al. |
| 2021 | 3 | On The Opportunities and Risks of Foundation Models IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. |
RISHI BOMMASANI et. al. |
| 2021 | 4 | Improved Denoising Diffusion Probabilistic Models IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that with a few simple modifications, DDPMs can also achieve competitive log-likelihoods while maintaining high sample quality. |
Alex Nichol; Prafulla Dhariwal; |
| 2021 | 5 | Training Verifiers to Solve Math Word Problems IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To increase performance, we propose training verifiers to judge the correctness of model completions. |
KARL COBBE et. al. |
| 2021 | 6 | Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We design models based off T5-Base and T5-Large to obtain up to 7x increases in pre-training speed with the same computational resources. |
William Fedus; Barret Zoph; Noam Shazeer; |
| 2021 | 7 | Multitask Prompted Training Enables Zero-Shot Task Generalization IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). |
VICTOR SANH et. al. |
| 2021 | 8 | Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper studies the long-term forecasting problem of time series. |
Haixu Wu; Jiehui Xu; Jianmin Wang; Mingsheng Long; |
| 2021 | 9 | Decision Transformer: Reinforcement Learning Via Sequence Modeling IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. |
LILI CHEN et. al. |
| 2021 | 10 | Efficiently Modeling Long Sequences with Structured State Spaces IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose the Structured State Space sequence model (S4) based on a new parameterization for the SSM, and show that it can be computed much more efficiently than prior approaches while preserving their theoretical strengths. |
Albert Gu; Karan Goel; Christopher Ré; |
| 2021 | 11 | Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Such a ‘geometric unification’ endeavour, in the spirit of Felix Klein’s Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. |
Michael M. Bronstein; Joan Bruna; Taco Cohen; Petar Veličković; |
| 2021 | 12 | Measuring Mathematical Problem Solving With The MATH Dataset IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To measure this ability in machine learning models, we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems. |
DAN HENDRYCKS et. al. |
| 2021 | 13 | Generalizing to Unseen Domains: A Survey on Domain Generalization IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. |
JINDONG WANG et. al. |
| 2021 | 14 | Ensemble Deep Learning: A Review IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. |
M. A. Ganaie; Minghui Hu; A. K. Malik; M. Tanveer; P. N. Suganthan; |
| 2021 | 15 | Tabular Data: Deep Learning Is Not All You Need IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our study shows that XGBoost outperforms these deep models across the datasets, including the datasets used in the papers that proposed the deep models. |
Ravid Shwartz-Ziv; Amitai Armon; |
| 2021 | 16 | The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we carefully study the performance of PPO in cooperative multi-agent settings. |
CHAO YU et. al. |
| 2021 | 17 | Variational Diffusion Models IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Diffusion-based generative models have demonstrated a capacity for perceptually impressive synthesis, but can they also be great likelihood-based models? We answer this in the affirmative, and introduce a family of diffusion-based generative models that obtain state-of-the-art likelihoods on standard image density estimation benchmarks. |
Diederik P. Kingma; Tim Salimans; Ben Poole; Jonathan Ho; |
| 2021 | 18 | A Survey of Uncertainty in Deep Neural Networks IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: For a practical application, we discuss different measures of uncertainty, approaches for the calibration of neural networks and give an overview of existing baselines and implementations. |
JAKOB GAWLIKOWSKI et. al. |
| 2021 | 19 | A Survey of Transformers IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey, we provide a comprehensive review of various X-formers. |
Tianyang Lin; Yuxin Wang; Xiangyang Liu; Xipeng Qiu; |
| 2021 | 20 | How Attentive Are Graph Attention Networks? IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To remove this limitation, we introduce a simple fix by modifying the order of operations and propose GATv2: a dynamic graph attention variant that is strictly more expressive than GAT. |
Shaked Brody; Uri Alon; Eran Yahav; |
| 2021 | 21 | E(n) Equivariant Graph Neural Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs). |
Victor Garcia Satorras; Emiel Hoogeboom; Max Welling; |
| 2021 | 22 | Model-Contrastive Federated Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose MOON: model-contrastive federated learning. |
Qinbin Li; Bingsheng He; Dawn Song; |
| 2021 | 23 | Domain Generalization: A Survey IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. |
Kaiyang Zhou; Ziwei Liu; Yu Qiao; Tao Xiang; Chen Change Loy; |
| 2021 | 24 | Federated Learning on Non-IID Data Silos: An Experimental Study IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, to help researchers better understand and study the non-IID data setting in federated learning, we propose comprehensive data partitioning strategies to cover the typical non-IID data cases. |
Qinbin Li; Yiqun Diao; Quan Chen; Bingsheng He; |
| 2021 | 25 | FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address this issue, we propose Curriculum Pseudo Labeling (CPL), a curriculum learning approach to leverage unlabeled data according to the model’s learning status. |
BOWEN ZHANG et. al. |
| 2021 | 26 | FedBN: Federated Learning on Non-IID Features Via Local Batch Normalization IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose an effective method that uses local batch normalization to alleviate the feature shift before averaging models. |
Xiaoxiao Li; Meirui Jiang; Xiaofei Zhang; Michael Kamp; Qi Dou; |
| 2021 | 27 | Offline Reinforcement Learning with Implicit Q-Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an offline RL method that never needs to evaluate actions outside of the dataset, but still enables the learned policy to improve substantially over the best behavior in the data through generalization. |
Ilya Kostrikov; Ashvin Nair; Sergey Levine; |
| 2021 | 28 | Graph Neural Network-Based Anomaly Detection in Multivariate Time Series IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Given high-dimensional time series data (e.g., sensor data), how can we detect anomalous events, such as system faults and attacks? |
Ailin Deng; Bryan Hooi; |
| 2021 | 29 | A Minimalist Approach to Offline Reinforcement Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we aim to make a deep RL algorithm work while making minimal changes. |
Scott Fujimoto; Shixiang Shane Gu; |
| 2021 | 30 | Towards Personalized Federated Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this survey, we explore the domain of Personalized FL (PFL) to address the fundamental challenges of FL on heterogeneous data, a universal characteristic inherent in all real-world datasets. |
Alysa Ziying Tan; Han Yu; Lizhen Cui; Qiang Yang; |
| 2020 | 1 | A Simple Framework For Contrastive Learning Of Visual Representations IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents SimCLR: a simple framework for contrastive learning of visual representations. |
Ting Chen; Simon Kornblith; Mohammad Norouzi; Geoffrey Hinton; |
| 2020 | 2 | Denoising Diffusion Probabilistic Models IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. |
Jonathan Ho; Ajay Jain; Pieter Abbeel; |
| 2020 | 3 | Bootstrap Your Own Latent: A New Approach To Self-supervised Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. |
JEAN-BASTIEN GRILL et. al. |
| 2020 | 4 | Denoising Diffusion Implicit Models IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To accelerate sampling, we present denoising diffusion implicit models (DDIMs), a more efficient class of iterative implicit probabilistic models with the same training procedure as DDPMs. |
Jiaming Song; Chenlin Meng; Stefano Ermon; |
| 2020 | 5 | Score-Based Generative Modeling Through Stochastic Differential Equations IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In particular, we introduce a predictor-corrector framework to correct errors in the evolution of the discretized reverse-time SDE. |
YANG SONG et. al. |
| 2020 | 6 | Supervised Contrastive Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we extend the self-supervised batch contrastive approach to the fully-supervised setting, allowing us to effectively leverage label information. |
PRANNAY KHOSLA et. al. |
| 2020 | 7 | A Tutorial on Learning With Bayesian Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. |
David Heckerman; |
| 2020 | 8 | Scaling Laws For Neural Language Models IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study empirical scaling laws for language model performance on the cross-entropy loss. |
JARED KAPLAN et. al. |
| 2020 | 9 | FixMatch: Simplifying Semi-Supervised Learning With Consistency And Confidence IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. |
KIHYUK SOHN et. al. |
| 2020 | 10 | Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a $ProbSparse$ self-attention mechanism, which achieves $O(L \log L)$ in time complexity and memory usage, and has comparable performance on sequences’ dependency alignment. |
HAOYI ZHOU et. al. |
| 2020 | 11 | Open Graph Benchmark: Datasets for Machine Learning on Graphs IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. |
WEIHUA HU et. al. |
| 2020 | 12 | Knowledge Distillation: A Survey IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. |
Jianping Gou; Baosheng Yu; Stephen John Maybank; Dacheng Tao; |
| 2020 | 13 | Big Self-Supervised Models Are Strong Semi-Supervised Learners IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We find that, the fewer the labels, the more this approach (task-agnostic use of unlabeled data) benefits from a bigger network. |
Ting Chen; Simon Kornblith; Kevin Swersky; Mohammad Norouzi; Geoffrey Hinton; |
| 2020 | 14 | Reformer: The Efficient Transformer IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce two techniques to improve the efficiency of Transformers. |
Nikita Kitaev; Łukasz Kaiser; Anselm Levskaya; |
| 2020 | 15 | Fourier Neural Operator for Parametric Partial Differential Equations IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. |
ZONGYI LI et. al. |
| 2020 | 16 | Big Bird: Transformers for Longer Sequences IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear. |
MANZIL ZAHEER et. al. |
| 2020 | 17 | Offline Reinforcement Learning: Tutorial, Review, And Perspectives On Open Problems IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. |
Sergey Levine; Aviral Kumar; George Tucker; Justin Fu; |
| 2020 | 18 | Meta-Learning In Neural Networks: A Survey IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta-learning aims to improve the learning algorithm itself, given the experience of multiple learning episodes. |
Timothy Hospedales; Antreas Antoniou; Paul Micaelli; Amos Storkey; |
| 2020 | 19 | Graph Contrastive Learning with Augmentations IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. |
YUNING YOU et. al. |
| 2020 | 20 | Reliable Evaluation Of Adversarial Robustness With An Ensemble Of Diverse Parameter-free Attacks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. |
Francesco Croce; Matthias Hein; |
| 2020 | 21 | Understanding Contrastive Representation Learning Through Alignment And Uniformity On The Hypersphere IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we identify two key properties related to the contrastive loss: (1) alignment (closeness) of features from positive pairs, and (2) uniformity of the induced distribution of the (normalized) features on the hypersphere. |
Tongzhou Wang; Phillip Isola; |
| 2020 | 22 | A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this regard, researchers have proposed different UQ methods and examined their performance in a variety of applications such as computer vision (e.g., self-driving cars and object detection), image processing (e.g., image restoration), medical image analysis (e.g., medical image classification and segmentation), natural language processing (e.g., text classification, social media texts and recidivism risk-scoring), bioinformatics, etc. |
MOLOUD ABDAR et. al. |
| 2020 | 23 | On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, optimizing the hyper-parameters of common machine learning models is studied. |
Li Yang; Abdallah Shami; |
| 2020 | 24 | Conservative Q-Learning For Offline Reinforcement Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose conservative Q-learning (CQL), which aims to address these limitations by learning a conservative Q-function such that the expected value of a policy under this Q-function lower-bounds its true value. |
Aviral Kumar; Aurick Zhou; George Tucker; Sergey Levine; |
| 2020 | 25 | Linformer: Self-Attention With Linear Complexity IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we demonstrate that the self-attention mechanism can be approximated by a low-rank matrix. |
Sinong Wang; Belinda Z. Li; Madian Khabsa; Han Fang; Hao Ma; |
| 2020 | 26 | Deep Reinforcement Learning for Autonomous Driving: A Survey IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This review summarises deep reinforcement learning (DRL) algorithms and provides a taxonomy of automated driving tasks where (D)RL methods have been employed, while addressing key computational challenges in real world deployment of autonomous driving agents. |
B RAVI KIRAN et. al. |
| 2020 | 27 | Self-supervised Learning: Generative or Contrastive IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this survey, we take a look into new self-supervised learning methods for representation in computer vision, natural language processing, and graph learning. |
XIAO LIU et. al. |
| 2020 | 28 | Rethinking Attention with Performers IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. |
KRZYSZTOF CHOROMANSKI et. al. |
| 2020 | 29 | Transformers Are RNNs: Fast Autoregressive Transformers With Linear Attention IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address this limitation, we express the self-attention as a linear dot-product of kernel feature maps and make use of the associativity property of matrix products to reduce the complexity from $\mathcal{O}\left(N^2\right)$ to $\mathcal{O}\left(N\right)$, where $N$ is the sequence length. |
Angelos Katharopoulos; Apoorv Vyas; Nikolaos Pappas; François Fleuret; |
| 2020 | 30 | Can AI Help In Screening Viral And COVID-19 Pneumonia? IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The aim of this paper is to propose a robust technique for automatic detection of COVID-19 pneumonia from digital chest X-ray images applying pre-trained deep-learning algorithms while maximizing the detection accuracy. |
MUHAMMAD E. H. CHOWDHURY et. al. |
| 2019 | 1 | PyTorch: An Imperative Style, High-Performance Deep Learning Library IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. |
ADAM PASZKE et. al. |
| 2019 | 2 | Exploring The Limits of Transfer Learning with A Unified Text-to-Text Transformer IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts all text-based language problems into a text-to-text format. |
COLIN RAFFEL et. al. |
| 2019 | 3 | EfficientNet: Rethinking Model Scaling For Convolutional Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. |
Mingxing Tan; Quoc V. Le; |
| 2019 | 4 | A Comprehensive Survey On Graph Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. |
ZONGHAN WU et. al. |
| 2019 | 5 | Advances and Open Problems in Federated Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges. |
PETER KAIROUZ et. al. |
| 2019 | 6 | NuScenes: A Multimodal Dataset For Autonomous Driving IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. |
HOLGER CAESAR et. al. |
| 2019 | 7 | Optuna: A Next-generation Hyperparameter Optimization Framework IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The purpose of this study is to introduce new design-criteria for next-generation hyperparameter optimization software. |
Takuya Akiba; Shotaro Sano; Toshihiko Yanase; Takeru Ohta; Masanori Koyama; |
| 2019 | 8 | Federated Learning: Challenges, Methods, And Future Directions IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this article, we discuss the unique characteristics and challenges of federated learning, provide a broad overview of current approaches, and outline several directions of future work that are relevant to a wide range of research communities. |
Tian Li; Anit Kumar Sahu; Ameet Talwalkar; Virginia Smith; |
| 2019 | 9 | Fast Graph Representation Learning With PyTorch Geometric IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present the library in detail and perform a comprehensive comparative study of the implemented methods in homogeneous evaluation scenarios. |
Matthias Fey; Jan Eric Lenssen; |
| 2019 | 10 | A Comprehensive Survey On Transfer Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Due to the rapid expansion of the transfer learning area, it is both necessary and challenging to comprehensively review the relevant studies. |
FUZHEN ZHUANG et. al. |
| 2019 | 11 | Transformer-XL: Attentive Language Models Beyond A Fixed-Length Context IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence. |
ZIHANG DAI et. al. |
| 2019 | 12 | A Survey on Bias and Fairness in Machine Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. |
Ninareh Mehrabi; Fred Morstatter; Nripsuta Saxena; Kristina Lerman; Aram Galstyan; |
| 2019 | 13 | Parameter-Efficient Transfer Learning For NLP IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: As an alternative, we propose transfer with adapter modules. |
NEIL HOULSBY et. al. |
| 2019 | 14 | Benchmarking Neural Network Robustness To Common Corruptions And Perturbations IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we establish rigorous benchmarks for image classifier robustness. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier’s robustness to common perturbations. |
Dan Hendrycks; Thomas Dietterich; |
| 2019 | 15 | Generative Modeling By Estimating Gradients Of The Data Distribution IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. |
Yang Song; Stefano Ermon; |
| 2019 | 16 | Simplifying Graph Convolutional Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. |
FELIX WU et. al. |
| 2019 | 17 | MixMatch: A Holistic Approach To Semi-Supervised Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-augmented unlabeled examples and mixing labeled and unlabeled data using MixUp. |
DAVID BERTHELOT et. al. |
| 2019 | 18 | Towards Federated Learning At Scale: System Design IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions. |
KEITH BONAWITZ et. al. |
| 2019 | 19 | On The Convergence Of Adam And Beyond IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We provide an explicit example of a simple convex optimization setting where Adam does not converge to the optimal solution, and describe the precise problems with the previous analysis of Adam algorithm. |
Sashank J. Reddi; Satyen Kale; Sanjiv Kumar; |
| 2019 | 20 | Theoretically Principled Trade-off Between Robustness And Accuracy IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we decompose the prediction error for adversarial examples (robust error) as the sum of the natural (classification) error and boundary error, and provide a differentiable upper bound using the theory of classification-calibrated loss, which is shown to be the tightest possible upper bound uniform over all probability distributions and measurable predictors. |
HONGYANG ZHANG et. al. |
| 2019 | 21 | Self-training With Noisy Student Improves ImageNet Classification IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Noisy Student Training, a semi-supervised learning approach that works well even when labeled data is abundant. |
Qizhe Xie; Minh-Thang Luong; Eduard Hovy; Quoc V. Le; |
| 2019 | 22 | Unsupervised Data Augmentation For Consistency Training IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present a new perspective on how to effectively noise unlabeled examples and argue that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in semi-supervised learning. |
Qizhe Xie; Zihang Dai; Eduard Hovy; Minh-Thang Luong; Quoc V. Le; |
| 2019 | 23 | SCAFFOLD: Stochastic Controlled Averaging for Federated Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: As a solution, we propose a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the `client-drift’ in its local updates. |
SAI PRANEETH KARIMIREDDY et. al. |
| 2019 | 24 | An Introduction To Variational Autoencoders IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we provide an introduction to variational autoencoders and some important extensions. |
Diederik P. Kingma; Max Welling; |
| 2019 | 25 | RotatE: Knowledge Graph Embedding By Relational Rotation In Complex Space IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. |
Zhiqing Sun; Zhi-Hong Deng; Jian-Yun Nie; Jian Tang; |
| 2019 | 26 | Certified Adversarial Robustness Via Randomized Smoothing IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This randomized smoothing technique has been proposed recently in the literature, but existing guarantees are loose. |
Jeremy M Cohen; Elan Rosenfeld; J. Zico Kolter; |
| 2019 | 27 | Deep Leakage From Gradients IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Abstract: Exchanging gradients is a widely used method in modern multi-node machine learning system (e.g., distributed training, collaborative learning). For a long time, people believed … |
Ligeng Zhu; Zhijian Liu; Song Han; |
| 2019 | 28 | Introduction to Online Convex Optimization IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: It is necessary as well as beneficial to take a robust approach, by applying an optimization method that learns as one goes along, learning from experience as more aspects of the problem are observed. |
Elad Hazan; |
| 2019 | 29 | Mastering Atari, Go, Chess And Shogi By Planning With A Learned Model IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. |
JULIAN SCHRITTWIESER et. al. |
| 2019 | 30 | On The Variance of The Adaptive Learning Rate and Beyond IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Pursuing the theory behind warmup, we identify a problem of the adaptive learning rate (i.e., it has problematically large variance in the early stage), suggest warmup works as a variance reduction technique, and provide both empirical and theoretical evidence to verify our hypothesis. |
LIYUAN LIU et. al. |
| 2018 | 1 | Representation Learning With Contrastive Predictive Coding IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. |
Aaron van den Oord; Yazhe Li; Oriol Vinyals; |
| 2018 | 2 | Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning With A Stochastic Actor IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. |
Tuomas Haarnoja; Aurick Zhou; Pieter Abbeel; Sergey Levine; |
| 2018 | 3 | How Powerful Are Graph Neural Networks? IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. |
Keyulu Xu; Weihua Hu; Jure Leskovec; Stefanie Jegelka; |
| 2018 | 4 | Large Scale GAN Training For High Fidelity Natural Image Synthesis IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale. |
Andrew Brock; Jeff Donahue; Karen Simonyan; |
| 2018 | 5 | Graph Neural Networks: A Review of Methods and Applications IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey, we propose a general design pipeline for GNN models and discuss the variants of each component, systematically categorize the applications, and propose four open problems for future research. |
JIE ZHOU et. al. |
| 2018 | 6 | Neural Ordinary Differential Equations IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a new family of deep neural network models. |
Ricky T. Q. Chen; Yulia Rubanova; Jesse Bettencourt; David Duvenaud; |
| 2018 | 7 | Spectral Normalization For Generative Adversarial Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. |
Takeru Miyato; Toshiki Kataoka; Masanori Koyama; Yuichi Yoshida; |
| 2018 | 8 | An Empirical Evaluation Of Generic Convolutional And Recurrent Networks For Sequence Modeling IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To assist related work, we have made code available at http://github.com/locuslab/TCN . |
Shaojie Bai; J. Zico Kolter; Vladlen Koltun; |
| 2018 | 9 | DARTS: Differentiable Architecture Search IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. |
Hanxiao Liu; Karen Simonyan; Yiming Yang; |
| 2018 | 10 | Federated Optimization In Heterogeneous Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce a framework, FedProx, to tackle heterogeneity in federated networks. |
TIAN LI et. al. |
| 2018 | 11 | The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. |
Jonathan Frankle; Michael Carbin; |
| 2018 | 12 | Obfuscated Gradients Give A False Sense Of Security: Circumventing Defenses To Adversarial Examples IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We describe characteristic behaviors of defenses exhibiting the effect, and for each of the three types of obfuscated gradients we discover, we develop attack techniques to overcome it. |
Anish Athalye; Nicholas Carlini; David Wagner; |
| 2018 | 13 | PointPillars: Fast Encoders For Object Detection From Point Clouds IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we consider the problem of encoding a point cloud into a format appropriate for a downstream detection pipeline. |
ALEX H. LANG et. al. |
| 2018 | 14 | Relational Inductive Biases, Deep Learning, And Graph Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a new building block for the AI toolkit with a strong relational inductive bias–the graph network–which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. |
PETER W. BATTAGLIA et. al. |
| 2018 | 15 | Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The goal of this paper is to explain the essential RNN and LSTM fundamentals in a single document. |
Alex Sherstinsky; |
| 2018 | 16 | Neural Tangent Kernel: Convergence And Generalization In Neural Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We prove the positive-definiteness of the limiting NTK when the data is supported on the sphere and the non-linearity is non-polynomial. |
Arthur Jacot; Franck Gabriel; Clément Hongler; |
| 2018 | 17 | Efficient Neural Architecture Search Via Parameter Sharing IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. |
Hieu Pham; Melody Y. Guan; Barret Zoph; Quoc V. Le; Jeff Dean; |
| 2018 | 18 | Continual Lifelong Learning With Neural Networks: A Review IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. |
German I. Parisi; Ronald Kemker; Jose L. Part; Christopher Kanan; Stefan Wermter; |
| 2018 | 19 | Deep Learning In Agriculture: A Survey IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we perform a survey of 40 research efforts that employ deep learning techniques, applied to various agricultural and food production challenges. |
Andreas Kamilaris; Francesc X. Prenafeta-Boldu; |
| 2018 | 20 | Deeper Insights Into Graph Convolutional Networks For Semi-Supervised Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we develop deeper insights into the GCN model and address its fundamental limits. |
Qimai Li; Zhichao Han; Xiao-Ming Wu; |
| 2018 | 21 | A Survey On Deep Transfer Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. |
CHUANQI TAN et. al. |
| 2018 | 22 | Deep Learning For Time Series Classification: A Review IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. |
Hassan Ismail Fawaz; Germain Forestier; Jonathan Weber; Lhassane Idoumghar; Pierre-Alain Muller; |
| 2018 | 23 | Generalized Cross Entropy Loss For Training Deep Neural Networks With Noisy Labels IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we present a theoretically grounded set of noise-robust loss functions that can be seen as a generalization of MAE and CCE. |
Zhilu Zhang; Mert R. Sabuncu; |
| 2018 | 24 | Federated Learning with Non-IID Data IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we focus on the statistical challenge of federated learning when local data is non-IID. |
YUE ZHAO et. al. |
| 2018 | 25 | On First-Order Meta-Learning Algorithms IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. |
Alex Nichol; Joshua Achiam; John Schulman; |
| 2018 | 26 | Soft Actor-Critic Algorithms And Applications IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we describe Soft Actor-Critic (SAC), our recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework. |
TUOMAS HAARNOJA et. al. |
| 2018 | 27 | Hierarchical Graph Representation Learning With Differentiable Pooling IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here we propose DiffPool, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion. |
REX YING et. al. |
| 2018 | 28 | Co-teaching: Robust Training Of Deep Neural Networks With Extremely Noisy Labels IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Therefore in this paper, we propose a new deep learning paradigm called Co-teaching for combating with noisy labels. |
BO HAN et. al. |
| 2018 | 29 | ProxylessNAS: Direct Neural Architecture Search On Target Task And Hardware IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present \emph{ProxylessNAS} that can \emph{directly} learn the architectures for large-scale target tasks and target hardware platforms. |
Han Cai; Ligeng Zhu; Song Han; |
| 2018 | 30 | Representation Learning On Graphs With Jumping Knowledge Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We analyze some important properties of these models, and propose a strategy to overcome those. |
KEYULU XU et. al. |
| 2017 | 1 | Decoupled Weight Decay Regularization IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: L$_2$ regularization and weight decay regularization are equivalent for standard stochastic gradient descent (when rescaled by the learning rate), but as we demonstrate this is \emph{not} the case for adaptive gradient algorithms, such as Adam. |
Ilya Loshchilov; Frank Hutter; |
| 2017 | 2 | Proximal Policy Optimization Algorithms IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a surrogate objective function using stochastic gradient ascent. |
John Schulman; Filip Wolski; Prafulla Dhariwal; Alec Radford; Oleg Klimov; |
| 2017 | 3 | GANs Trained By A Two Time-Scale Update Rule Converge To A Local Nash Equilibrium IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions. |
Martin Heusel; Hubert Ramsauer; Thomas Unterthiner; Bernhard Nessler; Sepp Hochreiter; |
| 2017 | 4 | Model-Agnostic Meta-Learning For Fast Adaptation Of Deep Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning. |
Chelsea Finn; Pieter Abbeel; Sergey Levine; |
| 2017 | 5 | Improved Training Of Wasserstein GANs IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. |
Ishaan Gulrajani; Faruk Ahmed; Martin Arjovsky; Vincent Dumoulin; Aaron Courville; |
| 2017 | 6 | Mixup: Beyond Empirical Risk Minimization IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose mixup, a simple learning principle to alleviate these issues. |
Hongyi Zhang; Moustapha Cisse; Yann N. Dauphin; David Lopez-Paz; |
| 2017 | 7 | Fashion-MNIST: A Novel Image Dataset For Benchmarking Machine Learning Algorithms IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Fashion-MNIST, a new dataset comprising of 28×28 grayscale images of 70,000 fashion products from 10 categories, with 7,000 images per category. |
Han Xiao; Kashif Rasul; Roland Vollgraf; |
| 2017 | 8 | Prototypical Networks For Few-shot Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose prototypical networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each new class. |
Jake Snell; Kevin Swersky; Richard S. Zemel; |
| 2017 | 9 | Neural Message Passing For Quantum Chemistry IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we reformulate existing models into a single common framework we call Message Passing Neural Networks (MPNNs) and explore additional novel variations within this framework. |
Justin Gilmer; Samuel S. Schoenholz; Patrick F. Riley; Oriol Vinyals; George E. Dahl; |
| 2017 | 10 | Axiomatic Attribution For Deep Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. |
Mukund Sundararajan; Ankur Taly; Qiqi Yan; |
| 2017 | 11 | On Calibration Of Modern Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We discover that modern neural networks, unlike those from a decade ago, are poorly calibrated. |
Chuan Guo; Geoff Pleiss; Yu Sun; Kilian Q. Weinberger; |
| 2017 | 12 | CARLA: An Open Urban Driving Simulator IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce CARLA, an open-source simulator for autonomous driving research. |
Alexey Dosovitskiy; German Ros; Felipe Codevilla; Antonio Lopez; Vladlen Koltun; |
| 2017 | 13 | Multi-Agent Actor-Critic For Mixed Cooperative-Competitive Environments IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We explore deep reinforcement learning methods for multi-agent domains. |
RYAN LOWE et. al. |
| 2017 | 14 | Neural Discrete Representation Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a simple yet powerful generative model that learns such discrete representations. |
Aaron van den Oord; Oriol Vinyals; Koray Kavukcuoglu; |
| 2017 | 15 | Quantization And Training Of Neural Networks For Efficient Integer-Arithmetic-Only Inference IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficiently than floating point inference on commonly available integer-only hardware. |
BENOIT JACOB et. al. |
| 2017 | 16 | An Overview Of Multi-Task Learning In Deep Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This article aims to give a general overview of MTL, particularly in deep neural networks. |
Sebastian Ruder; |
| 2017 | 17 | Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address these challenges, we propose to model the traffic flow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for traffic forecasting that incorporates both spatial and temporal dependency in the traffic flow. |
Yaguang Li; Rose Yu; Cyrus Shahabi; Yan Liu; |
| 2017 | 18 | Multimodal Machine Learning: A Survey And Taxonomy IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Instead of focusing on specific multimodal applications, this paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy. |
Tadas Baltrušaitis; Chaitanya Ahuja; Louis-Philippe Morency; |
| 2017 | 19 | Boosting Adversarial Attacks With Momentum IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address this issue, we propose a broad class of momentum-based iterative algorithms to boost adversarial attacks. |
YINPENG DONG et. al. |
| 2017 | 20 | Gradient Episodic Memory for Continual Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. |
David Lopez-Paz; Marc’Aurelio Ranzato; |
| 2017 | 21 | Continual Learning Through Synaptic Intelligence IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. |
Friedemann Zenke; Ben Poole; Surya Ganguli; |
| 2017 | 22 | Convolutional 2D Knowledge Graph Embeddings IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets. |
Tim Dettmers; Pasquale Minervini; Pontus Stenetorp; Sebastian Riedel; |
| 2017 | 23 | Deep Sets IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study the problem of designing models for machine learning tasks defined on \emph{sets}. |
MANZIL ZAHEER et. al. |
| 2017 | 24 | Curiosity-driven Exploration By Self-supervised Prediction IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We formulate curiosity as the error in an agent’s ability to predict the consequence of its own actions in a visual feature space learned by a self-supervised inverse dynamics model. |
Deepak Pathak; Pulkit Agrawal; Alexei A. Efros; Trevor Darrell; |
| 2017 | 25 | One Pixel Attack For Fooling Deep Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. |
Jiawei Su; Danilo Vasconcellos Vargas; Sakurai Kouichi; |
| 2017 | 26 | Hindsight Experience Replay IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a novel technique called Hindsight Experience Replay which allows sample-efficient learning from rewards which are sparse and binary and therefore avoid the need for complicated reward engineering. |
MARCIN ANDRYCHOWICZ et. al. |
| 2017 | 27 | Methods For Interpreting And Understanding Deep Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: It introduces some recently proposed techniques of interpretation, along with theory, tricks and recommendations, to make most efficient use of these techniques on real data. |
Grégoire Montavon; Wojciech Samek; Klaus-Robert Müller; |
| 2017 | 28 | Conditional Adversarial Domain Adaptation IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present conditional adversarial domain adaptation, a principled framework that conditions the adversarial adaptation models on discriminative information conveyed in the classifier predictions. |
Mingsheng Long; Zhangjie Cao; Jianmin Wang; Michael I. Jordan; |
| 2017 | 29 | SmoothGrad: Removing Noise By Adding Noise IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper makes two contributions: it introduces SmoothGrad, a simple method that can help visually sharpen gradient-based sensitivity maps, and it discusses lessons in the visualization of these maps. |
Daniel Smilkov; Nikhil Thorat; Been Kim; Fernanda Viégas; Martin Wattenberg; |
| 2017 | 30 | Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we address these challenges and finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters. |
NOAM SHAZEER et. al. |
| 2016 | 1 | XGBoost: A Scalable Tree Boosting System IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. |
Tianqi Chen; Carlos Guestrin; |
| 2016 | 2 | Semi-Supervised Classification With Graph Convolutional Networks IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. |
Thomas N. Kipf; Max Welling; |
| 2016 | 3 | Communication-Efficient Learning of Deep Networks from Decentralized Data IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. |
H. Brendan McMahan; Eider Moore; Daniel Ramage; Seth Hampson; Blaise Agüera y Arcas; |
| 2016 | 4 | Why Should I Trust You?: Explaining The Predictions Of Any Classifier IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally around the prediction. |
Marco Tulio Ribeiro; Sameer Singh; Carlos Guestrin; |
| 2016 | 5 | Asynchronous Methods For Deep Reinforcement Learning IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. |
VOLODYMYR MNIH et. al. |
| 2016 | 6 | Convolutional Neural Networks On Graphs With Fast Localized Spectral Filtering IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words’ embedding, represented by graphs. |
Michaël Defferrard; Xavier Bresson; Pierre Vandergheynst; |
| 2016 | 7 | Matching Networks For One Shot Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we employ ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories. |
Oriol Vinyals; Charles Blundell; Timothy Lillicrap; Koray Kavukcuoglu; Daan Wierstra; |
| 2016 | 8 | Overcoming Catastrophic Forgetting In Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We demonstrate our approach is scalable and effective by solving a set of classification tasks based on the MNIST hand written digit dataset and by learning several Atari 2600 games sequentially. |
JAMES KIRKPATRICK et. al. |
| 2016 | 9 | An Overview Of Gradient Descent Optimization Algorithms IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In the course of this overview, we look at different variants of gradient descent, summarize challenges, introduce the most common optimization algorithms, review architectures in a parallel and distributed setting, and investigate additional strategies for optimizing gradient descent. |
Sebastian Ruder; |
| 2016 | 10 | Neural Architecture Search With Reinforcement Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. |
Barret Zoph; Quoc V. Le; |
| 2016 | 11 | OpenAI Gym IF:9 Summary Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Abstract: OpenAI Gym is a toolkit for reinforcement learning research. It includes a growing collection of benchmark problems that expose a common interface, and a website where people can … |
GREG BROCKMAN et. al. |
| 2016 | 12 | Understanding Deep Learning Requires Rethinking Generalization IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. |
Chiyuan Zhang; Samy Bengio; Moritz Hardt; Benjamin Recht; Oriol Vinyals; |
| 2016 | 13 | Federated Learning: Strategies For Improving Communication Efficiency IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose two ways to reduce the uplink communication costs: structured updates, where we directly learn an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, where we learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling before sending it to the server. |
JAKUB KONEČNÝ et. al. |
| 2016 | 14 | InfoGAN: Interpretable Representation Learning By Information Maximizing Generative Adversarial Nets IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner. |
XI CHEN et. al. |
| 2016 | 15 | Gaussian Error Linear Units (GELUs) IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network activation function. |
Dan Hendrycks; Kevin Gimpel; |
| 2016 | 16 | Equality Of Opportunity In Supervised Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. |
Moritz Hardt; Eric Price; Nathan Srebro; |
| 2016 | 17 | Wide & Deep Learning For Recommender Systems IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present Wide & Deep learning—jointly trained wide linear models and deep neural networks—to combine the benefits of memorization and generalization for recommender systems. |
HENG-TZE CHENG et. al. |
| 2016 | 18 | Generative Adversarial Imitation Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning. |
Jonathan Ho; Stefano Ermon; |
| 2016 | 19 | On Large-Batch Training For Deep Learning: Generalization Gap And Sharp Minima IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We investigate the cause for this generalization drop in the large-batch regime and present numerical evidence that supports the view that large-batch methods tend to converge to sharp minimizers of the training and testing functions – and as is well known, sharp minima lead to poorer generalization. |
Nitish Shirish Keskar; Dheevatsa Mudigere; Jorge Nocedal; Mikhail Smelyanskiy; Ping Tak Peter Tang; |
| 2016 | 20 | Automatic Chemical Design Using A Data-driven Continuous Representation Of Molecules IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We report a method to convert discrete representations of molecules to and from a multidimensional continuous representation. |
RAFAEL GÓMEZ-BOMBARELLI et. al. |
| 2016 | 21 | End-to-end Sequence Labeling Via Bi-directional LSTM-CNNs-CRF IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. |
Xuezhe Ma; Eduard Hovy; |
| 2016 | 22 | The Concrete Distribution: A Continuous Relaxation Of Discrete Random Variables IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we introduce Concrete random variables—continuous relaxations of discrete random variables. |
Chris J. Maddison; Andriy Mnih; Yee Whye Teh; |
| 2016 | 23 | SeqGAN: Sequence Generative Adversarial Nets With Policy Gradient IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a sequence generation framework, called SeqGAN, to solve the problems. |
Lantao Yu; Weinan Zhang; Jun Wang; Yong Yu; |
| 2016 | 24 | Deep Transfer Learning With Joint Adaptation Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. |
Mingsheng Long; Han Zhu; Jianmin Wang; Michael I. Jordan; |
| 2016 | 25 | Progressive Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy. |
ANDREI A. RUSU et. al. |
| 2016 | 26 | Deep Networks With Stochastic Depth IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address these problems, we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time. |
Gao Huang; Yu Sun; Zhuang Liu; Daniel Sedra; Kilian Weinberger; |
| 2016 | 27 | Binarized Neural Networks: Training Deep Neural Networks With Weights And Activations Constrained To +1 Or -1 IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a method to train Binarized Neural Networks (BNNs) – neural networks with binary weights and activations at run-time. |
Matthieu Courbariaux; Itay Hubara; Daniel Soudry; Ran El-Yaniv; Yoshua Bengio; |
| 2016 | 28 | Hyperband: A Novel Bandit-Based Approach To Hyperparameter Optimization IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a novel algorithm, Hyperband, for this framework and analyze its theoretical properties, providing several desirable guarantees. |
Lisha Li; Kevin Jamieson; Giulia DeSalvo; Afshin Rostamizadeh; Ameet Talwalkar; |
| 2016 | 29 | Learning Hand-Eye Coordination For Robotic Grasping With Deep Learning And Large-Scale Data Collection IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images. |
Sergey Levine; Peter Pastor; Alex Krizhevsky; Deirdre Quillen; |
| 2016 | 30 | Revisiting Semi-Supervised Learning With Graph Embeddings IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a semi-supervised learning framework based on graph embeddings. |
Zhilin Yang; William W. Cohen; Ruslan Salakhutdinov; |
| 2015 | 1 | Batch Normalization: Accelerating Deep Network Training By Reducing Internal Covariate Shift IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. |
Sergey Ioffe; Christian Szegedy; |
| 2015 | 2 | Unsupervised Representation Learning With Deep Convolutional Generative Adversarial Networks IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. |
Alec Radford; Luke Metz; Soumith Chintala; |
| 2015 | 3 | Continuous Control With Deep Reinforcement Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. |
TIMOTHY P. LILLICRAP et. al. |
| 2015 | 4 | Show, Attend And Tell: Neural Image Caption Generation With Visual Attention IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. |
KELVIN XU et. al. |
| 2015 | 5 | Deep Reinforcement Learning With Double Q-learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we answer all these questions affirmatively. |
Hado van Hasselt; Arthur Guez; David Silver; |
| 2015 | 6 | Trust Region Policy Optimization IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement. |
John Schulman; Sergey Levine; Philipp Moritz; Michael I. Jordan; Pieter Abbeel; |
| 2015 | 7 | Character-level Convolutional Networks For Text Classification IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. |
Xiang Zhang; Junbo Zhao; Yann LeCun; |
| 2015 | 8 | Fast And Accurate Deep Network Learning By Exponential Linear Units (ELUs) IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce the exponential linear unit (ELU) which speeds up learning in deep neural networks and leads to higher classification accuracies. |
Djork-Arné Clevert; Thomas Unterthiner; Sepp Hochreiter; |
| 2015 | 9 | LINE: Large-scale Information Network Embedding IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a novel network embedding method called the LINE, which is suitable for arbitrary types of information networks: undirected, directed, and/or weighted. |
JIAN TANG et. al. |
| 2015 | 10 | Deep Unsupervised Learning Using Nonequilibrium Thermodynamics IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we develop an approach that simultaneously achieves both flexibility and tractability. |
Jascha Sohl-Dickstein; Eric A. Weiss; Niru Maheswaranathan; Surya Ganguli; |
| 2015 | 11 | Learning Transferable Features With Deep Adaptation Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a new Deep Adaptation Network (DAN) architecture, which generalizes deep convolutional neural network to the domain adaptation scenario. |
Mingsheng Long; Yue Cao; Jianmin Wang; Michael I. Jordan; |
| 2015 | 12 | DeepFool: A Simple And Accurate Method To Fool Deep Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we fill this gap and propose the DeepFool algorithm to efficiently compute perturbations that fool deep networks, and thus reliably quantify the robustness of these classifiers. |
Seyed-Mohsen Moosavi-Dezfooli; Alhussein Fawzi; Pascal Frossard; |
| 2015 | 13 | Prioritized Experience Replay IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently. |
Tom Schaul; John Quan; Ioannis Antonoglou; David Silver; |
| 2015 | 14 | Dueling Network Architectures For Deep Reinforcement Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a new neural network architecture for model-free reinforcement learning. |
ZIYU WANG et. al. |
| 2015 | 15 | End-to-End Training Of Deep Visuomotor Policies IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? |
Sergey Levine; Chelsea Finn; Trevor Darrell; Pieter Abbeel; |
| 2015 | 16 | Convolutional Networks On Graphs For Learning Molecular Fingerprints IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a convolutional neural network that operates directly on graphs. |
DAVID DUVENAUD et. al. |
| 2015 | 17 | Gated Graph Sequence Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we study feature learning techniques for graph-structured inputs. |
Yujia Li; Daniel Tarlow; Marc Brockschmidt; Richard Zemel; |
| 2015 | 18 | High-Dimensional Continuous Control Using Generalized Advantage Estimation IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We address the first challenge by using value functions to substantially reduce the variance of policy gradient estimates at the cost of some bias, with an exponentially-weighted estimator of the advantage function that is analogous to TD(lambda). |
John Schulman; Philipp Moritz; Sergey Levine; Michael Jordan; Pieter Abbeel; |
| 2015 | 19 | BinaryConnect: Training Deep Neural Networks With Binary Weights During Propagations IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce BinaryConnect, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of the stored weights in which gradients are accumulated. |
Matthieu Courbariaux; Yoshua Bengio; Jean-Pierre David; |
| 2015 | 20 | Empirical Evaluation Of Rectified Activations In Convolutional Network IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network: standard rectified linear unit (ReLU), leaky rectified linear unit (Leaky ReLU), parametric rectified linear unit (PReLU) and a new randomized leaky rectified linear units (RReLU). |
Bing Xu; Naiyan Wang; Tianqi Chen; Mu Li; |
| 2015 | 21 | Unsupervised Deep Embedding For Clustering Analysis IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Deep Embedded Clustering (DEC), a method that simultaneously learns feature representations and cluster assignments using deep neural networks. |
Junyuan Xie; Ross Girshick; Ali Farhadi; |
| 2015 | 22 | Unsupervised Learning Of Video Representations Using LSTMs IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We use multilayer Long Short Term Memory (LSTM) networks to learn representations of video sequences. |
Nitish Srivastava; Elman Mansimov; Ruslan Salakhutdinov; |
| 2015 | 23 | Generating Sentences From A Continuous Space IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce and study an RNN-based variational autoencoder generative model that incorporates distributed latent representations of entire sentences. |
SAMUEL R. BOWMAN et. al. |
| 2015 | 24 | A Critical Review Of Recurrent Neural Networks For Sequence Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this survey, we review and synthesize the research that over the past three decades first yielded and then made practical these powerful learning models. |
Zachary C. Lipton; John Berkowitz; Charles Elkan; |
| 2015 | 25 | Adversarial Autoencoders IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose the adversarial autoencoder (AAE), which is a probabilistic autoencoder that uses the recently proposed generative adversarial networks (GAN) to perform variational inference by matching the aggregated posterior of the hidden code vector of the autoencoder with an arbitrary prior distribution. |
Alireza Makhzani; Jonathon Shlens; Navdeep Jaitly; Ian Goodfellow; Brendan Frey; |
| 2015 | 26 | Deep Learning With Limited Numerical Precision IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study the effect of limited precision data representation and computation on neural network training. |
Suyog Gupta; Ankur Agrawal; Kailash Gopalakrishnan; Pritish Narayanan; |
| 2015 | 27 | Scheduled Sampling For Sequence Prediction With Recurrent Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a curriculum learning strategy to gently change the training process from a fully guided scheme using the true previous token, towards a less guided scheme which mostly uses the generated token instead. |
Samy Bengio; Oriol Vinyals; Navdeep Jaitly; Noam Shazeer; |
| 2015 | 28 | Autoencoding Beyond Pixels Using A Learned Similarity Metric IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an autoencoder that leverages learned representations to better measure similarities in data space. |
Anders Boesen Lindbo Larsen; Søren Kaae Sønderby; Hugo Larochelle; Ole Winther; |
| 2015 | 29 | Deep Multi-scale Video Prediction Beyond Mean Square Error IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we train a convolutional network to generate future frames given an input sequence. |
Michael Mathieu; Camille Couprie; Yann LeCun; |
| 2015 | 30 | Stacked Attention Networks For Image Question Answering IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents stacked attention networks (SANs) that learn to answer natural language questions from images. |
Zichao Yang; Xiaodong He; Jianfeng Gao; Li Deng; Alex Smola; |
| 2014 | 1 | Adam: A Method For Stochastic Optimization IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. |
Diederik P. Kingma; Jimmy Ba; |
| 2014 | 2 | Conditional Generative Adversarial Nets IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we introduce the conditional version of generative adversarial nets, which can be constructed by simply feeding the data, y, we wish to condition on to both the generator and discriminator. |
Mehdi Mirza; Simon Osindero; |
| 2014 | 3 | How Transferable Are Features In Deep Neural Networks? IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. |
Jason Yosinski; Jeff Clune; Yoshua Bengio; Hod Lipson; |
| 2014 | 4 | FitNets: Hints For Thin Deep Nets IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we extend this idea to allow the training of a student that is deeper and thinner than the teacher, using not only the outputs but also the intermediate representations learned by the teacher as hints to improve the training process and final performance of the student. |
ADRIANA ROMERO et. al. |
| 2014 | 5 | Semi-Supervised Learning With Deep Generative Models IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. |
Diederik P. Kingma; Danilo J. Rezende; Shakir Mohamed; Max Welling; |
| 2014 | 6 | A Tutorial On Principal Component Analysis IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The goal of this paper is to dispel the magic behind this black box. |
Jonathon Shlens; |
| 2014 | 7 | NICE: Non-linear Independent Components Estimation IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a deep learning framework for modeling complex high-dimensional densities called Non-linear Independent Component Estimation (NICE). |
Laurent Dinh; David Krueger; Yoshua Bengio; |
| 2014 | 8 | Deep Metric Learning Using Triplet Network IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we propose the triplet network model, which aims to learn useful representations by distance comparisons. |
Elad Hoffer; Nir Ailon; |
| 2014 | 9 | SAGA: A Fast Incremental Gradient Method With Support For Non-Strongly Convex Composite Objectives IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. |
Aaron Defazio; Francis Bach; Simon Lacoste-Julien; |
| 2014 | 10 | Collaborative Deep Learning For Recommender Systems IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address this problem, we generalize recent advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. |
Hao Wang; Naiyan Wang; Dit-Yan Yeung; |
| 2014 | 11 | Unifying Visual-Semantic Embeddings With Multimodal Neural Language Models IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by recent advances in multimodal learning and machine translation, we introduce an encoder-decoder pipeline that learns (a): a multimodal joint embedding space with images and text and (b): a novel language model for decoding distributed representations from our space. |
Ryan Kiros; Ruslan Salakhutdinov; Richard S. Zemel; |
| 2014 | 12 | OpenML: Networked Science In Machine Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce OpenML, a place for machine learning researchers to share and organize data in fine detail, so that they can work more effectively, be more visible, and collaborate with others to tackle harder problems. |
Joaquin Vanschoren; Jan N. van Rijn; Bernd Bischl; Luis Torgo; |
| 2014 | 13 | The Loss Surfaces Of Multilayer Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study the connection between the highly non-convex loss function of a simple model of the fully-connected feed-forward neural network and the Hamiltonian of the spherical spin-glass model under the assumptions of: i) variable independence, ii) redundancy in network parametrization, and iii) uniformity. |
Anna Choromanska; Mikael Henaff; Michael Mathieu; Gérard Ben Arous; Yann LeCun; |
| 2014 | 14 | Multiple Object Recognition With Visual Attention IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an attention-based model for recognizing multiple objects in images. |
Jimmy Ba; Volodymyr Mnih; Koray Kavukcuoglu; |
| 2014 | 15 | Neural Variational Inference And Learning In Belief Networks IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a fast non-iterative approximate inference method that uses a feedforward network to implement efficient exact sampling from the variational posterior. |
Andriy Mnih; Karol Gregor; |
| 2014 | 16 | Convolutional Neural Networks Over Tree Structures For Programming Language Processing IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a novel tree-based convolutional neural network (TBCNN) for programming language processing, in which a convolution kernel is designed over programs’ abstract syntax trees to capture structural information. |
Lili Mou; Ge Li; Lu Zhang; Tao Wang; Zhi Jin; |
| 2014 | 17 | Breaking The Curse Of Dimensionality With Convex Neural Networks IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: By letting the number of hidden units grow unbounded and using classical non-Euclidean regularization tools on the output weights, we provide a detailed theoretical analysis of their generalization performance, with a study of both the approximation and the estimation errors. |
Francis Bach; |
| 2014 | 18 | Deep Learning With Elastic Averaging SGD IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose synchronous and asynchronous variants of the new algorithm. |
Sixin Zhang; Anna Choromanska; Yann LeCun; |
| 2014 | 19 | New Insights And Perspectives On The Natural Gradient Method IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of approximate 2nd-order optimization method, where the Fisher information matrix can be viewed as an approximation of the Hessian. |
James Martens; |
| 2014 | 20 | Taming The Monster: A Fast And Simple Algorithm For Contextual Bandits IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a new algorithm for the contextual bandit learning problem, where the learner repeatedly takes one of $K$ actions in response to the observed context, and observes the reward only for that chosen action. |
ALEKH AGARWAL et. al. |
| 2014 | 21 | Label Distribution Learning IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper proposes six working LDL algorithms in three ways: problem transformation, algorithm adaptation, and specialized algorithm design. |
Xin Geng; |
| 2014 | 22 | Video (language) Modeling: A Baseline For Generative Models Of Natural Videos IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a strong baseline model for unsupervised feature learning using video data. |
MARCAURELIO RANZATO et. al. |
| 2014 | 23 | On The Computational Efficiency Of Training Neural Networks IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we revisit the computational complexity of training neural networks from a modern perspective. |
Roi Livni; Shai Shalev-Shwartz; Ohad Shamir; |
| 2014 | 24 | Guaranteed Matrix Completion Via Non-convex Factorization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we establish a theoretical guarantee for the factorization formulation to correctly recover the underlying low-rank matrix. |
Ruoyu Sun; Zhi-Quan Luo; |
| 2014 | 25 | Fastfood: Approximate Kernel Expansions In Loglinear Time IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we overcome this difficulty by proposing Fastfood, an approximation that accelerates such computation significantly. |
Quoc Viet Le; Tamas Sarlos; Alexander Johannes Smola; |
| 2014 | 26 | Deep Unfolding: Model-Based Inspiration Of Novel Deep Architectures IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This work aims to obtain the advantages of both approaches. |
John R. Hershey; Jonathan Le Roux; Felix Weninger; |
| 2014 | 27 | An Information-Theoretic Analysis Of Thompson Sampling IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We provide an information-theoretic analysis of Thompson sampling that applies across a broad range of online optimization problems in which a decision-maker must learn from partial feedback. |
Daniel Russo; Benjamin Van Roy; |
| 2014 | 28 | Lazier Than Lazy Greedy IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we develop the first linear-time algorithm for maximizing a general monotone submodular function subject to a cardinality constraint. |
Baharan Mirzasoleiman; Ashwinkumar Badanidiyuru; Amin Karbasi; Jan Vondrak; Andreas Krause; |
| 2014 | 29 | Differentially Private Empirical Risk Minimization: Efficient Algorithms And Tight Error Bounds IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we initiate a systematic investigation of differentially private algorithms for convex empirical risk minimization. |
Raef Bassily; Adam Smith; Abhradeep Thakurta; |
| 2013 | 1 | Playing Atari With Deep Reinforcement Learning IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. |
VOLODYMYR MNIH et. al. |
| 2013 | 2 | Spectral Networks And Locally Connected Networks On Graphs IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we consider possible generalizations of CNNs to signals defined on more general domains without the action of a translation group. |
Joan Bruna; Wojciech Zaremba; Arthur Szlam; Yann LeCun; |
| 2013 | 3 | Estimating Continuous Distributions In Bayesian Classifiers IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we abandon the normality assumption and instead use statistical methods for nonparametric density estimation. |
George H. John; Pat Langley; |
| 2013 | 4 | Probabilistic Latent Semantic Analysis IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In order to avoid overfitting, we propose a widely applicable generalization of maximum likelihood model fitting by tempered EM. |
Thomas Hofmann; |
| 2013 | 5 | Estimating Or Propagating Gradients Through Stochastic Neurons For Conditional Computation IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We examine this question, existing approaches, and compare four families of solutions, applicable in different settings. |
Yoshua Bengio; Nicholas Léonard; Aaron Courville; |
| 2013 | 6 | Do Deep Nets Really Need To Be Deep? IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this extended abstract, we show that shallow feed-forward networks can learn the complex functions previously learned by deep nets and achieve accuracies previously only achievable with deep models. |
Lei Jimmy Ba; Rich Caruana; |
| 2013 | 7 | Deep Learning For Detecting Robotic Grasps IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we apply a deep learning approach to solve this problem, which avoids time-consuming hand-design of features. |
Ian Lenz; Honglak Lee; Ashutosh Saxena; |
| 2013 | 8 | Rao-Blackwellised Particle Filtering For Dynamic Bayesian Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we show how we can exploit the structure of the DBN to increase the efficiency of particle filtering, using a technique known as Rao-Blackwellisation. |
Arnaud Doucet; Nando de Freitas; Kevin Murphy; Stuart Russell; |
| 2013 | 9 | Predicting Parameters In Deep Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We demonstrate that there is significant redundancy in the parameterization of several deep learning models. |
Misha Denil; Babak Shakibi; Laurent Dinh; Marc’Aurelio Ranzato; Nando de Freitas; |
| 2013 | 10 | Gaussian Processes For Big Data IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce stochastic variational inference for Gaussian process models. |
James Hensman; Nicolo Fusi; Neil D. Lawrence; |
| 2013 | 11 | A Survey On Multi-view Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In recent years, a great many methods of learning from multi-view data by considering the diversity of different views have been proposed. |
Chang Xu; Dacheng Tao; Chao Xu; |
| 2013 | 12 | Stochastic Pooling For Regularization Of Deep Convolutional Neural Networks IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a simple and effective method for regularizing large convolutional neural networks. |
Matthew D. Zeiler; Rob Fergus; |
| 2013 | 13 | Zero-Shot Learning By Convex Combination Of Semantic Embeddings IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Several recent publications have proposed methods for mapping images into continuous semantic embedding spaces. |
MOHAMMAD NOROUZI et. al. |
| 2013 | 14 | Deep Learning Using Linear Support Vector Machines IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we demonstrate a small but consistent advantage of replacing the softmax layer with a linear support vector machine. |
Yichuan Tang; |
| 2013 | 15 | Induction Of Selective Bayesian Classifiers IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we examine previous work on the naive Bayesian classifier and review its limitations, which include a sensitivity to correlated features. |
Pat Langley; Stephanie Sage; |
| 2013 | 16 | The Bayesian Structural EM Algorithm IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, I extend Structural EM to deal directly with Bayesian model selection. |
Nir Friedman; |
| 2013 | 17 | A Survey On Metric Learning For Feature Vectors And Structured Data IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. |
Aurélien Bellet; Amaury Habrard; Marc Sebban; |
| 2013 | 18 | A Semantic Matching Energy Function For Learning With Multi-relational Data IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we present a new neural network architecture designed to embed multi-relational graphs into a flexible continuous vector space in which the original data is kept and enhanced. |
Xavier Glorot; Antoine Bordes; Jason Weston; Yoshua Bengio; |
| 2013 | 19 | Inferring Parameters And Structure Of Latent Variable Models By Variational Bayes IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper I present the Variational Bayes framework, which provides a solution to these problems. |
Hagai Attias; |
| 2013 | 20 | Deep Learning Of Representations: Looking Forward IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Deep learning research aims at discovering learning algorithms that discover multiple levels of distributed representations, with higher levels representing more abstract concepts. |
Yoshua Bengio; |
| 2013 | 21 | Learning To Optimize Via Posterior Sampling IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our second theoretical contribution is a Bayesian regret bound for posterior sampling that applies broadly and can be specialized to many model classes. |
Daniel Russo; Benjamin Van Roy; |
| 2013 | 22 | Learning Bayesian Network Structure From Massive Datasets: The Sparse Candidate Algorithm IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce an algorithm that achieves faster learning by restricting the search space. |
Nir Friedman; Iftach Nachman; Dana Pe’er; |
| 2013 | 23 | One-Class Classification: Taxonomy Of Study And Review Of Techniques IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we present a unified view of the general problem of OCC by presenting a taxonomy of study for OCC problems, which is based on the availability of training data, algorithms used and the application domains applied. |
Shehroz S. Khan; Michael G. Madden; |
| 2013 | 24 | An Introductory Study On Time Series Modeling And Forecasting IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The aim of this dissertation work is to present a concise description of some popular time series forecasting models used in practice, with their salient features. |
Ratnadip Adhikari; R. K. Agrawal; |
| 2013 | 25 | Class Imbalance Problem In Data Mining Review IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper systematic study of each approach is define which gives the right direction for research in class imbalance problem. |
Rushi Longadge; Snehalata Dongre; |
| 2013 | 26 | Generalized Denoising Auto-Encoders As Generative Models IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose here a different attack on the problem, which deals with all these issues: arbitrary (but noisy enough) corruption, arbitrary reconstruction loss (seen as a log-likelihood), handling both discrete and continuous-valued variables, and removing the bias due to non-infinitesimal corruption noise (or non-infinitesimal contractive penalty). |
Yoshua Bengio; Li Yao; Guillaume Alain; Pascal Vincent; |
| 2013 | 27 | A Time Series Forest For Classification And Feature Extraction IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a tree ensemble method, referred to as time series forest (TSF), for time series classification. |
Houtao Deng; George Runger; Eugene Tuv; Martyanov Vladimir; |
| 2013 | 28 | Learning By Transduction IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We describe a method for predicting a classification of an object given classifications of the objects in the training set, assuming that the pairs object/classification are generated by an i.i.d. process from a continuous probability distribution. |
Alex Gammerman; Volodya Vovk; Vladimir Vapnik; |
| 2013 | 29 | Large-scale Multi-label Learning With Missing Labels IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we directly address both these problems by studying the multi-label problem in a generic empirical risk minimization (ERM) framework. |
Hsiang-Fu Yu; Prateek Jain; Purushottam Kar; Inderjit S. Dhillon; |
| 2013 | 30 | Comparing Bayesian Network Classifiers IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we empirically evaluate algorithms for learning four types of Bayesian network (BN) classifiers – Naive-Bayes, tree augmented Naive-Bayes, BN augmented Naive-Bayes and general BNs, where the latter two are learned using two variants of a conditional-independence (CI) based BN-learning algorithm. |
Jie Cheng; Russell Greiner; |
| 2012 | 1 | Scikit-learn: Machine Learning In Python IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. |
FABIAN PEDREGOSA et. al. |
| 2012 | 2 | Representation Learning: A Review And New Perspectives IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. |
Yoshua Bengio; Aaron Courville; Pascal Vincent; |
| 2012 | 3 | ADADELTA: An Adaptive Learning Rate Method IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a novel per-dimension learning rate method for gradient descent called ADADELTA. |
Matthew D. Zeiler; |
| 2012 | 4 | On The Difficulty Of Training Recurrent Neural Networks IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we attempt to improve the understanding of the underlying issues by exploring these problems from an analytical, a geometric and a dynamical systems perspective. |
Razvan Pascanu; Tomas Mikolov; Yoshua Bengio; |
| 2012 | 5 | Practical Recommendations For Gradient-based Training Of Deep Architectures IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This chapter is meant as a practical guide with recommendations for some of the most commonly used hyper-parameters, in particular in the context of learning algorithms based on back-propagated gradient and gradient-based optimization. |
Yoshua Bengio; |
| 2012 | 6 | Auto-WEKA: Combined Selection And Hyperparameter Optimization Of Classification Algorithms IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We consider the problem of simultaneously selecting a learning algorithm and setting its hyperparameters, going beyond previous work that addresses these issues in isolation. |
Chris Thornton; Frank Hutter; Holger H. Hoos; Kevin Leyton-Brown; |
| 2012 | 7 | Poisoning Attacks Against Support Vector Machines IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We investigate a family of poisoning attacks against Support Vector Machines (SVM). |
Battista Biggio; Blaine Nelson; Pavel Laskov; |
| 2012 | 8 | Tensor Decompositions For Learning Latent Variable Models IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). |
Anima Anandkumar; Rong Ge; Daniel Hsu; Sham M. Kakade; Matus Telgarsky; |
| 2012 | 9 | A Comparative Study Of Efficient Initialization Methods For The K-Means Clustering Algorithm IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Numerous initialization methods have been proposed to address this problem. |
M. Emre Celebi; Hassan A. Kingravi; Patricio A. Vela; |
| 2012 | 10 | Thompson Sampling For Contextual Bandits With Linear Payoffs IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we design and analyze a generalization of Thompson Sampling algorithm for the stochastic contextual multi-armed bandit problem with linear payoff functions, when the contexts are provided by an adaptive adversary. |
Shipra Agrawal; Navin Goyal; |
| 2012 | 11 | An MDP-based Recommender System IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We describe our predictive model in detail and evaluate its performance on real data. |
Guy Shani; Ronen I. Brafman; David Heckerman; |
| 2012 | 12 | Discriminative Probabilistic Models For Relational Data IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we present an alternative framework that builds on (conditional) Markov networks and addresses two limitations of the previous approach. |
Ben Taskar; Pieter Abbeel; Daphne Koller; |
| 2012 | 13 | Marginalized Denoising Autoencoders For Domain Adaptation IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose marginalized SDA (mSDA) that addresses two crucial limitations of SDAs: high computational cost and lack of scalability to high-dimensional features. |
Minmin Chen; Zhixiang Xu; Kilian Weinberger; Fei Sha; |
| 2012 | 14 | Large-Sample Learning Of Bayesian Networks Is NP-Hard IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we provide new complexity results for algorithms that learn discrete-variable Bayesian networks from data. |
David Maxwell Chickering; Christopher Meek; David Heckerman; |
| 2012 | 15 | Sum-Product Networks: A New Deep Architecture IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that if an SPN is complete and consistent it represents the partition function and all marginals of some graphical model, and give semantics to its nodes. |
Hoifung Poon; Pedro Domingos; |
| 2012 | 16 | Generalized Fisher Score For Feature Selection IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a generalized Fisher score to jointly select features. |
Quanquan Gu; Zhenhui Li; Jiawei Han; |
| 2012 | 17 | A Widely Applicable Bayesian Information Criterion IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In the present paper, we define a widely applicable Bayesian information criterion (WBIC) by the average log likelihood function over the posterior distribution with the inverse temperature $1/\log n$, where $n$ is the number of training samples. |
Sumio Watanabe; |
| 2012 | 18 | Modeling Temporal Dependencies In High-Dimensional Sequences: Application To Polyphonic Music Generation And Transcription IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a probabilistic model based on distribution estimators conditioned on a recurrent neural network that is able to discover temporal dependencies in high-dimensional sequences. |
Nicolas Boulanger-Lewandowski; Yoshua Bengio; Pascal Vincent; |
| 2012 | 19 | On Smoothing And Inference For Topic Models IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we highlight the close connections between these approaches. |
Arthur Asuncion; Max Welling; Padhraic Smyth; Yee Whye Teh; |
| 2012 | 20 | Expectation-Propogation For The Generative Aspect Model IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We develop an alternative approach that leads to higher accuracy at comparable cost. |
Thomas P. Minka; John Lafferty; |
| 2012 | 21 | Kernel-based Conditional Independence Test And Application In Causal Discovery IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a Kernel-based Conditional Independence test (KCI-test), by constructing an appropriate test statistic and deriving its asymptotic distribution under the null hypothesis of conditional independence. |
Kun Zhang; Jonas Peters; Dominik Janzing; Bernhard Schoelkopf; |
| 2012 | 22 | Stochastic Gradient Descent For Non-smooth Optimization: Convergence Results And Optimal Averaging Schemes IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate the performance of SGD without such smoothness assumptions, as well as a running average scheme to convert the SGD iterates to a solution with optimal optimization accuracy. |
Ohad Shamir; Tong Zhang; |
| 2012 | 23 | Algorithms For Learning Kernels Based On Centered Alignment IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents new and effective algorithms for learning kernels. |
Corinna Cortes; Mehryar Mohri; Afshin Rostamizadeh; |
| 2012 | 24 | Advances In Optimizing Recurrent Networks IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Experiments reported here evaluate the use of clipping gradients, spanning longer time ranges with leaky integration, advanced momentum techniques, using more powerful output probability models, and encouraging sparser gradients to help symmetry breaking and credit assignment. |
Yoshua Bengio; Nicolas Boulanger-Lewandowski; Razvan Pascanu; |
| 2012 | 25 | Learning Task Grouping And Overlap In Multi-task Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a framework for multi-task learn- ing that enables one to selectively share the information across the tasks. |
Abhishek Kumar; Hal Daume III; |
| 2012 | 26 | Efficiently Inducing Features Of Conditional Random Fields IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a feature induction method for CRFs. |
Andrew McCallum; |
| 2012 | 27 | Parallelizing Exploration-Exploitation Tradeoffs With Gaussian Process Bandit Optimization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We formalize the task as a multi-armed bandit problem, where the unknown payoff function is sampled from a Gaussian process (GP), and instead of a single arm, in each round we pull a batch of several arms in parallel. |
Thomas Desautels; Andreas Krause; Joel Burdick; |
| 2012 | 28 | A New Class Of Upper Bounds On The Log Partition Function IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a new class of upper bounds on the log partition function, based on convex combinations of distributions in the exponential domain, that is applicable to an arbitrary undirected graphical model. |
Martin Wainwright; Tommi S. Jaakkola; Alan Willsky; |
| 2012 | 29 | A Convex Formulation For Learning Task Relationships In Multi-Task Learning IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a regularization formulation for learning the relationships between tasks in multi-task learning. |
Yu Zhang; Dit-Yan Yeung; |
| 2012 | 30 | A Practical Algorithm For Topic Modeling With Provable Guarantees IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we present an algorithm for topic model inference that is both provable and practical. |
SANJEEV ARORA et. al. |
| 2011 | 1 | Natural Language Processing (almost) From Scratch IF:10 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. |
RONAN COLLOBERT et. al. |
| 2011 | 2 | Building High-level Features Using Large Scale Unsupervised Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We consider the problem of building high-level, class-specific feature detectors from only unlabeled data. |
QUOC V. LE et. al. |
| 2011 | 3 | Optimization With Sparsity-Inducing Penalties IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The goal of this paper is to present from a general perspective optimization tools and techniques dedicated to such sparsity-inducing penalties. |
Francis Bach; Rodolphe Jenatton; Julien Mairal; Guillaume Obozinski; |
| 2011 | 4 | Analysis Of Thompson Sampling For The Multi-armed Bandit Problem IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, for the first time, we show that Thompson Sampling algorithm achieves logarithmic expected regret for the multi-armed bandit problem. |
Shipra Agrawal; Navin Goyal; |
| 2011 | 5 | Domain Adaptation For Statistical Classifiers IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. |
H. Daume III; D. Marcu; |
| 2011 | 6 | Making Gradient Descent Optimal For Strongly Convex Stochastic Optimization IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we investigate the optimality of SGD in a stochastic setting. |
Alexander Rakhlin; Ohad Shamir; Karthik Sridharan; |
| 2011 | 7 | Doubly Robust Policy Evaluation And Learning IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we leverage the strength and overcome the weaknesses of the two approaches by applying the doubly robust technique to the problems of policy evaluation and optimization. |
Miroslav Dudik; John Langford; Lihong Li; |
| 2011 | 8 | Convergence Rates Of Inexact Proximal-Gradient Methods For Convex Optimization IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using proximal-gradient methods, where an error is present in the calculation of the gradient of the smooth term or in the proximity operator with respect to the non-smooth term. |
Mark Schmidt; Nicolas Le Roux; Francis Bach; |
| 2011 | 9 | Learning With Submodular Functions: A Convex Optimization Perspective IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this monograph, we present the theory of submodular functions from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems. |
Francis Bach; |
| 2011 | 10 | A Unified Framework For Approximating And Clustering Data IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Given a set $F$ of $n$ positive functions over a ground set $X$, we consider the problem of computing $x^*$ that minimizes the expression $\sum_{f\in F}f(x)$, over $x\in X$. |
Dan Feldman; Michael Langberg; |
| 2011 | 11 | Optimizing Dialogue Management With Reinforcement Learning: Experiments With The NJFun System IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which addresses the technical challenges in applying reinforcement learning to a working dialogue system with human users. |
M. Kearns; D. Litman; S. Singh; M. Walker; |
| 2011 | 12 | A Reliable Effective Terascale Linear Learning System IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a system and a set of techniques for learning linear predictors with convex losses on terascale datasets, with trillions of features, {The number of features here refers to the number of non-zero entries in the data matrix.} |
Alekh Agarwal; Olivier Chapelle; Miroslav Dudik; John Langford; |
| 2011 | 13 | Evolutionary Algorithms For Reinforcement Learning IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. |
J. J. Grefenstette; D. E. Moriarty; A. C. Schultz; |
| 2011 | 14 | Revisiting K-means: New Algorithms Via Bayesian Nonparametrics IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we revisit the k-means clustering algorithm from a Bayesian nonparametric viewpoint. |
Brian Kulis; Michael I. Jordan; |
| 2011 | 15 | Budget-Optimal Task Allocation For Reliable Crowdsourcing Systems IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we consider a general model of such crowdsourcing tasks and pose the problem of minimizing the total price (i.e., number of task assignments) that must be paid to achieve a target overall reliability. |
David R. Karger; Sewoong Oh; Devavrat Shah; |
| 2011 | 16 | Structured Sparsity Through Convex Optimization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. |
Francis Bach; Rodolphe Jenatton; Julien Mairal; Guillaume Obozinski; |
| 2011 | 17 | Better Mini-Batch Algorithms Via Accelerated Gradient Methods IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Mini-batch algorithms have been proposed as a way to speed-up stochastic convex optimization problems. |
Andrew Cotter; Ohad Shamir; Nathan Srebro; Karthik Sridharan; |
| 2011 | 18 | Parallel Coordinate Descent For L1-Regularized Loss Minimization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose Shotgun, a parallel coordinate descent algorithm for minimizing L1-regularized losses. |
Joseph K. Bradley; Aapo Kyrola; Danny Bickson; Carlos Guestrin; |
| 2011 | 19 | Risk-Sensitive Reinforcement Learning Applied To Control Under Constraints IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we consider Markov Decision Processes (MDPs) with error states. |
P. Geibel; F. Wysotzki; |
| 2011 | 20 | Learning To Order Things IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Nevertheless, we describe simple greedy algorithms that are guaranteed to find a good approximation. |
W. W. Cohen; R. E. Schapire; Y. Singer; |
| 2011 | 21 | Efficient Optimal Learning For Contextual Bandits IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We address the problem of learning in an online setting where the learner repeatedly observes features, selects among a set of actions, and receives reward for the action taken. |
MIROSLAV DUDIK et. al. |
| 2011 | 22 | Noise Tolerance Under Risk Minimization IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we explore noise tolerant learning of classifiers. |
Naresh Manwani; P. S. Sastry; |
| 2011 | 23 | Active Ranking Using Pairwise Comparisons IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). |
Kevin G. Jamieson; Robert D. Nowak; |
| 2011 | 24 | Active Learning With Multiple Views IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We focus here on active learning for multi-view domains, in which there are several disjoint subsets of features (views), each of which is sufficient to learn the target concept. |
C. A. Knoblock; S. Minton; I. Muslea; |
| 2011 | 25 | Trading Regret For Efficiency: Online Convex Optimization With Long Term Constraints IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper we propose a framework for solving constrained online convex optimization problem. |
Mehrdad Mahdavi; Rong Jin; Tianbao Yang; |
| 2011 | 26 | Learning Symbolic Models Of Stochastic Domains IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this article, we work towards the goal of developing agents that can learn to act in complex worlds. |
L. P. Kaelbling; H. M. Pasula; L. S. Zettlemoyer; |
| 2011 | 27 | From Bandits To Experts: On The Value Of Side-Observations IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We develop practical algorithms with provable regret guarantees, which depend on non-trivial graph-theoretic properties of the information feedback structure. |
Shie Mannor; Ohad Shamir; |
| 2011 | 28 | Accelerating Reinforcement Learning Through Implicit Imitation IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose and study a formal model of implicit imitation that can accelerate reinforcement learning dramatically in certain cases. |
C. Boutilier; B. Price; |
| 2011 | 29 | Trace Lasso: A Trace Norm Regularization For Correlated Designs IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. |
Edouard Grave; Guillaume Obozinski; Francis Bach; |
| 2011 | 30 | Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we present an empirical study of various semi-supervised learning techniques on a variety of datasets. |
N. V. Chawla; Grigoris Karakoulas; |
| 2010 | 1 | A Reduction Of Imitation Learning And Structured Prediction To No-Regret Online Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a new iterative algorithm, which trains a stationary deterministic policy, that can be seen as a no regret algorithm in an online learning setting. |
Stephane Ross; Geoffrey J. Gordon; J. Andrew Bagnell; |
| 2010 | 2 | A Contextual-Bandit Approach To Personalized News Article Recommendation IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we model personalized recommendation of news articles as a contextual bandit problem, a principled approach in which a learning algorithm sequentially selects articles to serve users based on contextual information about the users and articles, while simultaneously adapting its article-selection strategy based on user-click feedback to maximize total user clicks. |
Lihong Li; Wei Chu; John Langford; Robert E. Schapire; |
| 2010 | 3 | A Tutorial On Bayesian Optimization Of Expensive Cost Functions, With Application To Active User Modeling And Hierarchical Reinforcement Learning IF:9 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. |
Eric Brochu; Vlad M. Cora; Nando de Freitas; |
| 2010 | 4 | Asymptotic Equivalence Of Bayes Cross Validation And Widely Applicable Information Criterion In Singular Learning Theory IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In the present paper, we theoretically compare the Bayes cross-validation loss and the widely applicable information criterion and prove two theorems. |
Sumio Watanabe; |
| 2010 | 5 | GraphLab: A New Framework For Parallel Machine Learning IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. |
YUCHENG LOW et. al. |
| 2010 | 6 | Robust PCA Via Outlier Pursuit IF:8 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an efficient convex optimization-based algorithm we call Outlier Pursuit, that under some mild assumptions on the uncorrupted points (satisfied, e.g., by the standard generative assumption in PCA problems) recovers the exact optimal low-dimensional subspace, and identifies the corrupted points. |
Huan Xu; Constantine Caramanis; Sujay Sanghavi; |
| 2010 | 7 | Optimal Distributed Online Prediction Using Mini-Batches IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we present the \emph{distributed mini-batch} algorithm, a method of converting many serial gradient-based online prediction algorithms into distributed algorithms. |
Ofer Dekel; Ran Gilad-Bachrach; Ohad Shamir; Lin Xiao; |
| 2010 | 8 | Adaptive Submodularity: Theory And Applications In Active Learning And Stochastic Optimization IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce the concept of adaptive submodularity, generalizing submodular set functions to adaptive policies. |
Daniel Golovin; Andreas Krause; |
| 2010 | 9 | Unbiased Offline Evaluation Of Contextual-bandit-based News Article Recommendation Algorithms IF:7 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce a \emph{replay} methodology for contextual bandit algorithm evaluation. |
Lihong Li; Wei Chu; John Langford; Xuanhui Wang; |
| 2010 | 10 | Robustness And Generalization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We derive generalization bounds for learning algorithms based on their robustness: the property that if a testing sample is similar to a training sample, then the testing error is close to the training error. |
Huan Xu; Shie Mannor; |
| 2010 | 11 | X-Armed Bandits IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We consider a generalization of stochastic bandits where the set of arms, $\cX$, is allowed to be a generic measurable space and the mean-payoff function is locally Lipschitz with respect to a dissimilarity function that is known to the decision maker. |
Sébastien Bubeck; Rémi Munos; Gilles Stoltz; Csaba Szepesvari; |
| 2010 | 12 | Adaptive Bound Optimization For Online Convex Optimization IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a new online convex optimization algorithm that adaptively chooses its regularization function based on the loss functions observed so far. |
H. Brendan McMahan; Matthew Streeter; |
| 2010 | 13 | Settling The Polynomial Learnability Of Mixtures Of Gaussians IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In contrast, because pathological scenarios can arise when considering univariate projections of mixtures of more than two Gaussians, the bulk of the work in this paper concerns how to leverage an algorithm for learning univariate mixtures (of many Gaussians) to yield an efficient algorithm for learning in high dimensions. |
Ankur Moitra; Gregory Valiant; |
| 2010 | 14 | Contextual Bandit Algorithms With Supervised Learning Guarantees IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We address the problem of learning in an online, bandit setting where the learner must repeatedly select among $K$ actions, but only receives partial feedback based on its choices. |
Alina Beygelzimer; John Langford; Lihong Li; Lev Reyzin; Robert E. Schapire; |
| 2010 | 15 | Application Of K Means Clustering Algorithm For Prediction Of Students Academic Performance IF:6 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: A system for analyzing students results based on cluster analysis and uses standard statistical algorithms to arrange their scores data according to the level of their performance is described. |
O. J. Oyelade; O. O. Oladipupo; I. C. Obagbuwa; |
| 2010 | 16 | Trajectory Clustering And An Application To Airspace Monitoring IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents a framework aimed at monitoring the behavior of aircraft in a given airspace. |
Maxime Gariel; Ashok N. Srivastava; Eric Feron; |
| 2010 | 17 | Portfolio Allocation For Bayesian Optimization IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose several portfolio strategies, the best of which we call GP-Hedge, and show that this method outperforms the best individual acquisition function. |
Eric Brochu; Matthew W. Hoffman; Nando de Freitas; |
| 2010 | 18 | Distributed Autonomous Online Learning: Regrets And Intrinsic Privacy-Preserving Properties IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we consider online learning with {\em distributed} data sources. |
Feng Yan; Shreyas Sundaram; S. V. N. Vishwanathan; Yuan Qi; |
| 2010 | 19 | Learning From Logged Implicit Exploration Data IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We empirically verify our solution on two reasonably sized sets of real-world data obtained from Yahoo!. |
Alex Strehl; John Langford; Sham Kakade; Lihong Li; |
| 2010 | 20 | Collaborative Filtering In A Non-Uniform World: Learning With The Weighted Trace Norm IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce a weighted version of the trace-norm regularizer that works well also with non-uniform sampling. |
Ruslan Salakhutdinov; Nathan Srebro; |
| 2010 | 21 | Safe Feature Elimination In Sparse Supervised Learning IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We investigate fast methods that allow to quickly eliminate variables (features) in supervised learning problems involving a convex loss function and a $l_1$-norm penalty, leading to a potentially substantial reduction in the number of variables prior to running the supervised learning algorithm. |
Laurent El Ghaoui; Vivian Viallon; Tarek Rabbani; |
| 2010 | 22 | Polynomial Learning Of Distribution Families IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To estimate parameters of a Gaussian mixture distribution in high dimensions, we provide a deterministic algorithm for dimensionality reduction. |
Mikhail Belkin; Kaushik Sinha; |
| 2010 | 23 | An Inverse Power Method For Nonlinear Eigenproblems With Applications In 1-Spectral Clustering And Sparse PCA IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we show that a certain class of constrained optimization problems with nonquadratic objective and constraints can be understood as nonlinear eigenproblems. |
Matthias Hein; Thomas Bühler; |
| 2010 | 24 | Near-Optimal Bayesian Active Learning With Noisy Observations IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We tackle the fundamental problem of Bayesian active learning with noise, where we need to adaptively select from a number of expensive tests in order to identify an unknown hypothesis sampled from a known prior distribution. |
Daniel Golovin; Andreas Krause; Debajyoti Ray; |
| 2010 | 25 | A CHAID Based Performance Prediction Model In Educational Data Mining IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: As this academic performance is influenced by many factors, it is essential to develop predictive data mining model for students’ performance so as to identify the slow learners and study the influence of the dominant factors on their academic performance. |
M. Ramaswami; R. Bhaskaran; |
| 2010 | 26 | Manifold Elastic Net: A Unified Framework For Sparse Dimension Reduction IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we proposed the manifold elastic net or MEN for short. |
Tianyi Zhou; Dacheng Tao; Xindong Wu; |
| 2010 | 27 | Sparse Inverse Covariance Selection Via Alternating Linearization Methods IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a first-order method based on an alternating linearization technique that exploits the problem’s special structure; in particular, the subproblems solved in each iteration have closed-form solutions. |
Katya Scheinberg; Shiqian Ma; Donald Goldfarb; |
| 2010 | 28 | Supervised Classification Performance Of Multispectral Images IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here we compare the different classification methods and their performances. |
K. Perumal; R. Bhaskaran; |
| 2010 | 29 | Agnostic Active Learning Without Constraints IF:5 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present and analyze an agnostic active learning algorithm that works without keeping a version space. |
Alina Beygelzimer; Daniel Hsu; John Langford; Tong Zhang; |
| 2010 | 30 | Network Flow Algorithms For Structured Sparsity IF:4 Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose an efficient procedure which computes its solution exactly in polynomial time. |
Julien Mairal; Rodolphe Jenatton; Guillaume Obozinski; Francis Bach; |