r/mlpapers • • Mar 10 '18

I wrote a plain-English explanation of the original AlphaGo paper by DeepMind, published in Nature. Check it out!

Thumbnail
medium.com
23 Upvotes

r/mlpapers • • Mar 17 '17

Deep Forest: Towards An Alternative to Deep Neural Networks

Thumbnail
arxiv.org
18 Upvotes

r/mlpapers • • May 06 '16

A curated list of awesome TensorFlow experiments, libraries, and projects. Inspired by awesome-machine-learning.

Thumbnail
github.com
16 Upvotes

r/mlpapers • • Nov 06 '14

Machine Learning Journal Club

16 Upvotes

Hi all,

In continuation of this post on /r/MachineLearning I have created this subreddit.

I created this subreddit so that it might be easier to follow the paper discussions and make it easier to participate.

My suggestion is to simply make a little cross-post on /r/MachineLearning in order to announce a new voting round and new paper to discuss, but keep the main action here. This way older subjects that people find interesting but weren't there to discuss are easier to be found here.

I believe this will increase participation and make archivation possible.

Please subscribe, I am hoping to continue the ML paper discussions in this subreddit starting next round if possible.


r/mlpapers • • Dec 30 '18

List of arXiv review papers in machine learning

Thumbnail freenode-machinelearning.github.io
15 Upvotes

r/mlpapers • • Dec 25 '18

My list of code repos found from papers

15 Upvotes

Here is my list of known code repos for machine learning. I have found them from numerous papers. I find it more useful and practicable to keep up with new software than with papers without software. Even so, the list is not fully up date; I update it as I like. It is my personal and limited view of the world.

This is brought to you by the Freenode IRC channel on machine learning.


r/mlpapers • • Nov 08 '14

[Discussion] Algorithms for Non-negative Matrix Factorization

15 Upvotes

Here's the long-overdue discussion thread on Algorithms for Non-negative Matrix Factorization by Lee & Seung, NIPS 2000 (PDF link). This paper was awarded the NIPS classic paper award in 2013, and is one of the fundamental papers on the subject. Since it's publication, it has been cited over 3800 times (according to google).


r/mlpapers • • Nov 30 '23

Google announces 2.2M new materials discovered using GNN

12 Upvotes

Materials discovery is critical but tough. New materials enable big innovations like batteries or LEDs. But there are ~infinitely many combinations to try. Testing for them experimentally is slow and expensive.

So scientists and engineers want to simulate and screen materials on computers first. This can check way more candidates before real-world experiments. However, models historically struggled at accurately predicting if materials are stable.

Researchers at DeepMind made a system called GNoME that uses graph neural networks and active learning to push past these limits.

GNoME models materials' crystal structures as graphs and predicts formation energies. It actively generates and filters candidates, evaluating the most promising with simulations. This expands its knowledge and improves predictions over multiple cycles.

The authors introduced new ways to generate derivative structures that respect symmetries, further diversifying discoveries.

The results:

  1. GNoME found 2.2 million new stable materials - equivalent to 800 years of normal discovery.

  2. Of those, 380k were the most stable and candidates for validation.

  3. 736 were validated in external labs. These include a totally new diamond-like optical material and another that may be a superconductor.

Overall this demonstrates how scaling up deep learning can massively speed up materials innovation. As data and models improve together, it'll accelerate solutions to big problems needing new engineered materials.

TLDR: DeepMind made an AI system that uses graph neural networks to discover possible new materials. It found 2.2 million candidates, and over 300k are most stable. Over 700 have already been synthesized.

Full summary available here. Paper is here.


r/mlpapers • • Dec 18 '20

[D] 2020 in Review | 10 AI Papers That Made an Impact

12 Upvotes

Much of the world may be on hold, but AI research is still booming. The volume of peer-reviewed AI papers has grown by more than 300 percent over the last two decades, and attendance at AI conferences continues to increase significantly, according to the Stanford AI Index. In 2020, AI researchers made exciting progress on applying transformers to areas other than natural-language processing (NLP) tasks, bringing the powerful network architecture to protein sequences modelling and computer vision tasks such as object detection and panoptic segmentation. Improvements this year in unsupervised and self-supervised learning methods meanwhile evolved these into serious alternatives to traditional supervised learning methods.

As part of our year-end series, Synced highlights 10 artificial intelligence papers that garnered extraordinary attention and accolades in 2020.

Here is a quick read: 2020 in Review | 10 AI Papers That Made an Impact


r/mlpapers • • Nov 27 '19

A massive list of Monte Carlo tree search research papers

12 Upvotes

https://github.com/benedekrozemberczki/awesome-monte-carlo-tree-search-papers

It was compiled in a semi-automated way and covers content from the following conferences:


r/mlpapers • • Nov 07 '14

Fundamental Papers

12 Upvotes

Are there any fundamental papers a newbie should read to get caught up with the current state of ML?


r/mlpapers • • Jan 24 '15

Neural Machine Translation by Jointly Learning to Align and Translate

11 Upvotes

http://arxiv.org/abs/1409.0473

Hey everyone. I couldn't help posting this paper, and I think I'll start posting regularly from now on (time allowing). Most of the papers I post will be on deep learning, as that is my biggest area of interest; also, I feel as if it can be understood with the least amount of math for people that ML applications.

Paper Summary: The history behind this paper is that there's been a large interest lately in using recurrent neural networks (RNNs) to perform machine translation. The original idea by Quoc Le et. al (I forgot the specific name of the paper if anyone wants to link below), was to have a recurrent neural network trained to predict the next word given the previous word and the context, as follows: http://imgur.com/0ZMT6hm

To perform translation, the network outputs an EOS (end of sentence) token, and the network will now begin producing the first output for the translated sentence. The brilliant part about this is that it uses the final hidden state for the input (the sentence to be translated) as additional input to all the translation units. This is essentially compressing the input (the entire sentence) into N (#hidden_states) real numbers! Pretty neat!

The recurrent network uses LSTM gates for the "memory" units. It is then trained using stochastic gradient descent.

The paper I've attached is an extension of this idea that uses all of the hidden states instead of the final one.

Side Note: I really want to encourage discussion, so please ask questions and make comments in the light of

  • Clarification questions

  • Ideas this could be used for

  • Interesting things to think about

  • Other papers that have similar, but interesting ideas

  • Why this paper is interesting

  • Why I'm wrong about everything I wrote (Please! I learn the most when people tell me I'm wrong)

  • What makes X better than Y

  • What happens if they excluded X

  • Anything else you can think of

Also, when referencing the paper, be sure to include the section, as it will make it easiest for everyone to join in on the discussion!


r/mlpapers • • Nov 23 '14

[Discussion] How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation

11 Upvotes

r/mlpapers • • Nov 17 '14

[Discussion] Automating music composition and melody generation

11 Upvotes

As per the voting thread, next week's paper is going to be Automating music composition and melody generation (Related project on github), proposed by /u/CreativePunch .

The paper and the related project provides a hybrid way using a neural network to create a scoring for good music, and then using a statistical model to generate music and select the good music using the neural network, which is then optimized using a genetic algorithm.

It's a perfect example of how multiple simpler components can be chained together to create a more complex process capable of very complex tasks.


r/mlpapers • • Nov 08 '14

[META] How should we hold this Journal Club

11 Upvotes

Hi there!

/u/CreativePunch has kindly set up this subreddit to keep track of the papers discussed by the /r/MachineLearning journal club. I've taken the liberty of setting up a new voting- and discussion thread like in the old format that it had over in /r/MachineLearning

But maybe there's stuff we want to change about the format of the club? I'd appreciate any input you guys have on anything!

For now, this is the proposed process:

  1. Each week a new voting thread is set up

  2. The proposal with the most upvotes at the end of the week (say Friday or Saturday) will be the upcoming week's paper.

  3. A discussion thread for it will be created in /r/mlpapers , which will then be crossposted to /r/MachineLearning

What I'm unsure about:

  • should we crosspost the discussion threads to other subs as well?

  • should we crosspost the voting-threads as well?

  • should we pin the current voting or discussion thread?


r/mlpapers • • Sep 09 '19

A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019)

10 Upvotes

GitHub: https://github.com/benedekrozemberczki/ClusterGCN

Paper: https://arxiv.org/abs/1905.07953

Abstract:

Graph convolutional network (GCN) has been successfully applied to many graph-based applications; however, training a large-scale GCN remains challenging. Current SGD-based algorithms suffer from either a high computational cost that exponentially grows with number of GCN layers, or a large space requirement for keeping the entire graph and the embedding of each node in memory. In this paper, we propose Cluster-GCN, a novel GCN algorithm that is suitable for SGD-based training by exploiting the graph clustering structure. Cluster-GCN works as the following: at each step, it samples a block of nodes that associate with a dense subgraph identified by a graph clustering algorithm, and restricts the neighborhood search within this subgraph. This simple but effective strategy leads to significantly improved memory and computational efficiency while being able to achieve comparable test accuracy with previous algorithms. To test the scalability of our algorithm, we create a new Amazon2M data with 2 million nodes and 61 million edges which is more than 5 times larger than the previous largest publicly available dataset (Reddit). For training a 3-layer GCN on this data, Cluster-GCN is faster than the previous state-of-the-art VR-GCN (1523 seconds vs 1961 seconds) and using much less memory (2.2GB vs 11.2GB). Furthermore, for training 4 layer GCN on this data, our algorithm can finish in around 36 minutes while all the existing GCN training algorithms fail to train due to the out-of-memory issue. Furthermore, Cluster-GCN allows us to train much deeper GCN without much time and memory overhead, which leads to improved prediction accuracy---using a 5-layer Cluster-GCN, we achieve state-of-the-art test F1 score 99.36 on the PPI dataset, while the previous best result was 98.71.


r/mlpapers • • Dec 28 '21

NeurIPS 2021 - Curated papers - Part 2

9 Upvotes

In part-2 , I have discussed following papers :

  1. Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

  2. Attention Bottlenecks for Multimodal Fusion

  3. AugMax: Adversarial Composition of Random Augmentations for Robust Training

  4. Revisiting Model Stitching to Compare Neural Representations

https://rakshithv-deeplearning.blogspot.com/2021/12/neurips-2021-curated-papers-part2.html


r/mlpapers • • May 30 '21

ProteinBERT: A universal deep-learning model of protein sequence and function

Thumbnail
self.bioinformatics
9 Upvotes

r/mlpapers • • Dec 07 '20

[N] Open AI’s GPT-3 Paper Shares NeurIPS 2020 Best Paper Awards With Politecnico di Milano, CMU and UC Berkeley

9 Upvotes

OpenAI’s groundbreaking GPT-3 language model paper, a no-regret learning dynamics study from Politecnico di Milano & Carnegie Mellon University, and a UC Berkeley work on data summarization have been named the NeurIPS 2020 Best Paper Award winners. The organizing committee made the announcements this morning, along with their Test of Time Award, to kick off the thirty-fourth Conference on Neural Information Processing Systems.

NeurIPS 2020 continues through December 12. With 9,467 submitted papers, this has been another record-breaking year for NeurIPS — with 38 percent more paper submissions than 2019. A total of 1,903 papers were accepted, compared to 1,428 last year.

Over the course of the week, participants can virtually join the Expo, where top industry sponsors will provide talks, panels, and demos of academic interest. Tutorials will cover current lines of inquiry while general sessions will include talks, posters, and demonstrations. A full agenda can be found by visiting the NeurIPS conference schedule page.

Here is a quick read: Open AI’s GPT-3 Paper Shares NeurIPS 2020 Best Paper Awards With Politecnico di Milano, CMU and UC


r/mlpapers • • Oct 29 '20

[R] DeepMind Introduces Algorithms for Causal Reasoning in Probability Trees

9 Upvotes

Probability trees may have been around for decades, but they have received little attention from the AI and ML community. Until now. “Probability trees are one of the simplest models of causal generative processes,” explains the new DeepMind paper Algorithms for Causal Reasoning in Probability Trees, which the authors say is the first to propose concrete algorithms for causal reasoning in discrete probability trees.

Here is a quick read: DeepMind Introduces Algorithms for Causal Reasoning in Probability Trees

The paper Algorithms for Causal Reasoning in Probability Trees is on arXiv, and an interactive tutorial is available on GitHub.


r/mlpapers • • Mar 12 '20

Visual Explanation of SimCLR: State of the Art Self-Supervised Learning Method

Thumbnail
amitness.com
9 Upvotes

r/mlpapers • • Dec 18 '19

A repository of community detection (graph clustering) research papers with implementations (deep learning, spectral clustering, edge cuts, factorization)

9 Upvotes

Link: https://github.com/benedekrozemberczki/awesome-community-detection

The repository covers techniques such as deep learning, spectral clustering, edge cuts, factorization. I monthly update it with new papers when something comes out with code.


r/mlpapers • • Dec 12 '19

A massive list of gradient boosting research papers from major conferences

9 Upvotes

https://github.com/benedekrozemberczki/awesome-gradient-boosting-papers

A curated list of gradient and adaptive boosting papers with implementations from the following conferences:


r/mlpapers • • Dec 05 '18

Filtr.pub: Finding signals in noisy AI Research

9 Upvotes

AI research is moving at breakneck pace. State-of-the-art methods seem to become obsolete almost as quickly as they are found - as practitioners, it’s important to stay on top of the field.

Meanwhile, the quantity of papers uploaded on arXiv is outpacing Moore’s Law. With the sheer quantity of research published on a daily basis, and the lack of peer review for uploading - it’s becoming increasingly difficult to know what’s important and what isn’t.

How do you separate signal from noise?

Enter filtr.pub

A unique platform designed to prioritize quality over quantity. Upvoting. Email subscriptions. Intelligent filters. Everything you wished arXiv had but doesn’t - it’s all here. Brought to you by fellow practitioners - Data Scientists and Machine Learning Engineers equally frustrated with this problem. After a lot of looking, we were unable to find a viable solution. So we decided to build one.

Check us out! We’re working hard on the platform - and we’ll invite a select group of practitioners for a closed-beta, so we can iterate on feedback and get the product ready for the wider community! Sign up for news + updates + the opportunity to be a part of the beta program.


r/mlpapers • • Jan 29 '21

[N] NVIDIA, UToronto, McGill & Vector Study Delivers Real-Time SDF Rendering & SOTA Complex Geometry Reconstruction

8 Upvotes

A new study by NVIDIA, University of Toronto, McGill University and the Vector Institute introduces an efficient neural representation that enables real-time rendering of high-fidelity neural SDFs for the first time while delivering SOTA quality geometric reconstruction.

Here is a quick read: NVIDIA, UToronto, McGill & Vector Study Delivers Real-Time SDF Rendering & SOTA Complex Geometry Reconstruction

The paper Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Surfaces is on arXiv.