r/MachineLearning • • Mar 10 '20

AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

https://arxiv.org/pdf/2003.03384.pdf
22 Upvotes

11 comments sorted by

View all comments

0

u/[deleted] Mar 10 '20

[deleted]

2

u/svantana Mar 11 '20

Agreed. It looks interesting at first, but looking closer at the OP dictionary, basically the only thing it can do is affine transforms and various scalar nonlinearities, which means it will only come up with variations of NNs with potentially exotic activation functions.

The co-learning of predictor and learner is interesting, but very dumb: the learner should at least have access to the predictor in some sense, either through operations such as grad(), or some more advanced ability to traverse the computational graph.

2

u/bbateman2011 Mar 13 '20

it will only come up with variations of NNs with potentially exotic activation functions

I am not capable of attacking the detailed math, but the above statement covers a significant number of ArXiv papers!

3

u/svantana Mar 13 '20

Right, and nothing wrong with that. I'm just adding some nuance to the insinuation from the paper that it can come up with any algorithm possible.