>> Credit belongs to whoever actually makes it work.
That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.
> credit belongs to the people who deserve credit for the work they've done that was useful to others
Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets.
For example, just taking a random paper that Schmidhuber claims should give him credit for GANs, https://people.idsia.ch/~juergen/FKI-126-90ocr.pdf hopefully you can easily see that a lot of work would be needed to turn this into a realistic image generation model. And that is, even if you admit that the idea is strongly related to GANs, which I'm not convinced of but won't spend time on.
> Credit belongs to whoever actually makes it work.
>> That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice?
It is accepted practice in the ML community. If it weren't, Schmidhuber wouldn't be complaining.
I agree a significant amount of work (and often insight too) is needed to translate an architecture idea into something that works in practice, and there are certainly plenty of ideas that are obvious in the abstract. But I also think it's important to avoid dismissing work only on the basis that it doesn't involve "real life datasets".
Deep learning is a relatively unexplored field and there are many open mathematical and scientific questions to ask that involve only model equations or contrived datasets. Novel theoretical results are not just about some architecture idea but about proving facts that can be useful for understanding how the model class would perform in different scenarios. Which in turn can help shape the search space for applied work.
Additionally, I don't think credit assignment should be so discrete. 100% agree that vomiting out vague ideas shouldn't grant claims to credit, but academic science much too often gives only a single author the "real" credit.
Incidentally, in other fields the person who actually makes it work very well may not be the person that receives this credit. Like biology can involve a lot of hard manual work (that isn't really intellectual) in order to realize a project plan. It varies how much of the credit those people receive, and I'm not even sure how much they should receive. This topic is extremely nuanced.
"Introductory theoretical work in GAN was done by Schmidhuber [1], but it was not until large experimental efforts [2,3,4] on image generations that the power of GANs was revealed."
I don't buy that this is standard practice in the ML community, and even if it is it's BS. If the basic idea/principle has been published previously but in a different context you should cite it and say why the solution is not directly applicable or has not been evaluated in the current context. Anything else is unprofessional.
Agreed. The piece anticipated this straw man argument:
> "the inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it)." Nothing more or less than the standard elementary principles of scientific credit assignment.[T22] LBH, however, apparently aren't satisfied with credit for popularising the inventions of others; they also want the inventor's credit.[LEC]
Basically ideas are a dime a dozen. Sure, your idea might be a good one, but how do we spot your grain of sand is special when it looks the same as the rest of the desert? Essentially having an idea isn't useful to others. Demonstrating that your idea has legs is useful to others.
I don't have to deal with citing papers, but I once had to deal with people pitching me ideas, wanting me to sign an NDA, in exchange for 50% of the revenue after I did all the actual work. Just out of curiosity, I signed one once. It was a fart app, IIRC. They thought a fart app needed an NDA, and that I'd then go do all the work and give them 50% because they "had the idea". It was so laughably sad.
If you think these ideas are valuable, I have a beautiful clock for you. It is right twice a day. You'll have the same problem: you won't know when it's right. You'll need someone else's work to tell that.
There's a spectrum of ideas, from groundbreaking to "dime a dozen". In tech startups, and in almost all of computer science, most ideas are a dime a dozen, and the value is in the execution.
But clearly, some ideas are groundbreaking. Einstein rightfully gets the credit for an on-paper hypothesis that wasn't proved until decades later via a chain of critical discoveries and experimental innovations by other people. It's legit to call it Einstin's relativity, and not Mossbauer/Hay's relativity.
Ideas are a dime a dozen in the sense that the same idea will often occur to dozens of people, on a dime's worth of effort. Relativity theory wasn't anything like that. Einstein made predictions that no one else was making. When one of them from GR was confirmed a few years ago, Lenny Susskind famously marveled at the foresight, saying "they didn't call him Einstein for nothing!".
Problem then goes to how do I decide whether this particular idea is a dime a dozen or a unique idea... Everyone ends up going by feels when answering this question for any particular problem.
This is nothing to do with ideas. Schmidhuber is complaining that his published work was plagiarised. In machine learning research, in order to publish your work you have to show that your proposed approach works and to do that you have to beat some benchmarks and establish a new state of the art, otherwise there's no publication. That takes work and that's the work that Schmidhuber claims was inappropriately left uncited. In fact that's exactly the kind of work that Hinton, LeCun and Bengio have always done. That's what machine learning researchers do.
This ... idea that Schmidhuber is an ideas man who's never done any real work is Hinton's allegation, and it's clearly designed to misrepresent both Schmidhuber and his work in order to discredit his complaints. And I'm sorry to say that people on HN have fallen for it hook, line and sinker, I guess because that's what social media says.
Btw, the point I make, that you don't get published in machine learning without beating some benchmarks and establishing a new state of the art, I can attribute that to none other than Hinton himself, in an interview with Wired, whence I quote, by the by:
>> What we should be going for, particularly in the basic science conferences, is radically new ideas. Because we know a radically new idea in the long run is going to be much more influential than a tiny improvement.
So that's the guy accusing the other guy of being nothing but an ideas man and that you don't need to cite someone who first came up with an idea, saying that "new ideas" are important.
But that's just Hinton presenting things just the way he likes. Now ideas are important, now they're not, as he pleases.
I don't know enough about this area of research to have an opinion on this particular topic, but I've noticed a trend with this sort of thing with my colleagues. They both claim at different times, depending on whether it benefits them, that
(1) Ideas are a dime a dozen, and making it work or bringing it to fruition, is the important thing
and
(2) The idea is the important thing; the specific implementation by someone doesn't matter, as they're just doing what the idea creator or discoverer laid out for others to follow.
Sometimes I feel like there's a fundamental paradox there that arises a lot in numerous areas of work, business, and economics.
That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.