That can impossibly be true. JS is turing complete, turing completeness means that the halting problem is undecidable, which means that at best Google will execute JS with some random resource limit to avoid infinite resource consumption, which in turn means that whether Google will actually see the content is ultimately undefined. And even worse: other search engines might, due to lack of standardization of the available execution resources, see a different picture. That's just a braindead model for information storage and exchange.
Everything you said applies to humans using normal web browsers as well. And yet people still fill pages with JS, and browsers still execute it, and people still see stuff.
With one minor difference: Humans at least have full-blown human intelligence which they can use to heuristically determine when the execution is "complete". Not that that makes all that much more sense ...
You seem to be saying that page load is not deterministic but that is not true. A special version of a browser can easily keep track of http requests and other async operations and consider the page to be complete when async ops = 0 and the last paint is complete. The only hard part I can think of is a page that is recursively calling setTimeout endlessly, but even that can be coded around.
Right, so like you said they have a reasonable cut-off for this one most likely ultra-rare situation. It's not going to be perfect but it doesn't have to be, in order to be useful.
Yeah, that does make it better, but it's still not perfect. Witness the complaints right here about how you can't tell whether you got an empty page or it just hasn't finished loading yet.
Trouble awaits when you try to wedge a general-purpose application environment into a page-based document viewer.
I doubt Google is worried about unfairly weighting pages it indexes with Javascript that takes so long to run, hence the cutoff. It's more a question of where to place the cutoff than whether to have one at all, which is absurd.
No, the question is neither of those. The question is how to define the cutoff in a portable way that enables interoperability and long-term stability. A document format where every implementation has its own secret cutoff that also probably changes all the time is just idiotic if your goal is interoperability.
The exact same argument could apply to pages that take forever to load, because the web server sends you a chunk of bytes every two seconds. Would it be braindead for Google to apply an arbitrary timeout without standardization?
OK, so we have a pretty straightforward rule. Providing no further input to the program once it begins,
1) If execution terminates by some timeout T, where T is at least several seconds, then index that.
2) If execution has not yet terminated by T, whether or not we have any idea whether it will terminate in the future, don't index.
Tune T so that it will get the vast majority of reasonable web pages. (Hypothesize, and test, T = 5 seconds or 30 seconds or something. Have a human look at the highest-PageRanked timeouts to figure out what's going on.)
This applies whether "the program" is server-side or client-side. The halting problem cannot be reduced to the timeout problem (since the halting problem asks if it ever terminates), and the timeout problem is pretty clearly computable.
