Surely, they would throw current events, news articles, the latest snapshot of WikiPedia, etc...

I can't imagine it making sense to purposefully neglect to keep a model as up-to-date as possible!