It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent.
This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing you could usefully run on your own PC or server VM.
But these ML models are like neutron stars of computation density. You can’t really peek inside to see what’s going on either. An unknown stolen model’s properties would need to be discovered by experimentation.
I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.
Building them would be a challenge, but definitely not an insurmountable one. I’ve worked on a couple of C++ projects at a similar scale to Windows (millions of LOC) and the build systems were a major pain. But a determined engineer with the readme file and no other help could get it building in a week or so.
(This probably says more about how hard it is to build C++ than anything else)
Some years ago someone that worked at Microsoft told me he didn't think any individual engineer who already works on Windows could ever get Windows building by themselves with just the code.
Plus it already has been done before with Windows XP, without even any documentation and there is a guide on the internet on how to build Windows Server 2003.
This was done (if I remember right) when governments and big customers had access to the Windows source-code.
You’re also not trying to get a full CICD pipeline complete with unit/integration tests, crypto signatures, ability to flip features on and off with a click , monitoring of the cicd pipeline, scaled so 1000s of engineers can work at once etc
> But a determined engineer with the readme file and no other help could get it building in a week or so.
That's probably true, but I wouldn't be surprised if something like windows doesn't have a README file. And it does have build instructions they may well be in some wiki separated from the source code.
Well, it kind of is a language thing. Many newer languages (Rust and Go come to mind) are much more consistent in the way you interact with them as your project scales.
There are even nice timelapse videos of that process: https://vimeo.com/464644850 (you’ll need a Vimeo account to see it, because Vimeo is weird like that. This was on YouTube originally, but it was taken down.)
I guess you have never compiled a Linux/BSD distribution from scratch and supported it alongside its infrastructure and lead its maintenance process.
Even without that, if you want downloadable and runnable software platforms, look to public Git repositories. Some of the people who have no financial motivation will release what they do alongside installation procedures and quality of life scripts and architecture documentation.
Most of the open source platforms doesn't publish this documentation and doesn't make installation easy to keep a sizeable moat and protect the platform they have developed, hence this is why we have a division between "Free Software" and "Open Source".
In short, "The bulk of really valuable commercial code" is self contained, but not open source, or if open source it's not Free Software and made deployable for other parties. Otherwise it loses monetary value in the eyes of the people who develop that for the monies.
Otherwise we have have Elasticsearch incident, where they pivot and move to "Source Available" model to protect their castle.
"I guess you have never compiled a Linux/BSD distribution from scratch"
You guess, and so perhaps do other things as well, with poor acuity.
The incalculable value of open source software has approximately no bearing on this assertion.
Yes I love linux and bsd too I'm not defacing your religion. I'm actually quite Stallmanesque in making my own life harder by only using ooen source software as much as possible and being super fun a family gatherings talking about it.
> It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent.
I was trying to come to grasp with how much resource there is concentrated in one of these models. Somehow I come to the conclusion that it cost more than buying a jet airliner to train one of these models. And it is about the same order of money as commissioning and building a skyscraper in Manhattan. Is that correct approximately?
For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount.
That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.
Electricity costs are basically irrelevant because the cards are so expensive.
A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total.
Most dataceneters are between 7 and 10 cents per kilowatt hour for electricity. Some are below 4. At 10 cents, that's $53,000 in electricity costs, which is nothing next to $30 million in capital costs.
No, I'm willing to bet the CO2 cost of the cards is also way higher than the electricity. Those things are built on the global supply chain, with materials potentially making multiple thousands of kms journeys between each step.
Long term I also imagine it's much cheaper to run these large model trainings on renewables. It's a very centralized process that doesn't necessarily need 100% availability.
The manufacturing process, however, is totally decentralized, and NVIDIA mostly manufactures in China where coal is cheap.
US grid mix produces about 0.855 pounds of CO2 per kWh[0]. So 552,000 kWh 452,640 pounds of CO2 which is 205.31 metric tons. At a cost of $40 per tonne[1] of CO2 that works out to $8,212.40 which is still small compared to the capital cost of the cards.
AWS us-west-2 is housed in The Dalles and Prineville, Oregon. Not only are they near a massive wind farm in the Columbia Gorge, but also quite near the Columbia river's many hydro-electric dams. Facebook and Apple also have Prineville data centers. They are built there intentionally. Electricity at many data centers is quite carbon-lean.
I always feel there is an opportunity cost here though. If that green energy wasn’t being used for compute it could be available to heat someone’s home instead of them using dirty sources.
$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.
At the same time I guarantee you they didn’t get it right the first time. I’m sure there were multiple (both serially and in parallel) runs as they worked out kinks and tuned hyper parameters.
Not to mention, the kind of expertise to run this for a major corporation doesn't come for free either? Facebook employs quite a few high profile ML researchers who undoubtedly make mid-high six figure salaries.
The point was that if you only need to train once, then it's cheaper to rent the GPUs than to buy them. If you need to train it multiple times, then the cost of buying the GPUs is amortized among runs.
In any case the cost per run is going to be lower than 30m
I'm sure that's the case. The latest sku I'm responsible for QC testing now contains 4x A100's in a 2U chassis. And oh man the number of QSFP ports it utilizes..
Azure is generally a pretty terrible cloud (poor UX, very slow for anything, multiple highly critical cross-tenant security issues, etc.) far behind the market leader, AWS, so they have to compensate with pricing (same reason why Oracle Cloud is so reasonably price, they're already so far behind their usual pricing wouldn't make any sense).
There's no reasonable way to get an estimate of what it actually costs FB.
1) The GPU's are not single use, they will amortize it over 3 yrs and there are other things that it will be used for that generate revenue.
2) The cost of the servers for these GPU's to run in with massive CPU, RAM, and storage requirements.
3) The overhead of building and operating all of that infrastructure in terms of people, electricity, cooling, etc.
4) The overhead of having dozens or hundreds of engineers & scientists who contributed to this.
One way you can distill the first three is to use AWS/Azure/GCP costs. But then you are still missing a major factor which is the humans that worked on it, and the human may very well exceed the hardware cost.
Plus there's a lot of highly specialized engineers required to keep all those GPUs up and running during training and the ML engineers who are skilled in deep learning + hardware, plus the systems for gathering/cleaning/labelling data. Gather enough engineers and now you need managers, PMs, etc.
No you are probably overestimating the cost by 1-2 orders of magnitude. GPT-3 probably cost under $5 million, and this model is smaller and there have been algorithmic improvements to training transformers since then.
So do they estimate how much computing power/time they will need and then find some upper tier minimum $ amount to get the maximum discount possible or getting a certain resource availability commitment from Google? That's an interesting accounting problem.
> No you are probably overestimating the cost by 1-2 orders of magnitude.
You are right! Wow. Thank you for correcting me.
> GPT-3 probably cost under $5 million,
Is that one training run or includes all the fiddling to find the right hyperparameters? Or there aren't many of those in these training or they are not that sensitive?
I think they probably did a lot of hyperparameter searching to train the smaller models and then extrapolated for the largest model, but I'm just guessing. OpenAI had a finite amount of money when they were training GPT-3, they likely do it differently now that inference costs are significant compared to training costs.
“Brute forcing a really inefficient approximation/estimator” is a good way to summarize it.
It’s like having an overfit equation to a sample of data points, instead of the simpler actual line they fall near.
They end up being black boxes, we have almost no idea how they work inside, and we have no idea how overtrained they are when something simpler could do the same thing.
I don't think the term "brute forcing" is an adequate term to describe gradient descent. Brute forcing would be to try all random weights with no system imo.
Can "something simpler", for example, code correct function bodies from comments describing functions in natural language? I think people are too quick to dismiss the power of these models.
I am by no means dismissing the power. They are created very chaotically, however. Spaghetti thrown at a wall. They are brute force approximations.
They are wasteful. If LLaMa 13B is as powerful as previous 65B models, that's a significant amount of unnecessary paramaters lost/pruned in just this iterative upgrade alone. How small can they go? The fewest parameters that get the job done 99% as well is the way to go.
There is also the difference between the rules and use of language being directly compressed into the model, vs all the information known to humans compressed into the model. A smaller model that ingests relevant information on the fly (more like Bing, that supplements itself with search), may be less wasteful and perform better.
The current models being released are chosen because "they work" not because they are least fit and most performant optimized.
Finding a sha256 hash with N leading zeros is basically arbitrarily computationally expensive but could be written on a piece of paper. I don't see training an ML model as an egregious example of concentrating compute power
Certainly they retain not just information but compute capacity in a way that other expensive transformations don’t. I’m hard pressed to think of another example where compute spend now can be banked and used to reduce compute requirements later. Rainbow tables maybe? But they’re much less general purpose.
Not only can we bank computation, speed up physical simulations by 100x but I also saw some work on being able to design outcomes in GoL (game of life).
There was a paper on using a NN to build or predict arbitrary patters in GoL, but I can't find it right now.
It would be interesting to see an analysis of this. I see your point - otoh is there a reason to believe that more computation is being "banked" than say matrix inversion, or other optimizations that aren't gradient descent based?
The large datasets involved let us usefully (for some value of useful) bank lots of compute, but it's not obvious to me that it's done particularly efficiently compared to other things you might precompute.
For converged model training, training is often quite inefficient because the weight updates decay to zero and most epochs are having a very small individual effect. I think for e.g. stable diffusion, they dont train to anywhere near convergence so weight updates have a bigger average effect. Not sure if that applies to llms
Back when wavelet compression was still being developed, there was a joke that the best compression algorithm is "give an image to a grad student and tell them to figure out the best transform".
Whole new vistas open up to possible retaliation for piracy. Imagine how a bootlegged AI could have been set up to not just steal your info but manipulate you into ruining your life as revenge for bootlegging it...
I don't know about about this model, but usually with these ML models you download the static weights, but nothing is stopping you from fine tuning them to your needs/new information.
It's not automatic, would require some ML Engineering, but nothing is stopping you if you have the Pytorch graph and weights.
I mean the answer to the life is 42 but it took 7.5 million years for an advanced alien tech computer with the size of a building to calculate that
/s
Calculating things takes time and unrelated to output size. There are NP problems that simply outputs true or false yet requires more computational power than the universe can support
I fail to see how this is different from other software in that regard. If you have parameters but not the network architecture, then it's not very useful.
You do need to guess things like activation functions, number of attention heads, order of attention layers, etc. Often the parameter names reveal something about these.
Under Feist Publications, Inc., v. Rural Telephone Service Co. ... it gets tricky.
From Wikipedia:
> The ruling of the court was written by Justice Sandra Day O'Connor. It examined the purpose of copyright and explained the standard of copyrightability as based on originality.
> The case centered on two well-established principles in United States copyright law: that facts are not copyrightable, and that compilations of facts can be.
> "There is an undeniable tension between these two propositions", O'Connor wrote in her opinion. "Many compilations consist of nothing but raw data—i.e. wholly factual information not accompanied by any original expression. On what basis may one claim a copyright upon such work? Common sense tells us that 100 uncopyrightable facts do not magically change their status when gathered together in one place. … The key to resolving the tension lies in understanding why facts are not copyrightable: The ″Sine qua non of copyright is originality."
> ...
> The standard for creativity is extremely low. It need not be novel; it need only possess a "spark" or "minimal degree" of creativity to be protected by copyright.
> In regard to collections of facts, O'Connor wrote that copyright can apply only to the creative aspects of collection: the creative choice of what data to include or exclude, the order and style in which the information is presented, etc.—not to the information itself. If Feist were to take the directory and rearrange it, it would destroy the copyright owned in the data. "Notwithstanding a valid copyright, a subsequent compiler remains free to use the facts contained in another's publication to aid in preparing a competing work, so long as the competing work does not feature the same selection and arrangement", she wrote.
> The court held that Rural's directory was nothing more than an alphabetic list of all subscribers to its service, which it was required to compile under law, and that no creative expression was involved. That Rural spent considerable time and money collecting the data was irrelevant to copyright law, and Rural's copyright claim was dismissed.
---
And so, my (I am not a lawyer) take on this is that the numbers of the model are not copyrightable. The selection of the source material is... kind of. This gets into a "a recipe is not copyrightable, yet a recipe book is"
If you were to steal a chunk of source code or a binary from meta/Google, you could probably get it running inside a few weeks effort.
Sure, the binary probably depends on a lot of internal proprietary infrastructure, but also most of that infrastructure is easy to write a mock implementation of, as long as you are happy for it to be in-ram, not multi-homed and don't need it to scale to billions of users.
Most of the binaries have a standalone mode for running on a developers PC with few/no dependencies anyway.
-1: as an ex-googler, I can say it was hard enough for Google itself to get its code to run, given gonzo infrastructure assumptions, proprietary libraries/languages, etc.
sorry, but that's not how code works. It's true that code quality could be terrible but in fact Google is famous/notorious for extreme code review at the line-by-line granularity, plus comments, design docs and more.
The real issues are (again) in dependencies and complex tooling. You can have beautiful code and then in the middle of it, an ML inference call that assumes a crazy ML model and set of hardware to run it on.
Assume you got source for a game written in a proprietary game engine. If you don't have access to the game engine itself, nor the API documentation, etc, how long will it take to get this game running in your manner of choosing?
The infrastructure in these companies is a huge amount of scaffolding that's non-trivial to replicate.
Google has no incentives to allow an arbitrary component run standalone. Quite the contrary.
What they do get in return for the coupling is that they can evolve the common libraries and code patterns across the board (there are even automated code refactoring tools that help you do massive code changes, automating code review sessions across hundreds and hundreds of teams, with all changes tested against all reverse dependencies etc etc). All this allows for a level of internal code quality that is hard to see elsewhere.
Unless you really care a lot about that one requirement you seem to care about. In that case, yeah, you'd choose a different tradeoff.
Arguably it's not low quality code, but low quality system. Code can be correct, clear, and documented, and still be fragile and sensitive to platform configuration changes. E.g. "how many switches do I have to change in the build system before the code no longer builds?", "how many network jacks can I move this server over before I lobotomize the system?"
You ought to be able to arrive at the same conclusion, then, with these LLMs. Without arrays of GPUs, it would take thousands of years to train one. Without a corpus of billions or trillions of words, one would produce output of very limited utility.
I think you have to consider that some things are systems, and it is the assembly of their components that imparts the true quality.
Good luck even getting a google3-based Hello World to compile. I don't remember the exact numbers, but just #including the most basic libs resulted in a O(100M) binary.
And anything more complex than that would probably have dependencies on so many fat client libs, so much infrastructure, and so many external services, that you'll need months-years to even make sense of them, let alone mock them up.
This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing you could usefully run on your own PC or server VM.
But these ML models are like neutron stars of computation density. You can’t really peek inside to see what’s going on either. An unknown stolen model’s properties would need to be discovered by experimentation.