I would say that comedians have always been among the top professional A/B testers (long before the buzz around A/B testing). Their entire professional success or failure depends on constant small and large adjustments to their routines based on testing and feedback. And when it comes to being funny and staying funny, comedy is brutal and completely unforgiving.
Here's a great 50 minute segment, with Jerry Seinfeld, Chris Rock, Ricky Gervais and Louis CK, in which they discuss various aspects of their material and how they work with it (adding / discarding / making adjustments and so on):
Where do you draw the line between "A/B testing" and "learning and improving", which nearly everyone does. Homeless people will figure out how to get better results from begging, chefs will learn how to cook food that tastes better, and these things have been going on for a long time - what sets comedians apart?
As a comedian and a A/B tester, I feel somewhat qualified to answer this question. There are similarities with what comedians do and A/B testing, but the big difference is that A/B testing is (or, in some cases, should be) scientific. Comedians can't think about statistical significance. But what sets comedians apart from chefs is that comedians get instant feedback. Chefs, in most cases, don't serve the food and watch the faces for emotion (although I just watched Jiro Dreams of Sushi and he does, maybe that's part of his success?)
The extreme majority of homeless people don't do homelessness as a profession, they do it out of desperation. It's not a reasonable comparison (particularly given I referenced professional A/B testing).
Your typical homeless person does a form of A/B testing in trying out various schemes to stay alive (whether it comes to food, begging, shelter, etc).
A/B testing is applying a scientific method to learning, so I don't regard that you have to draw any distinctions there.
A good chef is absolutely a top notch A/B tester, but then I wasn't implying otherwise. It's not like comedians being good at it excludes chefs from being good at it.
I would say that there are a lot of working average and mediocre chefs in the world, given the number of restaurants of every type. There are radically fewer comedians earning a living at it. I would say that comedy is even less forgiving than food (and food is very unforgiving professionally compared to most things, as people are extremely picky about food).
You can get by being a bad chef, we have 600,000 restaurants in the US, and a huge number of them likely prove that every day. You cannot get by being a bad comedian for long. You're either funny or you're not. With food there is a huge lower and middle ground - people eat mediocre mass food all the time, and go back for more (see: McDonalds or Denny's). People will tolerate eating mediocre food to survive; they won't willingly pay to go see what they know to be terrible comedians.
If we're talking challenging A/B systems, I would argue it's radically harder to succeed like a comedian, than to beg for change or flip a burger at Denny's. To make it a little more similar, a chef would need to make food surrounded by all the day's customers, while getting real time feedback with every burger flip.
Comedians have one of the fastest feedback systems available: instantaneous mass customer response, that they absorb and gauge in real time. Customers either laugh at a joke or they don't.
An average chef that gets no compliments can keep working. An average comedian that gets no laughs generally cannot. I think comedy requires positive feedback, whereas cooking does not require it (it's ideal, but doesn't have to exist, just OK will do for most meals in America).
Your average chef at an average restaurant - nobody knows who you are, the customer never sees you, they're not going to attach your name to bad cooking, and the customers don't come to the restaurant because your name is on the board. This is often true even at nice restaurants. As a comedian, people know who you are, and they specifically will remember that (good or bad).
If you're Louis CK, you're competing with yourself every night, the audience likely knows your past material. If you're a high level chef, typically you don't throw out all of your best recipes and have to replace the entire menu every year. You can't go on YouTube and taste every recipe by Gordon Ramsay.
If begging is the way somebody earns enough money to live on, is that not pretty much the same thing - I know people with normal jobs who do it "out of desperation", not because they love the job but because they need money and can't find any other job.
But that's besides the point; I took two random examples not to highlight them but along the logic that everyone does this in all walks of life. Without putting any scientific thought into it, I have, whether purposefully or not, tried and improved my methods for cooking, for playing sports, for smoking cigarettes... anything.
> I would say that comedians have always been among the top professional A/B testers
The logic behind this statement, in your later reply to me, being that it's much easier to be a mediocre chef than a mediocre comic, but that says nothing about the quality of the testing, all it says is that there is a much longer tail on the graph of quality in cooking. The fact that it is easier to be a mediocre chef than it is to be a mediocre comic really only speaks to the demand of both services, not how much testing they do or how good it is.
Homelessness is one of the hardest professions out there, since it consumes every waking and sleeping moment of your existence. Where do I sleep tonight so I don't get robbed or hurt? Where can I find something to eat? Where can I go to get out of the weather without getting into trouble? Where can I wash the dirt and grime off me that comes from the life. Homelessness is a constant process of learning and adapting for those homeless that possess the mental faculties necessary.
You picked a terrible example to try to make your point.
I have no idea if this is at all related to the topic (and I realize this reads like typical HN smarter/hipper-than-thou criticism, so I apologize in advance) but while I find Aziz Ansari to be hilarious in roles (Parks & Rec, his small role in Flight of the Conchords, etc), I've seen a few of his standup shows and didn't find his own material to be particularly funny at all.
That's too bad, I'm a huge(huuuuuge) Parks and Rec fan, but I happen to also really enjoy some parts of his standup. I think some of his jokes suck, but some of them just crack me up. I watch a lot of standup and while I wouldn't put him at the top of my list, I definitely enjoy him.
Also he has mentioned on one of his AMA's that he loves standup more than acting/anything else so it's too bad that some folks don't like his standup. Oh well, to each their own!
Steve Martin talked about his process making his comedy routine in his autobiography. His process was essentially A/B testing, although not as precise as this approach can be today.
Getting statistically useful data out of this will be difficult. A/B tests can tell you which of a set if options performs better for a measurable metric, but not why, and without no visibility into 'metrics' you can't easily measure -- of which there are a lot.
Here's a great 50 minute segment, with Jerry Seinfeld, Chris Rock, Ricky Gervais and Louis CK, in which they discuss various aspects of their material and how they work with it (adding / discarding / making adjustments and so on):
https://www.youtube.com/watch?v=sAxoRh06XM4