Nothing scales well. scaling requires lots of effort. It doesn't matter what language you use, you'll rapidly find all its pain points.
> bad packaging when there's a lot of cross-cutting dependencies
Much as I hate it, docker solves this. Failing that poetry or if you must venv. (if you're being "clever" statically compile everything and ship the whole environment, including the interpreter) its packaging is a joy compared to node. Even better, enforce standard environments, which stops all of this. One version of everything. you want to change it? best upgrade it for everyone else.
> slow performance
Meh, again depends on your use case. If you're really into performance then dump out to C/C++ and pybind it. fronting performance critical code in python is a fairly decent way to allow non specialists handle and interface performance critical code. Its far cheaper to staff it that way too. standard python programmers are cheaper than performance experts.
If we are being realistic, most of the time 80% of python programs are spend waiting on network.
Granted, python is not overly fast, but then most of the time your bottleneck is the developer not the language.
> no concurrency
Yes, this is a pain. I would really like some non GIL based threading. However its not really been that much of a problem. multiprocessing Queues are useful here, if limited. Failing that, make more processes and use an rpc system.
> typing as second-class citizens
The annotation is under developed. being reliant on dataclass libraries to enforce typing is a bit poop.
> People stuck in sprawling enterprise codebases, with O(million) lines of code to wrangle, seem almost universally miserable with the language.
I work with a _massive_ monorepo. Python isn't the problem, its programmer being "clever" or making needless abstractions of abstractions. None of that is python's issues, its egotistical programmer not wanting to read other people's (un documented) code. And not wanting to spend time make other people's code better.
>Nothing scales well. scaling requires lots of effort. It doesn't matter what language you use, you'll rapidly find all its pain points.
This is very important. A lot of people think that just using go or rust or whatever other language is new fixes all of this. But with a big enough project, you'll find all the issues. It's just a matter of time.
Do not miss that one will find all of the language's pain points. I'd wager that a dynamically typed languages such as Python has quit a few more pain points at scale than a more principled language such as OCaml.
I love Python's bignum arithmetic when I write small prototypes for public key cryptography. I love Python's extensive standard library when I'm scrapping a couple web pages for easier local reading. But I would never willingly chose it for anything bigger than a few hundred lines. I'm simply not capable of dealing with large dynamically typed programs.
Now if people try Rust or OCaml with the mentality of an early startup's Lisp developer, they're going to get hurt right away ("fighting the language" and "pleasing the compiler" is neither pleasing nor productive), and they're going to get hurt in the long run (once you've worked around the language's annoying checks, you won't reap as much benefit).
If you'll allow the caricature, don't force Coq down Alan Kay's throat, and don't torture Edsger Dijkstra with TCL.
This is somewhat true - scaling is hard no matter what - but some things scale much better than others. I have been miserable working with ruby on rails codebases that are much smaller than java codebases I have been content working on. This is despite personally enjoying the ruby language far more than the java language.
> Much as I hate it, docker solves this. Failing that poetry or if you must venv. (if you're being "clever" statically compile everything and ship the whole environment, including the interpreter) its packaging is a joy compared to node. Even better, enforce standard environments, which stops all of this. One version of everything. you want to change it? best upgrade it for everyone else.
No, docker doesn't solve the fact that some packages just won't play nicely together. NPM actually does this better than the python ecosystem too since it will still work with different versions of the same dependency. You get larger bundle sizes but that's better than the alternative of it just flat not working.
Scalability is not just runtime, it's also developer time scalability. The larger the project, the more you have to split it up and write interface documentation between libraries - which adds complexity.
As for processing scalability - Python is OK, but it's considerably hampered by BDFL's own opinions. The result is a few third party libraries that implement parallelism in their own way. That functionality should be integral to the standard library already. The worst part is lack of standard API for data sharing between processes.
> packaging
Python's packaging issues only start with package management. Setuptools is a wholly mess of a system, that literally gave me headaches for the lack of "expected features". I hate it with every single cell in my body.
And then there are systems and libraries, where you literally cannot use docker (Hello PySpark!).
>read other people's (un documented) code
I lolled! Seriously... We get Python fanboys moan about how indentation makes everything more readable and it's a pleasure to write code in python. Give me a break!
When I have to revisit old code I've written, I occasionally encounter my "cleverness" at the time. I always hate that past version of me. I think I've mostly learned my lesson. I guess I'll know in a few years.
"When I have to revisit old code I've written, I occasionally encounter my "cleverness" at the time. I always hate that past version of me. I think I've mostly learned my lesson. I guess I'll know in a few years."
I'm sorry, but as a fellow Python "devop" too, this really reads like empty apologism.
>Nothing scales well. scaling requires lots of effort.
Sure, just like all PLs have their flaws, and most software has security vulnerabilities. But it's a question of degree and the tendency of the language. Different languages work better in different domains, and fail in others, and what Python is specifically bad at is scaling.
If only for the lack of (strong/static) typing and the relatively underpowered control flow mechanisms (e.g. Python often using exceptions in their stead)... While surely all languages have pain points that show up at scale, Python still has a notable lot of significant ones precisely in this area.
>docker, poetry, venv...
Yes, and this is exactly the point. There's at least three different complex solutions, none of which can really be considered a "go-to" choice. What is Rust doing differently? Hell, what are Linux distros doing differently?
>If you're really into performance then dump out to C/C++ and pybind it.
If you want performance, don't use Python - was the parent's point.
>If we are being realistic, most of the time 80% of python programs are spend waiting on network.
This really, really doesn't apply to all of programming (or even those domains Python is used in). Besides, what argument is that? If it were true for your workload, then it would be so for all other languages too, meaning discussion or caring about performance is practically meaningless.
>Granted, python is not overly fast, but then most of the time your bottleneck is the developer not the language.
Once again, this applies to all languages equally, yet, for example, Python web frameworks regularly score near the bottom of all benchmarks. I doubt it is because of the lack of bright programmers working in Python, or the lack of efforts to make the frameworks faster.
>Python isn't the problem, its programmer being "clever" or making needless abstractions of abstractions.
Just as C isn't the problem, it's the programmer forgetting to check for the size of the buffer, and PHP isn't the problem, it's the programmer not using the correct function for random number generation.
You can always trace any given error to a single individual making an honest mistake, that's really not a useful way to think about this. It's about a programming language (or an environment) leading the programmer into wrong directions, and the lack of safety measures for misguided "egotistical programmers" to do damage. You can blame the programmers all you want, but at the end of the day, the one commonality is the language.
Now Python is still one of my favorite languages, and I think that for a lot of domains, it really is the right choice, and I can't imagine doing my work without it. But performance and large, complex systems, is not one of those domains, and I honestly feel like all you've said in Python's favor is that other languages are like that too, and that it's the fault of the programmers anyway.
I have thought about what you've written. I broadly agree. I didn't mean for my post to be a "python is great really", It was more to illustrate that all programming languages have drawbacks.
The is a points that I think I've failed to get over:
> Just as C isn't the problem, it's the programmer forgetting to check for the size of the buffer, and PHP isn't the problem, it's the programmer not using the correct function for random number generation
I don't think I was arguing that point. of course all languages have their USP. My point I wanted to get across is that large python projects are not inherently hard to manage. That kind of scaling is really not that much of an issue. I've worked on large repos for C, C++, python, perl, node and as a punishment, php. The only language that had an issue with a large codebase was node, because it was impossible to build and manage security. The "solution" to that was to have thousands of repos hiding in github.
The biggest impediment to growth was people refusing to read code, followed swiftly by pointless abstractions. This lead to silly situations where there were 7-12(!) wrappers for s3 functions. none of them had documentation and only one had test coverage.
> doesn't scale well.
Nothing scales well. scaling requires lots of effort. It doesn't matter what language you use, you'll rapidly find all its pain points.
> bad packaging when there's a lot of cross-cutting dependencies
Much as I hate it, docker solves this. Failing that poetry or if you must venv. (if you're being "clever" statically compile everything and ship the whole environment, including the interpreter) its packaging is a joy compared to node. Even better, enforce standard environments, which stops all of this. One version of everything. you want to change it? best upgrade it for everyone else.
> slow performance
Meh, again depends on your use case. If you're really into performance then dump out to C/C++ and pybind it. fronting performance critical code in python is a fairly decent way to allow non specialists handle and interface performance critical code. Its far cheaper to staff it that way too. standard python programmers are cheaper than performance experts.
If we are being realistic, most of the time 80% of python programs are spend waiting on network.
Granted, python is not overly fast, but then most of the time your bottleneck is the developer not the language.
> no concurrency
Yes, this is a pain. I would really like some non GIL based threading. However its not really been that much of a problem. multiprocessing Queues are useful here, if limited. Failing that, make more processes and use an rpc system.
> typing as second-class citizens
The annotation is under developed. being reliant on dataclass libraries to enforce typing is a bit poop.
> People stuck in sprawling enterprise codebases, with O(million) lines of code to wrangle, seem almost universally miserable with the language.
I work with a _massive_ monorepo. Python isn't the problem, its programmer being "clever" or making needless abstractions of abstractions. None of that is python's issues, its egotistical programmer not wanting to read other people's (un documented) code. And not wanting to spend time make other people's code better.