Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Dstat project ended due to RedHat replacing it with its own dstat tool (github.com/dagwieers)
168 points by tanelpoder on May 22, 2019 | hide | past | favorite | 152 comments


Having read through this post and the previous comments[1] I have to say that RedHat did the right thing in the completely worst possible way.

Stealing the project name was a mistake. The authors are right to be upset by the fact that people used to get their project when they did "yum install dstat" but now are getting a completely different (and not completely compatible) project. I think a lot of the friction could have gone away if they had just named it "pcp-dstat" or something similar, but instead they essentially took this person's namespace without discussion, and really for no good reason that I can tell.

The other issue is that forking a project without even talking with the people running it is a very touchy subject. Ultimately it does look like the project was dead- even to the point where people were asking to contribute but not getting responded to, and the fact that python2 is EOL and python3 support wasn't added until after this drama started shows that the project wasn't really "active". So while I do think they should have reached out, I totally understand why that slipped their mind.

So at this point I think RedHat should apologize and fix the package name.

[1] https://github.com/dagwieers/dstat/issues/156


It didn't just slip their mind, they did not really care: https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c7

The decision to do this was made before June 2018, 18 months after the last activity. One of the reasons cited was lack of activity, but that is no thanks to them, I guess.

That is why I am convinced the goal was to replace it from the onset, there was no interest in helping out the project. In fact "no activity" was the right excuse to make their action seem legitimate. Attempting to contact the project could have jeopardized that plan.

They could have just removed "dstat" from the distribution, and added a note that users can now use pcp-dstat. But now they made it impossible for users to add the original dstat. Let alone the support nightmare of having a different tool with the same name. There's no winning this one.


I totally agree that they should have left a note on the issue tracker - even if they thought it would go unread. However I think you're being a little unfair on this one point.

>there was no interest in helping out the project.

They saw that people were filing issues and making pull requests and that those people were getting total radio silence. I can understand how someone might think "well, people are already trying to 'help the project' and the maintainer isn't even acknowledging it, so it's a waste of time".

And while again I totally understand and agree that they should have tried to contact you, you've known about the removal for like 9 months and you apparently never reached out to them either? Like, a couple words in reply to a bugzilla or an IRC message does not amount to picking a fight with a company.


Huh? Jumping to conclusions while not knowing all the facts. Please read up on the discussions that we had _after_ Red Hat made this decision. Their decision was made, end of discussion.

I am sure they made a sound business decision, and I think as a result of that I made the right personal decision. And here we are now having this meta-discussion with people not having a clue. Welcome !


You knew that it had been removed from Fedora months before they announced it was going to be replaced in RHEL. You seem totally confident that they wouldn't have just added the package back once you showed back up and addressed the issues that lead to it's removal. Maybe that's true, but I'm not sure it is.


There was a discussion in the Github issues. From that it was pretty clear this was a done deal. If the project is no longer maintained, why would they. And they had the PCP reimplementation ready.

You seem to imply there was no communication, and I stopped the project out of the blue. That is a misrepresentation.

Nobody stepped up to take over maintainership, and I don't see anyone doing that now. But if someone wants to try, I can unarchive the project and restore the PRs and issues.

If not, the king is dead, long live the king!


While Redhat did not handle this situation well you are also being very unfair to them. Why would they waste their time trying to contribute to a dead project? If I see that patches do not get merged and that bug reports go unanswered I will not bother doing anything because I know my time will be wasted, and I do not fault RedHat for doing the same.

What they should have done is tried to reach out to you before forking.


Unfair in what way?

They included in RHEL and sold it as part of their product when it suited them, and now they have replaced it because it suited them. That's fine, I don't have to agree.


Are you the person that used to maintain a bunch of rpm(s) ? Thanks a lot for doing that, it helped me a couple of times


Thanks :-)


I disagree, they were right to use the same namespace and its questionable how you conflate that to stealing. If something is abandoned, how can you steal it? The project was dead, PR's and issues were ignored and the lack of Python 3 support was a huge issue (which the developer pathetically tried to shut down discussion over by releasing a python 3 port just hours before closing the project).

Changing the name space would have been problematic, as so many people use automation tools now to install package dependencies, rather than break their work because someone has zero intention of maintaining a project, they did the right thing and used the same namespace.

Had this been a case of RH disagreed with the developers direction or any factor around an active project, it would have been a shitty move, but considering the developer ignored any offers to help and let the project to fall into a stale un-maintained state, I don't see any grounds for him to throw his rattle out of the pram and blame RH here.


Nobody stepped up to maintain the project, not even Red Hat. Some people have offered help, and I have send out a few GitHub invitations to people that offered, but no takers.

I assume people offering to help only wanted to see their own contributions merged.

I shouldn't be surprised people here take positions while not having been involved in the project or know the whole history.

In my opinion, stopping the project was the only sensible thing to do at this point. Red Hat replacing the tool with one of their own (which I wholeheartedly disagree with for various reasons) created a dead-end. It's the final blow to a dying project.

Wrt. Python 3 support, I simply did it because it was so easy to do, not because I thought this would be a game-changer. But hey, please do read into whatever fits your narrative :-)


You've misunderstood. The "dstat" package was removed from Fedora, rather than have its contents replaced. The package containing the new program is called "pcp-dstat", so they already did everything you think they should have done.


What makes you think he misunderstood? What happens on fc30 when you 'yum install dstat' as he suggested?

This webpage listing the contents of the fc30 pcp-system-tools package: https://fedora.pkgs.org/30/fedora-updates-x86_64/pcp-system-...

lists a file called '/usr/bin/dstat'. The pcp-system-tools package obsoletes a package called 'dstat'. If you go to https://pkgs.org/download/dstat, and look under "Fedora 30," you get pcp-system-tools.

[Spoiler: 'yum install dstat' installs pcp-system-tools, which has a symlink /usr/bin/dstat -> /usr/libexec/pcp/bin/pcp-dstat.]


An alias to the replacement is not the same as silently replacing the package. Yes, the user who care will notice - it's a tool for sysadmins, and the ones worth their salt are already familiar with precisely this provision in package managers, and it's included for precisely this reason. There's always politics involved in replacing a tool, but they didn't steal his namespace - they made the decision to package their own alternative, and yes, that editorial decision does lie with the distribution.


To quote a comment from a dstat collaborator from the above linked thread-

> That said I don't fault you for make a new tool that does all the new fancy things you mentioned above, I do think it's a little sketchy to make a new tool called dstat without at least asking here first. I know that when I hit Fedora 29 and did a yum install dstat and I get a bunch of pcp stuff I was pretty confused.

The way they have things aliased right now, if you type in `yum install dstat` you will get this new program (including all of its baggage).


Apparently reusing the name without talking to the author was a conscious decision on their behalf-

https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c7


meh. ultimately this is OSS. the maintainer was negligent. if you're not a good OSS maintainer, you don't deserve to maintain control. it's not reasonable to be indefinitely patient when the original author shows no initiative to maintain their project


They didn't even try to contact the author?!


dstat was basically presumed dead due to lack of upstream activity. I recall the internal discussion was something like

A: "Whoa, there is no dstat in RHEL8? Customers are gonna complain."

B: "Yes, it's because it's Python 2 only and upstream is dead. Tell customers to use PCP"

A: "Srsly? Nobody thinks of the scripts? Besides, dstat output is so nice."

C: "Well, there is some toy example to print pcp data in dstat format."

A: "Do we really want to tell customers 'there is a toy example'?"

C: "Give me a day or two to pimp it up."

And then C got a bit carried away. :-)


Why would you try to contact someone who's abandoned the project?


Maybe because it is the nice thing to do?

And it's not like I have disappeared from the face of the earth. I am quite active on GitHub.


Are all of those people commenting Red Hat employees ?


Well, I am at least. I care about when things like this happen, including figuring out how we can do better.


Start by fixing this issue- make it so `yum install dstat` installs the real dstat program.


Well, see below. Dag wants to just move on; the "real" dstat program is now defunct even if it wasn't before. So, we'll just keep things as they are for now.


I submitted this PR long time ago: https://github.com/dagwieers/dstat/pull/50 that fixed an important installation issue, and it took the author 5 months to look at it and simply close it without providing any hints how to fix the problem in some other way. So I was discouraged to contribute further to the project.


You added a security problem. The fix is obvious for everybody working in security, but it's questionable if it should be discussed there.


I can see how the original CVE patch removed using the current working directory for loading plugins, but it seems like less of a security issue to simply look up one directory relative to the current binary install, assuming the binary is stored in ./bin after being built and plugins would be stored in ./shared ... Now, it might be inconsiderate to look up a folder when one could be preconfigured based on a common root or home folder, but it doesn’t necessarily seem as bad to trust the parent folder of a binary as trusting the current working directory’s contents?


This isn't relative to the binary install, it's relative to argv[0]. It's the same security issue -- that argument is under user control. You can make a symlink to your target wherever you want and then argv[0] will be the symlink location. I don't think there is any secure way to get the current script path in Python.


so you install dstat in /usr/bin/dstat + /usr/share/dstat (because you are root), and an attacker creates /home/eve/bin/dstat with /home/eve/share/dstat/evil.py. why would you run /home/eve/bin/dstat? if eve can get you to run dstat from here ~/bin, why wouldn't she just have ~/bin/dstat with completely different contents?

i'm still convinced this is cargocult security.


You're right that this wouldn't matter if the program is run with the same privileges as the caller. I was imagining if that wasn't the case (e.g. the user gives dstat setuid permissions), then letting it load arbitrary code could produce a security hole. Admittedly I didn't think too hard about whether/why something like this might be done -- if it wouldn't be, then never mind. As a matter of general safe practice I err on the side of caution, i.e. not depending on argv[0] to have any particular value for the correctness of the program, because otherwise you have to think through all the possible attack/error scenarios and likely document them for the user, etc... and I thought in any case it was worth pointing out that argv[0] did not necessarily correspond to the program location regardless.


Edit/Update: You're right; I'm wrong -- I don't think this is the concern. Namely, I completely missed that the bug was already there in another line, and that the home directory was also being scanned. So it can't be a privilege escalation issue. Moreover I forgot that setuid doesn't work for scripts, so that shouldn't be the issue either. Not sure what's going on now either; I'm confused too.


Would something based on __file__ be more preferred, then?

If so, it could be argued the bug already existed and this patch simply uses the existing behaviour...


I don't think __file__ would be any different... it's not even an absolute path necessarily. That's why I said I don't think there's any secure way to do this in Python.

You're right though, I didn't see the issue was still there. The home directory should also not be there either. I think I looked at this too hastily but now I'm confused too.


    os.path.realpath(__file__)
That will give you the absolute path of the current python script, dereferencing any symlink and the .. notation.


It will also be insecure due to a race condition.


How do you arrive at the conclusion that looking at `./plugins/` is bad, but `../shared/dstat` is more acceptable? At first blush, the attack vector would seem to be exactly the same. Adding the extra layer of indirection does not seem, to my eyes, to add any significant extra layers of security. Both seem essentially untrustable from the perspective of the binary. Can you help me understand what I've missed in your explanation?

I can also see where someone might not want to endlessly re-debate an old security decision.


Sure. The original security bug was lines 31-32, but the new patch referring to the parent directory is a variation of line 30, untouched here: https://bugs.gentoo.org/attachment.cgi?id=210509&action=diff

CWD is far worse a risk, because it changes every time the command is run. By comparison it could be assumed that a secure install of the software is in a trusted location, and this just gets the parent root of that location. Of course, it’s better if it’s documented, but I see no difference between this and $JAVA_HOME’s approach of ./bin, ./share, etc. It could be argued this is better—no environment variables can override the default. If you install the app to /tmp/bin that’s your own fault...


Ah! Thank you for clarifying.

That said, going up a directory would seem to imply that dstat could easily be loading completely arbitrary code from a directory it has no exclusive claim to and thus cannot meaningfully trust. Indeed, as I understand `share/` that is exactly the point. This really doesn't strike me as being a safe thing to do, and I can readily see why the author would reject it on security grounds. This seems to me to be a distinction with little meaningful difference, though I can see where some people might disagree.

As I tell PMs, designers, and engineers I work with far too often for my own comfort, someone else's poor security decisions are no excuse for our own.


If absolute share/ access were the problem, the line below wouldn’t hard-code it twice: https://github.com/dagwieers/dstat/pull/50/files

The only bug in this PR that I can see is it might assume the system plugins are more important than the local overrides. Generally you’d put this sort of path in the least priority, to allow for /usr/local to override. That said, obviously hard-coding paths is worse than simply having a default, presumably relative to the current binary/file, and allowing end users and packagers to override/configure the defaults to suit their security preferences, with the default being that the entire package is unzipped with expected relative paths to files... or that’s how I would look at it. The only way to greater security would be to ship a Docker container, Snap package, or similar and mount your own filesystem overlays. :) Or, perhaps keeping a trusted list of plugins somewhere.


My read is that it's more that access control of `../share` in an unknown part of the filesystem is a matter for some concern. Given that the binary can be put basically anywhere, it would seem to be perilously close to CWD.


no, it's just cargocult security.

edit after your reply: if you install in /tmp, you'll end up with /tmp/bin/dstat and /tmp/share/dstat. you're concerned that an attacker could smuggle something into /tmp/share/dstat, but /tmp/bin/dstat is of no worry? what exactly is the threat here?

> access control of `../share` in an unknown part of the filesystem is a matter for some concern. Given that the binary can be put basically anywhere, it would seem to be perilously close to CWD.

another edit since i cannot reply to you: do you have any examples of the "threat model [which] includes that you can't trust every part of the filesystem you're working from"? something concrete, specific. a particular install prefix that would let you create $prefix/bin/dstat but $prefix/share/dstat would be dangerous.

aaand, see my reply at https://news.ycombinator.com/item?id=19989237


I don't know. If your threat model includes that you can't trust every part of the filesystem you're working from, being extra paranoid about it isn't the most unreasonable thing in the world.

EDIT: A sibling points out the issue in more detail - https://news.ycombinator.com/item?id=19989237


Devil's advocate: Isn't this kind of behavior how we ended up with what became the POSIX command-line utilities? A bunch of people re-implementing and extending each other's stuff without changing the name?

I know that the word has changed a bit since then and that it's a bit different to appropriate a soul-less corporation's command names vs an individual volunteer.. It just happened to strike me as odd at the moment I came across this thread that we're so up-in-arms about this when we're all happily using utilities that were produced by the very same process every day.


I don't know that massively popular utilities were ever "reimplemented" with the same name. Ported, or forked, or extended, but not reimplemented. It's not clear to me if pcp-dstat is a fork or a total rewrite.

Also, most of those utilities were named things which solely described what they did. It's a bit different to make a new "cut" vs a new "clhodapp".


Then there is a lot of history that you do not know, starting with BSD and progressing through the late 1980s and early 1990s when there was a torrent of "clones" of things.

The "Vixie cron" that you might even still be using is a "PD" clone of the Unix cron program, reimplemented from scratch with the same name. The van Smoorenburg "init" that you might have heard of was a clone of (an at the time sadly already out of date) Unix init program. All of the stuff in GNU coreutils and other similar GNU toolsets was a deliberate reimplementation from scratch.

And then there are the Thompson, Mashey, Bourne, Almquist, Bourne Again, Korn, Z, and Watanabe shells. And BusyBox and ToyBox. And /usr/lib/sendmail . And the "mailx" utility. And "awk". And "vncviewer". And "getty". And so on.

* https://unix.stackexchange.com/a/508142/5132

* http://jdebp.uk./FGA/inittab-getty-is-history.html

* https://news.ycombinator.com/item?id=19988382

* https://news.ycombinator.com/item?id=17005677

* https://unix.stackexchange.com/a/316279/5132


I'd like to add the common aliasing of vi->vim


You're too late. It's the fourth item in the first Stack Exchange answer. (-:


almost every BSD tool is a rewrite of the original unix tools, and every GNU one is a separate rewrite too.


this strongly reminds me of the hn post from five days back in which the main thread talked at some length about a very similar scenario

> "He missed a big one: you have no way to stop Linux distributions from hacking up your software, and you'll suffer the consequences of whatever they do."

https://news.ycombinator.com/item?id=19935648


Yeah I thought of the same comment. That is unfortunate.

Maybe Debian is more respectful since it's open source not backed by a company with enterprise customers?


"respectful" as in let's wait years for a maintainer to come up with a solution and when we try to propose our own the maintainer trows a fit and turns it down?


Is that what happened? I remember it differently...


Eh. Although we don't always do things perfectly, Fedora has a pretty strong policy of working with upstreams first.

By contrast in my personal experience as a developer of something that got packaged in Debian, I discovered later that that package had a bunch of patches added, a man page written (!) and so on, none of which was bad, but also none of which was fed back to me at all.

(In case not obvious: I work for Red Hat on Fedora.)


Not really, remember the ssh key fiasco which was also due to a Debian-only patch.

(Disclaimer: I work for Red Hat)


So, from a Fedora Project point of view, I'm not really sure how we could have done this better. This change was properly announced and publicized as https://fedoraproject.org/wiki/Changes/MergeDstatAndPerforma... and approved by FESCo (the Fedora Engineering Steering Committee) in August 2018 (https://pagure.io/fesco/issue/1956).

The change page says "The original dstat utility has reached end of life - it does not support python3 and there are no plans to update it." This may not have been right after all, but I don't think it's in bad faith. Perhaps FESCo should have dug a little deeper, but there aren't really any particular red flags indicating that that's warranted.

I'm open to suggestions, though -- we do want to continually improve our processes, and we do want to work well with upstreams. And, y'know, be better to long-time Fedora ecosystem friends.


Well, I had to find out this was planned and acted on after the decision was made and the alternative was written without any regards of the original project.

And that is fine if it would stay pcp-dstat as-is.

The most upsetting to me is that Dstat is no longer a python tool you can drop on a JeOS, Synology NAS or a WRT router to get it to work, you now have to install PCP and all its dependencies to make it work, which goes against the original design goals. (i.e. we used to support Python 1.5 for a long time to accommodate RHEL2.1 during its life-cycle)

Also, by taking the Dstat code, removing the plugin mechanism and replacing it with a PCP backend, writing a python plugin for Dstat is no longer possible. This is promoted as being a feature as "your plugin is now a config file" which is a bit disingenuous as you have to write a PCP backend which is a lot harder.

And it is not even a drop-in replacement, it only implements the built-in counters, not the full set of plugins (i.e. --top plugins are missing). So the argument that it needs to work as-is for existing customers does not hold true either.

By taking that name (with the Red Hat clout) there is no chance of anyone taking over maintenance without having to deal with 2 products using the same name, which I guess is forcing your wish on other distributions too.

Wrt. the change was properly announced. I bet the checklist was properly checked. Or to quote Douglas Adams:

“Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.”


Well, we do try to do better than the Cottington planning department. Like all changes, it was posted to devel-announce (https://lists.fedoraproject.org/archives/list/devel-announce...) to get more visibility than just a devel-list post (or a random wiki page).

In retrospect, it would have been nice for the devs working on this to contact you directly and explicitly about the proposed change. I'm sorry that didn't happen.

But here we are now -- at this point, what outcome would you like?


It has played out as it did, that's why I closed the project.

I don't have any wishes. Just a bad aftertaste but that will go. I don't like to dwell in the past, regardless of the fact I have to defend myself in this forum to total strangers who seem to know better :-)

And that will pass as well. We'll see...


Okay. Again, I'm sorry it came out like this. I would like us to do better.



The claim is that the original is no longer being maintained. Is that true? It changes a lot about what I would think of this move.


The issue below has more details, it does look more complex than this post suggests?

https://github.com/dagwieers/dstat/issues/156


From the git changelog, it doesn't appear to be completely unmaintained, but not very vital either: https://github.com/dagwieers/dstat/commits/master

Red Hat's announcement (posted in February 2019), also cites lack of Python 3 support as another reason for replacing the original dstat, but Python 3 compatibility was added last January.


Yeah, I added python 3 support after I learned that was the main reason for replacing it. It took me less than an hour to make it work on Python 3, including all (but 2) of the plugins.

And given that PCP is using most of the original code, they must have made the same changes to get Python 3 support.


maybe you shouldn't have waited until the absolute last minute to do this or it wouldn't have been replaced.


I am not sure what you are implying here. I have zero interest in maintaining the project. This was the last blow to a dying project.

The king is dead, long live the king.


pcp-dstat was implemented in 2018, FWIW. The blog post may have been written prior to the January 2019 dstat commits (before which the last commits were in 2016).

More color: https://github.com/dagwieers/dstat/issues/156


The package was removed in Fedora 29, released October 2018.


Yes, and the plan to do this was conceived before June 2018.


No, the original was finished.

Feature complete, bug free software doesn’t need to have patches released every week. That is the ultimate goal of a small utility like this.

It’s a shame that so many developers mistake this state for “unmaintained”or “abandoned”.


The software isn't finished if the runtime it runs on has been EOL'd.

dstat, up until earlier this year (a couple weeks before that announcement), did not support Python 3 - and the author may have just merged RedHat's changes anyways. RedHat's "fork" supported Python3 for months at that point.

Yes, it was a useful tool, and what RedHat did was shitty but I can understand the motivation for not wanting to keep a Python2 install around just for one tool.


> I can understand the motivation for not wanting to keep a Python2 install around just for one tool.

I don't use a Red Hat distro but would this really only be one tool? On Arch I see the following packages depend on python2: mercurial [required], git [optional], texlive-core [optional], graphviz [optional], ...


RHEL8 doesn't include a Python 2 runtime that is supported for the whole life of the distro. There are a couple exceptions where Python 2 is used at build time, but that's it.

At the point when people started to work on pcp-dstat, it was clear that if you wanted X in the base distribution it had to be Python 3 compatible.


Arch doesn't promise support. Red Hat does. If software in Arch is unsupported in the near future but currently works, that's less of a problem than software in RHEL becoming definitely unsupported during the release lifetime.


Yes but that doesn't really answer my question. My question was if these programs use Python 3 then why would Arch -- which generally has the latest versions of programs unmodified -- have them depending on Python 2. You just explained why depending on Python 2 is not a problem... which is answering a different question.


Anything python2 is a problem, because Python 2 is officially end of life in January 2020. It is being dropped from all major linux distributions, along any software that depends on it.

RedHat 8 that just got released this month doesn't have it. Not sure when is the next release/update cycle for Arch, that will drop it. Keeping in mind that Arch probably doesn't care about doing things last minute or even after the due date.


Mercurial 5.0 is Python 3 compatible. The Python code in Git is Python 3 compatible. I don't know about TeXLive or Graphviz, though...


Confused, so why is Python 2 a requirement?

  Name            : mercurial
  Version         : 5.0-1
  Description     : A scalable distributed SCM tool
  Depends On      : python2
  Optional Deps   : tk: for the hgk GUI [installed]
  Required By     : None
  Optional For    : None
  Conflicts With  : None
  Replaces        : None
  Installed Size  : 25.95 MiB


Possible options:

1. Some of mercurial's bundled plugins are not Py3 compatible (I recall reading a discussion as an outsider a while back that some plugins are planned to be ported to Rust, not Python 3, but that may have changed) 2. Nobody involved in packaging it for Arch has had time to verify mercurial 5 on python 3.


So looking into this a bit more (and my edit window has expired), it's because the mercurial team themselves suggest not to package mercurial 5.0 for python 3 yet:

https://www.mercurial-scm.org/wiki/Release5.0


Weird that it says Windows is the issue but I'm seeing this on Linux.


Python 2 is EOL? Interesting. https://pythonclock.org/


I don't understand what you're trying to do with your comment. It might not be EOL quite yet, but 7 months into the future isn't very long.

Especially since RHEL 8 is gonna stick around for the next decade, it's a good idea to move as much stuff as possible to a newer platform that will continue to have security updates at least provided and written by the authors of the platform.


The git repo linked here has 6 commits on Jan 16, 2019. And Nov 23, 2016 the last new commits before then.


To contrast, Redhat's pcp dstat looks like it was mostly developed during 2018 (a period during which dstat had been totally inactive for 2-3 years):

https://github.com/performancecopilot/pcp/commits/master/src...


18 months


oh god. it's based on pcp o.O


Why is this significant enough to call it out without explaining why you did so?


Even if it isn't being maintained, I wish they had changed the name if they were forking or reimplementing without permission. There's a reason trademark law exists and doesn't require registration if you can prove the situation.

But yeah, it's a lot harder to fight Red Hat (soon IBM!) over that if the project is inactively maintained and there's no registration.

Still, whether legal or not, the name squat is not a good look for Red Hat.


> I wish they had changed the name if they were forking or reimplementing without permission.

Has the name not changed to "pcp", or am I missing something? "pcp dstat" appears to be a command, but that seems like a reasonable abbreviation for "hey pcp, please give me a dstat-like interface".

[Edit] Correction. https://github.com/dagwieers/dstat/issues/156 says the following, so there is a compatibility symlink using the name "dstat". However the project name appears to be "pcp-dstat" or "pcp dstat".

"Since there had been no meaningful work on the old dstat code in years, github PRs were being ignored, community developers requesting access to help were being ignored, and we have customers using it ... we were asked to make this as seamless a transition as possible, provide that compatibility symlink for /usr/bin/dstat to pcp-dstat, and remove the old package from RHEL and Fedora."


> community developers requesting access to help were being ignored

This is not true, I have not been contacted. I learned from Fedora's decision to replace Dstat with PCP months after it was already decided.

So it's not like I have had a choice. The choices I have today are:

1. Continue with a project, while Fedora/RHEL is shipping a tool by the same name (with 90% of my code)

2. Rename the original Dstat project, which would be silly

3. Discontinue the project

Option 3 is the path of least resistance. Option 1 and 2 would not bring any joy. At least someone will be paid for maintaining the tool.


GitHub PRs being ignored seems totally true.

https://github.com/dagwieers/dstat/pulls?q=is%3Apr+is%3Aclos...


Sure, and Red Hat being paid for RHEL shipping Dstat for a decade could have helped out. But instead they decided to replace it.

And as a result I don't see a point continuing a project with the same name.


>>>> community developers requesting access to help were being ignored

>>> This is not true

>> GitHub PRs being ignored seems totally true.

> Sure, and Red Hat being paid for RHEL shipping Dstat for a decade could have helped out. But instead they decided to replace it.

It's one thing to be upset due to a belief that Red Hat/others are at fault, but why lie to make your point?


Who lied about this ?

It's pretty clear we didn't accepted any PRs since December 2016 until Red Hat decided to replace our code, which started early June 2018. That's 18 months.

If I am upset about anything, it is this: https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c7


> Who lied about this ?

> It's pretty clear we didn't accepted any PRs since December 2016 until Red Hat decided to replace our code, which started early June 2018. That's 18 months.

You are asking: "why didn't RH help out", you also didn't accept any pull requests. That surely would answer the question about why they didn't bother with pull request.

The question is then, how exactly do you think they have helped?


And where is the lie?

Maybe you should read the announcement and leave it at that.


Which PRs are you talking about? PRs I see there are either all from the last 4 hours or from 2016 and earlier that were merged.


4 hours ago was the last update to the PR, the pull requests are from years ago.


Oh wow, didn't realize. Yeah I see now. Thanks.


As part of archiving the project, I closed all open issues and PRs. That was what Github recommended me to do and I deemed best for everyone involved.


But isn't red hat calling their version 'pcp-dstat' and not simply 'dstat', so they aren't shipping with the exact same name.


They install a symlink from plain "dstat" to pcp-dstat. That's the exact same name part and plausibly objectionable.


That is done so that existing scripts keep working. Nobody complains about the LLVM linker installing itself as /usr/bin/ld, or systemd providing /sbin/init, do they?


Actually, yes they did. The use of /sbin/init was something that came up during the Debian Hoo-Hah. The conclusion was that it should be a symbolic link, and that there should be distinct packages providing that link and providing the real executable program image file that it points to.

* https://packages.debian.org/search?suite=jessie&searchon=con...


That's exactly what pcp-dstat does. /use/bin/dstat is a symbolic link.


I understand why they did it, but also why the author is complaining about it.


I am not really complaining, maybe unsatisfied of the direction it took and the consequences to other distributions without having a say.

But maybe people fancy the new PCP Dstat and accept their losses. In any case, money rules the world, Open Source ran by volunteers is dying and becomes less and less attractive.


The entire principle of a GNU based system is that everything is a reimplementation of original Unix tools with the same names but different provenances.


Like I said in a sibling comment, I understand why they did it, but also why the author is complaining about it. :-)


I only know enough to get me in trouble, but pcp is a general "framework" for performance monitoring... they have pcp versions/rewrites of all types of traditional monitoring utilities like vmstat, iostat, top.

Note: they don't overwrite/replace those utilities on the command line. You can still use the traditional utilities same as it ever was. The pcp versions are called like so: "pcp dstat" or "pcp atopsar".


Reminds me on moreutils parallel (C), hijacking GNU parallel (perl), and changing the command line options. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050


This sort of thing happens regularly. For another example of many, witness the effects on users of the change of the "mailx" command in Debian.

* https://unix.stackexchange.com/q/469780/5132

* https://unix.stackexchange.com/q/489477/5132


Honest question: aside from "they provide Enterprise support", does anyone have a reason they would actually use redhat or centos for new servers? Why not use a Debian-derived one, which seems to both be more common, and have less constant creepy corporate influences?


Disclaimer, I work for Red Hat. That being said...

The big things for our customers isn't just support, it is stability. When you build your systems on RHEL 8 (just released), you will get 10+ years of support and patches for that system. Those changes are all guarenteed to be binary compatible, so the risk of patching ever impacting systems is low. If there is a critical security vulnerability (say Heartbleed in openssl), Red Hat will backport the fix to every supported version of RHEL. That means you get the security fix without (1) having to upgrade the entire openssl dependency and risk breaking something else and (2) without pulling in newer changes that might have their own vulnerabilities.

Going along with the stability, vendors can QA their products against RHEL as a known target.

Last reason for now, if you are in an industry with any kind of compliance requirements, like FIPS, Red Hat jumps through the hoops to get the certification. That makes RHEL the only option in many environments.


I'd say both stability and update.

Debian never upgrades anything (unless security issue). All packages are frozen at the time of the initial major release. Have fun using curl/htop/nginx/libreoffice from 5-10 years ago, plenty of obscure bugs and missing command line flags.

RHEL actually maintain their system packages.


> which seems to be more common

I have exclusively used the Fedora line in enterprise environments (CentOS, OEL, RHEL).

The point about long term support is definitely #1.


We had a vendor that we bought software from require, as part of the software requirements, both Red Hat and Oracle. This is not unique as many small specialized software shops do the same. I still have some government software that requires Windows 2008 and no other version is supported.


Extremely long life span (10 years), support if you want it, great community, great package manager, and in rhel 8 really great new features like appstream.


Curious, i know dagwieers from being involved in the Ansible project (RedHat owned). Seems friction is occurring, given the language in his post.


What language ? It is being quite factual really.


Time to go to Puppet Dag? :)


:D


I literally use dstat all the time! This is very sad.


Why? What has changed? Do you use the plugin functionality of dstat, or do you just use the dstat command? If it's the latter, then you can do everything you could with dead-dstat and more with pcp-dstat.


It saddens me when a useful, good open source project closes. I realize I can still use it, and I will... That's not the point.


This is undoubtedly shady behavior. dstat is a widely regarded and mature tool. Looking at its repo for 'activity' without context - maybe nothing needs to be changed and its working as designed - is completely disingenious.

Why should repo activity without context be used as an indicator of anything in discussion instead of focusing on what Redhat has done?

This is openly hijacking an open source project by a billion dollar company because it can and makes a mockery of not only open source colloboration culture but basic professional behavior. Has Redhat reached out to the author, made any requests, tried to work out some way forward, offered to pay for the brand name? Cmon this is simply indefensible.


>Looking at its repo for 'activity' without context - maybe nothing needs to be changed and its working as designed - is completely disingenious.

>Why should repo activity without context be used as an indicator of anything in discussion instead of focusing on what Redhat has done?

"Activity" doesn't just refer to commit traffic. People were filing issues and making pull requests and not getting any responses for more than a year. And furthermore it was a Python 2 tool and the Python 2 EOL date is fast approaching.

I agree that the packagers should have at least left a note on GitHub (even if they thought it would go unread) - but clearly it was not a "maybe it's just mature and nothing needed to be changed" situation.


This is muddying the waters. Surely if you have gone through the trouble of looking at the repo and finding these pull requests you would also know the context and should then share it to give others here a context. Without that you are just here making 'vague' charges given dstat is a well known, highly regareded and mature tool. The Python3 port was already done by the author so the python2 eol seems a complete non sequitor. Again a rush to condemn without good reason.

Are you saying anyone running a Python2 project can expect to have Redhat reimplement it in Python3 secretly and have people defend it on HN? This kind of behavior is simply indefensible.

It's obvious there is a culture clash of open source developers sponsored or working for corporates conflating heavy activity on Github that they are paid to perform as their day jobs as the only open source model. But open source was traditionally about people having other jobs and using their spare time to develop open source without profit motives because they believed in the movement. They certainly did not face corporates and paid developers analysing their frequency of contributions to dismiss their efforts and justify an unethical takeover.


"dstat is a widely regarded and mature tool. Looking at its repo for 'activity' without context - maybe nothing needs to be changed and its working as designed - is completely disingenious."

Its based on a entire language which is about to go EOL (Python 2)


I have been using dstat for years. It's a great tool. But why kill the project just because some company has a project with the same name? There is more than one distro, you know


The project was already dead. But now the maintainer[s] found time to complain about how evil RH simply opted to provide a py3 compatible replacement and provide it as dstat.


That's not exactly what happened, but great to see you are making improvements in your fiction-writing skills :-) Keep it up !


It is the Big Gorilla, the silverback if you like. For other distros that would be even more confusing when some of their tools will be ported there.


dagwieers, sorry this happened man that really fucking sucks. They should have done this differently and reached out to you for communication and stuff. Pretty disappointed with them for this. Hope you keep on and find other projects that excite you to work on.


Why not come up with a new name for the project instead of rage quitting?


> Why not come up with a new name for the project instead of rage quitting?

Why should he change the project name when he originally have it first?

Also why did you have to frame it as rage quitting?


It’s the way of the world that larger fish get to do as they please.

A simple name change and he can continue to do his work. It’s the idea of turning the other cheek. That or trademark the name I guess.


> It’s the idea of turning the other cheek.

I had to look this up.

> refrain from retaliating when one has been attacked or insulted.

I don't think he have retaliated in anyway, at least from the github commit message and the blog post. He just decided it wasn't worth his free time to deal with this problem. Maybe there is a disagreement with the definition of rage quitting but it was his open source project and his free time. I don't think it's rage quitting if he wants to quit and do something else with his free time.

Rage quitting have a negative connotation such as flipping table or that scene from the movie Half baked where the dude cussed everybody out before quitting. He seems to have a career, whether you care or not, labeling at such may be detrimental to his career.

Regardless of if you care or not, I hope other people refrain from speculating or do huge leaps of conclusion base on little or no facts that can damage other people's ability to make a living.


Would redhat ship dstat under the new name?


They should have shipped the original devs version or forked it and worked with him to get it supported and submitted their RH changes upstream etc.


It was already dead, so pragmatically there was nothing to do, this is just a swan dance on RH's bugzilla.

Yes, RH should and could have filed and issue saying that they started working on a fork/port-to-PCP, and they intend to provide the dstat executable. But usually at that point, the work is already started, and there's almost nothing to be done. How long RH should wait for that ticket? What if the maintainer says, wow, cool, I'll make a py3 version in 3 months. But RH needs it in 2? What if the maintainer disappears again?

It's not great, but it was abandoned. :(


It wasn't abandoned, it didn't see any maintenance for 18 months.

I don't get paid for doing this work. Red Hat does. They get paid for offering it to customers that apparently demands that it ships with RHEL8. They have been shipping it with RHEL since RHEL3.

They haven't contributed to it ever in that timespan.

They could have paid me for maintaining it, they could have offered to maintain it for me. (Because it is a lot of work to accept and verify pull requests). I am a freelancer, I have to work for a living I don't have a lot of free time.

Instead, they ripped out the plugin backend, added the PCP backend and now you can't use it as a drop-in tool that can't run PCP. Your synology NAS, WRT router or JeOS platform. Because Red Hat does not care about anything other than RHEL. They are business-oriented. And that's fine.

Is the Open Source ecosystem better off ? I sincerely doubt it. It's being replaced by paid-for engineers with business interests. In the long term it is killing the community.


Thanks Dag alot for the dstat tool, I am using it till this day. As for opensource and community I think it would be reasonable to still have it under different name if you'll consider it after some time and still develop some functionality or curate and merge pull requests.


Performance monitoring tools for enterprises can be very lucrative. Surely they’ll move to an “open core”, closed source commercial license for the useful bits, if they haven’t already.


That isn't Redhat's model. They open the code (it's mostly 3rd party OSS anyway), including packages (hence, CentOS) and sell access to the updates system and support contracts.


That isn't Redhat's model.

Well... it hasn't been so far. But this isn't the old Red Hat anymore. This is IBM Hat. And for all the talk about keeping Red Hat mostly independent or whatever, I think we all know that's bullshit. Acquiring companies always give that little dog and pony show, and it never holds up in the long-term. There's no way that IBM culture won't wind up infecting Red Hat. Now whether that's a good thing or a bad thing is a question I'll leave to the philosophers. But I'd be very leery of assuming anything regarding what RH will or won't do going forward.


Every single thread someone has to bring it up. The acquisition isn’t even closed yet and suddenly every decision done is mysteriously pushed by IBM.

There is more than enough people who have their own minds. That is the RH culture, if this freedom is touched they will just quit.

IBM won’t affect RH in any way, Jim is not an idiot because he knows this would destroy RH and IBM are also not idiots because it would destroy they purchase.


The acquisition isn’t even closed yet and suddenly every decision done is mysteriously pushed by IBM.

I am not saying that "every decision is mysteriously pushed by IBM." I'm saying that IBM culture will - over the long run - infect Red Hat. To pretend otherwise is, IMO, pretty naive. Over the decades, company after company after company has been acquired, and nearly every time the acquiring company makes the same promises about maintaining autonomy for the acquired company. And sometimes it holds up for a few months, or even a few years. But every single time, at least that I can recall, in the long run the acquired company eventually gets totally absorbed by the parent and starts to take on their characteristics. I haven't heard any cogent argument yet to justify believing that this time will be different.

IBM are also not idiots because it would destroy they purchase.

Like the way IBM managed to avoid destroying Lotus, Informix, Tivoli, Rational, Sequent, Truven, Explorys, Phytel, etc.?


There's a couple of areas that's not actually true. For example their "insights" package which has rule based detection of problems to fix etc is open source but the actual ruleset is not.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: