I would like to know how this is different from jabber/xmpp and how likely it is that these differences are worthwhile. They say their API is more "pragmatic" and writing a full-blown XMPP client certainly is very tricky but I wonder if improving the way XMPP works wouldn't ultimately create a better result. After all, a lot of thought and experience has by now gone into it.
I see your problems are that core XMPP provides very little and you need extensions to actually build something interesting. Which is very true, however we have come to a state where some of them are included by default in any good server or client implementation)
> Not particularly web-friendly – you can’t easily speak XMPP from a web browser.
> Single server per MUC is a single point of control and availability
Very true. There has been attempts to federate MUCS (http://xmpp.org/extensions/xep-0289.html) but the problem here is about human roles and who can do what (notwithstanding end-to-end integrity)
> History synchronisation is very much a second class citizen feature
I could cite counterpoints for every one of your points but that's not interesting. The real issues I see with XMPP are:
* No efficient support for binary -- The official way to do it is to setup another transport with Jingle. Which is sad because a transport was already setup, and now we need another one...
* Because of previous point, any potential cryptography will waste a lot of space
* No framing (a point you cited), which is absurd considering that XMPP is a routing protocol
* XML is a non-joy to work with (Note: JSON would not be a complete solution, see previous points)
Even though it has deficiencies, I'm still betting (and working) on it. What we really lack is software, and tools built on a protocol, not a new protocol.
I've read through the blog, the comment would have been useful on differentiating Matrix from XMPP. Also, sort of off topic, but Skype has a XMPP bridge and WhatsApp is built on XMPP (they just don't support federation).
Oops - I should have called it out more clearly in the blog.
The biggest difference we have over XMPP is probably that messages in Matrix get synchronised over all the participants of a conversation... so you get distributed chat history for free, and no single points of failure on group chat (as you do with XMPP MUCs). And obviously Matrix is plain HTTP+JSON rather than messing around with XML.
One of the reasons we built Matrix is because, in practice, pretty much all the current big players started off using XMPP for developing their chat solutions: Google Talk was originally XMPP; I believe Facebook Messenger was built out at first on ejabberd; WhatsApp was originally XMPP etc; even APNS was originally XMPP!
But ALL of them have ended up mutating it to a proprietary closed standard - and nobody has even tried open federation other than Google's misadventure with Talk. So, unfortunately, it seems XMPP hasn't ended up being the interoperable web-for-IM that we all hoped.
Now, I have absolutely no idea if Matrix will be more successful in solving the problem; the hope is that by keeping it simple and using HTTP APIs there's more of a chance that players of all sizes will start exposing Matrix APIs for federation. Only time will tell. It's also worth noting that end-to-end encryption is a relatively recent potential obstacle: given iMessage and Telegram etc are all end-to-end encrypted, for them to ever federate with Matrix we'll need to support the same semantics and crypto. Hopefully we're going about this the right way (although end-to-end crypto isn't formally specified or implemented yet), but will be an interesting challenge. And one that XMPP hasn't tried to solve at all, as far as I know.
>I would like to know how this is different from jabber/xmpp and how likely it is that these differences are worthwhile.
I thought the question was fairly content free, and something that someone would be able to answer themselves if they had enough knowledge of XMPP to actually care, and 10 minutes to read the announcement blog.
>I wonder if improving the way XMPP works wouldn't ultimately create a better result. After all, a lot of thought and experience has by now gone into it.
A vague objection and a truism. Sorry if you felt bullied or something.
It's better than a downvote, if you ask me. A downvote is an anonymous put-down, and that's why people use it to punish anyone that disagrees with them, or if they don't like someone's opinion.
Perhaps if down-votes were displayed, and people could respond. Dunno, perhaps we're over-complicating it.
A downvote is a tool to provide community management of the signal-to-noise ratio on HN. Encouraging noise to trigger more noise would not be an effective means of serving that purpose.
That's not been my experience with downvotes here so far. I've regularly had meaningful comments, that would have been seen as "conversation" in a normal context, get downvoted here. Now I don't know what "noise" that got rid of, but it sure did serve a pretty powerful way of signaling others to jump on board with the downvotes and/or ignore my comment.
The fact that you thought they were meaningful does not mean that others found them as meaningful contributions to the discussion.
> but it sure did serve a pretty powerful way of signaling others to jump on board with the downvotes
Downvotes (because the fact that they have been given is visible and, with a small number, the comment text's visibility is not reduced much) do encourage others to moderate -- either with downvotes or upvotes.
If they are piling on with downvotes, that's perhaps a signal that lots of people here think your comment is inappropriate. I've had lots of posts of mine draw both hostile responses and a few downvotes -- but most of those get reverse by upvotes quickly. On the rare occasions where I've gotten more than a few downvotes that haven't been quickly countered by upvotes, I've usually been able to see a plausible explanation other than simple disagreement why people would see the post as not a substantive addition.
And that's been true of most of other people's posts I've seen downvoted more than slightly, so I am not inclined to think that there is a broad, general problem with downvoting on HN.
I think this is true when discussions tend to be quantifiable areas. But I have seen and experienced downvotes purely based on disagreement about political opinion which is why I wish downvotes mandated a comment as to why the downvote is happening.
> But I have seen and experienced downvotes purely based on disagreement about political opinion
You've seen downvotes and you've assumed were based purely on disagreement about political opinion.
> which is why I wish downvotes mandated a comment as to why the downvote is happening.
If they mandated a comment, then every downvote would produce discussion of the merits of the downvote, which is exactly the opposite of the purpose of having downvotes.
Ironically, this whole subthread has become a demonstration of why comments about downvotes are big distractions from whatever the substantive topic under discussion is.
The term 'real-time' is used _very_ loosely here.
In industrial settings, in order to claim realtime-properties, you need to make guarantees about the runtime behavior of server, client and the network infrastructure in between.
IP-based Ethernet -> not realtime capable (without extensions)
Python (garbage collected language) -> not realtime capable
You can at most claim to achieve soft-realtime.
As in "sub-500ms but only most of the time".. o_O
Different contexts. For weapon systems and industrial systems, real-time operating systems are needed to guarantee true real-time behavior.
That usage makes no sense in the context of internet messaging for the reasons you describe. For human-to-human messaging, I think they're referring to a property more akin to synchronous (versus asynchronous) messaging, i.e. chat versus email. I speculate that the matrix team thinks this is an acceptable overloading because, as a practical matter, the two concepts have nothing to do with one another.
rpdillon is totally right - the term is overloaded. We're using "realtime" exclusively here to mean synchronous human communication - like the RT in WebRTC. In other words, IM or VoIP - anything where you expect to get a response from the other user inmediately rather than waiting asynchronously.
This of course clashes with the RTOS style use of the word, and we aren't claiming that Matrix has any hard realtime timing guarantees at all, other than the soft realtime behaviour media stacks like WebRTC give you by means of high priority threads.
In retrospect it's a confusing term; we should probably avoid using it :)
disclaimer: use of the word realtime in the Matrix.org blurb is all my fault...
It uses WebRTC for media and JSON-RPC for signalling. On the server side it is Freeswitch module, so can have access to PSTN calling, VOIP and other signaling protocol.
Can't help but think "jesus, not another protocol I have to support". People bitch and moan about SIP but the goddamn thing works and is already extensible if you need your own unique features. Can't you build on that? Instead you're like phone manufacturers who each come up with their own protocol because hey, we've got those sweet PBX boxes to sell too!
I love your goal, but I really can't see why you can't use SIP to communicate and play nice with other endpoints out there. Surely that would be an advantage.
Given that XMPP hasn't taken off as it could have done - and the fact that each large IM or VoIP application seems to end up writing their own version (and putting that in their own walled garden) - we think a new, well-defined open standard, together with open source reference server and client codebases can be the catalyst to create an interoperable and federated "message transport" solution for a multitude of services!
So, for context: we ended up writing Matrix after literally 10 years of building SIP infrastructure, so we have some experience in SIP (and XMPP's) shortcomings :) The short answer to "why not extend SIP" is that SIP stacks are fairly fiddly to write and get right; and SIP/SIMPLE/MSRP is typically a lot more verbose and convoluted than HTTP (and hugely more verbose than SPDY and HTTP/2.0). There's no support in SIP for distributed storage of state, and there's no support for paginating and storing conversation history at all. In fact, some of the SIP community has even ended up bolting on IMAP(!) for storing conversation and call history :S Meanwhile, given HTTP is pretty simple and completely ubiquitous, so why bother with an alternative stack at all?
In terms of "jesus, not another protocol I have to support" - I completely sympathise, hence quoting http://xkcd.com/927/ in my blog post... We didn't do this lightly :)
One way of thinking of this is that right now you have LOADS of HTTP APIs out there for messaging - Twitter, FB, http://api.ihackernews.com, Reddit... they're sufficiently easy to implement that every new site goes and builds a new one, and developers have to go and learn each new one every time. Meanwhile, none of them will ever federate or interoperate - the whole thing's totally fragmented.
So whilst Matrix is indeed yet another API to support - hopefully it's one that might actually help solve this problem... so that the next time that someone wants to add a messaging HTTP API to their site, they can follow an interoperable pattern rather than reinvent the wheel again. Perhaps a better way of describing Matrix is "two-way RSS for arbitrary data, with distributed history"
Does this help justify things at all? (Disclaimer: Matrix is all my fault)
This is a very cool project, and one that I am very partial to. I agree that having a common application layer for structured data would be incredibly useful for developers and users across the board by freeing data from the big silos.
Question: Are you looking to expand beyond IM and VoIP, or do you see those as the main areas where the protocol will function?
as per the FAQ: The initial goal of Matrix is to fix the fragmentation of VoIP and IM, but Matrix’s real potential and ultimate mission is to be a new and truly open ecosystem on the Internet enabling services, devices and people to easily communicate with each other.
I'd say IM and VoIP would be the obvious common cases, but Matrix can and will be used to solve all kinds of problems!
Just adding that this is very much the first few steps of this project - if you think it sounds interesting we would love feedback and comments on #matrix:matrix.org (via matrix.org) or #matrix on freenode!
Matrix is not P2P, but is distributed: there is no single point of failure, the history is scattered across all servers which retrieve what they need from the others.
Did any of you (including Geary, I guess he blessed it, or does it go all the way up to Rami ?) considered the fact that maybe the issue isn't the absense of open standard (xmpp despite it's quirks does work) but lack of interest from companies to open up their walled gardens ?
Well ... HTTP is terribly complicated, and about as bad a fit for the problem as it can get? It has tons of overhead (at least compared to the mostly tiny payloads you can expect for IM), it doesn't support push delivery (except through horrible hacks like long polling), doesn't really support pipelining, in particular over TLS, none of the mechanisms that HTTP does provide seems to be of any use whatsoever for the purpose ... and then they complain about SIP!
Overall, it makes the impression on me like it was designed by someone who doesn't know much more than how to build web apps and therefore is afraid of anything not HTTP, and so will use HTTP no matter what, even if there are essentially only disadvantages compared to pretty much any alternative you could come up with.
(Oh, and also, they claim being "RESTful", without being anything like that ...)
SIP+SIMPLE/MRSP's overheads are just as bad as HTTP/1.x, if not worse. We decided to build Matrix after 10 years of fun doing commercial SIP and XMPP work; we have some experience. For instance, the worst scenario we've seen in a real-life SIP/MSRP user-agent in the wild was 50KB of SIP/MSRP nego to send the word "Heh" between two contacts (with a delivery report) :) Meanwhile SPDY and HTTP/2.x improve the overhead situation and pipelining situation enormously - although frankly we haven't seen any real-life problems using HTTP/1.1 yet at all...
The "simplicity" of HTTP I mentioned in the Matrix blurb refers to the fact that RFC2616 is relatively compact and self-contained, whereas SIP/SIMPLE/MSRP involves a huge number of RFCs, different protocols (SIP,SDP,MSRP...) and really is a lot more complicated to implement than just doing GETs and PUTs.
Now, the irony is that rather than being "designed by someone who doesn't know much more than how to build web apps" - it's more the other way round. Our experience is mainly with SIP/RTP/STUN/ICE/TURN etc; genuinely RESTful APIs are relatively new territory for us.
I would genuinely love to know what aspects of the Matrix client-server (or server-server) API is 'not anything like RESTful' - it's not too late for us to change the APIs (they've already been rewritten several times, oscillating between more or less RESTful purity), and half the point of releasing Matrix in its current early proto-form is to get detailed feedback from folks on whether we've Got It All Wrong :)
Oh, I certainly didn't mean to imply that SIP was somehow particularly efficient - I was just surprised that anyone would think HTTP was an improvement over SIP, in particular given that you just swap the SIP stack for an HTTP stack and the torture testing of SIP parsers for torture testing of HTTP parsers, especially so if you consider the history of SIP where people originally intended SIP to be parsable using HTTP parsers.
As for the huge number of RFCs: Well, those do cover a whole lot more than passing blobs around? That aspect is fully covered by the SIP RFC, and only a relatively small part of it, the rest is concerned with session establishment and teardown, which HTTP obviously isn't concerned with at all.
Which is why I am wondering: What is the point of using HTTP? What is the functionality that it provides you with that would justify requiring an HTTP stack in every endpoint, including the security risk from all that complexity?
As for RESTfulness: REST is in conflict with a fixed URI scheme, REST allows only for a fixed entry point, beyond that, all other relevant URIs should be discoverable from the returned representations. And no, I am not saying, your stuff should be RESTful, I'm just saying that it's not.
