Yes and no. Traditional software transactional memory as imagined in the 1980s and 1990s is still imaginary. The hardware is still too limited.

Instead, it's more akin to something like the "synchronized" method and block attribute in Java, where the compiler transparently acquires and releases a mutex. Except in Java the mutex is on a particular object, whereas in C++ it'll be on a per-block hidden global mutex.

For small enough blocks, and if the stars are aligned just right (i.e. regarding alignment, cache-line locality, etc), the compiler might be able to optimize away the mutex in favor of a series of LL/SC, CAS, etc statements. But then, so could the JVM for synchronized methods.

It basically makes multi-threading easier, but won't automagically make it possible to build lock-less data structures, except that maybe the compilers will be smart enough to optimize something as simple as a singly-linked list construct into atomic instructions.

This approach to "transactional memory" was mostly fleshed out years ago in proofs-of-concepts by Intel, GCC, etc developers. And the approach has driven Intel's design of their "transactional memory" instructions, which basically move aspects of traditional synchronization approaches (speculative mutex acquisition, dirty flags) into the microcode.

As far as I understand the mutex is not per block but is logically a single global mutex (synchronized blocks) or actual transactions with full rollback (atomic blocks).

It really is transactional memory and it is meant to be implemented using hardware acceleration like that available in recent server class x86, power and sparc CPUs.

GCC, I belive, supports both a pure software based implemenation and an hibrid one.

I guess we interpret the phrase differently. I interpret software transactional memory to mean being able to build complex lock-free, wait-free data structures. That doesn't require hardware transactional memory, but does require some strong hardware primitives.

Here are some good links which explain how TSX works and how Intel might have accomplished it.

  http://www.realworldtech.com/haswell-tm/
  http://www.realworldtech.com/haswell-tm-alt/

TL;DR: one way or another they're piggy-backing on the mechanisms needed to maintain x86's strong cache coherency. Fundamentally it's just speculative operations on a small amount of data--if conflicts are detected, or if you manipulate more than a couple of cache lines (because the processor will only be able to track and buffer a very limited number of cache lines for conflicting operations during the pendency of the transaction), the code generated by the compiler will either take a lock or loop.

So, yes, it will make much existing code faster. But it's not going to provide the ability to develop highly-concurrent lock-free data structures. If there's any serious contention (i.e. more than 2 or 3 threads), or your transactional blocks access more than a few words of shared state, the transaction will invariably abort. Which is why all software transactional implementations, including recent ones which make use of TSX, are typically _slower_ than similar lock-based approaches in real-world scenarios.

STM-light is still conceptually elegant from the programmer's perspective, but deep down there's much less magic then you'd think. That's because it's _very_ expensive to track conflicts in a fine-grained manner in hardware. LL/SC operations were proven in the 1980s to be universal primitives that could be used to implement arbitrarily complex lock-free, wait-free algorithms. And chips like ARM and POWER have LL/SC opcodes. But they're pale shadows of the constructs studied in the 1980s because trying to actually implement them in hardware is just too costly. So TSX, while cool and useful, is something of a hack (and I mean that in both the positive and pejorative senses).

The real speed gains from Intel's new architectural support come from hardware lock elison, and doubtless both GCC and LLVM will lean heavily on this as it'll be easier to work with. Like with VLIW and then auto-vectorization, don't put too much stock in promises that compilers will be able to transform typical application code into a form that's suitable for use by the specialized hardware instructions. Like with AVX2, for example, to really make good use of TSX programmers will still need to meticulously organize their data structures and code flow, and will need to be mindful of the hardware constraints from the very outset. And in most cases they'll be far better off using intrinsics or assembly in the critical sections of their code.

You can touch a lot of cache lines in a transaction. On POWER it's several hundred.

I totally agree that TSX works best with specialized data structures. But it's powerful enough that you can, for example, malloc() or free() a block of memory inside a transaction.

Interesting... according to this paper

  http://www.gsd.inesc-id.pt/~salaa/wttm2015/html/abstracts/Hasenplaugh.pdf

it's the opposite. The maximum reliable transaction write capacity if contiguous and well-aligned is small on POWER (63 cache lines) and high with TSX (400 lines). TSX is using L1 cache to buffer writes, and so the number of concurrent transactions only scales with cores. Whereas POWER uses a per-thread buffer which can scale linearly with the number of hardware threads (8 threads per CPU, 80 in their test).

So if I'm understanding this correctly, TSX has better capacity but poor concurrency, and POWER has poor capacity but better concurrency.

That's very cool.

Quick Side as the two commenters in this thread seem knowledge.

I assume xbegin is a memory barrier. But is it a full fence like mfence or lock prefix?

I see a lot of benchmarks using TSX for locking, but one of the nicer features of lock compxchg or lock xchg is they carried an implicit mfence this was nice because it forced reads/writes before the instructions to be completed.

I know xbegin/xend do _more_ then an mfence for reads/writes within the RTM region but do they provide fencing for instructions _after_ their execution?

No idea and don't want to dig into Intel docs right now, but I would be surprised if they were full fences as I think xbegin/xend would only require acquire/release semantics.

xacquire/xrelease can be used as modifiers to existing lock prefixed RMW instructions which are already full barriers, giving them optimistic locking capabilities.

Okay so I asking around on the intel forums. They fences for instruction ordering, and cache load/flush.

But not for load/stores. If you load/store from data within an RTM region (after _xend()) you have no atomic protections.

Indeed.

http://en.cppreference.com/w/cpp/language/transactional_memo...

Is this something we can expect in consumer hardware, i.e. Intel and/or ARM, anytime soon? I'm out of the loop for this one.

Intel's had transactional memory extensions since Haswell, but it was kind of buggy until Skylake, IIRC.

i too am curious