Compressed texture formats are underused. I think few people outside of game development understand the benefits. The linked post explains it but I'd like to take a stab at it too.
Regular image compression formats like PNG or JPEG save network bandwidth and disk storage but not RAM. They must be completely decompressed and stored uncompressed in RAM as long as they are displayed. Compressed texture formats do save RAM. They remain compressed in RAM while they are being displayed, typically 1/6 the size of regular images or even smaller. That saves not only RAM but memory bandwidth and power too; especially important on mobile. Switching to compressed textures can make applications significantly faster.
Compressed texture formats have two main disadvantages that Basis solves. One is that their compression ratios are fixed and not as good as JPEG. Basis adds extra compression to make them more comparable to JPEG on disk.
The other, larger issue is that compressed texture formats require hardware support on the GPU, and GPUs don't agree on what formats to support. There is no single format with guaranteed support on every platform, which means that cross platform applications must encode each image multiple times and ship different formats to different users. Basis solves this by efficient runtime transcoding to most common GPU formats. This capability is unique and a really big deal.
Anyone with images in their application should take a look at Basis. Now that you can ship a single compressed texture file that's supported everywhere, I hope that this technology can expand beyond games.
From this [1] it sounds like the texture is a modified variant of ETC1, which gets loaded up from disk into memory as such, and transcoded on the CPU into target GPU format (DXT, PVRTC, etc), and then uploaded to VRAM.
If so that sounds similar to their previous Crunch library [2], in that it does transcoding on the CPU before moving the texture to the GPU - but most likely much faster than Crunch because it starts with ETC as shared basis. (And artifacts are a common with transcoding between compressed formats, especially when transparency is present, but I have high hopes for this one considering the authors! :) )
Authors suggest that a version that transcodes on the GPU would be possible - that would be really interesting, I would imagine that would be even faster, although at the cost of higher texture memory usage.
Crunch only supports one family of formats, DXT. (Unity has a fork that supports the ETC family, but it does not support decoding ETC and DXT from the same compressed file). Efficiently transcoding one compressed file to multiple compressed texture formats at load time is the key new feature that only Basis provides.
Part of the issue is they have been patented to the hilt by parties like S3. It hasn't been an issue in video games because companies just agree to implement other S3 apis in return for a patent license, but in cases like this that obviously isn't an option.
I'm not sure where it would help. The encoding process is slow so it wouldn't make sense to encode anything that changes like the inputs or activations, or the weights during training. You could maybe encode the weights after training but you'd really have to massage the data to fit into these formats, and the lossy perceptual tricks used here are designed for the human visual system, not high-dimensional convolution filters.
No, compressed texture formats are lossy. Also the fixed bitrate that allows random access into the compressed pixels constrains the encoder a little more than the variable bitrate of JPEG or other image formats.
This is amazing to me from an outsider's point of view; I thought Binomial was a two-person shop (idk if they expanded recently). To be able to partner with Google and Chrome Media on an equal level just doesn't seem possible to me. Binomial must have some pretty nice tech. Congratulations on the partnership!
Well, they fixed a problem none of the GPU makers is interesting in solving directly: providing a common image compression format for 3D rendering. This is all a patent-encumbered mess, and the reason we are in this mess are the GPU vendors alone.
When Microsoft still had power over desktop computing, it pretty much dictated GPU vendors to use the DXT image compression format which is supported on all desktop GPUs, but nothing similar has happened on mobile.
In the current situation, this problem couldn't have been solved from the "inside". It isn't a trivial problem by far to solve (probably one of the few topics in programming that deserve the name "rocket science"), but it also doesn't require a huge workforce or standardization committee (and thank goodness this hasn't been "designed-by-committee"), "just" many years of focus and dedication.
> Internally, Basis files are composed of a non-uniform texture array of one or more 2D ETC1S texture "slices". ETC1S is a simple subset of the ETC1 texture format popular on Android.
Regular image compression formats like PNG or JPEG save network bandwidth and disk storage but not RAM. They must be completely decompressed and stored uncompressed in RAM as long as they are displayed. Compressed texture formats do save RAM. They remain compressed in RAM while they are being displayed, typically 1/6 the size of regular images or even smaller. That saves not only RAM but memory bandwidth and power too; especially important on mobile. Switching to compressed textures can make applications significantly faster.
Compressed texture formats have two main disadvantages that Basis solves. One is that their compression ratios are fixed and not as good as JPEG. Basis adds extra compression to make them more comparable to JPEG on disk.
The other, larger issue is that compressed texture formats require hardware support on the GPU, and GPUs don't agree on what formats to support. There is no single format with guaranteed support on every platform, which means that cross platform applications must encode each image multiple times and ship different formats to different users. Basis solves this by efficient runtime transcoding to most common GPU formats. This capability is unique and a really big deal.
Anyone with images in their application should take a look at Basis. Now that you can ship a single compressed texture file that's supported everywhere, I hope that this technology can expand beyond games.