We are comfortable accepting and including what we call permutations.
Clearly, our project is about testing web application frameworks and platforms (we just say "frameworks" for conservation of typing). Testing NIO vs. InputStreams, for example, in isolation is a different project.
But as part of a web framework—let's say you had a framework that provided support for both NIO and InputStreams—we'd be happy to include the multiple permutations.
Obviously, this is within reason. If you submitted 512 permutations, I might have a small heart attack. Also, a general principal that we ask all participants to bear in mind is that each test should represent a viable production-grade configuration. This guideline might rule out a great number of wilder permutations.
Clearly, our project is about testing web application frameworks and platforms (we just say "frameworks" for conservation of typing). Testing NIO vs. InputStreams, for example, in isolation is a different project.
But as part of a web framework—let's say you had a framework that provided support for both NIO and InputStreams—we'd be happy to include the multiple permutations.
Obviously, this is within reason. If you submitted 512 permutations, I might have a small heart attack. Also, a general principal that we ask all participants to bear in mind is that each test should represent a viable production-grade configuration. This guideline might rule out a great number of wilder permutations.