Re: Getting vendors using W3C tests

On Tue, May 31, 2011 at 6:52 AM, James Graham <jgraham@opera.com> wrote:
> * Getting the results out of the test into our systems. In theory this is
> solved for testharness.js tests by including a second <script>;
> testharnessreport.js that is blank on the W3C site and which vendors can use
> to convert the testharness.js results to whatever is needed
> result-collection system they use. However not all tests have consistently
> included this extra script.

I never got the point of that.  Why don't reusers just hack
testharness.js itself?  Or if we want it to be a separate file for
convenience, why doesn't testharness.js add the <script> to include
that file automatically?

> The second problem needs a bit of thought. Possibly the testsuite
> could have a build step and servers could be variables in the original
> files. That has some disadvantages though since there is non-negligible
> overhead to making everyone build the testsuite before they can run it.

If we make sure that the set of hosts is fixed and their roles are
well-documented, would it be unreasonable to ask implementers to just
edit the host file on their test machine to point the ___domain to
someplace they control?  That seems like it would be a lot simpler
than any of the alternatives I can think of.

Received on Tuesday, 31 May 2011 22:26:20 UTC