Re: Starting a test suite for Web Audio

On 05/21/2013 01:46 PM, Chris Lowis wrote:
>
> Tobie Langel writes:

> There's some really tricky complexities around testing audio, as it is
> quite unlike a lot of the other test suites I have seen. At the moment
> our discussions centre around using a ScriptProcesserNode vs an
> offlineAudioContext to allow reference tests. I think providing a bit of
> extra guidance and a single place to have tests reviewed and discussed
> is quite useful. Also, as our spec is a moving target at this stage, we'd
> like to try and keep our tests in lock-step with changes to the spec,
> via our bug tracker. I think having a place (our "fork") for tentative
> submissions where we can discuss them on our list might help with that.
>
> I don't think it precludes us from benefiting from your developments, as
> we'll regularly submit the tests our group is working on as a pull request when
> we've written tests we're happy with. I expect we'll do that frequently
> when we've come up with a sensible testing strategy. We just need a
> "sandbox" at this stage.
>
>>> I wanted the default branch on our github page to have some
>>> webaudio-specific information (especially contribution legalise and our
>>> "submission" workflow), hence the webaudio-readme branch. Does that make
>>> sense?
>>
>> We're aiming for a single, cross-group process and review system here.
>>
>> If you have special legal requirements or issues with the common
>> workflow, please let us now.
>
> No, we don't. Quite a lot of our members are invited experts who only
> engage with the W3C through the Web Audio group, I wanted to make the
> process clear to them. We do have a number of members who are keen
> to get involved with testing Audio specifically, and may need some help
> to know how to contribute. You will still be able to review the tests we
> write when we send a pull request to
> https://github.com/w3c/web-platform-tests, of course.

To be clear about this, there is nothing that stops you/people in your 
wg submitting PRs to web-platform-tests and having the policy that only 
other WG members should review things in the webaudio folder for now. 
That does have a number of advantages; e.g. your tests (including those 
in unaccepted pull requests) will automatically be mirrored on 
w3c-test.org so that people can run them without a local checkout. There 
is also an instance of the critic code review tool set up for the main 
web-platform-tests repository, and this would allow people in your wg to 
get notified of changes/submissions for webaudio tests alone (using the 
filter system). Using this ___location also makes it more likely that 
people familiar with the infrastructure will comment on whether you are 
following common idioms with testharness.js or idlharness.js.

On the other hand I don't really object if you iterate the testsuite in 
another ___location at first and then make a submission later. But it does 
mean that you will have worse tooling and could lead to unnecessary work 
later on.

Received on Tuesday, 21 May 2013 11:56:31 UTC