Extras-testing/QA Checklist/QA Improvements

(→Package Interface: link to wiki)
(→Check List: add description field)
Line 49: Line 49:
* UI usability issues cannot be used as a reason for vote down.
* UI usability issues cannot be used as a reason for vote down.
* Always test functionality - that is, run the program and try if it works as it should.
* Always test functionality - that is, run the program and try if it works as it should.
 +
* Pls consider adding "Package has description field" as additional item in the check list. Packages without or packages describing themselves as "''packagename'' application" shouldn't go to Extras as this is so easily done.
imaginary example:
imaginary example:

Revision as of 22:00, 18 January 2010

Note: Work in progress, none of the action points below are definitive.

Contents

Automatic checks/Autobuilder

  • Bugtracker field - TBD
  • Optified - TBD
    • also check opt. of other packages pulled in as dependencies

Package Interface

  • Changelog should be displayed.
  • Votes should be changeable.
  • Each package that enters or leaves testing triggers a e-mail for the testing squad list
  • Link to Wiki so that details of test criteria are always accessible to new testers

Thumbs Up

  • Collapsable check list will appear, testers should mark the fields tested.
  • Promotion should occur at +10 karma (if there's at least one completed check list ???)

Thumbs Down

  • Testers must comment on thumbs down.
  • Maintainers thumb down will demote the application from the testing queue.
  • X (fix me) thumbs down will demote the app.

Demotion

  • Packages can be demoted at any time by his maintainers or by a member of the testing squad (demotions should be advertised in the testing squad list).
  • When demoting a package there's a option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.

Check List 

1. [ ] Licensing ok.
2. [ ] No illegal/dubious content.
3. [ ] Working provided features.
4. [ ] No missing announced features.
5. [ ] Optification ok.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No known security risks.

Additional comments:
  • Put [x] for those tests you have done, elaborate on separate row if the test is FAIL.
  • Vote up if there were no FAILs. If there was even one FAIL, vote down.
  • UI usability issues cannot be used as a reason for vote down.
  • Always test functionality - that is, run the program and try if it works as it should.
  • Pls consider adding "Package has description field" as additional item in the check list. Packages without or packages describing themselves as "packagename application" shouldn't go to Extras as this is so easily done.

imaginary example: 

1. [x] Licensing ok.
2. [ ] No illegal/dubious content.
3. [x] Working provided features.
 FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123). 
 FAIL: When exporting file the program crashes (see bug: http://url/456)
4. [ ] No missing announced features.
5. [x] Optification ok.
 FAIL: the package uses 1512kb from root.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No known security risks.

Additional comments: 
I liked the program, even though, as is, I have to vote it down due to the bugs. I added few usability enhancements to bugzilla, see http://url/567 and http://url/678

Testing Squad

  • Can demote packages when there's blockers.
  • Can promote packages when they are stuck in the testing queue for a while without any known blocker.

Testing Squad mailing list

  • Public mailing list where are discussed concerns/improvements to the QA process.
  • Receives a notification for each packages that enters testing, is demoted or is promoted.
Retrieved from "http://bugs.maemo.org/Extras-testing/QA_Checklist/QA_Improvements"