Extras-testing/QA Checklist/QA Improvements

(→Package Interface)
(→Package Interface)
Line 8: Line 8:
== Package Interface ==
== Package Interface ==
-
* Changelog should be displayed
+
* Changelog should be displayed.
* Votes should be changeable.
* Votes should be changeable.
* Each package that enters or leaves testing triggers a e-mail for the testing squad list
* Each package that enters or leaves testing triggers a e-mail for the testing squad list

Revision as of 19:46, 9 January 2010

Note: Work in progress, none of the action points below are definitive.

Contents

Automatic checks/Autobuilder

  • Bugtracker field - TBD
  • Optified - TBD

Package Interface

  • Changelog should be displayed.
  • Votes should be changeable.
  • Each package that enters or leaves testing triggers a e-mail for the testing squad list

Thumbs Up

  • Collapsable check list will appear, testers should mark the fields tested.
  • Promotion should occur at +10 karma (if there's at least one completed check list ???)

Thumbs Down

  • Testers must comment on thumbs down.
  • Maintainers thumb down will demote the application from the testing queue.
  • X (fix me) thumbs down will demote the app.

Demotion

  • Packages can be demoted at any time by his maintainers or by a member of the testing squad (demotions should be advertised in the testing squad list).
  • When demoting a package there's a option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.

Check List 

1. [ ] Licensing ok.
2. [ ] No illegal/dubious content.
3. [ ] Working provided features.
4. [ ] No missing announced features.
5. [ ] Optification ok.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No known security risks.

Additional comments:
  • Put [x] for those tests you have done, elaborate on separate row if the test is FAIL.
  • Vote up if there were no FAILs. If there was even one FAIL, vote down.
  • UI usability issues cannot be used as a reason for vote down.
  • Always test functionality - that is, run the program and try if it works as it should.

imaginary example: 

1. [x] Licensing ok.
2. [ ] No illegal/dubious content.
3. [x] Working provided features.
 FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123). 
 FAIL: When exporting file the program crashes (see bug: http://url/456)
4. [ ] No missing announced features.
5. [x] Optification ok.
 FAIL: the package uses 1512kb from root.
6. [ ] No performance problems.
7. [ ] No power management issues.
8. [ ] No known security risks.

Additional comments: 
I liked the program, even though, as is, I have to vote it down due to the bugs. I added few usability enhancements to bugzilla, see http://url/567 and http://url/678

Testing Squad

  • Can demote packages when there's blockers.
  • Can promote packages when they are stuck in the testing queue for a while without any known blocker.

Testing Squad mailing list

  • Public mailing list where are discussed concerns/improvements to the QA process.
  • Receives a notification for each packages that enters testing, is demoted or is promoted.
Retrieved from "http://bugs.maemo.org/Extras-testing/QA_Checklist/QA_Improvements"