• Intro
• Downloads
• Community
• Development
• News
• TALK

Testing Squad

Latest revision as of 22:56, 28 July 2010

[edit] What is a Testing Squad?

Testing Squad is a team of Maemo community members that are involved in testing applications queued in Extras-testing repository in regular and systematic manner. The idea is to ensure high quality applications are promoted to end-user oriented Extras repository, instead of being stuck in Extras-testing forever, just because there is nobody willing to put enough effort to test these apps and make sure they are mature enough to hit Extras for everyone to use.

[edit] How everybody can help

• You can help testing beta software available in the Extras-testing repository, in order to provide high quality software to the Maemo users.
• If you don't feel ready to join the testing Squad or don't have enough skills, you can still help by testing the applications from a user point of view (functionalities, description, etc...). For more information read article Help testing software.
• Don't be shy to give feedback (both positive and negative) to author of an application, using packages interface or Bugzilla if you've found obvious bugs. Such activity does not require you to become Testing Squad member.

[edit] How to join the Testing Squad

• Read the "How everybody can help" section above.
• Subscribe the Testing Squad mailing list.
• Attend QA team meetings
  • Read minutes of our last meeting
• Welcome to the team! :-)

[edit] Mailing list

Mailing list for Testing Squad members is meant to keep track of apps in testing queue, so every member is aware which apps require most attention at given time. If you want to become Testing Squad member, subscribing to mailing list is essential.

There is also a mailing comments list that contains all the traffic from the comments on QA pages - it is a good reference to see how testing communication works, what the common issues are, etc.

[edit] How Testing Squad works

The Testing Squad follows a very simple process:

• Each week a member of the Testing Squad posts a list of applications that should be tested during that week.
• The testers test the applications freely in their free time.
• In the end of the week a evaluation is made and any pending issue is discussed among the team.

[edit] Super-testers

Super-testers are members of the testing-squad with a proven track record (currently testers with more than 100 package testing karma). Their votes (can) count a bit differently than those of regular testers. Three (3) votes down from the testers will demote an application when the 20 day limit is reached. This should take care of the obviously broken lingering packages. When 3 super-tester votes are up and 20 days have passed -> promotion unlock. This will only be able to influence promote/reject when at least 3 votes are the same and less than 3 differ.

See https://garage.maemo.org/pipermail/testingsquad-list/2010-July/000089.html for more details on the initial setup and some procedural detail on super-testers.

[edit] Packages marked for demotion

Let's keep the testing queue clean: Packages Marked for Demotion. So everyone please spend couple seconds and give thumbs down for those.

[edit] Code of Conduct

• Don't vote up an application before having a good understanding of its quality.
• Don't vote an application based in others comments, test it by yourself.
• Leave a comment or fill a bug report (preferred) every time you vote down an application.
• Don't vote down an application based in your personal preferences, always respect the Q&A rules.

[edit] Testers FAQ

[edit] Do I need special skills to become a tester?

No, anyone could become a tester (in fact you already are if you for example report bugs or give thumbs up/down to certain applications), however before becoming a member of Testing Squad you need to be ready to test various Extras-testing applications on more regular basis.

[edit] How to test applications properly?

Follow instructions at Q&A Checklist to ensure consistent testing procedure.

Help testing software is more thorough article about the same matter.

[edit] The application has a usable but very ugly UI, should I thumb it down?

No, developers are encouraged to adopt the Maemo 5 UI guidelines and optimize their application to finger use. However, lack of Maemo 5 UI optimization is not a reason to block an application in its way to Extras unless it is unusable even with stylus.

[edit] The application has no bugtracker, or an invalid one, should I thumb it down?

Yes: the bugtracker field is mandatory, applications without that field or with an invalid one can't be promoted to extras.

[edit] How should I decide if a bugtracker link is valid?

We are currently deciding some objective criteria for answering this question. Read article Tracking bugs in Extras for more info.

[edit] I found a critical bug during testing, what should I do?

First you should report the bug in the application bugtracker, after that ask the developer to thumb down the application(in order to demote it) and add it to this list.

[edit] Where to report bugs?

If the application has a regular product section in Maemo bugzilla, just use it: Enter a new bug report -> Extras -> application-name. Otherwise follow the link in the application bugtracker (or write an email to the developer if it contains a mailto link).

[edit] Should I always comment?

You don't need to comment if you don't have anything to add to the discussion; for example if someone already commented that the package needs a bugtracker or is not optified, you don't need to repeat the same, just vote down.