This tutorial is written to show you how to do the manual testing of open source application. Do note that most of the open source and free software don’t maintain the documentation for the testing. So my intention here is to guide you for the formal testing process of application with the example of Gedit. You don’t have to take this tutorial as standard for manual testing. It is my attempt to guide newbies to help get started with testing open source & free software. I’m open to improve this tutorial If you’ve suggestions.
Manual Testing
In this type of testing, we are not going to execute the software while designing the test cases. We make use of the specifications and requirements for the software to write the test cases. In case of Gedit, we assume that like most of the open source applications, it doesn’t have any standard specification but more of community driven specifications on how should software look and work. Goal for manual testing is to check that the software meets the specification and passes our test cases. Please note that as we lack the specifications and requirements for the open source applications, we may have to use that applications prior to writing test case. So manual test case writing is a bit more work in such case. However, once you write such test cases, then writing the future test cases based on revisions and feature changes becomes easy.
To Do List :
- Write Traceability Matrix (Test scenarios)
- Write Test Cases ( based on data from traceability matrix)
Use spreadsheet program to create table for test scenario and test cases. Take a look at below images for guidance on what the table should contain. The images are test scenario spreadsheet and test case spreadsheet.
Test Scenario Table
Test Case Table
Gather the requirements/specifications from the development mailing list. You are less likely to find the documentation for specification so you have to gather the feature changes and requirement from the developer discussion and testing the earlier builds.
What to test
- User Interface
- Fuctionality
- Configuration Settings
- Stress Test
User Interface
In this type of testing you’re going to verify the UI elements of the application. You need to find out if the application has all the UI elements on the place where it is supposed to be. Most of the UI changes per version are usually declared on mailing list of open source applications, so you have to keep tab on those announcements before writing test case to verify some of them. User interface for each file, settings, configuration and other feature windows will vary. So you need to write test cases to cover every small detail that goes for UI.
e.g.
- Test Scenario 1: Gedit should open with existing tab for untitled document.
- Test Scenario 2: Gedit should have following menus – File, Edit, View, Search, Tools, Documents, Help, in application menu bar at top (global menu bar in unity ubuntu).
Functionality
In this type of testing, you need to write test case for the expected functionality of the application. In case of Gedit, you’ll be writing about syntax highlighter, color schemes, saving file, opening file etc. You have to create detailed test cases for the functionality of the program. You can also include test cases by considering user point of view for the functionality. Do consult the application developers mailing list or forums while you’re writing these test cases.
e.g.
- Test Scenario 1 : Gedit should save the file, when ctrl+s is pressed.
- Test Scenario 2: Gedit should open print preview interface, if shift+ctrl+p is pressed or “print preview” option is selected.
Configuration Settings
Each application has settings and configuration in order for it to extend from it’s default behavior. You should write test cases for settings windows and dialog boxes. Also do note what should be the default behavior for such settings is usually written in the specification for most commercial applications. However, you have to either consult the developers or assume the behavior based on the understanding of the application.
e.g.
- Test Scenario 1: Gedit should change the color scheme when user performs following action – Edit > Preferences > Select “Font & Colors” tab > Select “Color scheme”.
- Test Scenario 2: Gedit should set the language when user performs following action – Tools > “Set Language”.
Stress Test
I suggest skipping this test for now if you’re newbie. In order to stress test application, you have to discuss with the developers regarding application and tools that you can use to stress test the application. You also need to know some of the fuctionality of application. Once you do that writing test case can make things clear to you. However, we’ll skip this part for the sake of simplicity.
Why You Should Do Manual Testing
Most of the freshers who are interested to land a job in testing career benefit from working on manual tests of open source applications. Manual tests also help improve the open source applications. Though there are very few opensource/free software that follow the manual test procedures, however, even if you do solo testing, that is enough to improve your skills.
Why you should avoid
Manual testing does get repetitive and boring. You need to consult with the mailing list developers to keep tab on specifications and feature changes to increase your test cases. In case of open source applications, it becomes hard get specifications too. Nobody is out there to read your test cases and validate the tests results either.
Now that you know what are the reasons you should test and should not test, it becomes clear what to do next. If not, drop me a line in comments and let me know what are your suggestions on open source application manual testing. What should be included and how should a newbie proceed with testing the applications developed under open source or free software.