Optimasi Remote Moderated Usability Testing Pada Low-Fidelity Prototype Dari E-Commerce Dengan Wawancara Pada Generasi Z Di Indonesia
DOI:
https://doi.org/10.33592/jutis.v11i1.3281Keywords:
Usability Testing, Remote Moderated Usability Testing, Think-Aloud, InterviewAbstract
The research aims to optimize the remote moderated usability testing on low-fidelity prototype of e-commerce with the interview among Indonesia’s generation Z. To achieve this objective, the research compares the number of negative findings from usability testing between remote moderated usability testing only and with additional interview. The usability testing was conducted online with think-aloud protocol, to five undergraduate students in Jabodetabek area, Indonesia. Qualitative data recorded in video, were observed, processed, and analyzed to obtain findings with negative sentiment. Usability testing with additional interview can increase the negative findings by 100%. Therefore, the interview technique significantly can be used to optimize the remote moderated usability among Indonesia’s generation Z.
References
[1] Andryanto, S. D. (2021). Memahami 6 Karakter Generasi Z, Bukan Cuma Familiar Teknologi Sejak Kecil. Retrieved February 4, 2022, from Tempo.co website: https://gaya.tempo.co/read/1467165/memahami-6-karakter-generasi-z-bukan-cuma-familiar-teknologi-sejak-kecil/full&view=ok
[2] Babich, N. (2017). Prototyping 101: The Difference between Low-Fidelity and High-Fidelity Prototypes and When to Use Each. Retrieved May 3, 2022, from Adobe website: https://blog.adobe.com/en/publish/2017/11/29/prototyping-difference-low-fidelity-high-fidelity-prototypes-use
[3] Cho, H., Powell, D., Pichon, A., Kuhns, L. M., Garofalo, R., & Schnall, R. (2019). Eye-tracking retrospective think-aloud as a novel approach for a usability evaluation. International Journal of Medical Informatics, 129, 366–373. https://doi.org/10.1016/j.ijmedinf.2019.07.010
[4] Cho, H., Powell, D., Pichon, A., Thai, J., Bruce, J., Kuhns, L. M., … Schnall, R. (2018). A Mobile Health Intervention for HIV Prevention Among Racially and Ethnically Diverse Young Men: Usability Evaluation. JMIR MHealth and UHealth, 6(9), e11450–e11450. https://doi.org/10.2196/11450
[5] Cho, H., Yen, P.-Y., Dowding, D., Merrill, J. A., & Schnall, R. (2018). A multi-level usability evaluation of mobile health applications: A case study. Journal of Biomedical Informatics, 86, 79–89. https://doi.org/10.1016/j.jbi.2018.08.012
[6] Davis, R., Gardner, J., & Schnall, R. (2020). A Review of Usability Evaluation Methods and Their Use for Testing eHealth HIV Interventions. Current HIV/AIDS Reports, 17(3), 203–218. https://doi.org/10.1007/s11904-020-00493-3
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. https://doi.org/10.1037/0033-295X.87.3.215
[7] Firdaus, H., & Zakiah, A. (2021). Implementation of usability testing methods to measure the usability aspect of management information system mobile application (Case study sukamiskin correctional institution). International Journal of Modern Education and Computer Science, 13(5), 58–67. https://doi.org/10.5815/ijmecs.2021.05.06
[8] Geisen, E., & Bergstorm, J. R. (2017). Usability Testing for Survey Research. Morgan Kaufmann.
Jibb, L. A., Cafazzo, J. A., Nathan, P. C., Seto, E., Stevens, B. J., Nguyen, C., & Stinson, J. N. (2017). Development of a mHealth Real-Time Pain Self-Management App for Adolescents With Cancer: An Iterative Usability Testing Study [Formula: see text]. Journal of Pediatric Oncology Nursing : Official Journal of the Association of Pediatric Oncology Nurses, 34(4), 283–294. https://doi.org/10.1177/1043454217697022
[9] Kawakyu, N., Nduati, R., Munguambe, K., Coutinho, J., Mburu, N., DeCastro, G., … Gimbel, S. (2019). Development and Implementation of a Mobile Phone-Based Prevention of Mother-To-Child Transmission of HIV Cascade Analysis Tool: Usability and Feasibility Testing in Kenya and Mozambique. JMIR MHealth and UHealth, 7(5), e13963. https://doi.org/10.2196/13963
[10] Lyzara, R., Purwandari, B., Zulfikar, M. F., Santoso, H. B., & Solichah, I. (2019). E-Government Usability Evaluation: Insights from A Systematic Literature Review. Proceedings of the 2nd International Conference on Software Engineering and Information Management, 249–253. https://doi.org/10.1145/3305160.3305178
[11] Menteri Dalam Negeri Republik Indonesia. INSTRUKSI MENTERI DALAM NEGERI NOMOR 27 TAHUN 2021 TENTANG PEMBERLAKUANPEMBERLAKUAN PEMBATASAN KEGIATAN MASYARAKAT LEVEL 4, LEVEL 3, DAN LEVEL 2 CORONA VIRUS DISEASE 2019 DI WILAYAH JAWA DAN BALI. , (2021).
[12] Minen, M. T., Jalloh, A., Ortega, E., Powers, S. W., Sevick, M. A., & Lipton, R. B. (2019). User Design and Experience Preferences in a Novel Smartphone Application for Migraine Management: A Think Aloud Study of the RELAXaHEAD Application. Pain Medicine (Malden, Mass.), 20(2), 369–377. https://doi.org/10.1093/pm/pny080
[13] Moran, K. (2019). Usability Testing 101. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/usability-testing-101/
[14] Moran, K. (2021). Remote Usability Testing: Study Guide. Retrieved from Nielsen Norman Group website: https://www.nngroup.com/articles/remote-usability-testing-study-guide/
[15] Moran, K., & Pernice, K. (2020a). Remote Moderated Usability Tests: How to Do Them. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/moderated-remote-usability-test/
[16] Moran, K., & Pernice, K. (2020b). Remote Moderated Usability Tests: Why to Do Them. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/moderated-remote-usability-test-why/
[17] Musiimenta, A., Atukunda, E. C., Tumuhimbise, W., Pisarski, E. E., Tam, M., Wyatt, M. A., … Haberer, J. E. (2018). Acceptability and Feasibility of Real-Time Antiretroviral Therapy Adherence Interventions in Rural Uganda: Mixed-Method Pilot Randomized Controlled Trial. JMIR MHealth and UHealth, 6(5), e122–e122. https://doi.org/10.2196/mhealth.9031
[18] Nuraeni, R. (2021). Kombinasi Usability Resting yang Optimum Untuk Pengembangan Aplikasi POS Bagi UMKM Saat Pandemi Covid-19 Di Indonesia. Sekolah Tinggi Ilmu Manajemen dan Ilmu Komputer ESQ.
[19] Paz, F., & Pow-Sang, J. A. (2016). Usability Evaluation Methods for Software Development: A Systematic Mapping Review. Proceedings - 8th International Conference on Advanced Software Engineering and Its Applications, ASEA 2015, 10(1), 1–4. https://doi.org/10.1109/ASEA.2015.8
[20] Pernice, K. (2018). User Interviews: How, When, and Why to Conduct Them. Retrieved February 26, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/user-interviews/
[21] Ramli, R., Jaafar, A., & Mohamed, H. (2009). Remote usability evaluation system (e-RUE). 2009 International Conference on Computer and Electrical Engineering, ICCEE 2009, 2, 639–643. https://doi.org/10.1109/ICCEE.2009.247
[22] Ranjit Kumar. (2011). Research Methodology A Step-by-Step Guide for Beginners (3rd ed.). Sage Publication, Inc.
Richardson, S., Mishuris, R., O’Connell, A., Feldstein, D., Hess, R., Smith, P., … Mann, D. (2017). “Think aloud†and “Near live†usability testing of two complex clinical decision support tools. International Journal of Medical Informatics, 106, 1–8. https://doi.org/10.1016/j.ijmedinf.2017.06.003
[23] Stonbraker, S., Cho, H., Hermosi, G., Pichon, A., & Schnall, R. (2018). Usability Testing of a mHealth App to Support Self-Management of HIV-Associated Non-AIDS Related Symptoms. Studies in Health Technology and Informatics, 250, 106–110.
[24] Tarkkanen, K., & Harkke, V. (2019). Scope for usability tests in IS development. AIS Transactions on Human-Computer Interaction, 11(3), 136–156. https://doi.org/10.17705/1thci.00117
[25] Umar, A., & Khan Tatari, K. (2008). Appropriate Web Usability Evaluation Method during Product Development. Retrieved from www.bth.se/tek
[26] Widman, L., Golin, C. E., Kamke, K., Massey, J., & Prinstein, M. J. (2017). Feasibility and acceptability of a web-based HIV/STD prevention program for adolescent girls targeting sexual communication skills. Health Education Research, 32(4), 343–352. https://doi.org/10.1093/her/cyx048
[27] Wirasasmiata, R., & Uska, M. (2019). Evaluation of E-Rapor Usability using Usability Testing Method. Advances in Social Science, Education and Humanities Research (ASSEHR), 330(Iceri 2018), 71–74. https://doi.org/10.2991/iceri-18.2019.15
[28] Yen, P.-Y., Lara, B., Lopetegui, M., Bharat, A., Ardoin, S., Johnson, B., … Curtis, J. (2016). Usability and Workflow Evaluation of “RhEumAtic Disease activitY†(READY). Applied Clinical Informatics, 07(04), 1007–1024. https://doi.org/10.4338/aci-2016-03-ra-0036
[29] Yen, P. Y., Walker, D. M., Smith, J. M. G., Zhou, M. P., Menser, T. L., & McAlearney, A. S. (2018). Usability evaluation of a commercial inpatient portal. International Journal of Medical Informatics, 110(June 2017), 10–18. https://doi.org/10.1016/j.ijmedinf.2017.11.007
[30] Zhang, T., Rau, P.-L. P., Salvendy, G., & Zhou, J. (2013). Comparing Low and High-Fidelity Prototypes in Mobile Phone Evaluation. International Journal of Technology Diffusion, 3(4), 1–19. https://doi.org/10.4018/jtd.2012100101
[2] Babich, N. (2017). Prototyping 101: The Difference between Low-Fidelity and High-Fidelity Prototypes and When to Use Each. Retrieved May 3, 2022, from Adobe website: https://blog.adobe.com/en/publish/2017/11/29/prototyping-difference-low-fidelity-high-fidelity-prototypes-use
[3] Cho, H., Powell, D., Pichon, A., Kuhns, L. M., Garofalo, R., & Schnall, R. (2019). Eye-tracking retrospective think-aloud as a novel approach for a usability evaluation. International Journal of Medical Informatics, 129, 366–373. https://doi.org/10.1016/j.ijmedinf.2019.07.010
[4] Cho, H., Powell, D., Pichon, A., Thai, J., Bruce, J., Kuhns, L. M., … Schnall, R. (2018). A Mobile Health Intervention for HIV Prevention Among Racially and Ethnically Diverse Young Men: Usability Evaluation. JMIR MHealth and UHealth, 6(9), e11450–e11450. https://doi.org/10.2196/11450
[5] Cho, H., Yen, P.-Y., Dowding, D., Merrill, J. A., & Schnall, R. (2018). A multi-level usability evaluation of mobile health applications: A case study. Journal of Biomedical Informatics, 86, 79–89. https://doi.org/10.1016/j.jbi.2018.08.012
[6] Davis, R., Gardner, J., & Schnall, R. (2020). A Review of Usability Evaluation Methods and Their Use for Testing eHealth HIV Interventions. Current HIV/AIDS Reports, 17(3), 203–218. https://doi.org/10.1007/s11904-020-00493-3
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. https://doi.org/10.1037/0033-295X.87.3.215
[7] Firdaus, H., & Zakiah, A. (2021). Implementation of usability testing methods to measure the usability aspect of management information system mobile application (Case study sukamiskin correctional institution). International Journal of Modern Education and Computer Science, 13(5), 58–67. https://doi.org/10.5815/ijmecs.2021.05.06
[8] Geisen, E., & Bergstorm, J. R. (2017). Usability Testing for Survey Research. Morgan Kaufmann.
Jibb, L. A., Cafazzo, J. A., Nathan, P. C., Seto, E., Stevens, B. J., Nguyen, C., & Stinson, J. N. (2017). Development of a mHealth Real-Time Pain Self-Management App for Adolescents With Cancer: An Iterative Usability Testing Study [Formula: see text]. Journal of Pediatric Oncology Nursing : Official Journal of the Association of Pediatric Oncology Nurses, 34(4), 283–294. https://doi.org/10.1177/1043454217697022
[9] Kawakyu, N., Nduati, R., Munguambe, K., Coutinho, J., Mburu, N., DeCastro, G., … Gimbel, S. (2019). Development and Implementation of a Mobile Phone-Based Prevention of Mother-To-Child Transmission of HIV Cascade Analysis Tool: Usability and Feasibility Testing in Kenya and Mozambique. JMIR MHealth and UHealth, 7(5), e13963. https://doi.org/10.2196/13963
[10] Lyzara, R., Purwandari, B., Zulfikar, M. F., Santoso, H. B., & Solichah, I. (2019). E-Government Usability Evaluation: Insights from A Systematic Literature Review. Proceedings of the 2nd International Conference on Software Engineering and Information Management, 249–253. https://doi.org/10.1145/3305160.3305178
[11] Menteri Dalam Negeri Republik Indonesia. INSTRUKSI MENTERI DALAM NEGERI NOMOR 27 TAHUN 2021 TENTANG PEMBERLAKUANPEMBERLAKUAN PEMBATASAN KEGIATAN MASYARAKAT LEVEL 4, LEVEL 3, DAN LEVEL 2 CORONA VIRUS DISEASE 2019 DI WILAYAH JAWA DAN BALI. , (2021).
[12] Minen, M. T., Jalloh, A., Ortega, E., Powers, S. W., Sevick, M. A., & Lipton, R. B. (2019). User Design and Experience Preferences in a Novel Smartphone Application for Migraine Management: A Think Aloud Study of the RELAXaHEAD Application. Pain Medicine (Malden, Mass.), 20(2), 369–377. https://doi.org/10.1093/pm/pny080
[13] Moran, K. (2019). Usability Testing 101. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/usability-testing-101/
[14] Moran, K. (2021). Remote Usability Testing: Study Guide. Retrieved from Nielsen Norman Group website: https://www.nngroup.com/articles/remote-usability-testing-study-guide/
[15] Moran, K., & Pernice, K. (2020a). Remote Moderated Usability Tests: How to Do Them. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/moderated-remote-usability-test/
[16] Moran, K., & Pernice, K. (2020b). Remote Moderated Usability Tests: Why to Do Them. Retrieved February 4, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/moderated-remote-usability-test-why/
[17] Musiimenta, A., Atukunda, E. C., Tumuhimbise, W., Pisarski, E. E., Tam, M., Wyatt, M. A., … Haberer, J. E. (2018). Acceptability and Feasibility of Real-Time Antiretroviral Therapy Adherence Interventions in Rural Uganda: Mixed-Method Pilot Randomized Controlled Trial. JMIR MHealth and UHealth, 6(5), e122–e122. https://doi.org/10.2196/mhealth.9031
[18] Nuraeni, R. (2021). Kombinasi Usability Resting yang Optimum Untuk Pengembangan Aplikasi POS Bagi UMKM Saat Pandemi Covid-19 Di Indonesia. Sekolah Tinggi Ilmu Manajemen dan Ilmu Komputer ESQ.
[19] Paz, F., & Pow-Sang, J. A. (2016). Usability Evaluation Methods for Software Development: A Systematic Mapping Review. Proceedings - 8th International Conference on Advanced Software Engineering and Its Applications, ASEA 2015, 10(1), 1–4. https://doi.org/10.1109/ASEA.2015.8
[20] Pernice, K. (2018). User Interviews: How, When, and Why to Conduct Them. Retrieved February 26, 2022, from Nielsen Norman Group website: https://www.nngroup.com/articles/user-interviews/
[21] Ramli, R., Jaafar, A., & Mohamed, H. (2009). Remote usability evaluation system (e-RUE). 2009 International Conference on Computer and Electrical Engineering, ICCEE 2009, 2, 639–643. https://doi.org/10.1109/ICCEE.2009.247
[22] Ranjit Kumar. (2011). Research Methodology A Step-by-Step Guide for Beginners (3rd ed.). Sage Publication, Inc.
Richardson, S., Mishuris, R., O’Connell, A., Feldstein, D., Hess, R., Smith, P., … Mann, D. (2017). “Think aloud†and “Near live†usability testing of two complex clinical decision support tools. International Journal of Medical Informatics, 106, 1–8. https://doi.org/10.1016/j.ijmedinf.2017.06.003
[23] Stonbraker, S., Cho, H., Hermosi, G., Pichon, A., & Schnall, R. (2018). Usability Testing of a mHealth App to Support Self-Management of HIV-Associated Non-AIDS Related Symptoms. Studies in Health Technology and Informatics, 250, 106–110.
[24] Tarkkanen, K., & Harkke, V. (2019). Scope for usability tests in IS development. AIS Transactions on Human-Computer Interaction, 11(3), 136–156. https://doi.org/10.17705/1thci.00117
[25] Umar, A., & Khan Tatari, K. (2008). Appropriate Web Usability Evaluation Method during Product Development. Retrieved from www.bth.se/tek
[26] Widman, L., Golin, C. E., Kamke, K., Massey, J., & Prinstein, M. J. (2017). Feasibility and acceptability of a web-based HIV/STD prevention program for adolescent girls targeting sexual communication skills. Health Education Research, 32(4), 343–352. https://doi.org/10.1093/her/cyx048
[27] Wirasasmiata, R., & Uska, M. (2019). Evaluation of E-Rapor Usability using Usability Testing Method. Advances in Social Science, Education and Humanities Research (ASSEHR), 330(Iceri 2018), 71–74. https://doi.org/10.2991/iceri-18.2019.15
[28] Yen, P.-Y., Lara, B., Lopetegui, M., Bharat, A., Ardoin, S., Johnson, B., … Curtis, J. (2016). Usability and Workflow Evaluation of “RhEumAtic Disease activitY†(READY). Applied Clinical Informatics, 07(04), 1007–1024. https://doi.org/10.4338/aci-2016-03-ra-0036
[29] Yen, P. Y., Walker, D. M., Smith, J. M. G., Zhou, M. P., Menser, T. L., & McAlearney, A. S. (2018). Usability evaluation of a commercial inpatient portal. International Journal of Medical Informatics, 110(June 2017), 10–18. https://doi.org/10.1016/j.ijmedinf.2017.11.007
[30] Zhang, T., Rau, P.-L. P., Salvendy, G., & Zhou, J. (2013). Comparing Low and High-Fidelity Prototypes in Mobile Phone Evaluation. International Journal of Technology Diffusion, 3(4), 1–19. https://doi.org/10.4018/jtd.2012100101
Downloads
Published
2023-04-18
How to Cite
Kelana, B., L.R., A. F., Firmansyah, F., & A, K. S. (2023). Optimasi Remote Moderated Usability Testing Pada Low-Fidelity Prototype Dari E-Commerce Dengan Wawancara Pada Generasi Z Di Indonesia. Jutis (Jurnal Teknik Informatika), 11(1), 1–14. https://doi.org/10.33592/jutis.v11i1.3281
Issue
Section
Articles
