Performance Evaluation Web Scraping BeautifulSoup and Lxml in the ConvexView Application

Achmad Ulul Azmi Wafiqi; Mochamad  Taufiqurrochman Abdul Aziz Zein; Tri  Anggoro

doi:10.33365/jtk.v20i1.472

Achmad Ulul Azmi Wafiqi Universitas Nahdlatul Ulama Al Ghazali
Mochamad Taufiqurrochman Abdul Aziz Zein Universitas Nahdlatul Ulama Al Ghazali
Tri Anggoro Universitas Nahdlatul Ulama Al Ghazali

DOI: https://doi.org/10.33365/jtk.v20i1.472

Keywords: Web Scraping, Himawari-9 satellite imagery, convective clouds, web applications, prototyping.

Abstract

Weather is a crucial factor in human life, especially in agriculture, shipping, and other activities. One of the main indicators of extreme weather conditions is convective clouds, which are clouds formed by warm air convection and have the potential to cause heavy rain, lightning, and strong winds. Monitoring convective clouds can currently be done using Himawari-9 satellite imagery, which provides infrared imagery data to detect the presence of clouds based on their peak temperature. However, this data is still raw and difficult for the general public to interpret. To address this issue, this study aims to design and develop a web-based ConvexView application capable of automatically visualizing Himawari-9 satellite imagery, specifically for the Cilacap region. This study also compares two Web Scraping techniques, BeautifulSoup and lxml, to determine the most optimal data extraction technique. The programming language used is Python, with FastAPI support for the backend and React JS for the frontend. The development method used is Prototyping, as it allows development to be carried out in stages and involves users in the design process. This application is designed so that users can view real-time visualizations of convective clouds and download the images. It is hoped that the results of this research will not only contribute in the form of a web-based application, but also serve as a reference for the development of satellite image-based meteorological applications in the future.

References

A. R. S. Maghriza, Y. D. Haryanto, Munawar, O. Yosafat, and N. F. Silalahi, “ANALISIS KEJADIAN HUJAN LEBAT DI WILAYAH CILACAP,” vol. 5, no. 2, pp. 144–154, 2024, doi: 10.53682/gjppg.v5i2.8499.

M. O. R. Hutagalung, “ANALISIS KEJADIAN HUJAN LEBAT MENGGUNAKAN DATA CITRA SATELIT HIMAWARI-8 (Studi Kasus Kota Manado, 16 Januari 2021),” Fisitek J. Ilmu Fis. dan Teknol., vol. 6, no. 2, pp. 16–22, 2023, doi: 10.30821/fisitekfisitek.v6i2.14445.

R. F. Ramdani, “Analisis Kejadian Hujan Lebat dan Banjir Kabupaten Pati Menggunakan Metode Cloud Convective Overlays dan Red Green Blue Convective Storms pada Satelit Himawari 8,” J. Penelit. Sains, vol. 23, no. 3, p. 150, 2021, doi: 10.56064/jps.v23i3.647.

R. Kurniati, S. L. H. Pakpahan, and A. Mulya, “Analisis Kejadian Hujan Lebat Menggunakan Citra Satelit HIMAWARI-8 (Studi Kasus Pulau Bintan, 7 November 2020),” Pros. Semin. Nas., vol. 3, no. 1, pp. 130–141, 2021.

A. Stevan Yondra, D. Triyanto, and S. Bahri, “Implementasi Web Scraping Untuk Mengumpulkan Informasi Produk Dari Situs E-Commerce,” J. Komput. Dan Apl., vol. 10, no. 01, 2022.

K. Dwicahyo and C. Indah Ratnasari, “Perbandingan Metode Web Scraping Dalam Pengambilan Data: Kajian Literatur,” Automata, vol. 4, 2023.

dan G. Badan Meteorologi, Klimatologi, “Citra Himawari-9 IR Enhanced - Indonesia.” https://www.bmkg.go.id/cuaca/satelit/himawari-ir-enhanced

U. Khandelwal and A. KB, “FastAPI vs. The Competition: A Security Feature Showdown with a Proposed Model for Enhanced Protection,” Interantional J. Sci. Res. Eng. Manag., vol. 08, no. 06, pp. 1–5, 2024, doi: 10.55041/ijsrem35726.

F. Sembiring and D. P. Sari, “Penerapan teknik scraping python pada website marketplace indonesia,” Integr. (Journal Inf. Technol. Vocat. Educ., vol. 2, no. 1, pp. 15–22, 2020, doi: 10.17509/integrated.v2i1.28243.

K. Sharma and G. M. Borkar, “Comparative Analysis of Dynamic Web Scraping Strategies: Evaluating Techniques for Enhanced Data Acquisition,” Adv. Commun. Syst., pp. 241–252, 2024, doi: 10.56155/978-81-955020-7-3-22.

E. Uzun, T. Yerlikaya, and O. Kırat, “Comparison of Python Libraries used for Web Data Extraction,” J. Tech. Univ. - Sofia Plovdiv branch, Bulg., vol. 24, no. May, pp. 87–92, 2018, [Online]. Available: https://erdincuzun.com/wp-content/uploads/download/plovdiv_journal_2018_01.pdf

M. D. Al Farizi, R. Hidayat, and M. Abdi, “OPTIMASI METODE SCRAPING DATA PRODUK DARI PLATFORM OPTIMIZATION OF DATA SCRAPING METHODS FOR PRODUCTS FROM TOKOPEDIA . COM PLATFORM,” vol. 2, no. September, pp. 1172–1181, 2023.

S. T. Aji, S. Rosad, and M. Abror, “Penerapan Teknik Scraping Pada Prototype Papan,” CENTIVE, vol. 3, no. Dec, 2023.

D. Ardiyansah, O. Pahlevi, and T. Santoso, “Implementasi Metode Prototyping Pada Sistem Informasi Pengadaan Barang Cetakan Berbasis Web,” Hexag. J. Tek. dan Sains, vol. 2, no. 2, pp. 17–22, 2021, doi: 10.36761/hexagon.v2i2.1083.

A. Fikriyya and R. T. Dirgahayu, “Implementasi Prototyping dalam Perancangan Sistem Informasi Sekolah Desa Pendar Foundation Yogyakarta,” J. UII Autom., vol. 1, no. 2, pp. 1–9, 2020.

E. W. Fridayanthie, H. Haryanto, and T. Tsabitah, “Penerapan Metode Prototype Pada Perancangan Sistem Informasi Penggajian Karyawan (Persis Gawan) Berbasis Web,” Paradig. - J. Komput. dan Inform., vol. 23, no. 2, pp. 151–157, 2021, doi: 10.31294/p.v23i2.10998.

S. A. Rismawan and Y. Syahidin, “Implementasi Website Berita Online Menggunakan Metode Crawling Data Dengan Bahasa Pemrograman Python,” J. Tek. Inform. dan Sist. Inf. , vol. 10, no. 3, pp. 167–178, 2023, [Online]. Available: http://jurnal.mdp.ac.id

R. Yusuf Azhari, “Web Service Framework : Flask Dan Fastapi,” Technol. Informatics Insight J., vol. 1, no. 1, pp. 58–65, 2022, doi: 10.32639/tiij.v1i1.54.

J. Chen, “Application Technology of Atmospheric Dispersion Models Based on FastAPI+Vue,” Acad. J. Eng. Technol. Sci., vol. 6, no. 11, pp. 120–126, 2023, doi: 10.25236/ajets.2023.061118.