IMPLEMENTASI CHATBOT DENGAN LARGE LANGUAGE MODEL LLAMA 3.1 DENGAN TEKNOLOGI RETRIEVAL-AUGMENTED GENERATION (RAG) UNTUK LAYANAN AKADEMIK MAHASISWA INFORMATIKA UPN “VETERAN” YOGYAKARTA

Prayanto, Farel Abid Yasser (2025) IMPLEMENTASI CHATBOT DENGAN LARGE LANGUAGE MODEL LLAMA 3.1 DENGAN TEKNOLOGI RETRIEVAL-AUGMENTED GENERATION (RAG) UNTUK LAYANAN AKADEMIK MAHASISWA INFORMATIKA UPN “VETERAN” YOGYAKARTA. Skripsi thesis, UPN Veteran Yogyakarta.

	Text ABSTRAK Skripsi Farel Abid Yasser Prayanto 123210076.pdf Download (239kB)
	Text COVER Skripsi Farel Abid Yasser Prayanto 123210076.pdf Download (216kB)
	Text DAFTAR ISI Skripsi Farel Abid Yasser Prayanto 123210076.pdf Download (352kB)
	Text DAFTAR PUSTAKA Skripsi Farel Abid Yasser Prayanto 123210076.pdf Download (205kB)
	Text HALAMAN PENGESAHAN PEMBIMBING.pdf Download (4MB)
	Text HALAMAN PENGESAHAN PENGUJI.pdf Download (4MB)
	Text Skripsi Full Farel Abid Yasser Prayanto 123210076.pdf Restricted to Repository staff only Download (3MB)

Abstract

ABSTRAK
Dalam era digital saat ini, akses terhadap informasi akademik yang cepat, akurat, dan
efisien menjadi kebutuhan mendasar bagi mahasiswa. Layanan akademik tradisional
seringkali terbatas oleh jam operasional dan ketersediaan staf, sehingga tidak selalu mampu
memenuhi kebutuhan informasi mahasiswa secara real-time. Keterbatasan ini dapat
menghambat proses pembelajaran dan administrasi akademik. Penelitian ini bertujuan untuk
mengembangkan dan mengimplementasikan chatbot akademik berbasis Large Language
Model (LLM) Llama 3.1 dengan teknologi Retrieval-Augmented Generation (RAG) untuk
layanan informasi mahasiswa di Program Studi Informatika UPN "Veteran" Yogyakarta,
yang mampu memberikan respons akurat, natural, dan kontekstual terhadap pertanyaan
pertanyaan seputar informasi akademik.
Metodologi penelitian yang digunakan adalah eksperimental dengan pendekatan
kuantitatif, meliputi pengumpulan data melalui ekstraksi konten dari dokumen PDF resmi
universitas, pra-pemrosesan data dengan teknik pembersihan teks dan chunking
menggunakan RecursiveCharacterTextSplitter, pengembangan sistem berbasis Rapid
Application Development (RAD), dan implementasi teknologi RAG dengan pendekatan
hybrid retrieval yang menggabungkan BM25 dan semantic search. Arsitektur sistem terdiri
dari empat komponen utama: Document Store untuk penyimpanan dan pengindeksan
dokumen, Chatbot Utama yang mengintegrasikan Llama 3.1 melalui Ollama dan teknologi
RAG, User Interface berbasis Streamlit, dan Evaluation System untuk menilai performa
sistem secara komprehensif. Evaluasi dilakukan dengan menggunakan kerangka pengukuran
multidimensi yang mencakup metrik RAG-specific (context relevance, answer faithfulness),
metrik akurasi (token F1, ROUGE, BLEU), metrik kalibrasi, metrik keadilan, serta metrik
ketahanan dan konsistensi.
Kata Kunci: Large Language Model, Llama 3.1, Retrieval-Augmented Generation, chatbot
akademik, hybrid retrieval
vii
ABSTRACT
In today's digital era, rapid, accurate, and efficient access to academic information
has become a fundamental need for students. Traditional academic services are often limited
by operating hours and staff availability, making them unable to consistently meet students'
real-time information needs. These limitations can impede learning processes and academic
administration. This research aims to develop and implement an academic chatbot based on
the Llama 3.1 Large Language Model (LLM) with Retrieval-Augmented Generation (RAG)
technology for student information services at the Informatics Study Program of UPN
"Veteran" Yogyakarta, capable of providing accurate, natural, and contextual responses to
questions about academic information.
The methodology employed is experimental with a quantitative approach, including
data collection through content extraction from official university PDF documents, data pre
processing
with
text
cleaning
techniques
and
chunking
using
RecursiveCharacterTextSplitter, system development based on Rapid Application
Development (RAD), and implementation of RAG technology with a hybrid retrieval
approach combining BM25 and semantic search. The system architecture consists of four
main components: Document Store for document storage and indexing, Main Chatbot
integrating Llama 3.1 via Ollama and RAG technology, a Streamlit-based User Interface,
and an Evaluation System to comprehensively assess system performance. Evaluation was
conducted using a multidimensional measurement framework that includes RAG-specific
metrics (context relevance, answer faithfulness), accuracy metrics (token F1, ROUGE,
BLEU), calibration metrics, fairness metrics, as well as robustness and consistency metrics.
Keywords: Large Language Model, Llama 3.1, Retrieval-Augmented Generation, academic
chatbot, hybrid retrieval
viii

Item Type:	Tugas Akhir (Skripsi)
Uncontrolled Keywords:	Large Language Model, Llama 3.1, Retrieval-Augmented Generation, academic chatbot, hybrid retrieval
Subjek:	Z Bibliography. Library Science. Information Resources > ZA Information resources
Divisions:	Fakultas Teknik Industri > (S1) Informatika
Depositing User:	Eko Yuli
Date Deposited:	05 Nov 2025 02:43
Last Modified:	05 Nov 2025 02:43
URI:	http://eprints.upnyk.ac.id/id/eprint/45305

Actions (login required)

View Item