GSoC 2026 Week 14

14 Aug 21, 2026 – Aug 28, 2026

Week 14 was the closing chapter of GSoC 2026. The four benchmark experiments — Issues #3 through #6 — had given me a full empirical picture of how different LLM strategies perform on the Amharic property mapping task. My job for this final week was to communicate those results clearly: a structured progress report, an interactive 12-slide presentation for mentors and the broader DBpedia community, and a deployed infrastructure piece that removes the last major technical blocker for the Amharic DBpedia website's interactive features.

The progress report

I wrote PROGRESS_REPORT.md in the LLMIntegration repository as a nine-section document covering everything from the problem statement to the final cross-experiment comparison, aimed at someone familiar with NLP but not necessarily with DBpedia or Amharic: it introduces the task with concrete Ge'ez script examples, explains the mathematical foundations of the cosine similarity retriever and the fuzzy accuracy metric, and presents all four experiment results in tables alongside plain-language interpretations.

The report also includes a cost-versus-accuracy trade-off analysis: larger models and more complex strategies give higher accuracy, but the gains shrink each step. The ensemble of five members (Issue #6) hits the highest measured accuracy at 67.62%, but adds roughly 15× the inference cost of a single zero-shot call. For a production pipeline that needs to process every Wikipedia dump, that trade-off matters.

Benchmark results across all four experiments

I don't read the 67.62% ceiling on Issue #6 as a failure — it's a measurement. It tells me exactly where the current approach tops out and what the next phase of work needs to target: better-quality training data for few-shot examples (curated, diverse demonstrations rather than random samples), a retriever fine-tuned on the Amharic DBpedia vocabulary rather than a general-purpose multilingual encoder, and fixing the extraction framework bugs so the ground truth labels themselves are more reliable.

The presentation

Alongside the written report, I published a 12-slide interactive presentation on this portfolio website called Multilingual LLM Benchmarking. The slides walk through the problem statement, the pipeline architecture, the mathematical foundations, and all four experiment results, with animated bar charts and data tables that make the cross-experiment comparison easy to read at a glance.

I designed it to be self-contained: someone watching without any DBpedia or Amharic background should understand the problem and the results by the final slide. Every slide has a specific purpose — nothing purely decorative — and the information density matches what you can absorb in about thirty seconds per slide at a normal presentation pace.

View the presentation →

kgproxy: solving the VPN problem

Since Week 8, a recurring blocker for the Amharic DBpedia website was the VPN requirement for the SPARQL endpoint. The internal endpoint is only accessible from within the institutional network, so the public website — hosted on GitHub Pages as static files — couldn't make live SPARQL queries to it. Every interactive feature that depended on real data was either faked with static snapshots or just deferred.

The endpoint failing from outside the VPN wasn't a one-off outage — it was the default state for any caller without institutional network access, and it kept coming back as a blocker every time a new feature needed real data. That's what pushed me to read James Hamilton's "On Designing and Deploying Internet-Scale Services" (LISA '07), whose first design tenet is design for failure: expect a dependency to be unreachable at any time and build the calling side to handle that, instead of treating it as an edge case. So instead of waiting on DBpedia to change how the endpoint is hosted, I built a small piece of infrastructure on the calling side that assumes the endpoint can be unreachable and works around it.

kgproxy solves this by acting as an authenticated intermediary. It runs inside the VPN (on an EC2 instance peered into the institutional network) and exposes a public HTTP endpoint for SPARQL queries. Any client — including a JavaScript function running in a visitor's browser on the Amharic DBpedia website — can send a SPARQL query to kgproxy's public URL and get a response back, no VPN credentials needed.

The kgproxy repository is at github.com/Nama21yo/kgproxy. Deploying it this week unblocked the interactive data features on the Amharic DBpedia website and proved that the VPN problem, while real, was solvable with a small piece of infrastructure rather than a change to the DBpedia endpoint itself.

Looking back and forward

Fourteen weeks is a short time to make measurable progress on a genuinely hard problem: building a pipeline that maps low-resource-language text to a large, structured ontology with limited training data, limited GPU access for most of the summer, and a base infrastructure (the extraction framework) that needed debugging before I could trust it. The benchmarks showed the two-stage retrieve-then-rerank approach works — the best ensemble hits 67.62% accuracy on a 279-example test set — and the experiments told me which levers move accuracy most reliably: better few-shot examples, chain-of-thought reasoning, and cross-lingual signal from Amharic-to-English translation.

The extraction framework work produced seven formally filed bug reports that are concrete, actionable improvements to how DBpedia processes Amharic Wikipedia content. The kgproxy deployment solved an infrastructure problem that had been blocking the public website for six weeks. And the template mappings I added throughout the summer incrementally grew the fraction of Amharic Wikipedia content that DBpedia can represent as structured Linked Data.

The work isn't finished — the extraction framework bugs are upstream fixes waiting to be merged, the LLM pipeline has a ceiling that better training data and a fine-tuned retriever should be able to raise, and the Amharic DBpedia graph has a long way to go before it gets close to the coverage of its English counterpart. But the summer left a foundation behind: documented, reproducible, and open-source, ready for whoever picks it up next.