Ognjen Jovanović

Independent Project - Local-First Knowledge Graph with LLM Intelligence

Built an intelligent, local-first platform that converts unstructured data (web pages, PDFs) into a structured markdown-based knowledge vault. Features interactive force-directed graph visualization, hybrid semantic search (vector + BM25), and an automated Mint & Lint governance mechanism for structural health checks.

• Built an automated ingestion pipeline using the grapper engine with section-aware chunking that respects markdown heading boundaries while maintaining contextual overlap.
• Implemented interactive 2D force-directed knowledge graph visualization with nodes grouped by classification (concepts, entities, sources) for intuitive semantic exploration.
• Integrated qmd hybrid semantic search with retry-logic and contextually balanced JSON abstraction for high-fidelity local queries.
• Designed a Mint & Lint governance workflow that detects orphan concepts, contradictions, and syntax errors, proposing diffs for user review via async mutex-backed Git automation.

Independent Project - Desktop AI Agent Workspace with Rust Backend

Built a local-first desktop AI agent workspace combining Next.js, Tauri, and Rust. Features a command-deck interface with specialist agents (Software Engineer, Cybersecurity Specialist), multi-provider LLM routing, and explicit approval-gated tool execution where every action requires user confirmation.

• Built a Tauri 2 desktop application with a Next.js 16 frontend and Rust backend handling provider routing, local persistence, and tool execution via IPC.
• Implemented a provider-flexible LLM gateway supporting Anthropic, OpenAI, DeepSeek, Google Gemini, and Ollama-compatible local gateways with streaming output.
• Designed an explicit approval-gated tool execution system where every tool request is previewed before execution, with execution history persisted for auditability.
• Implemented persistent local state using SQLite for conversations and settings, plus OS keyring for credential storage, with project-scoped directory grants.

Independent Project - Grounded Diligence Workspace with Human-in-the-Loop Review

Built a review-oriented diligence workspace that turns source material into maintained, reviewable knowledge. Sources are ingested and normalized, accepted wiki pages become a maintained knowledge base, follow-on evidence becomes a review proposal requiring explicit human acceptance, and all output stays grounded in accepted knowledge.

• Built a Next.js workbench with Supabase (auth, Postgres, storage) and an Inngest worker surface for background workflow processing.
• Implemented model-backed wiki compilation into accepted diligence pages with narrow follow-on proposals for new evidence requiring explicit human acceptance.
• Designed a grounded Ask mode that generates responses anchored to accepted knowledge, preventing hallucination beyond the maintained corpus.
• Added workflow telemetry, diagnostics, and evaluation tooling with checked-in fixtures for repeatable quality assessment.

Independent Project - LLM-Guided Autonomous Training and Experimentation

Built a framework-agnostic autonomous training platform that modifies ML code, runs experiments locally or on remote GPU machines, evaluates results, reverts regressions via git, and repeats until a target metric or budget limit is reached.

• Built an autonomous experiment loop where an LLM agent proposes training changes, runs jobs, evaluates metrics, and keeps or reverts iterations using git-backed experiment history.
• Implemented framework detection and strategy modules spanning Ultralytics, Hugging Face Transformers, Keras, Lightning, scikit-learn, XGBoost, and generic PyTorch workflows.
• Added remote GPU execution with SSH plus rsync, crash recovery, budget enforcement, checkpoint resume, and structured logging for long-running training jobs.
• Built a React plus FastAPI dashboard with live WebSocket updates, GPU telemetry, training curves, iteration comparison, and agent reasoning history.

Independent Project - YOLOv11, ByteTrack, and Traffic Analytics

Built an end-to-end vehicle detection, multi-object tracking, and analytics pipeline that works on video files, webcams, RTSP streams, and YouTube input using YOLOv11 fine-tuned on KITTI.

• Fine-tuned YOLOv11n on KITTI and achieved mAP@0.5 = 0.850, with a large improvement over the pretrained baseline after correcting dataset and class alignment.
• Built real-time tracking with persistent IDs via ByteTrack, plus overlays for boxes, trails, FPS, line-crossing counts, zone occupancy, and class breakdowns.
• Added a unified vtrack CLI covering demo, detection, evaluation, benchmark-track, and remote training workflows with normalized artifact bundles.
• Supported Apple Silicon inference and remote CUDA training, including tracker preset benchmarking and export of CSV and JSON analytics.

Graduation Thesis — End-to-End Deep Learning on Embedded Hardware

Built a complete autonomous driving pipeline: dataset collection (~20,000 labeled image-command pairs), PilotNet-inspired CNN training on NVIDIA GPU, dynamic quantization (FP32 → INT8, ~1.2x speedup), and TorchScript deployment on Raspberry Pi 4.

• Achieved real-time inference at ~20–25 Hz on ARM hardware, enabling autonomous hallway navigation using monocular vision with behavioral cloning.
• Integrated four HC-SR04 ultrasonic sensors as an independent safety layer with emergency obstacle avoidance, direction-aware recovery maneuvers, and a 500 ms watchdog timer.
• Developed a multi-threaded Python control server (drive_server.py) with dedicated threads for TCP control, telemetry, and MJPEG video streaming.
• Created a cross-platform Flutter mobile application with dual-joystick control, live camera feed, real-time sensor HUD, and seamless manual/autonomous mode switching.

Current Research Project — Supervised Mammography Classification on CBIS-DDSM

Developing a research-grade mammography decision-support pipeline for benign-vs-malignant classification on CBIS-DDSM. The project started as a semi-supervised learning study, but rigorous baseline correction and multi-seed sweeps led to a stronger supervised direction built around EfficientNet-B0 at 512px with label smoothing, AdamW, and clinical-style evaluation.

• Built a config-driven PyTorch research framework with patient-aware splitting, reproducible sweeps, TensorBoard logging, and cross-device workflows spanning Apple Silicon development and a CUDA workstation.
• Implemented supervised, FixMatch, and Mean Teacher training paths, then uncovered and corrected a backbone-freezing baseline bug that changed the scientific conclusion of the project.
• Ran multi-seed sweeps across label budgets, resolution, backbone, regularization, and optimizer variants; promoted a 512px EfficientNet-B0 + label smoothing + AdamW baseline after a final Adam vs AdamW head-to-head.
• Added clinical-style evaluation tooling including PR AUC, calibration, specificity at fixed sensitivity, exam-level aggregation, failure-analysis exports, and research retrospectives in notebooks and PDF reports.

Independent Project — QLoRA Fine-Tuned LLM for Veterinary Diagnosis

QLoRA fine-tuned Qwen2.5-3B-Instruct for structured veterinary differential diagnosis. Given patient signalment and symptoms, produces ranked differentials, clinical reasoning, triage recommendations, and suggested next steps across dogs, cats, cattle, pigs, and sheep.

• Fine-tuned Qwen2.5-3B-Instruct using 4-bit QLoRA (NF4) on an 8GB RTX 3070, achieving structured diagnostic outputs within ≤512-token sequences.
• Built an end-to-end pipeline: dataset curation from HuggingFace sources, Ollama-based synthetic livestock case generation, SFTTrainer fine-tuning, and ROUGE-L/BERTScore evaluation with LLM-as-judge.
• Ran systematic ablation studies across LoRA rank (r=8/16/32) and dataset size (25%/50%/100%) to identify optimal training configuration.
• Deployed a Gradio demo UI for interactive symptom-to-diagnosis inference with the merged LoRA adapter.

Production‑Grade ML Pipeline for Fraud Detection

Built a production‑grade ML pipeline for credit card fraud detection with full MLOps tooling. Implemented feature engineering, model training (XGBoost, LightGBM, CatBoost) with Optuna hyperparameter optimization, ensemble methods, MLflow tracking, FastAPI inference service, and Evidently monitoring.

• Achieved PR‑AUC 0.867 with XGBoost on highly imbalanced data (0.172% fraud)
• Implemented end‑to‑end MLOps pipeline with Docker, MLflow, FastAPI
• Created stacking ensemble improving recall to 0.847 while maintaining precision
• Built full test suite (69 tests) and reproducible configuration

Personal Project — Local AI Research Environment

Architected a dedicated Ubuntu 24.04 workstation with NVIDIA RTX GPU acceleration for ML experimentation, LLM serving, and generative AI research.

• Deployed containerized ML infrastructure via Docker: Ollama, OpenWebUI, ComfyUI, and Lobe Chat with NVIDIA container toolkit passthrough.
• Configured CUDA/cuDNN toolchain and integrated PyTorch, TensorFlow, Keras, and OpenCV for end-to-end model development and benchmarking.
• Implemented a remote workstation-client workflow over LAN, eliminating cloud dependency for GPU-intensive research.

Independent Project — CNN Binary Classifier for Veterinary Diagnostics

Designing a Jupyter-based ML system for classifying trichinoscopy slide images as Trichinella-positive or negative, targeting ≥95% sensitivity for food safety applications.

• Architecture: EfficientNet-B3 with custom classifier head, Grad-CAM explainability overlays, Albumentations augmentation pipeline, and Docker-containerized GPU environment.

Research Paper — Library Evaluation with Practical Demonstration

Authored a research paper surveying widely-used ML/AI libraries, analyzing their strengths, ecosystem fit, and industry adoption.

• Developed a companion Jupyter Notebook with an image recognition model trained on CIFAR-10 using PyTorch and NumPy with NVIDIA GPU acceleration.

Client Work — Full-Stack Websites

Designed and deployed production websites including autizamjuznibanat.com and boggyart.com using Astro.js with contact form integrations (Formspree) and responsive design.