Ognjen Jovanović

• Build production AI across agents/tool workflows, retrieval & grounding, knowledge systems, and LLM/model evaluation.
• Work on training & experimentation, model serving/inference/optimization, and observability with human-in-the-loop controls.

Independent Project - Local-First Knowledge Graph with LLM Intelligence

Built an intelligent, local-first platform that converts unstructured data (web pages, PDFs) into a structured markdown-based knowledge vault. Features interactive force-directed graph visualization, local semantic search via qmd, and an automated Mint & Lint governance mechanism for structural health checks.

• Built an automated ingestion pipeline using the grapper engine with section-aware chunking that respects markdown heading boundaries while maintaining contextual overlap.
• Implemented interactive 2D force-directed knowledge graph visualization with nodes grouped by classification (concepts, entities, sources) for intuitive semantic exploration.
• Integrated qmd local semantic search with retry-logic and contextually balanced JSON abstraction for high-fidelity local queries.
• Designed a human-reviewed Mint & Lint governance workflow that detects orphan concepts, contradictions, and syntax errors, proposing diffs for user review via async mutex-backed Git automation.

Independent Project - Desktop AI Agent Workspace with Rust Backend

Built a local-first desktop AI agent workspace combining Next.js, Tauri, and Rust. Features a command-deck interface with specialist agents (Software Engineer, Cybersecurity Specialist), multi-provider LLM routing, and explicit approval-gated tool execution where every action requires user confirmation.

• Built a Tauri 2 desktop application with a Next.js 16 frontend and Rust backend handling provider routing, local persistence, and tool execution via IPC.
• Implemented a provider-flexible LLM gateway supporting Anthropic, OpenAI, DeepSeek, Google Gemini, and Ollama-compatible local gateways with streaming output.
• Designed an explicit approval-gated tool execution system where every tool request is previewed before execution, with execution history persisted for auditability.
• Implemented persistent local state using SQLite for conversations and settings, plus OS keyring for credential storage, with project-scoped directory grants.

Independent Project - Grounded Diligence Workspace with Human-in-the-Loop Review

Built a review-oriented diligence workspace that turns source material into maintained, reviewable knowledge. Sources are ingested and normalized, accepted wiki pages become a maintained knowledge base, follow-on evidence becomes a review proposal requiring explicit human acceptance, and all output stays grounded in accepted knowledge.

• Built a Next.js workbench with Supabase (auth, Postgres, storage) and an Inngest worker surface for background workflow processing.
• Implemented model-backed wiki compilation into accepted diligence pages with narrow follow-on proposals for new evidence requiring explicit human acceptance.
• Designed a grounded Ask mode that generates responses anchored to accepted knowledge, preventing hallucination beyond the maintained corpus.
• Added workflow telemetry, diagnostics, and evaluation tooling with checked-in fixtures for repeatable quality assessment.

Independent Project - LLM-Guided Autonomous Training and Experimentation

Built a framework-agnostic autonomous training platform that modifies ML code, runs experiments locally or on remote GPU machines, evaluates results, reverts regressions via git, and repeats until a target metric or budget limit is reached.

• Built an autonomous experiment loop where an LLM agent proposes training changes, runs jobs, evaluates metrics, and keeps or reverts iterations using git-backed experiment history.
• Implemented framework detection and strategy modules spanning Ultralytics, Hugging Face Transformers, Keras, Lightning, scikit-learn, XGBoost, and generic PyTorch workflows.
• Added remote GPU execution with SSH plus rsync, crash recovery, budget enforcement, checkpoint resume, and structured logging for long-running training jobs.
• Built a React plus FastAPI dashboard with live WebSocket updates, GPU telemetry, training curves, iteration comparison, and agent reasoning history.

Independent Project - YOLOv11, ByteTrack, and Traffic Analytics

Built an end-to-end vehicle detection, multi-object tracking, and analytics pipeline that works on video files, webcams, RTSP streams, and YouTube input using YOLOv11 fine-tuned on KITTI.

• Fine-tuned YOLOv11n on KITTI and achieved mAP@0.5 = 0.850 (39× over the pretrained COCO baseline after correcting dataset and class alignment).
• Built real-time tracking with persistent IDs via ByteTrack and BoT-SORT, plus overlays for boxes, trails, FPS, line-crossing counts, zone occupancy, and class breakdowns.
• Added a unified vtrack CLI covering demo, detection, evaluation, benchmark-track, and remote training workflows with normalized artifact bundles.
• Supported Apple Silicon MPS inference and remote CUDA training, including tracker preset benchmarking and export of CSV and JSON analytics.

Graduation Thesis — End-to-End Deep Learning on Embedded Hardware

Built a complete autonomous driving pipeline: dataset collection (~20,000 labeled image-command pairs), PilotNet-inspired CNN training on NVIDIA GPU, dynamic INT8 quantization, and TorchScript deployment on Raspberry Pi 4.

• Achieved real-time inference at ~20 Hz on ARM hardware, enabling autonomous navigation using monocular vision with behavioral cloning.
• Integrated four HC-SR04 ultrasonic sensors as an independent safety layer with emergency obstacle avoidance, direction-aware recovery maneuvers, and a 500 ms watchdog timer.
• Developed a multi-threaded Python control server (drive_server.py) with dedicated threads for TCP control, telemetry, and MJPEG video streaming.
• Created a cross-platform Flutter mobile application with dual-joystick control, live camera feed, real-time sensor HUD, and seamless manual/autonomous mode switching.

Current Research Project — Supervised Mammography Classification on CBIS-DDSM

Developing a research-grade mammography decision-support pipeline for benign-vs-malignant classification on CBIS-DDSM. The project started as a semi-supervised learning study, but rigorous baseline correction and multi-seed sweeps led to a stronger supervised direction built around EfficientNet-B0 at 512px with label smoothing, AdamW, and clinical-style evaluation.

• Built a config-driven PyTorch research framework with patient-aware splitting, reproducible sweeps, TensorBoard logging, and cross-device workflows spanning Apple Silicon development and a CUDA workstation.
• Implemented supervised, FixMatch, and Mean Teacher training paths, then uncovered and corrected a backbone-freezing baseline bug that changed the scientific conclusion of the project.
• Ran multi-seed sweeps across label budgets, resolution, backbone, regularization, and optimizer variants; promoted a 512px EfficientNet-B0 + label smoothing + AdamW baseline after a final Adam vs AdamW head-to-head.
• Added clinical-style evaluation tooling including PR AUC, calibration, specificity at fixed sensitivity, exam-level aggregation, failure-analysis exports, and research retrospectives in notebooks and PDF reports.

Independent Project — QLoRA Fine-Tuned LLM for Veterinary Diagnosis

QLoRA fine-tuned Qwen2.5-3B-Instruct for structured veterinary differential diagnosis. Given patient signalment and symptoms, produces ranked differentials, clinical reasoning, triage recommendations, and suggested next steps across dogs, cats, cattle, pigs, and sheep.

• Fine-tuned Qwen2.5-3B-Instruct using 4-bit QLoRA (NF4) on an 8GB RTX 3070, achieving structured diagnostic outputs within ≤512-token sequences.
• Built an end-to-end pipeline: dataset curation from HuggingFace sources, Ollama-based synthetic livestock case generation, SFTTrainer fine-tuning, and ROUGE-L/BERTScore evaluation with LLM-as-judge.
• Ran systematic ablation studies across LoRA rank (r=8/16/32) and dataset size (25%/50%/100%) to identify optimal training configuration.
• Deployed a Gradio demo UI for interactive symptom-to-diagnosis inference with the merged LoRA adapter.

End-to-End MLOps Pipeline for Fraud Detection

Built an end-to-end tabular ML pipeline for credit card fraud detection with feature engineering, Optuna-tuned gradient boosting, ensemble methods, MLflow tracking, FastAPI inference, and Evidently monitoring. Portfolio/academic demo on the public Kaggle dataset.

• Achieved test Average Precision 0.867 and ROC-AUC 0.977 with XGBoost on highly imbalanced data (0.172% fraud)
• Implemented end-to-end MLOps-style stack with Docker, MLflow, FastAPI, and Evidently drift monitoring
• Created stacking ensemble improving recall to 0.847 at ROC-AUC 0.978
• Built full pytest suite and reproducible YAML-driven configuration

Personal Project — Local AI Research Environment

Architected a dedicated Ubuntu 24.04 workstation with NVIDIA RTX GPU acceleration for ML experimentation, LLM serving, and generative AI research.

• Deployed containerized ML infrastructure via Docker: Ollama, OpenWebUI, ComfyUI, and Lobe Chat with NVIDIA container toolkit passthrough.
• Configured CUDA/cuDNN toolchain and integrated PyTorch, TensorFlow, Keras, and OpenCV for end-to-end model development and benchmarking.
• Implemented a remote workstation-client workflow over LAN, eliminating cloud dependency for GPU-intensive research.

Independent Project — CNN Binary Classifier for Veterinary Diagnostics

Designing a Jupyter-based ML system for classifying trichinoscopy slide images as Trichinella-positive or negative, targeting ≥95% sensitivity for food safety applications.

• Architecture: EfficientNet-B3 with custom classifier head, Grad-CAM explainability overlays, Albumentations augmentation pipeline, and Docker-containerized GPU environment.

Research Paper — Library Evaluation with Practical Demonstration

Authored a research paper surveying widely-used ML/AI libraries, analyzing their strengths, ecosystem fit, and industry adoption.

• Developed a companion Jupyter Notebook with an image recognition model trained on CIFAR-10 using PyTorch and NumPy with NVIDIA GPU acceleration.

Client Work — Full-Stack Websites

Designed and deployed production websites including autizamjuznibanat.com and boggyart.com using Astro.js with contact form integrations (Formspree) and responsive design.