📖 AgenticQA Documentation

Welcome to AgenticQA documentation. This guide covers the architecture and implementation of AI-native test automation systems combining LLMs, RAG, and Agentic AI with Playwright.

11
AI Projects
3
Layer Architecture
5
Core Patterns
11
AI Projects
◉ Living Documentation

This documentation evolves with projects. Subscribe to YouTube @AgenticQA for tutorials.

🏗️ Architecture Overview

AgenticQA follows a three-layer architecture that separates AI planning from deterministic execution for CI/CD reliability.

Three-Layer Architecture

1 Intent Generation Layer AI-Powered
📝
Natural Language
Test Requirements
🔍
RAG Context
Domain Knowledge
🧠
LLM Planning
Claude / GPT-4
📋
Test Intent JSON
Structured Output
2 Validation Layer Critic Agent
✅
Schema Validation
JSON Structure Check
🎯
Selector Verification
DOM Existence
⚠️
Safety Guards
Action Limits
3 Execution Layer Deterministic No LLM Here!
🔌
MCP Server
Tool Protocol
🎭
Playwright
Browser Actions
📸
Screenshots
Evidence Capture
📊
Test Results
Pass/Fail Report
🔴 Critical Design Rule

NO LLM calls in Layer 3 - EVER. This ensures 100% deterministic CI/CD execution. Same intent = Same actions = Reproducible results.

⚙️ Tech Stack

🧠 AI/ML Stack

LangChain
LLM Orchestration & Chains
Core
ChromaDB
Vector Database
RAG
Ollama / OpenAI / Claude
LLM Providers
Flexible

🎭 Automation Stack

Playwright
Browser Automation
Core
TypeScript
Type-safe Test Code
Language
MCP Server
Model Context Protocol
Tools

🐍 Backend Stack

Python 3.11+
RAG & Agent Logic
Core
Pydantic
Data Validation
Validation
Streamlit
Web UI for AI Apps
UI

🚀 DevOps Stack

GitHub Actions
CI/CD Pipeline
CI/CD
Docker
Containerization
Deploy
Playwright Reports
HTML Reports & Artifacts
Reports

RAG Pipeline

Retrieval-Augmented Generation (RAG) enhances LLM responses with relevant context from your documentation, eliminating hallucinations.

RAG Pipeline Flow

📄
1. Document Ingestion
PDF → Chunk → Embed → Store
🔍
2. Query Processing
Query → Embed → Similarity Search
🎯
3. Context Assembly
Retrieved Chunks + Query → LLM

Embeddings

How Embeddings Work

"login button" → [0.23, -0.45, 0.78, ...]
"sign in link" → [0.21, -0.42, 0.81, ...] ← Similar!

Agentic AI

👁️
Observe
Analyze page DOM, current state, screenshots
🧠
Think
Plan next action based on test goal
⚡
Act
Execute via MCP tools (click, type, assert)

MCP Server

Available MCP Tools

🔗
navigate
Go to URL
👆
click
Click element
⌨️
fill
Type into input
✅
assert
Verify element
📸
screenshot
Capture page
🔍
smart_locate
AI selector healing

ResumeGenie AI

ResumeGenie AI Architecture

What It Does

  • • Analyzes resume vs job description
  • • Calculates match score (0-100)
  • • Provides actionable feedback
  • • Identifies missing keywords

Tech Stack

  • • Streamlit (Web UI)
  • • Python (Core Logic)
  • • Ollama / OpenAI (AI Analysis)
  • • Smart Prompt Engineering

StudyMate AI

StudyMate AI Architecture

What It Does

  • • Transforms PDFs into interactive learning tools
  • • Generates summaries, quizzes, flashcards & Q&A
  • • RAG-powered intelligence for accurate answers
  • • Creates interactive mind map visualizations

Tech Stack

  • • 7 Core Packages (Ultra-lightweight)
  • • Markmap.js (Mind Map Visualizations)
  • • RAG (Retrieval-Augmented Generation)
  • • OpenAI API (~$0.02/session)
  • • Privacy-first in-memory processing

FeedbackPulse AI

Turning Customer Sentiment into Actionable Intelligence

FeedbackPulse AI - Customer Sentiment Analysis

What It Does

  • • AI Sentiment Analysis with 85-90% accuracy
  • • Customer Health Scores for quick assessment
  • • Sentiment Classification (Positive/Neutral/Negative)
  • • Theme Extraction - identifies recurring topics
  • • Urgent Alert Detection for critical feedback
  • • Multi-Source Comparison (E-commerce, App Stores, Social)
  • • Interactive Visualizations with drill-down charts
  • • Multi-Format Export (JSON, CSV, Markdown)

Tech Stack

  • • Python (Core processing engine)
  • • Pandas (Data manipulation & analysis)
  • • Streamlit (Interactive web interface)
  • • Plotly (Interactive visualizations)
  • • OpenAI / Ollama (LLM-driven analysis)

⚡ Key Metrics

  • • 90-second rapid analysis (70+ reviews)
  • • 85-90% sentiment accuracy
  • • Free Local LLM Support included

MeetingMind AI

Reclaiming the $37 Billion Lost to Unproductive Meetings

MeetingMind AI - Enterprise Meeting Intelligence

What It Does

  • • Risk & Blocker Detection with severity levels
  • • Real-Time Meeting Cost Calculator
  • • Ask AI (Natural language Q&A on meetings)
  • • AI Action Items - automates takeaways & follow-ups
  • • Sentiment Analysis for participant mood
  • • Risk & Cost Tracking dashboard

Tech Stack

  • • Local AI Engine (Privacy-first processing)
  • • Python (Core backend logic)
  • • Streamlit (Interactive dashboard UI)
  • • Ollama (Free local LLM provider)
  • • NLP Processing (Advanced text extraction)

💰 Enterprise Power, Zero Cost

  • • MeetingMind AI: FREE
  • • Competitors (Fireflies/Otter): $5,000+

Interview Prep

Q: Why Playwright over Selenium?
Playwright offers auto-wait, better TypeScript support, multiple browser contexts, and network interception. It's Anthropic's MCP protocol's native choice for browser automation.
Q: What is RAG and why use it?
RAG retrieves relevant context from documents before LLM generates response. It eliminates hallucinations, provides sources, and can be updated instantly without retraining.
Q: Why separate intent from execution?
LLMs are non-deterministic. If in execution path, tests become flaky. By generating intent JSON first and executing deterministically, we get CI/CD reliability with AI flexibility.

Architecture Decision Records

ADR-001: Intent-Execution Separation

Decision: LLMs generate test intents (JSON), deterministic code executes them.
Rationale: CI/CD tests must be 100% deterministic.

ADR-002: Three-Layer Architecture

Decision: Separate Intent → Validation → Execution.
Key Rule: NO LLM calls in Layer 3 (Execution). Ever.

ADR-003: RAG over Fine-tuning

Decision: Use RAG (not fine-tuning) for domain knowledge.
Benefits: Instant updates, lower cost, transparent retrieval.