Ch 1: AI Concepts

• Artificial Intelligence, Machine Learning and Deep Learning
• Generative AI concepts and applications
• Traditional AI vs Generative AI
• Generative AI use cases in Data Engineering

Ch 2: Large Language Models & Prompt Engineering

• Introduction to LLMs and foundation models
• Tokens, context windows and inference
• Prompt engineering techniques
• Structured outputs and JSON responses
• Introduction to embeddings and similarity search

Ch 3: Generative AI with Databricks

• Databricks AI ecosystem overview
• Databricks AI Playground
• Foundation Model APIs
• Model serving endpoints
• Accessing LLMs using Python

Ch 4: Databricks AI Functions

• Introduction to Databricks AI functions
• Using ai_query() in SQL
• Calling foundation models from SQL
• AI-powered text processing
• Structured response generation

Ch 5: Intelligent Data Processing

• Text classification using AI
• Sentiment analysis
• AI-based information extraction
• Data enrichment using LLMs
• Working with customer reviews and feedback

Ch 6: Unstructured Data Engineering

• Working with PDF, JSON and text files
• Ingesting unstructured data
• Document parsing and preprocessing
• PySpark pipelines for AI-ready datasets
• Storing processed data in Delta Lake

Ch 7: AI Data Pipelines & Governance

• Delta Lake for AI datasets
• Unity Catalog for data governance
• Incremental data processing
• Embedding generation and refresh pipelines
• Automating AI-based ETL workflows

Ch 8: Introduction to RAG

• What is Retrieval-Augmented Generation?
• Limitations of traditional LLM applications
• RAG vs fine-tuning
• RAG architecture and components
• Enterprise RAG use cases

Ch 9: Document Processing for RAG

• PDF and document ingestion
• Extracting text from documents
• Document parsing and cleaning
• Text chunking strategies
• Preparing documents for vector search

Ch 10: Vector Embeddings

• Understanding text embeddings
• Embedding models
• Generating vector embeddings
• Vector similarity concepts
• Storing document embeddings

Ch 11: Databricks AI Search

• Introduction to Databricks AI Search (Vector Search)
• Vector indexes and index creation
• Delta Sync indexes
• Similarity search queries
• Retrieving relevant document chunks

Ch 12: Building RAG Applications

• Integrating vector retrieval with LLMs
• Context retrieval and prompt augmentation
• Generating context-aware responses
• Building a PDF-based AI chatbot
• Source citations and hallucination reduction

Ch 13: Advanced Retrieval Techniques

• Semantic search vs keyword search
• Hybrid search techniques
• Reranking retrieved documents
• Metadata-based filtering
• Chunk-size optimization

Ch 14: Intelligent RAG Pipelines

• Query rewriting
• Multi-query retrieval
• Structured RAG with SQL data
• Combining Delta tables and document retrieval
• Building hybrid structured/unstructured RAG applications

Ch 15: RAG Performance Optimization

• Evaluating retrieval relevance
• Answer groundedness
• Improving retrieval accuracy
• Reducing hallucinations
• Latency and token

Project Title
Retail Sales Analytics Platform using Databricks (Medallion Architecture)

Project Overview
Students will build a production-style data platform using Lakehouse Architecture (Bronze, Silver,
and Gold), implementing modern ETL practices, data quality validation, incremental processing,
orchestration, optimization, and reporting.

Business Scenario
A multinational retail company receives daily data from multiple operational systems. The
company wants to build a centralized analytics platform to:

• Consolidate data from different business domains
• Improve data quality and consistency with AI
• Track sales performance across regions
• Analyse customer purchasing behaviour
• Monitor inventory levels with AI
• Build AI Agents, RAG Artifacts
• Generate business KPIs for decision-makers
• Deliver Power BI dashboards with near real-time insights

Source Systems
Students will work with multiple datasets including:

• Customer Master
• Product Master
• Orders
• Inventory
• Region Master

Resume Highlights
After completing this project, you can confidently showcase experience in:

• PySpark & Spark SQL with AI
• Delta Lake with AI
• Medallion Architecture with LLMs
• ETL Pipeline Development with RAG
• Incremental Data Processing with Genie AI

This course provides hands-on training in Databricks, Spark SQL, Python, PySpark, Delta Lake, Generative AI, LLMs, AI Agents, and RAG applications. It focuses on building real-world data engineering and AI-powered solutions.

SQL Developers, Data Engineers, ETL Developers, Python Developers, Cloud Professionals, and IT professionals looking to upgrade their skills in Databricks, Generative AI, and RAG can join this training.

The course duration is 8 weeks, covering Databricks Data Engineering, Generative AI, LLMs, RAG applications, real-time projects, and certification guidance.

Yes. You will work on a real-world E-Commerce Data Engineering project involving Medallion Architecture, ETL pipelines, PySpark, Delta Lake, data quality, workflow automation, and AI-powered solutions.

Yes. The curriculum includes Generative AI, Large Language Models, Databricks AI Functions, prompt engineering, embeddings, Vector Search, RAG pipelines, and AI-powered applications.

Yes. SQL School provides Databricks Data Engineer Associate certification guidance, practice assessments, mock exams, scenario-based interview questions, resume project explanations, and technical mock interviews.