miva-lms-study-assistant

byDieke Sydney

Here is the fully updated, complete master prompt reflecting all aspects of your system: the Miva SIS \rightarrow LMS SSO authentication redirect, Playwright PDF scraper, Vimeo caption network interceptor, pdfplumber document parser, and Gemini 2.5 Flash note generator. Master Prompt (Copy & Paste into Bolt.new, Replit Agent, or Cursor): Build a full-stack, modular web application called "Miva LMS Study Assistant" that automates logging into Miva Open University's portal, downloading PDF slides, intercepting closed-caption transcripts from embedded Vimeo video lectures, and generating comprehensive Markdown study notes using Gemini 2.5 Flash. --- ### 1. Technology Stack & Requirements - **Backend / Automation:** Python 3.11+ using `playwright` (Async API) - **PDF Extraction:** `pdfplumber` (with `pypdf` fallback) - **Video Transcript Capture:** Playwright background network listener filtering for `.vtt` / Vimeo text-track endpoints. - **AI Model:** `google-genai` SDK using `gemini-2.5-flash` - **Dashboard UI:** `streamlit` (or React/TypeScript full-stack equivalent) - **Environment & Secrets:** `PORTAL_USER`, `PORTAL_PASS`, and `GEMINI_API_KEY` stored in `.env`. --- ### 2. Multi-Stage Pipeline Architecture #### Stage A: SSO Authentication & Persistence (`scraper.py`) 1. **Login Flow:** Start at `https://sis.miva.university`. Fill in `PORTAL_USER` and `PORTAL_PASS`, submit the form, and wait for the cross-domain OAuth redirect to `https://lms.miva.university/**` using `page.wait_for_url()`. 2. **Session Storage:** Save the browser context to `miva_session.json` using `context.storage_state()`. 3. **Session Reuse:** On subsequent runs, check if `miva_session.json` exists. Load it into `browser.new_context(storage_state="miva_session.json")` to bypass the login form. If the session has expired, catch the error, delete `miva_session.json`, and re-authenticate. #### Stage B: Media & Material Extraction Engine 1. **Vimeo Transcript Interception:** - Attach an event listener to Playwright's network responses: `page.on("response", handle_response)`. - Inspect all incoming response URLs for `.vtt`, `texttrack`, or Vimeo caption tracks. - Extract raw WebVTT text and clean it by stripping `WEBVTT` headers, timestamp cues (`00:01:20 --> 00:01:23`), and inline HTML tags (`<c>`, `<i>`). - Find all embedded `<iframe>` elements matching `vimeo.com` on the LMS page and trigger play button clicks to force the player to load caption network requests. 2. **PDF Downloads:** - Locate all downloadable PDF links (`a[href*='.pdf']` or `a[href*='resource/view.php']`). - Download files into a local folder: `downloads/week_content/`. #### Stage C: Document Text Extraction (`pdf_processor.py`) - Read every PDF in `downloads/week_content/` using `pdfplumber`. Fall back to `pypdf` if `pdfplumber` fails. - Format all extracted PDF texts and intercepted Vimeo transcripts into a unified context payload: === VIMEO LECTURE TRANSCRIPT === [Cleaned transcript text from video] === DOCUMENT: Lecture_Slides_Week1.pdf === [Extracted PDF page text] - If a PDF yields zero characters, append: `[WARNING: Scanned/Image PDF detected for <filename>]`. #### Stage D: Gemini Synthesis Engine (`generator.py`) - Initialize the Google GenAI client: `from google import genai`. - Call `gemini-2.5-flash` with `temperature=0.2` and the following strict system instruction: ```text You are an academic tutor and research assistant. Analyze the provided PDF slides and Vimeo lecture transcripts for a given week and generate a unified, highly detailed study guide in clean Markdown. Structure: # [Course Name] — Week [Number] Study Notes ## 1. Executive Summary - 3 to 4 sentences synthesizing spoken lecture points and slide readings. ## 2. Key Terminology & Definitions - **Term**: Definition and context referenced in lectures or slides. ## 3. Core Concepts & Lecture Breakdown (Group topics logically into subsections. Synthesize slide theories alongside key verbal points or emphasis made by the lecturer during the video.) ## 4. Practice & Exam Review Questions (5 high-yield revision questions with brief answer hints to test comprehension.) 3. Dashboard UI & Workflows (app.py) * Sidebar Inputs: * Status check for PORTAL_USER, PORTAL_PASS, and GEMINI_API_KEY. * Fields for Course Name (e.g., "Data Science"), Week Identifier (e.g., "Week 1"), and Target LMS Week URL. * Button to "Clear Saved Session" (removes miva_session.json). * Main View & Progress Updates: * Action button: "Run Auto-Extraction & Generate Notes". * Live progress tracker showing: * Authenticating at sis.miva.university / Restoring session... * Intercepting Vimeo transcripts & downloading PDFs from LMS... * Parsing PDF text buffers & cleaning VTT captions... * Synthesizing notes with Gemini 2.5 Flash... * Output display with 3 tabs: * Tab 1: Study Notes — Rendered Markdown preview with a "Download Notes (.md)" button. * Tab 2: Extracted Context — View combined PDF text + intercepted Vimeo transcripts. * Tab 3: Downloaded Files — Summary list of saved PDF files and captured audio tracks. 4. Error Resilience & Setup * Execute playwright install chromium automatically if browser binaries are missing. * Ensure all directories (downloads/week_content/, output_notes/) are created if missing. * Wrap all network navigation and selector interactions in try/except blocks to prevent crashes on portal DOM updates.

LandingPipelineLoginNotesDashboardSessionSign UpFilesContext
Landing

Comments (0)

No comments yet. Be the first!

Project Tasks

110 tasks
#1

Generate system requirement document

0m 43s0.1 cr used
Done
#2

Generate personas & user flows

0m 7s0.1 cr used
Done
#7

Create flow for Miva Open University Student

0m 6sCredits in parent
Done
#8

Landing

8m 25sCredits in subtasks
Done
#24

Repair Landing JSX

1m 21s0.2 cr used
Done
#20

Landing / Navigation

0m 24s0.8 cr used
Done
#17

Landing / Pipeline Hero

0m 49s0.8 cr used
Done
#18

Landing / Sso Redirect Diagram

0m 41s0.8 cr used
Done
#19

Landing / Entry Actions

0m 41s0.8 cr used
Done
#21

Landing / Footer

0m 18s0.8 cr used
Done
#22

Navigation

0m 24sCredits in parent
Done
#23

Footer

0m 18sCredits in parent
Done
#9

Sign Up

4m 17sCredits in subtasks
Done
#33

Repair Sign Up JSX

0m 27s0.2 cr used
Done
#25

Sign Up / Navigation Slot

0m 0s0.8 cr used
Done
#26

Sign Up / Enrollment Form

0m 32s0.8 cr used
Done
#27

Sign Up / Identity Context

0m 18s0.8 cr used
Done
#28

Sign Up / Footer Slot

0m 0s0.8 cr used
Done
#10

Login

20m 14sCredits in subtasks
Done
#55

Fix Login review finding: Measured horizontal overflow at desktop: the page is…

0m 13s0.4 cr used
Done
#29

Login / Navigation Slot

0m 0s0.8 cr used
Done
#30

Login / Credential Form

0m 32s0.8 cr used
Done
#31

Login / Identity Context

0m 20s0.8 cr used
Done
#32

Login / Footer Slot

0m 0s0.8 cr used
Done
#11

Dashboard

29m 52sCredits in subtasks
Done
#73

Fix Dashboard review finding: Dashboard workspace is clipped in the mobile viewport and…

0m 24s0.4 cr used
Done
#72

Fix Dashboard review finding: Dashboard workspace is clipped in the mobile viewport and…

0m 22s0.4 cr used
Done
#70

Fix Dashboard review finding: Desktop dashboard content is missing from the visible page;

1m 8s0.4 cr used
Done
#69

Fix Dashboard review finding: Desktop dashboard content is missing from the visible page;

0m 20s0.4 cr used
Done
#68

Fix Dashboard review finding: Desktop dashboard content is missing from the visible page;

0m 34s0.4 cr used
Done
#67

Fix Dashboard review findings: Desktop dashboard content is missing from the visible page; (+3 more)

1m 25s0.4 cr used
Done
#63

Dashboard / Navigation

0m 0s1.6 cr used
Backlog
#64

Dashboard / Engine Sidebar

0m 22s1.6 cr used
Done
#65

Dashboard / Setup Workspace

0m 37s1.6 cr used
Done
#66

Dashboard / Footer

0m 0s1.6 cr used
Backlog
#12

Session

39m 18sCredits in subtasks
Done
#60

Fix Session review finding: Session filename breaks awkwardly within the artifact row…

0m 20s0.4 cr used
Done
#57

Fix Session review findings: Desktop session content is missing; (+2 more)

0m 46s0.4 cr used
Done
#34

Session / Shell Header

0m 1s0.8 cr used
Done
#35

Session / Engine Sidebar

0m 29s0.8 cr used
Done
#36

Session / Clear Panel

0m 47s0.8 cr used
Done
#37

Session / Shell Footer

0m 2s0.8 cr used
Done
#13

Pipeline

39m 49sCredits in subtasks
Done
#61

Fix Pipeline review finding: Mobile Run log content is clipped inside a fixed-height…

0m 17s0.4 cr used
Done
#59

Fix Pipeline review finding: Mobile Run log content is clipped inside a fixed-height…

0m 20s0.4 cr used
Done
#38

Pipeline / Navigation

0m 2s0.8 cr used
Done
#39

Pipeline / Engine Sidebar

0m 36s0.8 cr used
Done
#40

Pipeline / Run Console

1m 20s0.8 cr used
Done
#41

Pipeline / Output Tabs

0m 25s0.8 cr used
Done
#42

Pipeline / Footer

0m 1s0.8 cr used
Done
#14

Notes

22m 19sCredits in subtasks
Done
#45

Notes / Navigation

0m 1s0.8 cr used
Done
#43

Notes / Engine Sidebar

0m 47s0.8 cr used
Done
#44

Notes / Study Guide Pane

0m 53s0.8 cr used
Done
#46

Notes / Footer

0m 1s0.8 cr used
Done
#15

Context

38m 15sCredits in subtasks
Done
#58

Fix Context review findings: Context Payload Pane.jsx renders a hardcoded `status` state… (+2 more)

1m 15s0.4 cr used
Done
#47

Context / Navigation Slot

0m 0s0.8 cr used
Done
#48

Context / Engine Sidebar

0m 29s0.8 cr used
Done
#49

Context / Payload Pane

0m 46s0.8 cr used
Done
#50

Context / Footer Slot

0m 1s0.8 cr used
Done
#16

Files

35m 51sCredits in subtasks
Done
#56

Fix Files review findings: Desktop layout omits the sidebar and places the guard and… (+3 more)

13m 28s0.4 cr used
Done
#51

Files / Shared Header

0m 2s0.8 cr used
Done
#52

Files / Engine Sidebar

0m 26s0.8 cr used
Done
#53

Files / Results Column

0m 44s0.8 cr used
Done
#54

Files / Shared Footer

0m 1s0.8 cr used
Done
#62

Retry Failed Pages

9m 58s0.4 cr used
Error
#71

Retry Failed Pages

4m 5s0.4 cr used
Done
#5

Architecture

2m 17s0.1 cr used
Done
#74

Create backend_modules diagram

1m 6sCredits in parent
Done
#75

Create er_diagram diagram

1m 6sCredits in parent
Done
#76

Create frontend_modules diagram

1m 6sCredits in parent
Done
#77

Create sequence diagram

1m 6sCredits in parent
Done
#78

Create service_network diagram

1m 6sCredits in parent
Done
#6

Workspace task plan

2m 11s0.1 cr used
Done
#79

Run the weekly study-material pipeline with a live four-stage tracker

140 cr estimated
To Do
#80

Drive extraction stages, iframe caption interception, and session persistence

140 cr estimated
To Do
#81

Show run files and let the student download the finished study guide

100 cr estimated
To Do
#82

Persist and reuse the LMS session across runs

80 cr estimated
To Do
#83

Extract PDF text, capture Vimeo captions, and synthesize the study guide

140 cr estimated
To Do
#84

Render and download the generated study notes

100 cr estimated
To Do
#85

Show environment status and let the student watch progress and consume results

100 cr estimated
To Do
#86

Download, parse, and clean weekly course materials

120 cr estimated
To Do
#87

Application identity gates saved run configuration and generated results

120 cr estimated
To Do
#88

Automate SSO login, caption interception, and tracker stage control

140 cr estimated
To Do
#89

Assemble and serve the unified context payload for inspection

120 cr estimated
To Do
#90

Configure, save, and run study-material extraction from the Dashboard

160 cr estimated
To Do
#91

Show combined PDF text and Vimeo transcripts in Extracted Context

80 cr estimated
To Do
#92

Persist, reuse, and clear the separate Miva SSO session

120 cr estimated
To Do
#93

Generate, render, and download the Markdown study guide

120 cr estimated
To Do
#94

Run the four-stage pipeline and surface live progress and failures

160 cr estimated
To Do
#95

Run the four-stage pipeline with saved-session reuse and re-authentication

140 cr estimated
To Do
#96

the Markdown study guide from the unified context

120 cr estimated
To Do
#97

Establish the application-owned identity layer

120 cr estimated
To Do
#98

Flag missing required environment variables with .env guidance

60 cr estimated
To Do
#99

Let a student create an application account and reach the Dashboard

120 cr estimated
To Do
#100

Drive the provider-owned Miva and Vimeo surfaces from the automation layer

120 cr estimated
To Do
#101

Verify credentials and complete the OAuth handshake with Miva SIS

120 cr estimated
To Do
#102

Send the assembled payload to Gemini

80 cr estimated
To Do
#103

Show Notes, Context, or Files once a run produces results

80 cr estimated
To Do
#112

Make Landing entry actions navigate to Sign Up and Login

40 cr estimated
To Do
#104

Wire Study Notes to live run output

120 cr estimated
To Do
#105

Make Sign Up create a real application account

120 cr estimated
To Do
#106

Wire Session page to real saved-session state

120 cr estimated
To Do
#107

Make Login verify identity and restore saved access

120 cr estimated
To Do
#108

Wire Files page to real run results

120 cr estimated
To Do
#109

Wire Dashboard to live configuration, status, and run control

120 cr estimated
To Do
#110

Wire Context page to the live unified payload

120 cr estimated
To Do
#111

Wire Pipeline page to live run, progress, and results

140 cr estimated
To Do
Landing design preview
Landing: Read the pipeline overview
Sign Up: Create application account
Login: Enter credentials
Sign Up: Fix flagged field and resubmit
Login: Re-enter valid credentials
Dashboard: 1. Review environment variable status
Dashboard: 2. Fill missing configuration fields
Dashboard: Save course, week and URL
Session: Clear saved LMS session
Session: Retry removal after failure
Pipeline: 1. Start auto-extraction run
Pipeline: 2. Watch four-stage progress tracker
Dashboard: 3. Correct configuration and re-run
Session: 4. Clear session and re-run
Notes: 5. Read generated study guide
Notes: Download notes .md
Context: 6. Inspect unified context payload
Files: 7. Review downloaded files list
Pipeline: 8. Re-run pipeline from error
Landing design preview
Landing: Read the pipeline overview
Sign Up: Create application account
Login: Enter credentials
Sign Up: Fix flagged field and resubmit
Login: Re-enter valid credentials
Dashboard: 1. Review environment variable status
Dashboard: 2. Fill missing configuration fields
Dashboard: Save course, week and URL
Session: Clear saved LMS session
Session: Retry removal after failure
Pipeline: 1. Start auto-extraction run
Pipeline: 2. Watch four-stage progress tracker
Dashboard: 3. Correct configuration and re-run
Session: 4. Clear session and re-run
Notes: 5. Read generated study guide
Notes: Download notes .md
Context: 6. Inspect unified context payload
Files: 7. Review downloaded files list
Pipeline: 8. Re-run pipeline from error