jacson

JacSON Logo

JacSON — Overview, Setup & Configuration Guide

Welcome to JacSON: a Javascript and JSON-based scraper and publishing tool for UQ Jac Course Profiles.
This guide explains everything that must be edited if you’re adapting JacSON for another school, faculty, or institution.

Overview

JacSON automates the retrieval of learning outcomes, assessment details, and weekly learning activities from the current (non-archived) version of the UQ course profiles website. These outputs can then be used to pre-populate components in Blackboard Ultra using JavaScript, enabling a more efficient course build process for teaching teams and learning designers.

Key Features

Limitations

🚀 Getting Started

Here’s how to replicate JacSON from scratch in your school or faculty.

1. Download Repo as ZIP file

If you haven’t already, download the Repo as a ZIP file. Instructions below assume you’ve saved it to: %USERPROFILE%\Documents\JacSON

2. 🐍 Install Python

Make sure Python 3.10 or later is installed.

Once that is installed, open a command prompt window (CMD.exe, not PowerShell!) and change to the folder: e.g. cd %USERPROFILE%\Documents\JacSON

Then install dependencies:

pip install -r requirements.txt

3. 📄 Google Sheet Setup

JacSON expects this layout:

A (Course Code) B (Auto) C (Status) D (Manual) E F (Notes)
ACCT7804 TRUE        

Getting your SECRETS ready

4. 🔐 Google Sheets Setup

JacSON reads and writes to a shared Google Sheet. To enable this:

a. Create a Service Account:

b. Save the key file

Place your credentials JSON here:

./secrets/credentials.json

🔐 How to Set Up a GitHub Personal Access Token

🧱 Prerequisites

✅ Step-by-Step Instructions

a. Log in to GitHub

Go to: https://github.com/login

b. Open Developer Settings
c. Generate a New Token
d. Configure the Token

Fill out the form:

This includes:

e. Generate and Save the Token

🔒 Using the Token in JacSON

  1. Open your JacSON folder.

  2. Create a file:

    ./secrets/github_token.txt

  3. Paste the token into the file:

    ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXX

✔️ Now JacSON’s upload_profiles.py will authenticate and upload .json

🛠 Manual Controls

Automated Controls

✅ Option 1: Windows — Use Task Scheduler

🔧 Script File (Windows Batch Script)

Save this as run_jacson.bat in the same folder as your JacSON project:

@echo off cd /d "C:\Path\To\Your\JacSON" python scraper_runner.py

🔄 Replace C:\Path\To\Your\JacSON with your actual folder path.

📅 Setup in Task Scheduler

  1. Open Task Scheduler
  2. Click Create Basic Task
  3. Name it: Run JacSON Scraper
  4. Choose Daily
  5. Set your time (e.g., 9:00 PM)
  6. Action: Start a program
  7. Program/script: C:\Path\To\Your\JacSON\run_jacson.bat
  8. Finish ✅

✅ Option 2: macOS / Linux — Use Cron

🔧 Script File

Save this as run_jacson.sh and make it executable:

#!/bin/bash cd /path/to/your/JacSON /usr/bin/python3 scraper_runner.py

Use which python3 to confirm the Python path

Then make it executable:

chmod +x run_jacson.sh

📅 Setup Cron Job

  1. Open terminal

  2. Edit your crontab:

    crontab -e

  3. Add a line like this (for 9:00 PM daily):

    0 21 * * * /path/to/your/JacSON/run_jacson.sh >> /path/to/your/JacSON/logs/cron.log 2>&1

🔧 Editable Configuration Overview

Below is a complete list of files, variables, and settings that you will need to modify to get JacSON running in your own environment.

1. ✅ Google Sheets Configuration

File: scripts/sheets_updater.py
Lines:

SPREADSHEET_ID = '1tJ04EY1AtyS-7iKlmgZhmom0f97xK3DsZ88wkqmRwNs'
RANGE_NAME     = 'Sheet1!A2:F'
CREDENTIALS_FILE = os.path.join('secrets', 'credentials.json')

2. 🔐 Google Service Account Setup

Files:

These scripts expect a service account credential file saved to:

./secrets/credentials.json

3. ☁️ GitHub Repository Settings

File: scripts/upload_profiles.py
Lines:

GITHUB_OWNER = "uqgblaze"
GITHUB_REPO  = "jacson"

4. 🔑 GitHub Personal Access Token

File: scripts/upload_profiles.py
Line:

TOKEN_PATH = os.path.join(PROJECT_ROOT, "secrets", "github_token.txt")

5. 🧾 Google Sheet Column Logic

All Google Sheets integrations assume:

🔄 Adjust in scripts if your Sheet layout differs.

6. 📄 Target Scraping Domain

File: jacson.py
Line:

target_domain = "https://course-profiles.uq.edu.au/course-profiles/"

7. 📁 Output File Structure

File: jacson.py → inside save_course_data()
Line:

profiles_root = os.path.join(base_directory, "profiles")

8. 📋 CSV Filename

Files: jacson.py, generate_course_list.py, sheets_updater.py

9. 🕒 GitHub Action Schedule (Optional)

File: .github/workflows/run-jacson.yml
Line:

cron: '0 17 * * *'  # 3:00 AM AEST

📦 Summary Table

What to Change File(s) Location/Note
Google Sheets ID sheets_updater.py SPREADSHEET_ID
Sheets API credentials All Google-related scripts Path: ./secrets/credentials.json
GitHub repo details upload_profiles.py GITHUB_OWNER, GITHUB_REPO
GitHub token upload_profiles.py ./secrets/github_token.txt
Course sheet column logic Sheets-related scripts B = Auto, C = Status, D = Manual, F = Notes
Target scrape domain jacson.py Inside get_course_profile_links()
Output folder jacson.py Inside save_course_data()
CSV filename Most scripts course-list.csv
GitHub Action schedule run-jacson.yml cron: section

🙏 Credits

JacSON was built by the UQ Business School Learning Design team with both love 💖 and spite 🤬 for Ultra’s Assessment Overview tables. Here’s to never needing to manually update those tables ever again!

This site is open source. Improve this page.