diff --git a/GIT-REFERENCE.md b/GIT-REFERENCE.md deleted file mode 100644 index d7c5814..0000000 --- a/GIT-REFERENCE.md +++ /dev/null @@ -1,542 +0,0 @@ -# Gitea How-To Guide - -Quick reference guide for using Gitea at http://100.120.125.113:3000 to manage your homelab projects. - -## Table of Contents -- [Getting Started](#getting-started) -- [Terminology](#terminology) -- [Cloning Repositories](#cloning-repositories) -- [Making Changes](#making-changes) -- [Pushing Changes](#pushing-changes) -- [Common Workflows](#common-workflows) -- [Troubleshooting](#troubleshooting) - ---- - -## Getting Started - -### About Your Gitea Server - -**Note:** Access via Pangolin Tunnel at https://git.pdmarf.co.uk if outside the local network. - -- **Server URL: http://100.120.125.113:3000 (or https://git.pdmarf.co.uk via Pangolin) -- **User**: pdm -- **Available Repositories**: - - `homelab-agents` - Shared AI agent prompts (pull-only on VPS) - - `vps-system-apps` - VPS system configuration (push for backup) - - Additional project repositories as needed - ---- - -## Terminology - -### Workstation -Your **main development machine** where you do most of your work. This could be: -- Your laptop (MacBook, Windows laptop, Linux laptop) -- Your desktop computer -- Any machine you consider your primary development environment - -**Not** a VPS - it's a machine you have physical access to or directly control. - -### VPS -A **Virtual Private Server** - a remote machine hosted in the cloud or on another system. Examples: -- Your system-apps VPS at your hosting provider -- Remote servers you manage -- Machines you SSH into rather than use directly - -### Repository -A project folder tracked by git with version history. Each repository can be: -- `homelab-agents` - The shared agents repository -- `vps-system-apps` - Your VPS configuration repository -- Any other project you create - ---- - -## Cloning Repositories - -Cloning creates a local copy of a repository on your workstation or VPS. - -### Clone homelab-agents (Shared Prompts) - -Use this to get the latest AI agent prompts on your VPS: - -```bash -git clone http://100.120.125.113:3000/pdm/homelab-agents.git ~/.homelab-agents -``` - -This creates a `~/.homelab-agents` directory with all shared agent definitions and templates. - -**Use Cases:** -- Getting agent prompts on a new VPS -- Keeping agents synchronized across systems -- Read-only reference for development - -### Clone vps-system-apps (Project Configuration) - -Use this to get your VPS configuration repository: - -```bash -git clone http://100.120.125.113:3000/pdm/vps-system-apps.git ~/projects/system-apps -``` - -This creates a `~/projects/system-apps` directory with Docker, scripts, and documentation. - -**Use Cases:** -- Setting up a new VPS with existing configuration -- Cloning on your workstation for editing -- Working on system configuration with version history - -### Clone Any Other Project Repository - -For future projects, use the same pattern: - -```bash -git clone http://100.120.125.113:3000/pdm/[project-name].git ~/projects/[project-name] -``` - ---- - -## Making Changes - -### Checking Repository Status - -See what files have changed: - -```bash -cd ~/projects/system-apps -git status -``` - -Output shows: -- Modified files (red) - existing files you changed -- Untracked files (red) - new files not yet in git -- Staged files (green) - files ready to commit - -### Viewing Specific Changes - -See exactly what changed in a file: - -```bash -git diff docker-compose/docker-compose.yml -``` - -Or see all changes: - -```bash -git diff -``` - -### Creating a Local Branch (Optional) - -For experimental work, create a separate branch: - -```bash -git checkout -b feature/new-monitoring-stack -# ... make your changes ... -git add . -git commit -m "Add Prometheus monitoring stack" -``` - -To switch back to main branch: - -```bash -git checkout main -``` - ---- - -## Pushing Changes - -### Basic Push Workflow - -After making changes, follow these three steps: - -#### Step 1: Stage Your Changes - -Add all modified files to staging: - -```bash -cd ~/projects/system-apps -git add -A -``` - -Or add specific files: - -```bash -git add docker-compose/docker-compose.yml -git add scripts/deploy.sh -``` - -#### Step 2: Commit Your Changes - -Create a commit with a descriptive message: - -```bash -git commit -m "Add health checks to Docker services" -``` - -**Good commit message practices:** -- Start with action verb: "Add", "Fix", "Update", "Refactor" -- Be specific: what changed and why -- Keep it concise (under 72 characters for first line) - -**Examples:** -```bash -git commit -m "Add Prometheus scrape config for Docker metrics" -git commit -m "Fix disk space alert threshold in monitoring" -git commit -m "Update deployment documentation with new workflow" -git commit -m "Refactor docker-compose.yml for better organization" -``` - -#### Step 3: Push to Gitea - -Send your commits to the Gitea server: - -```bash -git push origin main -``` - -**Success looks like:** -``` -Enumerating objects: 5, done. -Counting objects: 100% (5/5), done. -Delta compression using up to 4 threads -Compressing objects: 100% (3/3), done. -Writing objects: 100% (3/3), 456 bytes, done. -Total 3 (delta 1), reused 0 (delta 0) -To http://100.120.125.113:3000/pdm/vps-system-apps.git - abc1234..def5678 main -> main -``` - ---- - -## Common Workflows - -### Workflow 1: Edit on Your Workstation, Push to Gitea - -**Workstation** = Your main development machine (laptop, desktop, etc.) - -```bash -# Step 1: On your workstation, clone the repository -# (e.g., your laptop or desktop computer) -git clone http://100.120.125.113:3000/pdm/vps-system-apps.git ~/projects/system-apps - -# Step 2: Edit files on your workstation using vim, VS Code, or any editor -cd ~/projects/system-apps -vim docker-compose/docker-compose.yml -vim scripts/deploy.sh - -# Step 3: Check what you changed -git status - -# Step 4: Stage and commit -git add -A -git commit -m "Update Docker services with new health checks" - -# Step 5: Push to Gitea -git push origin main - -# Now your changes are backed up on Gitea and can be deployed to your VPS -``` - -**When to use this workflow:** -- Editing configuration files on your computer -- Writing scripts and documentation -- Testing changes before deploying to VPS - -### Workflow 2: Update Agents on VPS - -```bash -# On your VPS (remote server) -cd ~/.homelab-agents - -# Get latest agents from Gitea -git pull origin main - -# Now use the agents in your work -cat agents/sysadmin-session-closer.md -``` - -### Workflow 3: Work on VPS, Backup to Gitea - -```bash -# On your VPS (remote server) -cd ~/projects/system-apps - -# Make changes directly on the VPS -vim scripts/backup.sh -vim docker-compose/docker-compose.yml - -# Back up your work to Gitea -git add -A -git commit -m "Update backup script with better error handling" -git push origin main - -# Now your work is backed up and version-controlled on Gitea! -``` - -### Workflow 4: Create and Test New Feature - -```bash -# Create a new branch for experimental work -git checkout -b feature/new-setup - -# Make your changes -# Test everything locally - -# If it works, merge back to main -git checkout main -git merge feature/new-setup - -# Push to Gitea -git push origin main - -# Delete the feature branch (optional) -git branch -d feature/new-setup -``` - ---- - -## Useful Git Commands - -### View Commit History - -See what changed and when: - -```bash -# Show last 10 commits -git log --oneline -10 - -# Show commits with files changed -git log --stat -5 - -# Show specific commit details -git show abc1234 -``` - -### Undo Changes - -If you made a mistake: - -```bash -# Discard changes to a specific file (not yet committed) -git checkout -- scripts/deploy.sh - -# Discard all uncommitted changes -git reset --hard HEAD - -# Undo the last commit (keep your changes) -git reset --soft HEAD~1 - -# Undo the last commit (discard changes) -git reset --hard HEAD~1 -``` - -### Compare Versions - -See differences between versions: - -```bash -# Changes not yet committed -git diff - -# Changes between two commits -git diff abc1234 def5678 - -# Changes between branches -git diff main feature/new-setup -``` - -### Sync with Latest Changes - -Get updates from Gitea: - -```bash -# Fetch latest without merging -git fetch origin - -# Pull latest and merge (recommended) -git pull origin main -``` - ---- - -## Troubleshooting - -### "fatal: could not read Username" - -**Problem:** Git is asking for authentication but you don't have credentials saved. - -**Solution 1 - Use HTTPS with saved credentials:** -```bash -git config --global credential.helper store -git clone http://100.120.125.113:3000/pdm/homelab-agents.git -# Enter username (pdm) and password when prompted -# Credentials are saved for future use -``` - -**Solution 2 - Configure git user:** -```bash -git config --global user.name "pdm" -git config --global user.email "your@email.com" -``` - -### "Permission denied" on push - -**Problem:** Your push was rejected, possibly due to: -- Network access to Gitea server -- Repository permissions -- Authentication issue - -**Check network access:** -```bash -ping 100.120.125.113 -curl -I http://100.120.125.113:3000 -``` - -**Verify remote URL:** -```bash -git remote -v -# Should show: origin http://100.120.125.113:3000/pdm/[project].git -``` - -**Test Gitea access:** -```bash -# Try to fetch (read-only test) -git fetch origin -``` - -### "branch is behind" or merge conflicts - -**Problem:** Someone else pushed changes, and you have local changes too. - -**Solution:** -```bash -# Get latest from Gitea -git pull origin main - -# If conflicts occur, manually edit the conflicted files -git add . -git commit -m "Resolve merge conflicts" -git push origin main -``` - -### Repository not found - -**Problem:** Repository URL is incorrect or repository doesn't exist on Gitea. - -**Solution:** -```bash -# Verify correct URL format: -# http://100.120.125.113:3000/pdm/[exact-project-name].git - -# Check what repositories exist by visiting: -http://100.120.125.113:3000/pdm - -# Update remote if needed -git remote set-url origin http://100.120.125.113:3000/pdm/correct-name.git -``` - -### Lost changes or need to recover - -**Problem:** You made changes but accidentally ran `git reset --hard` - -**Solution - Find your work:** -```bash -# See recent commits even if hidden -git reflog - -# Restore to a specific commit -git reset --hard abc1234 -``` - ---- - -## Quick Reference Card - -```bash -# Cloning -git clone http://100.120.125.113:3000/pdm/homelab-agents.git ~/.homelab-agents -git clone http://100.120.125.113:3000/pdm/vps-system-apps.git ~/projects/system-apps - -# Status and viewing changes -git status # See what changed -git diff # See exact changes -git log --oneline -10 # See recent commits - -# Committing and pushing -git add -A # Stage all changes -git commit -m "Your message" # Create commit -git push origin main # Push to Gitea - -# Getting updates -git pull origin main # Get latest from Gitea -git fetch origin # Check for updates without merging - -# Branches -git checkout -b feature-name # Create new branch -git checkout main # Switch to main branch -git merge feature-name # Merge branch into main - -# Undoing -git reset --hard HEAD~1 # Undo last commit -git checkout -- filename # Undo changes to file -git revert abc1234 # Create new commit undoing old one -``` - ---- - -## Tips and Best Practices - -### ✓ Do -- **Pull before you push** - Get latest changes first: `git pull origin main` -- **Commit frequently** - Small, focused commits are easier to understand -- **Write good commit messages** - Future you will thank you -- **Test before pushing** - Verify changes work before committing -- **Use meaningful branch names** - `feature/monitoring-stack` not `test123` -- **Keep backups** - Push to Gitea regularly for safety - -### ✗ Don't -- **Don't commit secrets** - Never commit passwords, API keys, or tokens -- **Don't push broken code** - Test locally first -- **Don't commit generated files** - Use `.gitignore` for compiled code, dependencies -- **Don't force push to main** - Unless you really know what you're doing -- **Don't let changes sit locally** - Push to Gitea for backup regularly -- **Don't ignore errors** - Read error messages carefully before asking for help - ---- - -## Getting Help - -### Git Help -```bash -git help [command] # Get help for a command -git help commit # Example: help for commit command -``` - -### Check Gitea Web Interface - -Visit http://100.120.125.113:3000 to: -- Browse repository files -- View commit history -- See who changed what -- Manage repository settings - -### Verify Your Setup - -```bash -# Check git version -git --version - -# Check global config -git config --global -l - -# Test Gitea access -git clone http://100.120.125.113:3000/pdm/homelab-agents.git test-repo -# If successful, you're all set! -rm -rf test-repo -``` - ---- - -**Happy coding! Your homelab now has version control.** 🚀 diff --git a/WORKFLOW.md b/WORKFLOW.md deleted file mode 100644 index 9c86883..0000000 --- a/WORKFLOW.md +++ /dev/null @@ -1,61 +0,0 @@ -# Homelab Agents Workflow - -Simple workflow for using agents across multiple VPS instances. - ---- - -## Setup New VPS (One Time) - -```bash -# Install agents from Gitea -source <(curl -s http://100.120.125.113:3000/pdm/homelab-agents/raw/branch/main/scripts/bootstrap-agents.sh) -``` - -Done! Agents are now available in Claude Code. - ---- - -## Working on Projects - -### 1. Navigate to Your Project Directory -```bash -cd ~/vps-system-apps -# or -cd ~/my-project -``` - -### 2. Do Your Work -Edit files, configure systems, write code... - -### 3. When Done, Run Finish Up -In Claude Code, say: **"finish up"** - -The agent automatically: -- Creates session summary: `{hostname}-session-summary.md` -- Updates CLAUDE.md and README.md -- Commits and pushes to: `http://100.120.125.113:3000/pdm/{project-name}.git` - ---- - -## Key Rules - -**✓ DO:** Run `finish-up` from your project directory (~/vps-system-apps) -- Session summary goes to: vps-system-apps repository - -**✗ DON'T:** Run `finish-up` from ~/.homelab-agents -- Would go to homelab-agents repository (only for agent updates) - ---- - -## Quick Reference - -| Task | Command | -|------|---------| -| Install agents on new VPS | `source <(curl -s http://100.120.125.113:3000/pdm/homelab-agents/raw/branch/main/scripts/bootstrap-agents.sh)` | -| Check current directory | `pwd` | -| End work session | "finish up" in Claude Code | -| Update agents | `cd ~/.homelab-agents && git pull` | - -**Repository Pattern:** -- Agents: `http://100.120.125.113:3000/pdm/homelab-agents.git` -- Projects: `http://100.120.125.113:3000/pdm/{directory-name}.git` diff --git a/home-apps-session-summary.md b/home-apps-session-summary.md deleted file mode 100644 index 35488a8..0000000 --- a/home-apps-session-summary.md +++ /dev/null @@ -1,88 +0,0 @@ -# Session Summary - home-apps -**Date**: 2026-04-12 -**Hostname**: home-apps -**Repository**: homelab-agents (shared) - -## Session Overview - -Debugging and fixing Emacs configuration to work correctly on the Linux VM -accessed via terminal from an iPad. Config is authored on a Mac and synced -to the VM via Syncthing. - -## Accomplishments - -### Emacs Init Error Fixed (flashcard-config.el) -- Identified `void-variable flash-card-mode-map` error on startup -- Root cause: mismatched parenthesis caused `(define-key flash-card-mode-map ...)` - to fall outside the `use-package` block and execute before the package loaded -- Fixed by moving the closing `)` to after the second `define-key` - -### Cross-Platform Load Paths Fixed -- `flashcard-config.el` and `task-config.el` both referenced `~/config/sam_emacs_packages/` - which does not exist on this VM (packages live at `~/sam_emacs_packages/`) -- Applied the conditional `darwin`/Linux pattern: - ```elisp - (add-to-list 'load-path (expand-file-name - (if (eq system-type 'darwin) - "~/config/sam_emacs_packages/" - "~/sam_emacs_packages/"))) - ``` - -### secrets.el Symlink Added -- `init.el` loads `secrets.el` via `user-emacs-directory` but no symlink existed -- Added: `~/.emacs.d/secrets.el` → `~/emacs_config/secrets.el` - -### ELPA Package Visibility Fixed (ob-mermaid) -- On Mac, `~/.emacs.d` symlinks to `~/config/emacs_config` so packages install into - `~/emacs_config/elpa/`. On the VM, `~/.emacs.d/elpa/` is a separate real directory. -- Fixed by adding `~/emacs_config/elpa/` to `package-directory-list` in `package-setup.el` - before `(package-initialize)`, Linux-only - -### mermaid-mode Added -- Added `mermaid-mode` alongside existing `ob-mermaid` in `org-base.el` -- Guarded by same `(when my/mmdc-path ...)` check — only loads if `mmdc` is available -- Reuses `my/mmdc-path` variable for cross-platform CLI path - -### Puppeteer Sandbox Fix (VM only) -- `mmdc` uses headless Chromium which requires sandboxing — blocked on this Linux VM -- Created `~/.config/mermaid/puppeteer-config.json` with `{"args": ["--no-sandbox"]}` -- `org-base.el` detects this file at load time and sets `ob-mermaid-puppeteer-config` - and `mermaid-flags` accordingly; Mac is unaffected (file doesn't exist there) - -### Notion Credentials Investigated -- "Notion Task DBID not set" error traced to credentials path resolution -- `notion-config.el` resolves path relative to parent of `user-emacs-directory` - (following symlinks): Mac → `~/config/credentials/`, VM → `~/credentials/` -- File confirmed present at `~/credentials/notion-credentials.env` — already synced - -### .netrc Set Up for Gitea -- Bootstrap script at `https://gitea.pdmarf.co.uk` required authentication -- Created `~/.netrc` with API token for both the public domain and internal IP - (`100.120.125.113`) — git operations inside the bootstrap use the internal address -- Documented in `~/netrc-explainer.org` - -## Key Decisions - -- **Conditional load paths** rather than VM-only overrides — keeps one config file working on both platforms -- **`package-directory-list`** rather than symlinking elpa directories — safer, non-destructive -- **Puppeteer config file as feature flag** — presence of the file enables the workaround, absence is safe -- **`.netrc` for Gitea auth** — token stored once, not embedded in scripts or history - -## Files Modified - -| File | Change | -|------|--------| -| `my-packages/flashcard-config.el` | Fixed paren bug; conditional load path | -| `my-packages/task-config.el` | Conditional load path | -| `my-packages/package-setup.el` | Added `~/emacs_config/elpa` to `package-directory-list` on Linux | -| `my-packages/org-base.el` | Added `mermaid-mode`; Puppeteer config detection | -| `~/.emacs.d/secrets.el` | New symlink → `~/emacs_config/secrets.el` | -| `~/.config/mermaid/puppeteer-config.json` | New file, VM only | -| `~/.netrc` | New file, Gitea credentials | -| `~/netrc-explainer.org` | New documentation file | - -## Next Steps - -- Restart Emacs on Mac to let `:ensure t` install `mermaid-mode` from MELPA -- Test `mermaid-compile` and Babel `#+begin_src mermaid` blocks on the iPad -- Verify Notion Tasks opens cleanly after credentials fix diff --git a/scripts/README.md b/scripts/README.md deleted file mode 100644 index 65e5fee..0000000 --- a/scripts/README.md +++ /dev/null @@ -1,151 +0,0 @@ -# Scripts - -Utility scripts for agent installation and project management. - -## bootstrap-agents.sh - -**Purpose:** Ensures the latest homelab agents and tools are available on any VPS instance. - -**What it does:** -- Clones the homelab-agents repository if not present -- Updates the repository to latest version if it exists -- Installs helper scripts (like init-project) -- Adds scripts to your PATH -- Verifies agents are available for use - -**Usage:** - -Run this **once** on your new VPS: -```bash -source <(curl -s http://100.120.125.113:3000/pdm/homelab-agents/raw/branch/main/scripts/bootstrap-agents.sh) -``` - -**Why use `source`?** -- Uses `source` instead of `bash` so PATH updates immediately in your current shell -- `init-project` and other commands are available right away -- No need to manually run `source ~/.bashrc` afterward - ---- - -## init-project - -**Purpose:** Quickly initialize new VPS projects with automatic Gitea remote configuration and agent setup. - -**Important:** The Gitea repository must be created FIRST on the web UI. - -**What it does:** -1. Creates a new project directory -2. Initializes git repository -3. Sets up git user config -4. **Automatically configures SSH remote** to your Gitea repo -5. Creates initial main branch -6. **Copies agents to `.claude/agents/`** for Claude Code -7. Provides clear next steps - -**Workflow:** - -### Step 1: Create Repository on Gitea (First!) - -Go to: http://100.120.125.113:3000/repo/create - -- **Repository name**: `my-new-app` -- **Visibility**: Public or Private -- **Initialize repository**: Leave unchecked (init-project will set it up) -- Click "Create Repository" - -### Step 2: Initialize Local Project - -```bash -cd ~/projects -init-project my-new-app -``` - -**Example output:** -``` -📁 Creating project directory: my-new-app -🔧 Initializing git repository... -🔗 Configuring Gitea SSH remote... -📚 Setting up Claude Code agents... -✅ Copied 2 agent(s) to .claude/agents/ - -✅ Project initialized successfully! - -📋 Project Details: - Name: my-new-app - Directory: /home/pdm/projects/my-new-app - Remote: git@100.120.125.113:pdm/my-new-app.git - Branch: main - Agents: .claude/agents/ - -🚀 Next steps: - 1. Start working in this directory - 2. Create files and make changes - 3. When done, use the summary agent: .claude/agents/finish-up.md - 4. Agent will automatically commit and push to Gitea (via SSH) -``` - -**What happens next:** -- Work normally in your project directory -- Git remote is already configured to use SSH -- Agents are locally available in `.claude/agents/` -- When you use the summary agent, it pushes via SSH (no passwords!) -- No more "What's the Gitea remote URL?" prompts! - -**Project Repository:** - -Each project gets its own independent Gitea repository: -- Project: `my-new-app` → Repository: `git@100.120.125.113:pdm/my-new-app.git` -- Project: `project-2` → Repository: `git@100.120.125.113:pdm/project-2.git` - -Projects on the same VPS are separate and independent. - -**Agent Location:** - -Agents are available at: -``` -.claude/agents/finish-up.md -.claude/agents/[other-agents].md -``` - -Claude Code will automatically find them. - -**Installation:** - -`init-project` is automatically installed by bootstrap-agents.sh to: -``` -~/.homelab-scripts/init-project -``` - -It's added to your PATH, so you can run it from anywhere. - ---- - -## Workflow Summary - -### First Time on a VPS -```bash -source <(curl -s http://100.120.125.113:3000/pdm/homelab-agents/raw/branch/main/scripts/bootstrap-agents.sh) -``` - -### For Each New Project -```bash -# 1. Go to Gitea and create the repo -# http://100.120.125.113:3000/repo/create - -# 2. Initialize project locally -cd ~/projects -init-project my-app -cd my-app -# Start working... -``` - -### Closing a Session -Use the summary agent at: -``` -.claude/agents/finish-up.md -``` - -The agent will: -- Create session summary -- Commit changes -- Push to Gitea via SSH (automated!) \ No newline at end of file diff --git a/session-logs/session-2025-11-23-gitea-config.md b/session-logs/session-2025-11-23-gitea-config.md deleted file mode 100644 index 2c9eb3c..0000000 --- a/session-logs/session-2025-11-23-gitea-config.md +++ /dev/null @@ -1,162 +0,0 @@ -# Sysadmin Session Summary -## Date: 2025-11-23 -## Host: git-repro (Gitea LXC) -## Session Type: Configuration and Backup Setup - ---- - -## Session Overview - -This session focused on completing the Gitea configuration for external access via Pangolin tunnel and establishing automated backup procedures. - ---- - -## Tasks Completed - -### 1. Gitea Domain Configuration Fix - -**Issue:** Gitea login was returning 404 errors when accessed via the Pangolin tunnel domain. - -**Root Cause:** The Gitea configuration had the internal IP address (100.120.125.113) set for `DOMAIN`, `ROOT_URL`, and `SSH_DOMAIN` instead of the external domain. - -**Solution Applied:** -Updated `/etc/gitea/app.ini` with the following settings in the `[server]` section: - -```ini -[server] -SSH_DOMAIN = git.pdmarf.co.uk -DOMAIN = git.pdmarf.co.uk -HTTP_PORT = 3000 -ROOT_URL = https://git.pdmarf.co.uk/ -``` - -**Verification:** -- Gitea service restarted successfully -- Login via https://git.pdmarf.co.uk now works correctly -- AppURL correctly shows: https://git.pdmarf.co.uk/ - ---- - -### 2. Pangolin Container Restart - -**Action:** Restarted the Pangolin docker container to ensure the tunnel proxy was properly forwarding requests to the updated Gitea configuration. - -**Result:** External access via git.pdmarf.co.uk confirmed working. - ---- - -### 3. Automated Database Backup System - -**Implementation Details:** - -**Backup Script:** `/usr/local/bin/gitea-backup.sh` -```bash -#!/bin/bash -BACKUP_DIR="/var/backups/gitea" -DATE=$(date +%Y%m%d-%H%M%S) - -mkdir -p $BACKUP_DIR - -# Backup the database -cp /var/lib/gitea/data/gitea.db "$BACKUP_DIR/gitea.db-$DATE" - -# Compress -gzip "$BACKUP_DIR/gitea.db-$DATE" - -# Keep only last 7 days -find $BACKUP_DIR -name "*.gz" -mtime +7 -delete - -echo "Gitea backup completed: $BACKUP_DIR/gitea.db-$DATE.gz" -``` - -**Cron Schedule:** -``` -0 2 * * * /usr/local/bin/gitea-backup.sh -``` -- Runs daily at 2:00 AM -- 7-day retention policy -- Backups stored in `/var/backups/gitea/` - -**Backup Location:** `/var/backups/gitea/` - ---- - -### 4. Backup/Restore Testing - -**Test Performed:** Verified backup and restore process works correctly. - -**Verification Steps:** -1. Manual backup execution confirmed working -2. Backup file created with timestamp: `gitea.db-20251123-174012.gz` -3. Compression working (original database compressed to ~33KB) -4. Restore procedure documented and tested - ---- - -## Current System State - -### Gitea Service -- **Status:** Active (running) -- **Service:** `gitea.service` -- **Start Time:** 2025-11-23 17:43:18 UTC -- **Memory Usage:** ~112MB -- **Config File:** `/etc/gitea/app.ini` - -### Gitea Configuration Summary -| Setting | Value | -|---------|-------| -| App Name | Homelab Git | -| Domain | git.pdmarf.co.uk | -| ROOT_URL | https://git.pdmarf.co.uk/ | -| SSH_DOMAIN | git.pdmarf.co.uk | -| HTTP_PORT | 3000 | -| Database | SQLite3 | -| DB Path | /var/lib/gitea/data/gitea.db | -| Repository Root | /home/git/gitea-repositories | -| LFS Enabled | Yes | -| Mail Server | smtp.fastmail.com:465 | - -### Active Repositories -1. `pdm/homelab-agents.git` - Centralized AI agent prompts -2. `pdm/vps-system-apps.git` - System apps VPS configuration -3. `pdm/test.git` - Test repository - -### Backup System -| Component | Status | -|-----------|--------| -| Backup Script | /usr/local/bin/gitea-backup.sh | -| Backup Directory | /var/backups/gitea/ | -| Cron Schedule | Daily at 2 AM | -| Retention | 7 days | -| Latest Backup | gitea.db-20251123-174012.gz | - ---- - -## Files Modified This Session - -1. `/etc/gitea/app.ini` - Updated DOMAIN, ROOT_URL, SSH_DOMAIN -2. `/usr/local/bin/gitea-backup.sh` - Created backup script -3. `/var/spool/cron/crontabs/root` - Added backup cron job - ---- - -## Recommendations for Future Sessions - -1. **Off-site Backup:** Consider syncing backups to remote storage (S3, rsync to another server) -2. **Repository Backup:** The current backup only covers the database; consider backing up `/home/git/gitea-repositories/` as well -3. **Monitoring:** Set up alerting for backup failures -4. **Full Backup Script:** Create a comprehensive backup that includes: - - Database (`/var/lib/gitea/data/gitea.db`) - - Repositories (`/home/git/gitea-repositories/`) - - Configuration (`/etc/gitea/app.ini`) - - Custom files (`/var/lib/gitea/custom/`) - ---- - -## Session Timeline -- Session Start: 2025-11-23 (afternoon UTC) -- Session End: 2025-11-23 17:48 UTC - ---- - -*This session summary was generated by Claude Code following the sysadmin-session-closer process.* diff --git a/sys-apps-git-quick-reference.md b/sys-apps-git-quick-reference.md deleted file mode 100644 index 791c848..0000000 --- a/sys-apps-git-quick-reference.md +++ /dev/null @@ -1,87 +0,0 @@ -# Git Basics - Quick Reference - -Simple commands you'll use all the time. - ---- - -## Daily Workflow - -### Check what changed -```bash -git status # See what files changed -git diff # See exact changes -``` - -### Save your work -```bash -git add -A # Stage ALL changes (new, modified, deleted) -git commit -m "your message" # Save with description -git push origin main # Upload to Gitea -``` - -### Get latest changes -```bash -git pull origin main # Download updates from Gitea -``` - ---- - -## What Each Command Does - -| Command | What it does | -|---------|-------------| -| `git add -A` | Mark all changes as ready to save | -| `git commit -m "msg"` | Save marked changes with a description | -| `git push origin main` | Upload commits to Gitea | -| `git pull origin main` | Download latest from Gitea | -| `git status` | Show what changed | -| `git diff` | Show exact line changes | - ---- - -## Common Patterns - -### Made changes, want to save and upload: -```bash -git add -A -git commit -m "Add health checks to services" -git push origin main -``` - -### Want to see what you changed before committing: -```bash -git status # See which files changed -git diff # See exact changes -git add -A # Stage everything -git commit -m "..." # Commit -git push # Push -``` - -### Get updates from Gitea: -```bash -git pull origin main -``` - ---- - -## Remember - -- **git add** = Mark files (staging) -- **git commit** = Save with message (local) -- **git push** = Upload to Gitea (remote) -- **-A** = ALL files (new, modified, deleted) -- **-m** = Message (inline, no editor) - ---- - -## When to Use Finish-Up Agent vs Manual Git - -**Use finish-up agent:** -- Ending work session -- Want comprehensive documentation -- Need session summary - -**Use manual git:** -- Quick commits during work -- Custom commit messages -- Frequent small saves diff --git a/sys-apps-session-summary.md b/sys-apps-session-summary.md deleted file mode 100644 index e0f011b..0000000 --- a/sys-apps-session-summary.md +++ /dev/null @@ -1,200 +0,0 @@ -# Homelab Agents Session Summary - sys-apps -**Date:** 2025-11-30 -**VPS:** sys-apps -**Repository:** homelab-agents (shared agent repository) -**Session Type:** NetBox DHCP pool configuration - -## Session Overview - -This session focused on configuring NetBox to properly mark DHCP pool IP ranges as reserved, preventing them from showing as "available" for static assignment. Created automation scripts to set up DHCP pool ranges across all VLANs. - -## Problem Statement - -User noticed that NetBox showed IPs in the DHCP pool range (.100-.150) as "available" for static assignment. This created confusion about which IPs could actually be assigned statically vs. which were reserved for DHCP dynamic assignment. - -**Issue:** NetBox was showing 133 "available" IPs in VLAN 20, but most of those (.100-.150) were actually in the DHCP pool. - -## Solution Implemented - -### Two-Step Approach: - -1. **Create IP Ranges** - Logical grouping of DHCP pool -2. **Reserve Individual IPs** - Mark each IP .100-.150 as "Reserved" status - -## Scripts Created - -### 1. sys-apps-netbox-dhcp-setup.py -**Purpose:** Create IP Range objects for DHCP pools - -**What it does:** -- Connects to NetBox via API -- Fetches all IPv4 prefixes -- Creates IP Range for .100-.150 in each subnet -- Status: Active -- Description: "DHCP Pool - Dynamic Assignment Range (.100-.150)" - -**Results:** -- Created/verified 6 IP ranges -- Ranges: 192.168.0.100-150, 192.168.2.100-150, 192.168.20.100-150, 192.168.30.100-150, 192.168.40.100-150, 192.168.50.100-150 - -### 2. sys-apps-netbox-reserve-dhcp-ips.py -**Purpose:** Create individual IP objects and mark as reserved - -**What it does:** -- Connects to NetBox via API -- Fetches all /24 prefixes (skips /16) -- Creates IP object for each address .100-.150 -- Sets status to "Reserved" -- Description: "DHCP Pool - Reserved for dynamic assignment" - -**Results:** -- Created 255 IP objects across 5 VLANs -- 51 IPs per /24 subnet (.100 to .150 inclusive) -- All marked as "Reserved" status - -## VLANs Configured - -| VLAN | Subnet | DHCP Range | IPs Reserved | -|------|--------|------------|--------------| -| (No VLAN) | 192.168.2.0/24 | .100-.150 | 51 | -| Work Secure | 192.168.20.0/24 | .100-.150 | 51 | -| Family | 192.168.30.0/24 | .100-.150 | 51 | -| Guest | 192.168.40.0/24 | .100-.150 | 51 | -| IoT | 192.168.50.0/24 | .100-.150 | 51 | -| **Total** | | | **255** | - -## Technical Details - -### NetBox API Integration -- **URL:** https://netbox.pdmarf.co.uk -- **Authentication:** API Token (stored in /opt/docker/netbox/netbox_initial_populate.py) -- **Endpoints Used:** - - `/api/ipam/prefixes/` - List all network prefixes - - `/api/ipam/ip-ranges/` - Create IP range objects - - `/api/ipam/ip-addresses/` - Create individual IP objects - -### Script Features -- Environment variable or command-line argument for API token -- Error handling for duplicate entries -- Progress indicators for long-running operations -- Automatic network validation (IPs must be within subnet) -- Skips non-/24 networks (like /16) for IP reservation - -### Initial Issue: Tags Not Found -First run failed because script tried to create tag "dhcp-pool" which didn't exist: -``` -{'tags': ["Related object not found using the provided attributes: {'name': 'dhcp-pool'}"]} -``` - -**Fix:** Removed tags from IP Range creation, used only description field. - -## Verification - -**Before:** -- NetBox showed IPs .100-.150 as "Available" -- User saw 133 available IPs in VLAN 20 subnet - -**After:** -- IPs .100-.150 marked as "Reserved" -- Available count reduced significantly -- Only IPs outside DHCP pool show as available for static assignment - -**View Results:** -- IP Ranges: https://netbox.pdmarf.co.uk/ipam/ip-ranges/ -- VLAN 20 IPs: https://netbox.pdmarf.co.uk/ipam/prefixes/2/ip-addresses/ - -## Files Created - -**Scripts moved to system-apps repository:** -``` -http://100.120.125.113:3000/pdm/system-apps -├── sys-apps-netbox-dhcp-setup.py -├── sys-apps-netbox-reserve-dhcp-ips.py -└── README.md -``` - -**Note:** Scripts were initially created in homelab-agents but moved to system-apps repository as they are VPS-specific configuration tools, not shared agents. - -**Both scripts:** -- Follow naming convention: `{hostname}-{description}.py` -- Are executable (`chmod +x`) -- Accept API token as argument or environment variable -- Include comprehensive error handling and progress output - -## DHCP Pool Standard - -**Established standard for all VLANs:** -- DHCP pool range: `.100-.150` (51 IPs) -- Static assignment ranges: `.1-.99` and `.151-.254` -- Gateway typically: `.1` - -This provides: -- 99 static IPs before DHCP pool -- 51 DHCP dynamic IPs -- 104 static IPs after DHCP pool -- Total usable in /24: 254 IPs - -## Usage for Future VLANs - -When adding a new VLAN, run both scripts to configure DHCP pool: - -```bash -# Clone system-apps repository (if not already cloned) -git clone git@100.120.125.113:pdm/system-apps.git ~/system-apps -cd ~/system-apps - -# 1. Create IP Range -python3 sys-apps-netbox-dhcp-setup.py 'YOUR_API_TOKEN' - -# 2. Reserve Individual IPs -python3 sys-apps-netbox-reserve-dhcp-ips.py 'YOUR_API_TOKEN' -``` - -Or set environment variable once: -```bash -export NETBOX_TOKEN='YOUR_API_TOKEN' -python3 sys-apps-netbox-dhcp-setup.py -python3 sys-apps-netbox-reserve-dhcp-ips.py -``` - -## Key Decisions - -### Why Two Scripts? -1. **IP Ranges** - Logical grouping, visual representation in NetBox -2. **Individual IPs** - Actual reservation, prevents "available" status - -Both are needed because IP Ranges alone don't change IP availability status. - -### Why Mark as "Reserved" Not "DHCP"? -NetBox standard status values: -- Active = In use -- Reserved = Allocated but not assigned -- **Reserved is correct** for DHCP pool (allocated to DHCP server, not assigned to specific device) - -### Script Location -**Moved to `system-apps` repository** because: -- VPS-specific configuration tools (not shared agents) -- Belongs with other sys-apps infrastructure code -- `homelab-agents` = shared Claude Code agents only -- `system-apps` = VPS-specific automation and config -- Makes more sense in project-specific repo vs shared agent repo - -## Next Session Recommendations - -1. **Document NetBox token management** - Where tokens are stored, rotation policy -2. **Create tag for DHCP pools** - Add "dhcp-pool" tag in NetBox UI for easier filtering -3. **Consider automation** - Run scripts automatically when new VLAN is created -4. **Verify other subnets** - Check if any other networks need DHCP pool configuration - -## Session Learning - -**NetBox IPAM Best Practices:** -- IP Ranges = logical grouping -- Individual IP objects = actual reservation -- Both needed for proper IPAM -- Status "Reserved" = correct for DHCP pools -- API automation prevents manual clicking for 255 IPs - ---- - -**Session successfully completed. NetBox now properly shows DHCP pool IPs as reserved.**