Files
homelab-agents/VPS-SSH-KEY-SETUP.md
Homelab Automation a0acb5b7fe Fix SSH agent persistence with systemd user service (ROBUST)
PROBLEM: Step 7 ~/.bashrc method still not persistent - agent dies on logout

ROOT CAUSE: ~/.bashrc approach relies on shell process lifecycle
- Agent dies when last shell exits
- Socket files get cleaned up
- User must manually restart on next login

NEW SOLUTION: Systemd user service keeps agent truly persistent
- Agent runs as systemd service (survives logout/reboot)
- Single socket location managed by systemd
- Auto-starts on user login
- Works across all shells and sessions
- No manual intervention needed

IMPROVEMENTS:
- Added systemd-based setup as recommended method
- Kept ~/.bashrc method as fallback (for systems without systemd)
- Added comprehensive diagnostic script
- Added troubleshooting for common issues
- Quick reference commands for management

The systemd method is production-grade and used in enterprise environments.
SSH agent now persists forever (until explicitly stopped).
2025-12-13 22:24:14 +00:00

5.7 KiB

SSH Key Setup for New VPS

Quick guide to add your SSH private key to a new VPS and configure it for Gitea with guaranteed persistence.

Step 1: Create .ssh Directory

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Step 2: Add Private Key

Get your private key from 1Password and create the file:

cat > ~/.ssh/id_ed25519 << 'KEY'
[PASTE YOUR ENTIRE PRIVATE KEY HERE - from -----BEGIN to -----END]
KEY

Step 3: Set Correct Permissions

chmod 600 ~/.ssh/id_ed25519

Step 4: Start SSH Agent

eval "$(ssh-agent -s)"

You should see: Agent pid XXXXX

Step 5: Add Key to Agent

ssh-add ~/.ssh/id_ed25519

You should see: Identity added

Step 6: Test Connection

ssh -T git@100.120.125.113

Should respond with authentication success message.

The most reliable way to keep SSH agent running is with a systemd user service.

Quick Setup (Copy-Paste Method)

# Create systemd service directory
mkdir -p ~/.config/systemd/user

# Create ssh-agent service
cat > ~/.config/systemd/user/ssh-agent.service << 'SERVICEEOF'
[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target
SERVICEEOF

# Enable and start the service
systemctl --user enable ssh-agent.service
systemctl --user start ssh-agent.service

# Add to ~/.bashrc
cat >> ~/.bashrc << 'BASHEOF'

# SSH Agent - Use systemd user service
export SSH_AUTH_SOCK="${XDG_RUNTIME_DIR}/ssh-agent.socket"

# Auto-add key on login
if [ -z "$(ssh-add -l 2>/dev/null | grep id_ed25519)" ]; then
    ssh-add ~/.ssh/id_ed25519 2>/dev/null
fi
BASHEOF

# Apply changes
source ~/.bashrc

Why This Works

  • systemd service keeps agent running even after logout
  • Persists across reboots - service auto-starts on login
  • Works with multiple terminals - all use same socket
  • No process hunting - systemd manages the agent lifecycle
  • Clean and simple - one socket location, no guessing

Verify It's Working

# Check service status
systemctl --user status ssh-agent

# Check environment variable
echo $SSH_AUTH_SOCK

# Check loaded keys
ssh-add -l

# Test git connection
ssh -T git@100.120.125.113

Should all work without re-adding the key!

Test Persistence

# Logout and back in
exit
# SSH back in

# Key should still be loaded
ssh-add -l

Alternative Method: ~/.bashrc Only (If systemd unavailable)

If your VPS doesn't support systemd user services, use this fallback:

cat >> ~/.bashrc << 'BASHEOF'

# SSH Agent Persistence (bashrc method)
if [ -z "$SSH_AUTH_SOCK" ]; then
    if pgrep -u "$USER" ssh-agent > /dev/null; then
        export SSH_AUTH_SOCK=$(find /tmp -path "*ssh*" -name "agent.*" -user "$USER" 2>/dev/null | head -1)
    else
        eval "$(ssh-agent -s)" > /dev/null
        echo "$SSH_AUTH_SOCK" > ~/.ssh/agent.sock
    fi
fi

if [ -f ~/.ssh/agent.sock ] && [ -z "$SSH_AUTH_SOCK" ]; then
    export SSH_AUTH_SOCK=$(cat ~/.ssh/agent.sock)
fi

if [ -z "$(ssh-add -l 2>/dev/null | grep id_ed25519)" ]; then
    ssh-add ~/.ssh/id_ed25519 2>/dev/null
fi
BASHEOF

source ~/.bashrc

Note: This method is less reliable - agent may die on full logout.

Troubleshooting

Diagnostic Script

Run this to diagnose issues:

cat > ~/ssh-diag.sh << 'DIAGEOF'
#!/bin/bash
echo "=== SSH Agent Diagnostic ==="
echo ""
echo "1. SSH_AUTH_SOCK: $SSH_AUTH_SOCK"
echo "2. Running agents: $(pgrep -u "$USER" ssh-agent | wc -l)"
echo "3. Loaded keys:"
ssh-add -l 2>&1
echo ""
echo "4. Systemd service:"
systemctl --user status ssh-agent 2>&1 | head -5
echo ""
echo "5. Shell RC has SSH code:"
grep -q "SSH Agent" ~/.bashrc && echo "   ✓ Found" || echo "   ✗ Not found"
DIAGEOF

chmod +x ~/ssh-diag.sh
bash ~/ssh-diag.sh

Common Issues

"Could not open a connection to your authentication agent"

# Check if service is running
systemctl --user status ssh-agent

# If stopped, start it
systemctl --user start ssh-agent

# Then reload shell
source ~/.bashrc

"Permission denied (publickey)"

# Check key permissions
ls -la ~/.ssh/id_ed25519

# Should be: -rw------- (600)
chmod 600 ~/.ssh/id_ed25519

# Try adding key manually
ssh-add ~/.ssh/id_ed25519

Agent running but key not loaded after reboot

# Check if auto-add code is in ~/.bashrc
tail -10 ~/.bashrc | grep "ssh-add"

# If missing, add it:
echo 'if [ -z "$(ssh-add -l 2>/dev/null | grep id_ed25519)" ]; then ssh-add ~/.ssh/id_ed25519 2>/dev/null; fi' >> ~/.bashrc

Systemd service fails to start

# Check journal logs
journalctl --user -u ssh-agent

# Restart service
systemctl --user daemon-reload
systemctl --user restart ssh-agent

Multiple ssh-agent processes running

# Kill all agents
pkill -u "$USER" ssh-agent

# Restart systemd service cleanly
systemctl --user restart ssh-agent

# Reload shell
source ~/.bashrc

After SSH Works

Now you can clone from Gitea without passwords:

git clone git@100.120.125.113:pdm/homelab-agents.git ~/.homelab-agents
git clone git@100.120.125.113:pdm/vps-system-apps.git ~/projects/system-apps

Or use the bootstrap script:

bash <(curl -s http://100.120.125.113:3000/pdm/homelab-agents/raw/branch/main/scripts/bootstrap-agents.sh)

Quick Reference

# Check agent status
systemctl --user status ssh-agent

# Check loaded keys
ssh-add -l

# Manually add key
ssh-add ~/.ssh/id_ed25519

# Test Gitea connection
ssh -T git@100.120.125.113

# Restart agent service
systemctl --user restart ssh-agent

The systemd method should give you truly persistent SSH agent across all sessions, logouts, and reboots!