Jose Henrique
43351abb11
- Add BACKUP_STAGING_PATH + RSYNC_BIN for staging before archive - Add cleanup_run_staging to EXIT trap to remove staging dir - Extend test harness for staging scenarios
87 lines
4.5 KiB
Markdown
87 lines
4.5 KiB
Markdown
# K8s Script Pack
|
|
|
|
Kubernetes-focused shell scripts intended for cronjobs and operational utilities.
|
|
|
|
## Scripts
|
|
|
|
### `automated-nfs-backup.sh`
|
|
|
|
Backs up each top-level folder found in `NFS_SOURCE_PATH` into a single encrypted `.7z` archive per run, with optional Kubernetes workload quiescing when a folder name exactly matches a namespace name.
|
|
|
|
Behavior:
|
|
- Exact folder-to-namespace mapping only.
|
|
- `kube-system`, `kube-public`, `kube-node-lease`, and `default` are excluded from scale actions by default.
|
|
- Unmapped folder: backup still runs, Kubernetes scale actions are skipped.
|
|
- Mapped folder: saves replicas, scales selected workloads down, waits, adds that folder to the shared run archive, restores replicas, waits again.
|
|
- When `BACKUP_STAGING_PATH` is configured, a live rsync pre-copy is followed by a final offline delta; the namespace is restored before the staged copy is compressed.
|
|
- Scale-down issues are warnings by policy (backup still runs).
|
|
- Restore issues are warnings by policy (run can still complete successfully).
|
|
- Cleanup retains the `N` most recent matching archives by modification time and deletes older ones.
|
|
|
|
## Environment Variables
|
|
|
|
Required:
|
|
- `NFS_SOURCE_PATH`: Root path containing folders to back up.
|
|
- `BACKUP_OUTPUT_PATH`: Destination path for generated `.7z` archives.
|
|
|
|
Optional:
|
|
- `BACKUP_PASSWORD` (default: empty): When set, archive uses password protection; when empty, archive is not encrypted.
|
|
- `KUBECTL_BIN` (default: `kubectl`)
|
|
- `KUBE_CONTEXT` (default: empty)
|
|
- `WORKLOAD_KINDS` (default: `deployment,statefulset,replicaset,replicationcontroller`)
|
|
- `EXCLUDED_NAMESPACES` (default: `kube-system,kube-public,kube-node-lease,default`): Comma-separated namespace names that are always backed up without scale actions. Set an explicit empty value to disable the default exclusions.
|
|
- `BACKUP_STAGING_PATH` (default: empty, disabled): Local staging parent for two-pass rsync. It must be outside `NFS_SOURCE_PATH` and have enough free space for the largest folder being backed up.
|
|
- `RSYNC_BIN` (default: `rsync`)
|
|
- `ARCHIVE_PREFIX` (default: `nfs-backup`)
|
|
- `ARCHIVE_TS_FORMAT` (default: `%Y%m%d_%H%M%S`)
|
|
- `SEVENZ_METHOD` (default: `lzma2`)
|
|
- `SEVENZ_LEVEL` (default: `9`)
|
|
- `SEVENZ_HEADER_ENCRYPT` (default: `on`, only applied when `BACKUP_PASSWORD` is set)
|
|
- `SEVENZ_THREADS` (default: `on`)
|
|
- `SCALE_RETRY_COUNT` (default: `3`)
|
|
- `SCALE_RETRY_DELAY_SECONDS` (default: `5`)
|
|
- `SCALE_WAIT_SECONDS` (default: `30`): Fixed delay after scaling down and after restoring a mapped namespace.
|
|
- `LOG_LEVEL` (default: `info`)
|
|
- `TMP_STATE_DIR` (default: `/tmp/k8s-nfs-backup`): Parent directory for fresh per-run replica-state directories. A run never restores state files belonging to an earlier run.
|
|
- `NOTIFY_SUCCESS_URL` (default: empty, disabled)
|
|
- `NOTIFY_FAILURE_URL` (default: empty, disabled)
|
|
- `NOTIFY_TITLE` (default: `Kubernetes`)
|
|
- `NOTIFY_ASSET` (default: `kube config`)
|
|
- `CLEANUP_KEEP_COUNT` (default: `4`)
|
|
|
|
Notification payload (success and failure):
|
|
|
|
```json
|
|
{
|
|
"title": "Kubernetes",
|
|
"asset": "kube config",
|
|
"backupSizeInMB": 123
|
|
}
|
|
```
|
|
|
|
## Cronjob Notes
|
|
|
|
- Script is designed to run sequentially (one folder at a time).
|
|
- Provide Kubernetes RBAC allowing `get`, `list`, and `scale` on configured workload kinds in target namespaces.
|
|
- Ensure `kubectl` context and credentials are present in the runtime.
|
|
- Ensure `7z` is installed in the runtime image/host.
|
|
- When `BACKUP_STAGING_PATH` is enabled, ensure `rsync` is installed and place staging on fast local storage for the shortest offline delta.
|
|
|
|
## Failure Semantics
|
|
|
|
- Missing required env vars, missing commands, invalid paths, or inability to list namespaces: script exits non-zero immediately.
|
|
- Folder backup failures are counted. One or more successful folder backups continues with cleanup and exits zero; zero successful folder backups exits non-zero.
|
|
- Scale-down warnings/timeouts: logged and counted, backup continues.
|
|
- Restore warnings/timeouts: logged and counted, script does not fail solely because of restore warnings.
|
|
- If `NOTIFY_SUCCESS_URL` is set, success notification is sent at the end of a successful run.
|
|
- If `NOTIFY_FAILURE_URL` is set, a failure notification is sent when any folder backup failure is detected, including partial failures.
|
|
- On runs with at least one successful folder backup, cleanup retains the `CLEANUP_KEEP_COUNT` most recent matching archives and deletes older ones.
|
|
- Final summary always logs:
|
|
- `processed`
|
|
- `mapped`
|
|
- `unmapped`
|
|
- `backup_successes`
|
|
- `backup_failures`
|
|
- `scale_warnings`
|
|
- `restore_warnings`
|