CLI Reference¶

All commands are available via ops-hpc <command>.

Quick reference¶

$ ops-hpc --help

usage: ops-hpc [-h] [--verbose]
               {setup,build,run,run-staged,stage,log,results,status,vm,image,terminate,cleanup,config} ...

OpenSees HPC Pipeline Controller for GCP

positional arguments:
  {config,setup,build,run,run-staged,stage,log,results,status,vm,image,terminate,cleanup}
    config              Manage user profiles (project, zone, username)
    setup               One-time setup: create bucket, check auth and quota
    build               Build OpenSees image on a spot VM
    run                 Run analysis on GCP
    run-staged          Stage + create VM + start analysis (one command)
    stage               Zip + upload analysis files to GCS
    log                 Show run log or status from GCS
    results             GCS results: list or download
    status              Show image, VM, bucket, and cost status
    vm                  VM lifecycle: list, create, delete, upload, download, push, monitor
    image               Image lifecycle: list, create, archive, restore, delete
    terminate           Delete a VM (shortcut for vm delete)
    cleanup             Force-wipe bucket and kill orphan VMs

options:
  -h, --help            show this help message and exit
  --verbose, -v         Enable debug logging

$ ops-hpc vm --help

usage: ops-hpc vm [-h] {list,create,delete,upload,download,push,monitor} ...

positional arguments:
  {list,create,delete,upload,download,push,monitor}
    list                List all OpenSees VMs
    create              Create VM from OpenSees image
    delete              Delete a VM (or --all to delete all)
    upload              Upload files/directories to a VM (--from-file, --home)
    download            Download files/directories from a VM (SCP, small files)
    push                Push results from VM to GCS bucket (large results)
    monitor             Monitor a running VM

$ ops-hpc vm monitor --help

usage: ops-hpc vm monitor [-h] {stats,ps,log,disk,opsconsole} ...

positional arguments:
  {stats,ps,log,disk,opsconsole}
    stats               CPU, RAM, disk, network, processes
    ps                  Show running processes (top-like)
    log                 Tail a log file on the VM
    disk                Show disk usage breakdown
    opsconsole          Stream live OpenSees output

$ ops-hpc image --help

usage: ops-hpc image [-h] {list,create,archive,restore,delete} ...

positional arguments:
  {list,create,archive,restore,delete}
    list                List all OpenSees images
    create              Create image from a VM's disk
    archive             Export image to cold storage
    restore             Restore image from cold storage
    delete              Permanently delete image everywhere

Global options¶

ops-hpc --help              # show help
ops-hpc --verbose <command> # enable debug logging

Profile commands¶

All configuration (project, zone, SSH username, costs) is stored in named profiles. You must create a profile before any other commands work.

Profiles are stored at ~/.ops-hpc/profiles/<name>.json.

config create¶

Create a new profile interactively. Prompts for required fields.

ops-hpc config create us-central
ops-hpc config create europe-west

config use¶

Switch the active profile.

ops-hpc config use europe-west        # all commands now use europe-west4-b
ops-hpc config use us-central      # switch back to us-central1-a

config show¶

Show the active profile with all values and their sources (profile, default, or auto-derived).

ops-hpc config show

config list¶

List all profiles. Active profile is marked with *.

ops-hpc config list

config set / unset¶

Modify a value in the active profile.

ops-hpc config set zone europe-west4-c
ops-hpc config set local_results_dir "H:\OpenSeesAnalyses\Results"
ops-hpc config unset hourly_cost     # revert to default

config copy / delete¶

ops-hpc config copy us-central us-east    # copy a profile
ops-hpc config delete us-east             # delete a profile

config path¶

Print the profile directory and active profile file location.

ops-hpc config path

Profile keys¶

Key	Required	Default	Description
`project`	yes	—	GCP project ID
`zone`	yes	—	GCP zone (e.g. `us-central1-a`, `europe-west4-b`)
`username`	yes	—	OS Login SSH username (e.g. `user_example_com`)
`bucket`	no	`{project}-opensees-results`	GCS results bucket name
`machine_type`	no	`h3-standard-88`	VM machine type
`image_name`	no	`opensees-rocky8-ready`	OpenSees image name
`image_family`	no	`opensees-hpc`	Image family
`boot_disk_size_gb`	no	50	Build VM disk size
`run_disk_size_gb`	no	100	Analysis VM disk size
`boot_disk_type`	no	`pd-balanced`	Disk type
`num_cores`	no	88	CPU cores on the machine
`hourly_cost`	no	4.92	On-demand cost ($/hr)
`hourly_cost_spot`	no	2.34	Spot cost ($/hr)
`local_results_dir`	no	`./results`	Local download directory

Example profiles¶

us-central (us-central1-a, $4.92/hr on-demand):

{
  "project": "my-project-id",
  "zone": "us-central1-a",
  "username": "myuser",
  "hourly_cost": 4.92,
  "hourly_cost_spot": 2.34
}

europe-west (europe-west4-b, $5.17/hr on-demand):

{
  "project": "my-project-id",
  "zone": "europe-west4-b",
  "username": "myuser",
  "hourly_cost": 5.17,
  "hourly_cost_spot": 2.46
}

Commands¶

setup¶

One-time project setup. Checks auth, OS Login, bucket, storage permissions, and H3 quota.

ops-hpc setup

Checks (5 steps):

GCP authentication
OS Login enabled (project-level enable-oslogin=TRUE)
GCS results bucket exists (creates if needed)
VM service account has storage write access (fixes if needed)
H3 CPU quota (88 needed for h3-standard-88)

build¶

Build the OpenSees image on a spot VM.

ops-hpc build                              # default image name
ops-hpc build --name my-custom-image       # custom image name

Creates spot h3-standard-88 (50 GB disk)
Uploads and runs build_opensees_full.sh
Stops VM, creates image, deletes VM
Asks for confirmation before starting

Flag	Description
`--name`	Custom image name (default: `opensees-rocky8-ready`)

run¶

Run analyses from a jobs file.

ops-hpc run --jobs jobs.txt           # on-demand VM (safe)
ops-hpc run --jobs jobs.txt --spot    # spot VM (cheap, for testing)

Flag	Description
`--jobs`, `-j`	Path to jobs file (required)
`--spot`	Use spot VM (cheaper, may preempt)

run-staged¶

Stage analysis files to GCS, create a VM with a startup script, and start analysis on boot. No SSH needed -- the VM downloads the staged zip and runs automatically via GCP startup script mechanism.

ops-hpc run-staged batch.txt                          # on-demand VM
ops-hpc run-staged batch.txt --spot                   # spot VM (cheaper)
ops-hpc run-staged batch.txt --max-hours 8            # custom safety cap
ops-hpc run-staged batch.txt --max-minutes 90         # safety cap in minutes
ops-hpc run-staged batch.txt --name my-run            # custom VM name

Flag	Description
`batch_file`	Path to batch file listing analysis directories (required)
`--spot`	Use spot VM (cheaper, may preempt)
`--max-hours`	Auto-delete safety cap in hours (default: 12)
`--max-minutes`	Safety cap in minutes (overrides --max-hours)
`--name`	Custom VM name (default: `opensees-run-<batch>-YYYYMMDD`)

How it works:

Reads directories from the batch file
Zips and uploads them to GCS _uploads/ prefix (same as stage)
Creates a VM with a startup script that calls run_analysis.sh <batch_name>
Analysis starts on boot -- no SSH needed
VM name auto-derived from batch filename (underscores replaced with hyphens)
If VM creation fails, the staged zip is cleaned up automatically
If API times out, checks whether the VM actually exists before erroring

Monitor progress with ops-hpc log. Results appear in the GCS bucket as zip files.

stage¶

Zip analysis directories and upload to GCS _uploads/ prefix. Replaces the old vm stage subcommand.

ops-hpc stage --from-file batch.txt                   # zip name derived from filename
ops-hpc stage --from-file batch.txt --name mybatch    # custom zip name
ops-hpc stage ./model_A ./model_B --name mybatch      # positional paths
ops-hpc stage --list                                  # list staged zips
ops-hpc stage --delete batch                          # delete a staged file
ops-hpc stage --clear                                 # delete all staged files

Flag	Description
`paths`	Local files or directories (positional)
`--from-file`, `-f`	Read paths from a text file (one per line)
`--name`, `-n`	Zip name (default: derived from --from-file filename)
`--list`, `-l`	List staged zips in GCS
`--delete`, `-d`	Delete a staged zip from GCS (handles both zip and non-zip names)
`--clear`	Delete all staged files from GCS

After staging, the command prints a one-liner to paste in SSH-in-browser:

mkdir -p /tmp/ops-hpc-analysis && gsutil cp gs://bucket/_uploads/batch.zip /tmp/ && ...

log¶

View run logs and status from GCS. Logs are written by run_analysis.sh to the _logs/ prefix and status to _status/latest.txt.

ops-hpc log                    # show latest status
ops-hpc log --list             # list all available logs
ops-hpc log my-run-log         # show a specific log

Flag	Description
`name`	Log name to show (optional, default: latest status)
`--list`, `-l`	List all available logs

results list¶

Show what's in the GCS bucket without downloading.

ops-hpc results list

results download¶

Download results from GCS bucket to your PC. Shows progress percentage.

ops-hpc results download                                # one-shot download to ./results/
ops-hpc results download --dir H:\MyResults             # custom directory
ops-hpc results download --watch                        # armed: poll + download as results appear
ops-hpc results download --watch --interval 120         # poll every 2 minutes

Flag	Description
`--watch`, `-w`	Keep polling for new results (Ctrl+C to stop)
`--interval`	Poll interval in seconds (default: 60)
`--dir`, `-d`	Download directory (default: ./results)

See Results & Downloads for the full explanation of download modes and tracking.

status¶

Show image, VMs, bucket, and cost information. Bucket listing shows one line per result set (not per file).

ops-hpc status                # quick status
ops-hpc status --usage        # include per-VM compute cost breakdown (slower)

Flag	Description
`--usage`, `-u`	Include 30-day compute cost breakdown

VM commands¶

Manage VMs directly. All VM commands follow ops-hpc vm <action>.

vm list¶

List all OpenSees VMs in the project.

ops-hpc vm list

vm create¶

Create a VM from the OpenSees image and wait for SSH. Asks for confirmation.

ops-hpc vm create                          # auto-generated name (opensees-run-YYYYMMDD)
ops-hpc vm create my-vm                    # custom name
ops-hpc vm create my-vm --spot             # spot pricing
ops-hpc vm create my-vm --max-hours 4      # 4h safety cap
ops-hpc vm create my-vm --max-minutes 90   # 90 min safety cap
ops-hpc vm create my-vm --disk-size 50     # 50 GB boot disk

Argument/Flag	Description
`name`	VM name (optional, auto-generated if omitted)
`--spot`	Use spot VM
`--max-hours`	Auto-delete after N hours, supports decimals e.g. 1.5 (default: 6)
`--max-minutes`	Auto-delete after N minutes (overrides --max-hours)
`--disk-size`	Boot disk size in GB (default: 100)

VM naming convention: opensees-<purpose>-<model>-<YYYYMMDD> (e.g. opensees-run-ttflat-20260406).

vm delete¶

Delete a single VM or all VMs. Asks for confirmation.

ops-hpc vm delete my-vm      # delete one VM
ops-hpc vm delete --all      # delete ALL VMs + wipe bucket (runs cleanup)

Argument/Flag	Description
`name`	VM name to delete
`--all`	Delete all OpenSees VMs and wipe bucket

vm upload¶

Upload local files or directories to a running VM via SCP. Automatically fixes permissions after upload.

ops-hpc vm upload my-vm ./my_model
ops-hpc vm upload my-vm file1.tcl file2.tcl
ops-hpc vm upload my-vm ./model --remote-dir /opt/analysis
ops-hpc vm upload my-vm --from-file paths.txt          # read paths from a file
ops-hpc vm upload my-vm ./script.sh --home              # upload to ~ instead of analysis dir

Argument/Flag	Description
`name`	Target VM name
`paths`	Local files or directories to upload
`--from-file`, `-f`	Read paths from a text file (one per line)
`--remote-dir`, `-r`	Remote destination (default: `/tmp/ops-hpc-analysis`)
`--home`	Upload to home directory (`~`) instead of analysis dir

vm download¶

Download files or directories from a running VM via SCP. Best for small files. For large result sets, use vm push + results instead.

ops-hpc vm download my-vm /tmp/ops-hpc-analysis/results
ops-hpc vm download my-vm /tmp/output.txt --dir D:\MyResults

Argument/Flag	Description
`name`	Source VM name
`remote_path`	Remote file or directory to download
`--dir`, `-d`	Local destination directory (default: ./results)

vm push¶

Push results from the VM to the GCS bucket. Uses Google's internal network (fast). Each push creates a timestamped prefix to prevent overwrites. Download later with ops-hpc results.

ops-hpc vm push my-vm                  # push all analysis dirs to GCS

This is the recommended way to get large results off the VM: 1. ops-hpc vm push my-vm (VM -> GCS, fast internal network) 2. ops-hpc results --dir H:\MyResults (GCS -> your PC) 3. Bucket auto-cleaned after verified download

Argument	Description
`name`	Source VM name

vm monitor¶

Monitor a running VM. Five modes available.

vm monitor stats¶

CPU, RAM, disk, disk I/O, network, uptime, and OpenSees process count. Matches the GUI metrics display.

ops-hpc vm monitor stats my-vm           # one-shot snapshot
ops-hpc vm monitor stats my-vm -w        # keep refreshing every 15s
ops-hpc vm monitor stats my-vm -w --interval 5  # refresh every 5s

Flag	Description
`--watch`, `-w`	Keep refreshing
`--interval`	Refresh interval in seconds (default: 15)

vm monitor ps¶

Show running processes on the VM (top-like, sorted by CPU usage).

ops-hpc vm monitor ps my-vm

vm monitor log¶

Tail a log file on the VM. Auto-detects .log and .out files in the analysis directory.

ops-hpc vm monitor log my-vm                     # last 50 lines (auto-detect file)
ops-hpc vm monitor log my-vm -f                  # follow in real-time (Ctrl+C to stop)
ops-hpc vm monitor log my-vm --file /path/to/log # specific file
ops-hpc vm monitor log my-vm -n 100              # last 100 lines

Flag	Description
`--follow`, `-f`	Follow output in real-time
`--file`	Remote log file path (auto-detect if omitted)
`--lines`, `-n`	Number of lines to show (default: 50)

vm monitor disk¶

Show disk usage breakdown on the VM.

ops-hpc vm monitor disk my-vm                    # df + analysis dir breakdown
ops-hpc vm monitor disk my-vm --path /opt        # breakdown of specific path

Flag	Description
`--path`	Remote path for breakdown (default: analysis dir)

vm monitor opsconsole¶

Stream live OpenSees console output by attaching to the running process.

ops-hpc vm monitor opsconsole my-vm              # Ctrl+C to stop

Image commands¶

Manage OpenSees disk images. All image commands follow ops-hpc image <action>.

image list¶

List all OpenSees images in the project.

ops-hpc image list

image create¶

Create a new image from a VM's boot disk. Stops the VM first if running. Asks for confirmation.

ops-hpc image create my-vm                     # default image name
ops-hpc image create my-vm --name my-image     # custom image name

Argument/Flag	Description
`vm_name`	VM to snapshot (will be stopped first)
`--name`	Custom image name (default: from config)

image archive¶

Export image to Cloud Storage (cold) and delete from Compute Engine. Asks for confirmation.

ops-hpc image archive

image restore¶

Restore image from Cloud Storage back to Compute Engine.

ops-hpc image restore

image delete¶

Permanently delete the image from both Compute Engine and Cloud Storage. Asks for confirmation.

ops-hpc image delete

Shortcuts¶

terminate¶

Shortcut for vm delete. Asks for confirmation.

ops-hpc terminate my-vm      # same as: ops-hpc vm delete my-vm
ops-hpc terminate --all      # same as: ops-hpc vm delete --all

cleanup¶

Emergency cleanup: kill all OpenSees VMs, wipe bucket. Asks for confirmation.

ops-hpc cleanup

Manual workflow cheat sheet¶

SSH workflow (upload + SSH + run manually)¶

ops-hpc vm list                          # what VMs are running?
ops-hpc results list                     # what's in the bucket?
ops-hpc status                           # full status (image, VMs, bucket, costs)

ops-hpc vm create <name> [--spot]        # start VM
ops-hpc vm upload <name> <path>          # upload model
# SSH in: gcloud compute ssh <user>@<name> --zone=<zone>
# Run:    openseesmp <NP> main.tcl
ops-hpc vm monitor stats <name> -w       # watch metrics
ops-hpc vm push <name>                   # results -> GCS
ops-hpc results download --dir <path>    # GCS -> your PC
ops-hpc terminate <name>                 # kill VM

No-SSH workflow (stage + auto-start)¶

ops-hpc run-staged batch.txt             # stage, create VM, auto-start analysis
ops-hpc log                              # check progress
ops-hpc results download --dir <path>    # GCS -> your PC when done

Or step-by-step:

ops-hpc stage --from-file batch.txt      # zip + upload to GCS
ops-hpc vm create <name>                 # start VM
# SSH in and run: bash ~/run_analysis.sh batchname
ops-hpc log                              # check progress
ops-hpc results download --dir <path>    # GCS -> your PC
ops-hpc terminate <name>                 # kill VM

Core pinning with `I_MPI_PIN_PROCESSOR_LIST`¶

The I_MPI_PIN_PROCESSOR_LIST environment variable controls which CPU cores an MPI job uses. Set it before calling openseesmp to pin ranks to specific cores.

h3-standard-88 layout¶

Socket 0                          Socket 1
├── NUMA node 0: cores 0-21      ├── NUMA node 2: cores 44-65
└── NUMA node 1: cores 22-43     └── NUMA node 3: cores 66-87

2 sockets x 44 cores (Intel Xeon Platinum 8481C @ 2.70GHz)
4 NUMA nodes (SNC2: 22 cores per NUMA node)
No hyperthreading (SMT off)
Total: 88 physical cores

Single model — use all cores¶

No pinning needed. The wrapper assigns ranks to cores 0-87 automatically:

cd /tmp/ops-hpc-analysis/my_model
openseesmp 88 main.tcl

If you want to be explicit (e.g., leave 2 cores free for OS):

export I_MPI_PIN_PROCESSOR_LIST=0-85
openseesmp 86 main.tcl

Single model — restrict to one socket¶

Useful for testing or when sharing the machine:

export I_MPI_PIN_PROCESSOR_LIST=0-43
cd /tmp/ops-hpc-analysis/my_model
openseesmp 44 main.tcl

Two models simultaneously — one per socket¶

Each model gets its own socket with dedicated memory bandwidth. Run from two separate SSH terminals (or web console sessions):

Terminal 1:

export I_MPI_PIN_PROCESSOR_LIST=0-43
cd /tmp/ops-hpc-analysis/model_A
openseesmp 44 main.tcl

Terminal 2:

export I_MPI_PIN_PROCESSOR_LIST=44-87
cd /tmp/ops-hpc-analysis/model_B
openseesmp 44 main.tcl

Or run both from one terminal in the background:

(export I_MPI_PIN_PROCESSOR_LIST=0-43 && cd /tmp/ops-hpc-analysis/model_A && openseesmp 44 main.tcl) &
(export I_MPI_PIN_PROCESSOR_LIST=44-87 && cd /tmp/ops-hpc-analysis/model_B && openseesmp 44 main.tcl) &
wait

Three models — split across cores¶

(export I_MPI_PIN_PROCESSOR_LIST=0-27 && cd /tmp/ops-hpc-analysis/model_A && openseesmp 28 main.tcl) &
(export I_MPI_PIN_PROCESSOR_LIST=28-55 && cd /tmp/ops-hpc-analysis/model_B && openseesmp 28 main.tcl) &
(export I_MPI_PIN_PROCESSOR_LIST=56-83 && cd /tmp/ops-hpc-analysis/model_C && openseesmp 28 main.tcl) &
wait

Important notes¶

Partition count must match NP — repartition in STKO to match the number of ranks you want per model. The number of .mpco.cdata files must equal the NP argument
The pin list overrides the wrapper — I_MPI_PIN_PROCESSOR_LIST has highest priority in Intel MPI, so it overrides I_MPI_PIN_DOMAIN and other settings in the openseesmp wrapper
Check if simultaneous is actually faster — some models are memory-bandwidth bound, not compute-bound. Run a quick test: start model A alone, note the timestep pace, then start model B. If model A slows down significantly, run sequentially instead
Scaling test — run a few timesteps at different core counts (22, 44, 86) to find the sweet spot. If 44 cores is nearly as fast as 86, the model doesn't benefit from more cores
Monitor from another terminal: htop or ps aux | grep OpenSees
Push results after both finish: ops-hpc vm push my-vm uploads all directories