1 - Command Line
ClusterCockpit Command Line Options
This page describes the command line options for the cc-backend
executable.
-add-user <username>:[admin,support,manager,api,user]:<password>
Function: Adds a new user to the database. Only one role can be assigned.
Example: -add-user abcduser:manager:somepass
Function: Specifies alternative path to application configuration file.
Default: ./config.json
Example: -config ./configfiles/configuration.json
Function: Removes a user from the database by username.
Example: -del-user abcduser
Function: Enables development components: GraphQL Playground and Swagger UI.
Function: Go server listens via github.com/google/gops/agent (for debugging).
-import-job <path-to-meta.json>:<path-to-data.json>, ...
Function: Import one or more jobs by comma seperated list of paths to meta.json
and data.json
.
Example: -import-job ./to-import/job1-meta.json:./to-import/job1-data.json,./to-import/job2-meta.json:./to-import/job2-data.json
Function: Setups var
directory. Initializes sqlite database file, config.json
and .env
environment variable file.
Function: Iterates the job-archive and re-initializes the ‘job’, ’tag’, and ‘jobtag’ tables based on archived jobs.
Caution: All running jobs will be lost!
Function: Generates and prints a JWT for the user specified by its username.
Example: -jwt abcduser
Function: Set this flag to add date and time to log messages.
Function: Sets the loglevel of the running ClusterCockpit instance. “Debug” will print all levels, “Crit” will only log critical log messages.
Arguments: debug | info | warn | err | crit
Default: info
Example: -loglevel debug
Function: Migrate database to latest supported version and exit.
Function: Start a server, continues listening on configured port (Default: :8080
) after initialization and argument handling.
Function: Synchronizes the ‘user’ table with LDAP.
Function: Shows version information and exits.
2 - Configuration
ClusterCockpit Configuration Option References
CC-Backend requires a JSON configuration file that specifies the cluster systems to be used. The schema of the configuration is described at the schema documentation.
To override the default, specify the location of a JSON configuration file with the -config <file path>
command line option.
Configuration Options
addr
: Type string. Address where the http (or https) server will listen on (for example: ’localhost:80’). Default :8080
.apiAllowedIPs
: Type string array. Addresses from which the secured API endpoints (/users and other auth related endpoints) can be reacheduser
: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.group
: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.disable-authentication
: Type bool. Disable authentication (for everything: API, Web-UI, …). Default false
.embed-static-files
: Type bool. If all files in web/frontend/public
should be served from within the binary itself (they are embedded) or not. Default true
.static-files
: Type string. Folder where static assets can be found, if embed-static-files
is false
. No default.db-driver
: Type string. ‘sqlite3’ or ‘mysql’ (mysql will work for mariadb as well). Default sqlite3
.db
: Type string. For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!). Default: ./var/job.db
.job-archive
: Type object.kind
: Type string. At them moment only file is supported as value.path
: Type string. Path to the job-archive. Default: ./var/job-archive
.compression
: Type integer. Setup automatic compression for jobs older than number of days.retention
: Type object.policy
: Type string (required). Retention policy. Possible values none, delete,
move.includeDB
: Type boolean. Also remove jobs from database.age
: Type integer. Act on jobs with startTime older than age (in days).location
: Type string. The target directory for retention. Only applicable for retention policy move.
disable-archive
: Type bool. Keep all metric data in the metric data repositories, do not write to the job-archive. Default false
.validate
: Type bool. Validate all input json documents against json schema.session-max-age
: Type string. Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! Default 168h
.https-cert-file
and https-key-file
: Type string. If both those options are not empty, use HTTPS using those certificates.redirect-http-to
: Type string. If not the empty string and addr
does not end in “:80”, redirect every request incoming at port 80 to that url.machine-state-dir
: Type string. Where to store MachineState files. TODO: Explain in more detail!stop-jobs-exceeding-walltime
: Type int. If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. Default 0
.short-running-jobs-duration
: Type int. Do not show running jobs shorter than X seconds. Default 300
.jwts
: Type object (required). For JWT Authentication.max-age
: Type string (required). Configure how long a token is valid. As string parsable by time.ParseDuration().cookieName
: Type string. Cookie that should be checked for a JWT token.vaidateUser
: Type boolean. Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.trustedIssuer
: Type string. Issuer that should be accepted when validating external JWTs.syncUserOnLogin
: Type boolean. Add non-existent user to DB at login attempt with values provided in JWT.
ldap
: Type object. For LDAP Authentication and user synchronisation. Default nil
.url
: Type string (required). URL of LDAP directory server.user_base
: Type string (required). Base DN of user tree root.search_dn
: Type string (required). DN for authenticating LDAP admin account with general read rights.user_bind
: Type string (required). Expression used to authenticate users via LDAP bind. Must contain uid={username}
.user_filter
: Type string (required). Filter to extract users for syncing.username_attr
: Type string. Attribute with full user name. Defaults to gecos
if not provided.sync_interval
: Type string. Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.sync_del_old_users
: Type boolean. Delete obsolete users in database.syncUserOnLogin
: Type boolean. Add non-existent user to DB at login attempt if user exists in Ldap directory.
clusters
: Type array of objects (required)name
: Type string. The name of the cluster.metricDataRepository
: Type object with properties: kind
(Type string, can be one of cc-metric-store
, influxdb
), url
(Type string), token
(Type string)filterRanges
Type object. This option controls the slider ranges for the UI controls of numNodes, duration, and startTime. Example:
"filterRanges": {
"numNodes": { "from": 1, "to": 64 },
"duration": { "from": 0, "to": 86400 },
"startTime": { "from": "2022-01-01T00:00:00Z", "to": null }
}
ui-defaults
: Type object. Default configuration for ui views. If overwritten, all options must be provided! Most options can be overwritten by the user via the web interface.analysis_view_histogramMetrics
: Type string array. Metrics to show as job count histograms in analysis view. Default ["flops_any", "mem_bw", "mem_used"]
.analysis_view_scatterPlotMetrics
: Type array of string array. Initial
scatter plot configuration in analysis view. Default [["flops_any", "mem_bw"], ["flops_any", "cpu_load"], ["cpu_load", "mem_bw"]]
.job_view_nodestats_selectedMetrics
: Type string array. Initial metrics shown in node statistics table of single job view. Default ["flops_any", "mem_bw", "mem_used"]
.job_view_polarPlotMetrics
: Type string array. Metrics shown in polar plot of single job view. Default ["flops_any", "mem_bw", "mem_used", "net_bw", "file_bw"]
.job_view_selectedMetrics
: Type string array. Default ["flops_any", "mem_bw", "mem_used"]
.plot_general_colorBackground
: Type bool. Color plot background according to job average threshold limits. Default true
.plot_general_colorscheme
: Type string array. Initial color scheme. Default "#00bfff", "#0000ff", "#ff00ff", "#ff0000", "#ff8000", "#ffff00", "#80ff00"
.plot_general_lineWidth
: Type int. Initial linewidth. Default 3
.plot_list_jobsPerPage
: Type int. Jobs shown per page in job lists. Default 50
.plot_list_selectedMetrics
: Type string array. Initial metric plots shown in jobs lists. Default "cpu_load", "ipc", "mem_used", "flops_any", "mem_bw"
.plot_view_plotsPerRow
: Type int. Number of plots per row in single job view. Default 3
.plot_view_showPolarplot
: Type bool. Option to toggle polar plot in single job view. Default true
.plot_view_showRoofline
: Type bool. Option to toggle roofline plot in single job view. Default true
.plot_view_showStatTable
: Type bool. Option to toggle the node statistic table in single job view. Default true
.system_view_selectedMetric
: Type string. Initial metric shown in system view. Default cpu_load
.
Some of the ui-defaults
values can be appended by :<clustername>
in order to have different settings depending on the current cluster. Those are notably job_view_nodestats_selectedMetrics
, job_view_polarPlotMetrics
, job_view_selectedMetrics
and plot_list_selectedMetrics
.
3 - Environment
ClusterCockpit Environment Variables
All security-related configurations, e.g. keys and passwords, are set using environment variables. It is supported to set these by means of a .env
file in the project root.
Environment Variables
An example env file is found in this directory. Copy it as .env
into the project root and adapt it for your needs.
JWT_PUBLIC_KEY
and JWT_PRIVATE_KEY
: Base64 encoded Ed25519 keys used for JSON Web Token (JWT) authentication. You can generate your own keypair using go run ./cmd/gen-keypair/gen-keypair.go
. For more information, see the JWT documentation.SESSION_KEY
: Some random bytes used as secret for cookie-based sessions.LDAP_ADMIN_PASSWORD
: The LDAP admin user password (optional).CROSS_LOGIN_JWT_HS512_KEY
: Used for token based logins via another authentication service.LOGLEVEL
: Can be crit
, err
, warn
, info
or debug
. Can be used to reduce logging. Default is info
.
4 - REST API
ClusterCockpit RESTful API Endpoint Reference
Usage of Swagger UI
To use the Swagger UI for testing you have to run an instance of cc-backend on localhost
(and use the default port 8080):
You may want to start the demo as described here .
This Swagger UI is also available as part of cc-backend
if you start it with
the dev
option:
./cc-backend -server -dev
You may access it at this URL.
Swagger API Reference
Non-Interactive Documentation
This reference is rendered using the
swaggerui
plugin based on the original definition file found in the ClusterCockpit
repository,
but without a serving backend.This means that all interactivity (“Try It Out”) will not return actual data. However, a
Curl
call and a compiled
Request URL
will still be displayed, if an API endpoint is executed.
5 - Authentication Handbook
How to configure and use the authentication backends
6 - Job Archive Handbook
All you need to know about the ClusterCockpit Job Archive
7 - Schemas
ClusterCockpit Schema References
ClusterCockpit Schema References for
- Application Configuration
- Cluster Configuration
- Job Data
- Job Statistics
- Units
- Job Archive Job Metadata
- Job Archive Job Metricdata
The schemas in their raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schemas found in the repository are not automatically rendered in this reference documentation.The raw JSON schemas are parsed and rendered for better readability using the
json-schema-for-humans utility.
Last Update: 02.02.2024
7.1 - Application Config Schema
ClusterCockpit Application Config Schema Reference
A detailed description of each of the application configuration options can be found in the config documentation.
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024cc-backend configuration file schema
Title: cc-backend configuration file schema
1. [Optional] Property cc-backend configuration file schema > addr
Description: Address where the http (or https) server will listen on (for example: ’localhost:80’).
2. [Optional] Property cc-backend configuration file schema > user
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
3. [Optional] Property cc-backend configuration file schema > group
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
4. [Optional] Property cc-backend configuration file schema > disable-authentication
Description: Disable authentication (for everything: API, Web-UI, …).
5. [Optional] Property cc-backend configuration file schema > embed-static-files
Description: If all files in web/frontend/public
should be served from within the binary itself (they are embedded) or not.
6. [Optional] Property cc-backend configuration file schema > static-files
Description: Folder where static assets can be found, if embed-static-files is false.
7. [Optional] Property cc-backend configuration file schema > db-driver
| |
---|
Type | enum (of string) |
Required | No |
Description: sqlite3 or mysql (mysql will work for mariadb as well).
Must be one of:
8. [Optional] Property cc-backend configuration file schema > db
Description: For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!).
9. [Optional] Property cc-backend configuration file schema > job-archive
Description: Configuration keys for job-archive
9.1. [Required] Property cc-backend configuration file schema > job-archive > kind
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Backend type for job-archive
Must be one of:
9.2. [Optional] Property cc-backend configuration file schema > job-archive > path
Description: Path to job archive for file backend
9.3. [Optional] Property cc-backend configuration file schema > job-archive > compression
Description: Setup automatic compression for jobs older than number of days
9.4. [Optional] Property cc-backend configuration file schema > job-archive > retention
Description: Configuration keys for retention
9.4.1. [Required] Property cc-backend configuration file schema > job-archive > retention > policy
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Retention policy
Must be one of:
9.4.2. [Optional] Property cc-backend configuration file schema > job-archive > retention > includeDB
Description: Also remove jobs from database
9.4.3. [Optional] Property cc-backend configuration file schema > job-archive > retention > age
Description: Act on jobs with startTime older than age (in days)
9.4.4. [Optional] Property cc-backend configuration file schema > job-archive > retention > location
Description: The target directory for retention. Only applicable for retention move.
10. [Optional] Property cc-backend configuration file schema > disable-archive
Description: Keep all metric data in the metric data repositories, do not write to the job-archive.
11. [Optional] Property cc-backend configuration file schema > validate
Description: Validate all input json documents against json schema.
12. [Optional] Property cc-backend configuration file schema > session-max-age
Description: Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire!
13. [Optional] Property cc-backend configuration file schema > https-cert-file
Description: Filepath to SSL certificate. If also https-key-file is set use HTTPS using those certificates.
14. [Optional] Property cc-backend configuration file schema > https-key-file
Description: Filepath to SSL key file. If also https-cert-file is set use HTTPS using those certificates.
15. [Optional] Property cc-backend configuration file schema > redirect-http-to
Description: If not the empty string and addr does not end in :80, redirect every request incoming at port 80 to that url.
16. [Optional] Property cc-backend configuration file schema > stop-jobs-exceeding-walltime
Description: If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job.
17. [Optional] Property cc-backend configuration file schema > short-running-jobs-duration
Description: Do not show running jobs shorter than X seconds.
18. [Required] Property cc-backend configuration file schema > jwts
Description: For JWT token authentication.
18.1. [Required] Property cc-backend configuration file schema > jwts > max-age
Description: Configure how long a token is valid. As string parsable by time.ParseDuration()
18.2. [Optional] Property cc-backend configuration file schema > jwts > cookieName
Description: Cookie that should be checked for a JWT token.
18.3. [Optional] Property cc-backend configuration file schema > jwts > validateUser
Description: Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
18.4. [Optional] Property cc-backend configuration file schema > jwts > trustedIssuer
Description: Issuer that should be accepted when validating external JWTs
18.5. [Optional] Property cc-backend configuration file schema > jwts > syncUserOnLogin
Description: Add non-existent user to DB at login attempt with values provided in JWT.
19. [Optional] Property cc-backend configuration file schema > ldap
Description: For LDAP Authentication and user synchronisation.
19.1. [Required] Property cc-backend configuration file schema > ldap > url
Description: URL of LDAP directory server.
19.2. [Required] Property cc-backend configuration file schema > ldap > user_base
Description: Base DN of user tree root.
19.3. [Required] Property cc-backend configuration file schema > ldap > search_dn
Description: DN for authenticating LDAP admin account with general read rights.
19.4. [Required] Property cc-backend configuration file schema > ldap > user_bind
Description: Expression used to authenticate users via LDAP bind. Must contain uid={username}.
19.5. [Required] Property cc-backend configuration file schema > ldap > user_filter
Description: Filter to extract users for syncing.
19.6. [Optional] Property cc-backend configuration file schema > ldap > username_attr
Description: Attribute with full username. Default: gecos
19.7. [Optional] Property cc-backend configuration file schema > ldap > sync_interval
Description: Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
19.8. [Optional] Property cc-backend configuration file schema > ldap > sync_del_old_users
Description: Delete obsolete users in database.
19.9. [Optional] Property cc-backend configuration file schema > ldap > syncUserOnLogin
Description: Add non-existent user to DB at login attempt if user exists in Ldap directory
20. [Required] Property cc-backend configuration file schema > clusters
| |
---|
Type | array of object |
Required | Yes |
Description: Configuration for the clusters to be displayed.
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
20.1. cc-backend configuration file schema > clusters > clusters items
20.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > name
Description: The name of the cluster.
20.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository
Description: Type of the metric data repository for this cluster
20.1.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > kind
| |
---|
Type | enum (of string) |
Required | Yes |
Must be one of:
- “influxdb”
- “prometheus”
- “cc-metric-store”
- “test”
20.1.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > url
20.1.2.3. [Optional] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > token
20.1.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges
Description: This option controls the slider ranges for the UI controls of numNodes, duration, and startTime.
20.1.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes
Description: UI slider range for number of nodes
20.1.3.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > from
20.1.3.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > to
20.1.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration
Description: UI slider range for duration
20.1.3.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > from
20.1.3.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > to
20.1.3.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime
Description: UI slider range for start time
20.1.3.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > from
| |
---|
Type | string |
Required | Yes |
Format | date-time |
20.1.3.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > to
21. [Optional] Property cc-backend configuration file schema > ui-defaults
Description: Default configuration for web UI
21.1. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorBackground
Description: Color plot background according to job average threshold limits
21.2. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_lineWidth
Description: Initial linewidth
21.3. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_jobsPerPage
Description: Jobs shown per page in job lists
21.4. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_plotsPerRow
Description: Number of plots per row in single job view
21.5. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showPolarplot
Description: Option to toggle polar plot in single job view
21.6. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showRoofline
Description: Option to toggle roofline plot in single job view
21.7. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showStatTable
Description: Option to toggle the node statistic table in single job view
21.8. [Required] Property cc-backend configuration file schema > ui-defaults > system_view_selectedMetric
Description: Initial metric shown in system view
21.9. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics
| |
---|
Type | array of string |
Required | Yes |
Description: Metrics to show as job count histograms in analysis view
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.9.1. cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics > analysis_view_histogramMetrics items
21.10. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics
| |
---|
Type | array of array |
Required | Yes |
Description: Initial scatter plto configuration in analysis view
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.10.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items
| |
---|
Type | array of string |
Required | No |
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.10.1.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items > analysis_view_scatterPlotMetrics items items
21.11. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics
| |
---|
Type | array of string |
Required | Yes |
Description: Initial metrics shown in node statistics table of single job view
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.11.1. cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics > job_view_nodestats_selectedMetrics items
21.12. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics
| |
---|
Type | array of string |
Required | Yes |
Description: Metrics shown in polar plot of single job view
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.12.1. cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics > job_view_polarPlotMetrics items
21.13. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics
| |
---|
Type | array of string |
Required | Yes |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.13.1. cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics > job_view_selectedMetrics items
21.14. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorscheme
| |
---|
Type | array of string |
Required | Yes |
Description: Initial color scheme
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.14.1. cc-backend configuration file schema > ui-defaults > plot_general_colorscheme > plot_general_colorscheme items
21.15. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics
| |
---|
Type | array of string |
Required | Yes |
Description: Initial metric plots shown in jobs lists
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.15.1. cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics > plot_list_selectedMetrics items
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.2 - Cluster Schema
ClusterCockpit Cluster Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024HPC cluster description
Title: HPC cluster description
Description: Meta data information of a HPC cluster
1. [Required] Property HPC cluster description > name
Description: The unique identifier of a cluster
2. [Required] Property HPC cluster description > metricConfig
| |
---|
Type | array of object |
Required | Yes |
Description: Metric specifications
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
2.1. HPC cluster description > metricConfig > metricConfig items
2.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > name
Description: Metric name
2.1.2. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit
Description: Metric unit
2.1.2.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
2.1.2.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
2.1.3. [Required] Property HPC cluster description > metricConfig > metricConfig items > scope
Description: Native measurement resolution
2.1.4. [Required] Property HPC cluster description > metricConfig > metricConfig items > timestep
Description: Frequency of timeseries points
2.1.5. [Required] Property HPC cluster description > metricConfig > metricConfig items > aggregation
| |
---|
Type | enum (of string) |
Required | Yes |
Description: How the metric is aggregated
Must be one of:
2.1.6. [Required] Property HPC cluster description > metricConfig > metricConfig items > peak
Description: Metric peak threshold (Upper metric limit)
2.1.7. [Required] Property HPC cluster description > metricConfig > metricConfig items > normal
Description: Metric normal threshold
2.1.8. [Required] Property HPC cluster description > metricConfig > metricConfig items > caution
Description: Metric caution threshold (Suspicious but does not require immediate action)
2.1.9. [Required] Property HPC cluster description > metricConfig > metricConfig items > alert
Description: Metric alert threshold (Requires immediate action)
2.1.10. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters
| |
---|
Type | array of object |
Required | No |
Description: Array of cluster hardware partition metric thresholds
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
2.1.10.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items
2.1.10.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name
Description: Hardware partition name
2.1.10.1.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak
2.1.10.1.3. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal
2.1.10.1.4. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution
2.1.10.1.5. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert
2.1.10.1.6. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove
3. [Required] Property HPC cluster description > subClusters
| |
---|
Type | array of object |
Required | Yes |
Description: Array of cluster hardware partitions
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1. HPC cluster description > subClusters > subClusters items
3.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > name
Description: Hardware partition name
3.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > processorType
Description: Processor type
3.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > socketsPerNode
Description: Number of sockets per node
3.1.4. [Required] Property HPC cluster description > subClusters > subClusters items > coresPerSocket
Description: Number of cores per socket
3.1.5. [Required] Property HPC cluster description > subClusters > subClusters items > threadsPerCore
Description: Number of SMT threads per core
3.1.6. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateScalar
Description: Theoretical node peak flop rate for scalar code in GFlops/s
3.1.6.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit
Description: Metric unit
3.1.6.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value
3.1.7. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateSimd
Description: Theoretical node peak flop rate for SIMD code in GFlops/s
3.1.7.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit
Description: Metric unit
3.1.7.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value
3.1.8. [Required] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth
Description: Theoretical node peak memory bandwidth in GB/s
3.1.8.1. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit
Description: Metric unit
3.1.8.2. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value
3.1.9. [Required] Property HPC cluster description > subClusters > subClusters items > nodes
Description: Node list expression
3.1.10. [Required] Property HPC cluster description > subClusters > subClusters items > topology
Description: Node topology
3.1.10.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > node
| |
---|
Type | array of integer |
Required | Yes |
Description: HwTread lists of node
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
node items | - |
3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items
3.1.10.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > socket
| |
---|
Type | array of array |
Required | Yes |
Description: HwTread lists of sockets
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items
| |
---|
Type | array of integer |
Required | No |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items
3.1.10.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain
| |
---|
Type | array of array |
Required | Yes |
Description: HwTread lists of memory domains
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items
| |
---|
Type | array of integer |
Required | No |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items
3.1.10.4. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > die
| |
---|
Type | array of array |
Required | No |
Description: HwTread lists of dies
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
die items | - |
3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die items
| |
---|
Type | array of integer |
Required | No |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items
3.1.10.5. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > core
| |
---|
Type | array of array |
Required | No |
Description: HwTread lists of cores
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
core items | - |
3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core items
| |
---|
Type | array of integer |
Required | No |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items
3.1.10.6. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > accelerators
| |
---|
Type | array of object |
Required | No |
Description: List of of accelerator devices
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items
3.1.10.6.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id
Description: The unique device id
3.1.10.6.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type
| |
---|
Type | enum (of string) |
Required | Yes |
Description: The accelerator type
Must be one of:
- “Nvidia GPU”
- “AMD GPU”
- “Intel GPU”
3.1.10.6.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model
Description: The accelerator model
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.3 - Job Data Schema
ClusterCockpit Job Data Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Job metric data list
Title: Job metric data list
Description: Collection of metric data of a HPC job
1. [Required] Property Job metric data list > mem_used
Description: Memory capacity used
1.1. [Required] Property Job metric data list > mem_used > node
| |
---|
Type | object |
Required | Yes |
Additional properties | [Any type: allowed] |
Defined in | job-metric-data.schema.json |
Description: Metric data of a HPC job
1.1.1. [Required] Property Job metric data list > mem_used > node > unit
Description: Metric unit
1.1.1.1. [Required] Property Job metric data list > mem_used > node > unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
1.1.1.2. [Optional] Property Job metric data list > mem_used > node > unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
1.1.2. [Required] Property Job metric data list > mem_used > node > timestep
Description: Measurement interval in seconds
1.1.3. [Optional] Property Job metric data list > mem_used > node > thresholds
Description: Metric thresholds for specific system
1.1.3.1. [Optional] Property Job metric data list > mem_used > node > thresholds > peak
1.1.3.2. [Optional] Property Job metric data list > mem_used > node > thresholds > normal
1.1.3.3. [Optional] Property Job metric data list > mem_used > node > thresholds > caution
1.1.3.4. [Optional] Property Job metric data list > mem_used > node > thresholds > alert
1.1.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries
Description: Statistics series across topology
1.1.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > min
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
min items | - |
1.1.4.1.1. Job metric data list > mem_used > node > statisticsSeries > min > min items
1.1.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > max
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
max items | - |
1.1.4.2.1. Job metric data list > mem_used > node > statisticsSeries > max > max items
1.1.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > mean
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
mean items | - |
1.1.4.3.1. Job metric data list > mem_used > node > statisticsSeries > mean > mean items
1.1.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles
1.1.4.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 10
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
10 items | - |
1.1.4.4.1.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 10 > 10 items
1.1.4.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 20
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
20 items | - |
1.1.4.4.2.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 20 > 20 items
1.1.4.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 30
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
30 items | - |
1.1.4.4.3.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 30 > 30 items
1.1.4.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 40
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
40 items | - |
1.1.4.4.4.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 40 > 40 items
1.1.4.4.5. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 50
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
50 items | - |
1.1.4.4.5.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 50 > 50 items
1.1.4.4.6. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 60
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
60 items | - |
1.1.4.4.6.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 60 > 60 items
1.1.4.4.7. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 70
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
70 items | - |
1.1.4.4.7.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 70 > 70 items
1.1.4.4.8. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 80
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
80 items | - |
1.1.4.4.8.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 80 > 80 items
1.1.4.4.9. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 90
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
90 items | - |
1.1.4.4.9.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 90 > 90 items
1.1.4.4.10. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 25
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
25 items | - |
1.1.4.4.10.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 25 > 25 items
1.1.4.4.11. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 75
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
75 items | - |
1.1.4.4.11.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 75 > 75 items
1.1.5. [Required] Property Job metric data list > mem_used > node > series
| |
---|
Type | array of object |
Required | Yes |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
1.1.5.1. Job metric data list > mem_used > node > series > series items
1.1.5.1.1. [Required] Property Job metric data list > mem_used > node > series > series items > hostname
1.1.5.1.2. [Optional] Property Job metric data list > mem_used > node > series > series items > id
1.1.5.1.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics
Description: Statistics across time dimension
1.1.5.1.3.1. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > avg
Description: Series average
1.1.5.1.3.2. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > min
Description: Series minimum
1.1.5.1.3.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > max
Description: Series maximum
1.1.5.1.4. [Required] Property Job metric data list > mem_used > node > series > series items > data
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
1.1.5.1.4.1. At least one of the items must be
2. [Required] Property Job metric data list > flops_any
Description: Total flop rate with DP flops scaled up
2.1. [Optional] Property Job metric data list > flops_any > node
Description: Metric data of a HPC job
2.2. [Optional] Property Job metric data list > flops_any > socket
Description: Metric data of a HPC job
2.3. [Optional] Property Job metric data list > flops_any > memoryDomain
Description: Metric data of a HPC job
2.4. [Optional] Property Job metric data list > flops_any > core
Description: Metric data of a HPC job
2.5. [Optional] Property Job metric data list > flops_any > hwthread
Description: Metric data of a HPC job
3. [Required] Property Job metric data list > mem_bw
Description: Main memory bandwidth
3.1. [Optional] Property Job metric data list > mem_bw > node
Description: Metric data of a HPC job
3.2. [Optional] Property Job metric data list > mem_bw > socket
Description: Metric data of a HPC job
3.3. [Optional] Property Job metric data list > mem_bw > memoryDomain
Description: Metric data of a HPC job
4. [Required] Property Job metric data list > net_bw
Description: Total fast interconnect network bandwidth
4.1. [Required] Property Job metric data list > net_bw > node
Description: Metric data of a HPC job
5. [Optional] Property Job metric data list > ipc
Description: Instructions executed per cycle
5.1. [Optional] Property Job metric data list > ipc > node
Description: Metric data of a HPC job
5.2. [Optional] Property Job metric data list > ipc > socket
Description: Metric data of a HPC job
5.3. [Optional] Property Job metric data list > ipc > memoryDomain
Description: Metric data of a HPC job
5.4. [Optional] Property Job metric data list > ipc > core
Description: Metric data of a HPC job
5.5. [Optional] Property Job metric data list > ipc > hwthread
Description: Metric data of a HPC job
6. [Required] Property Job metric data list > cpu_user
Description: CPU user active core utilization
6.1. [Optional] Property Job metric data list > cpu_user > node
Description: Metric data of a HPC job
6.2. [Optional] Property Job metric data list > cpu_user > socket
Description: Metric data of a HPC job
6.3. [Optional] Property Job metric data list > cpu_user > memoryDomain
Description: Metric data of a HPC job
6.4. [Optional] Property Job metric data list > cpu_user > core
Description: Metric data of a HPC job
6.5. [Optional] Property Job metric data list > cpu_user > hwthread
Description: Metric data of a HPC job
7. [Required] Property Job metric data list > cpu_load
Description: CPU requested core utilization (load 1m)
7.1. [Required] Property Job metric data list > cpu_load > node
Description: Metric data of a HPC job
8. [Optional] Property Job metric data list > flops_dp
Description: Double precision flop rate
8.1. [Optional] Property Job metric data list > flops_dp > node
Description: Metric data of a HPC job
8.2. [Optional] Property Job metric data list > flops_dp > socket
Description: Metric data of a HPC job
8.3. [Optional] Property Job metric data list > flops_dp > memoryDomain
Description: Metric data of a HPC job
8.4. [Optional] Property Job metric data list > flops_dp > core
Description: Metric data of a HPC job
8.5. [Optional] Property Job metric data list > flops_dp > hwthread
Description: Metric data of a HPC job
9. [Optional] Property Job metric data list > flops_sp
Description: Single precision flops rate
9.1. [Optional] Property Job metric data list > flops_sp > node
Description: Metric data of a HPC job
9.2. [Optional] Property Job metric data list > flops_sp > socket
Description: Metric data of a HPC job
9.3. [Optional] Property Job metric data list > flops_sp > memoryDomain
Description: Metric data of a HPC job
9.4. [Optional] Property Job metric data list > flops_sp > core
Description: Metric data of a HPC job
9.5. [Optional] Property Job metric data list > flops_sp > hwthread
Description: Metric data of a HPC job
10. [Optional] Property Job metric data list > vectorization_ratio
Description: Fraction of arithmetic instructions using SIMD instructions
10.1. [Optional] Property Job metric data list > vectorization_ratio > node
Description: Metric data of a HPC job
10.2. [Optional] Property Job metric data list > vectorization_ratio > socket
Description: Metric data of a HPC job
10.3. [Optional] Property Job metric data list > vectorization_ratio > memoryDomain
Description: Metric data of a HPC job
10.4. [Optional] Property Job metric data list > vectorization_ratio > core
Description: Metric data of a HPC job
10.5. [Optional] Property Job metric data list > vectorization_ratio > hwthread
Description: Metric data of a HPC job
11. [Optional] Property Job metric data list > cpu_power
Description: CPU power consumption
11.1. [Optional] Property Job metric data list > cpu_power > node
Description: Metric data of a HPC job
11.2. [Optional] Property Job metric data list > cpu_power > socket
Description: Metric data of a HPC job
12. [Optional] Property Job metric data list > mem_power
Description: Memory power consumption
12.1. [Optional] Property Job metric data list > mem_power > node
Description: Metric data of a HPC job
12.2. [Optional] Property Job metric data list > mem_power > socket
Description: Metric data of a HPC job
13. [Optional] Property Job metric data list > acc_utilization
Description: GPU utilization
13.1. [Required] Property Job metric data list > acc_utilization > accelerator
Description: Metric data of a HPC job
14. [Optional] Property Job metric data list > acc_mem_used
Description: GPU memory capacity used
14.1. [Required] Property Job metric data list > acc_mem_used > accelerator
Description: Metric data of a HPC job
15. [Optional] Property Job metric data list > acc_power
Description: GPU power consumption
15.1. [Required] Property Job metric data list > acc_power > accelerator
Description: Metric data of a HPC job
16. [Optional] Property Job metric data list > clock
Description: Average core frequency
16.1. [Optional] Property Job metric data list > clock > node
Description: Metric data of a HPC job
16.2. [Optional] Property Job metric data list > clock > socket
Description: Metric data of a HPC job
16.3. [Optional] Property Job metric data list > clock > memoryDomain
Description: Metric data of a HPC job
16.4. [Optional] Property Job metric data list > clock > core
Description: Metric data of a HPC job
16.5. [Optional] Property Job metric data list > clock > hwthread
Description: Metric data of a HPC job
17. [Optional] Property Job metric data list > eth_read_bw
Description: Ethernet read bandwidth
17.1. [Required] Property Job metric data list > eth_read_bw > node
Description: Metric data of a HPC job
18. [Optional] Property Job metric data list > eth_write_bw
Description: Ethernet write bandwidth
18.1. [Required] Property Job metric data list > eth_write_bw > node
Description: Metric data of a HPC job
19. [Required] Property Job metric data list > filesystems
| |
---|
Type | array of object |
Required | Yes |
Description: Array of filesystems
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
19.1. Job metric data list > filesystems > filesystems items
19.1.1. [Required] Property Job metric data list > filesystems > filesystems items > name
19.1.2. [Required] Property Job metric data list > filesystems > filesystems items > type
| |
---|
Type | enum (of string) |
Required | Yes |
Must be one of:
- “nfs”
- “lustre”
- “gpfs”
- “nvme”
- “ssd”
- “hdd”
- “beegfs”
19.1.3. [Required] Property Job metric data list > filesystems > filesystems items > read_bw
Description: File system read bandwidth
19.1.3.1. [Required] Property Job metric data list > filesystems > filesystems items > read_bw > node
Description: Metric data of a HPC job
19.1.4. [Required] Property Job metric data list > filesystems > filesystems items > write_bw
Description: File system write bandwidth
19.1.4.1. [Required] Property Job metric data list > filesystems > filesystems items > write_bw > node
Description: Metric data of a HPC job
19.1.5. [Optional] Property Job metric data list > filesystems > filesystems items > read_req
Description: File system read requests
19.1.5.1. [Required] Property Job metric data list > filesystems > filesystems items > read_req > node
Description: Metric data of a HPC job
19.1.6. [Optional] Property Job metric data list > filesystems > filesystems items > write_req
Description: File system write requests
19.1.6.1. [Required] Property Job metric data list > filesystems > filesystems items > write_req > node
Description: Metric data of a HPC job
19.1.7. [Optional] Property Job metric data list > filesystems > filesystems items > inodes
Description: File system write requests
19.1.7.1. [Required] Property Job metric data list > filesystems > filesystems items > inodes > node
Description: Metric data of a HPC job
19.1.8. [Optional] Property Job metric data list > filesystems > filesystems items > accesses
Description: File system open and close
19.1.8.1. [Required] Property Job metric data list > filesystems > filesystems items > accesses > node
Description: Metric data of a HPC job
19.1.9. [Optional] Property Job metric data list > filesystems > filesystems items > fsync
Description: File system fsync
19.1.9.1. [Required] Property Job metric data list > filesystems > filesystems items > fsync > node
Description: Metric data of a HPC job
19.1.10. [Optional] Property Job metric data list > filesystems > filesystems items > create
Description: File system create
19.1.10.1. [Required] Property Job metric data list > filesystems > filesystems items > create > node
Description: Metric data of a HPC job
19.1.11. [Optional] Property Job metric data list > filesystems > filesystems items > open
Description: File system open
19.1.11.1. [Required] Property Job metric data list > filesystems > filesystems items > open > node
Description: Metric data of a HPC job
19.1.12. [Optional] Property Job metric data list > filesystems > filesystems items > close
Description: File system close
19.1.12.1. [Required] Property Job metric data list > filesystems > filesystems items > close > node
Description: Metric data of a HPC job
19.1.13. [Optional] Property Job metric data list > filesystems > filesystems items > seek
Description: File system seek
19.1.13.1. [Required] Property Job metric data list > filesystems > filesystems items > seek > node
Description: Metric data of a HPC job
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.4 - Job Statistics Schema
ClusterCockpit Job Statistics Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Job statistics
Title: Job statistics
Description: Format specification for job metric statistics
1. [Required] Property Job statistics > unit
Description: Metric unit
1.1. [Required] Property Job statistics > unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
1.2. [Optional] Property Job statistics > unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
2. [Required] Property Job statistics > avg
Description: Job metric average
3. [Required] Property Job statistics > min
Description: Job metric minimum
4. [Required] Property Job statistics > max
Description: Job metric maximum
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.5 - Unit Schema
ClusterCockpit Unit Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Metric unit
Title: Metric unit
Description: Format specification for job metric units
1. [Required] Property Metric unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
2. [Optional] Property Metric unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.6 - Job Archive Metadata Schema
ClusterCockpit Job Archive Metadata Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Title: Job meta data
Description: Meta data information of a HPC job
1. [Required] Property Job meta data > jobId
Description: The unique identifier of a job
2. [Required] Property Job meta data > user
Description: The unique identifier of a user
3. [Required] Property Job meta data > project
Description: The unique identifier of a project
4. [Required] Property Job meta data > cluster
Description: The unique identifier of a cluster
5. [Required] Property Job meta data > subCluster
Description: The unique identifier of a sub cluster
6. [Optional] Property Job meta data > partition
Description: The Slurm partition to which the job was submitted
7. [Optional] Property Job meta data > arrayJobId
Description: The unique identifier of an array job
8. [Required] Property Job meta data > numNodes
Description: Number of nodes used
9. [Optional] Property Job meta data > numHwthreads
Description: Number of HWThreads used
10. [Optional] Property Job meta data > numAcc
Description: Number of accelerators used
11. [Required] Property Job meta data > exclusive
Description: Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user
Restrictions | |
---|
Minimum | ≥ 0 |
Maximum | ≤ 2 |
12. [Optional] Property Job meta data > monitoringStatus
Description: State of monitoring system during job run
13. [Optional] Property Job meta data > smt
Description: SMT threads used by job
14. [Optional] Property Job meta data > walltime
Description: Requested walltime of job in seconds
15. [Required] Property Job meta data > jobState
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Final state of job
Must be one of:
- “completed”
- “failed”
- “cancelled”
- “stopped”
- “out_of_memory”
- “timeout”
16. [Required] Property Job meta data > startTime
Description: Start epoch time stamp in seconds
17. [Required] Property Job meta data > duration
Description: Duration of job in seconds
18. [Required] Property Job meta data > resources
| |
---|
Type | array of object |
Required | Yes |
Description: Resources used by job
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
18.1.1. [Required] Property Job meta data > resources > resources items > hostname
18.1.2. [Optional] Property Job meta data > resources > resources items > hwthreads
| |
---|
Type | array of integer |
Required | No |
Description: List of OS processor ids
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
18.1.3. [Optional] Property Job meta data > resources > resources items > accelerators
| |
---|
Type | array of string |
Required | No |
Description: List of of accelerator device ids
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
18.1.4. [Optional] Property Job meta data > resources > resources items > configuration
Description: The configuration options of the node
19. [Optional] Property Job meta data > metaData
Description: Additional information about the job
19.1. [Optional] Property Job meta data > metaData > jobScript
Description: The batch script of the job
19.2. [Optional] Property Job meta data > metaData > jobName
Description: Slurm Job name
19.3. [Optional] Property Job meta data > metaData > slurmInfo
Description: Additional slurm infos as show by scontrol show job
20. [Optional] Property Job meta data > tags
| |
---|
Type | array of object |
Required | No |
Description: List of tags
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | True |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
tags items | - |
20.1.1. [Required] Property Job meta data > tags > tags items > name
20.1.2. [Required] Property Job meta data > tags > tags items > type
21. [Required] Property Job meta data > statistics
Description: Job statistic data
21.1. [Required] Property Job meta data > statistics > mem_used
| |
---|
Type | object |
Required | Yes |
Additional properties | [Any type: allowed] |
Defined in | job-metric-statistics.schema.json |
Description: Memory capacity used (required)
21.1.1. [Required] Property Job meta data > statistics > mem_used > unit
Description: Metric unit
21.1.1.1. [Required] Property Job meta data > statistics > mem_used > unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
21.1.1.2. [Optional] Property Job meta data > statistics > mem_used > unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
21.1.2. [Required] Property Job meta data > statistics > mem_used > avg
Description: Job metric average
21.1.3. [Required] Property Job meta data > statistics > mem_used > min
Description: Job metric minimum
21.1.4. [Required] Property Job meta data > statistics > mem_used > max
Description: Job metric maximum
21.2. [Required] Property Job meta data > statistics > cpu_load
Description: CPU requested core utilization (load 1m) (required)
21.3. [Required] Property Job meta data > statistics > flops_any
Description: Total flop rate with DP flops scaled up (required)
21.4. [Required] Property Job meta data > statistics > mem_bw
Description: Main memory bandwidth (required)
21.5. [Optional] Property Job meta data > statistics > net_bw
Description: Total fast interconnect network bandwidth (required)
21.6. [Optional] Property Job meta data > statistics > file_bw
Description: Total file IO bandwidth (required)
21.7. [Optional] Property Job meta data > statistics > ipc
Description: Instructions executed per cycle
21.8. [Required] Property Job meta data > statistics > cpu_user
Description: CPU user active core utilization
21.9. [Optional] Property Job meta data > statistics > flops_dp
Description: Double precision flop rate
21.10. [Optional] Property Job meta data > statistics > flops_sp
Description: Single precision flops rate
21.11. [Optional] Property Job meta data > statistics > rapl_power
Description: CPU power consumption
21.12. [Optional] Property Job meta data > statistics > acc_used
Description: GPU utilization
21.13. [Optional] Property Job meta data > statistics > acc_mem_used
Description: GPU memory capacity used
21.14. [Optional] Property Job meta data > statistics > acc_power
Description: GPU power consumption
21.15. [Optional] Property Job meta data > statistics > clock
Description: Average core frequency
21.16. [Optional] Property Job meta data > statistics > eth_read_bw
Description: Ethernet read bandwidth
21.17. [Optional] Property Job meta data > statistics > eth_write_bw
Description: Ethernet write bandwidth
21.18. [Optional] Property Job meta data > statistics > ic_rcv_packets
Description: Network interconnect read packets
21.19. [Optional] Property Job meta data > statistics > ic_send_packets
Description: Network interconnect send packet
21.20. [Optional] Property Job meta data > statistics > ic_read_bw
Description: Network interconnect read bandwidth
21.21. [Optional] Property Job meta data > statistics > ic_write_bw
Description: Network interconnect write bandwidth
21.22. [Optional] Property Job meta data > statistics > filesystems
| |
---|
Type | array of object |
Required | No |
Description: Array of filesystems
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
21.22.1.1. [Required] Property Job meta data > statistics > filesystems > filesystems items > name
21.22.1.2. [Required] Property Job meta data > statistics > filesystems > filesystems items > type
| |
---|
Type | enum (of string) |
Required | Yes |
Must be one of:
- “nfs”
- “lustre”
- “gpfs”
- “nvme”
- “ssd”
- “hdd”
- “beegfs”
21.22.1.3. [Required] Property Job meta data > statistics > filesystems > filesystems items > read_bw
Description: File system read bandwidth
21.22.1.4. [Required] Property Job meta data > statistics > filesystems > filesystems items > write_bw
Description: File system write bandwidth
21.22.1.5. [Optional] Property Job meta data > statistics > filesystems > filesystems items > read_req
Description: File system read requests
21.22.1.6. [Optional] Property Job meta data > statistics > filesystems > filesystems items > write_req
Description: File system write requests
21.22.1.7. [Optional] Property Job meta data > statistics > filesystems > filesystems items > inodes
Description: File system write requests
21.22.1.8. [Optional] Property Job meta data > statistics > filesystems > filesystems items > accesses
Description: File system open and close
21.22.1.9. [Optional] Property Job meta data > statistics > filesystems > filesystems items > fsync
Description: File system fsync
21.22.1.10. [Optional] Property Job meta data > statistics > filesystems > filesystems items > create
Description: File system create
21.22.1.11. [Optional] Property Job meta data > statistics > filesystems > filesystems items > open
Description: File system open
21.22.1.12. [Optional] Property Job meta data > statistics > filesystems > filesystems items > close
Description: File system close
21.22.1.13. [Optional] Property Job meta data > statistics > filesystems > filesystems items > seek
Description: File system seek
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100
7.7 - Job Archive Metrics Data Schema
ClusterCockpit Job Archive Metrics Data Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Job metric data
Title: Job metric data
Description: Metric data of a HPC job
1. [Required] Property Job metric data > unit
Description: Metric unit
1.1. [Required] Property Job metric data > unit > base
| |
---|
Type | enum (of string) |
Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
1.2. [Optional] Property Job metric data > unit > prefix
| |
---|
Type | enum (of string) |
Required | No |
Description: Unit prefix
Must be one of:
2. [Required] Property Job metric data > timestep
Description: Measurement interval in seconds
3. [Optional] Property Job metric data > thresholds
Description: Metric thresholds for specific system
3.1. [Optional] Property Job metric data > thresholds > peak
3.2. [Optional] Property Job metric data > thresholds > normal
3.3. [Optional] Property Job metric data > thresholds > caution
3.4. [Optional] Property Job metric data > thresholds > alert
4. [Optional] Property Job metric data > statisticsSeries
Description: Statistics series across topology
4.1. [Optional] Property Job metric data > statisticsSeries > min
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
min items | - |
4.1.1. Job metric data > statisticsSeries > min > min items
4.2. [Optional] Property Job metric data > statisticsSeries > max
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
max items | - |
4.2.1. Job metric data > statisticsSeries > max > max items
4.3. [Optional] Property Job metric data > statisticsSeries > mean
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
mean items | - |
4.3.1. Job metric data > statisticsSeries > mean > mean items
4.4. [Optional] Property Job metric data > statisticsSeries > percentiles
4.4.1. [Optional] Property Job metric data > statisticsSeries > percentiles > 10
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
10 items | - |
4.4.1.1. Job metric data > statisticsSeries > percentiles > 10 > 10 items
4.4.2. [Optional] Property Job metric data > statisticsSeries > percentiles > 20
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
20 items | - |
4.4.2.1. Job metric data > statisticsSeries > percentiles > 20 > 20 items
4.4.3. [Optional] Property Job metric data > statisticsSeries > percentiles > 30
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
30 items | - |
4.4.3.1. Job metric data > statisticsSeries > percentiles > 30 > 30 items
4.4.4. [Optional] Property Job metric data > statisticsSeries > percentiles > 40
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
40 items | - |
4.4.4.1. Job metric data > statisticsSeries > percentiles > 40 > 40 items
4.4.5. [Optional] Property Job metric data > statisticsSeries > percentiles > 50
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
50 items | - |
4.4.5.1. Job metric data > statisticsSeries > percentiles > 50 > 50 items
4.4.6. [Optional] Property Job metric data > statisticsSeries > percentiles > 60
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
60 items | - |
4.4.6.1. Job metric data > statisticsSeries > percentiles > 60 > 60 items
4.4.7. [Optional] Property Job metric data > statisticsSeries > percentiles > 70
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
70 items | - |
4.4.7.1. Job metric data > statisticsSeries > percentiles > 70 > 70 items
4.4.8. [Optional] Property Job metric data > statisticsSeries > percentiles > 80
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
80 items | - |
4.4.8.1. Job metric data > statisticsSeries > percentiles > 80 > 80 items
4.4.9. [Optional] Property Job metric data > statisticsSeries > percentiles > 90
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
90 items | - |
4.4.9.1. Job metric data > statisticsSeries > percentiles > 90 > 90 items
4.4.10. [Optional] Property Job metric data > statisticsSeries > percentiles > 25
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
25 items | - |
4.4.10.1. Job metric data > statisticsSeries > percentiles > 25 > 25 items
4.4.11. [Optional] Property Job metric data > statisticsSeries > percentiles > 75
| |
---|
Type | array of number |
Required | No |
| Array restrictions |
---|
Min items | 3 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
Each item of this array must be | Description |
---|
75 items | - |
4.4.11.1. Job metric data > statisticsSeries > percentiles > 75 > 75 items
5. [Required] Property Job metric data > series
| |
---|
Type | array of object |
Required | Yes |
| Array restrictions |
---|
Min items | N/A |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
5.1. Job metric data > series > series items
5.1.1. [Required] Property Job metric data > series > series items > hostname
5.1.2. [Optional] Property Job metric data > series > series items > id
5.1.3. [Required] Property Job metric data > series > series items > statistics
Description: Statistics across time dimension
5.1.3.1. [Required] Property Job metric data > series > series items > statistics > avg
Description: Series average
5.1.3.2. [Required] Property Job metric data > series > series items > statistics > min
Description: Series minimum
5.1.3.3. [Required] Property Job metric data > series > series items > statistics > max
Description: Series maximum
5.1.4. [Required] Property Job metric data > series > series items > data
| Array restrictions |
---|
Min items | 1 |
Max items | N/A |
Items unicity | False |
Additional items | False |
Tuple validation | See below |
5.1.4.1. At least one of the items must be
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100