package api.v1.beta1

Get desktop application:
View/edit binary Protocol Buffers messages

* DBManager service defines APIs to manage Katib database.

• rpc ReportObservationLog (ReportObservationLogRequest, ReportObservationLogReply)

  api.proto:20

  * Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.

  message ReportObservationLogRequest

  api.proto:272

  • string trial_name = 1

  • optional ObservationLog observation_log = 2

  message ReportObservationLogReply

  api.proto:277

  (message has no fields)

• rpc GetObservationLog (GetObservationLogRequest, GetObservationLogReply)

  api.proto:25

  * Get all log of Observations for a Trial.

  message GetObservationLogRequest

  api.proto:289

  • string trial_name = 1

  • string metric_name = 2

  • string start_time = 3

    /The start of the time range. RFC3339 format

  • string end_time = 4

    /The end of the time range. RFC3339 format

  message GetObservationLogReply

  api.proto:296

  • optional ObservationLog observation_log = 1

• rpc DeleteObservationLog (DeleteObservationLogRequest, DeleteObservationLogReply)

  api.proto:30

  * Delete all log of Observations for a Trial.

  message DeleteObservationLogRequest

  api.proto:300

  • string trial_name = 1

  message DeleteObservationLogReply

  api.proto:304

  (message has no fields)

* EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

• rpc GetEarlyStoppingRules (GetEarlyStoppingRulesRequest, GetEarlyStoppingRulesReply)

  api.proto:45

  message GetEarlyStoppingRulesRequest

  api.proto:340

  • optional Experiment experiment = 1

  • repeated Trial trials = 2

  • string db_manager_address = 3

  message GetEarlyStoppingRulesReply

  api.proto:346

  • repeated EarlyStoppingRule early_stopping_rules = 1

• rpc SetTrialStatus (SetTrialStatusRequest, SetTrialStatusReply)

  api.proto:46

  message SetTrialStatusRequest

  api.proto:379

  • string trial_name = 1

  message SetTrialStatusReply

  api.proto:383

  (message has no fields)

• rpc ValidateEarlyStoppingSettings (ValidateEarlyStoppingSettingsRequest, ValidateEarlyStoppingSettingsReply)

  api.proto:47

  message ValidateEarlyStoppingSettingsRequest

  api.proto:362

  • optional EarlyStoppingSpec early_stopping = 1

  message ValidateEarlyStoppingSettingsReply

  api.proto:369

  * Return INVALID_ARGUMENT Error if Early Stopping Settings are not Valid

  (message has no fields)

* Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

• rpc GetSuggestions (GetSuggestionsRequest, GetSuggestionsReply)

  api.proto:37

  message GetSuggestionsRequest

  api.proto:307

  • optional Experiment experiment = 1

  • repeated Trial trials = 2

    All completed trials owned by the experiment.

  • int32 current_request_number = 4

    The number of Suggestions requested at one time. When you set 3 to current_request_number, you get three Suggestions at one time.

  • int32 total_request_number = 5

    The number of Suggestions requested till now.

  message GetSuggestionsReply

  api.proto:316

  • repeated GetSuggestionsReply.ParameterAssignments parameter_assignments = 1

  • optional AlgorithmSpec algorithm = 2

  • repeated EarlyStoppingRule early_stopping_rules = 3

• rpc ValidateAlgorithmSettings (ValidateAlgorithmSettingsRequest, ValidateAlgorithmSettingsReply)

  api.proto:38

  message ValidateAlgorithmSettingsRequest

  api.proto:330

  • optional Experiment experiment = 1

  message ValidateAlgorithmSettingsReply

  api.proto:337

  * Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

  (message has no fields)

* HP or NAS algorithm settings.

Used in: AlgorithmSpec

• string name = 1

• string value = 2

* HP or NAS algorithm specification.

Used in: ExperimentSpec, GetSuggestionsReply

• string algorithm_name = 1

• repeated AlgorithmSetting algorithm_settings = 2

Used in: EarlyStoppingRule

• UNKNOWN_COMPARISON = 0

  Unknown comparison, not used

• EQUAL = 1

  Equal comparison, e.g. accuracy = 0.7

• LESS = 2

  Less comparison, e.g. accuracy < 0.7

• GREATER = 3

  Greater comparison, e.g. accuracy > 0.7

* Distribution types for HyperParameter.

Used in: FeasibleSpace

• DISTRIBUTION_UNSPECIFIED = 0

• UNIFORM = 1

• LOG_UNIFORM = 2

• NORMAL = 3

• LOG_NORMAL = 4

* EarlyStoppingRule represents single early stopping rule.

Used in: GetEarlyStoppingRulesReply, GetSuggestionsReply

• string name = 1

  Name of the rule. Usually, metric name.

• string value = 2

  Value of the metric.

• ComparisonType comparison = 3

  Correlation between name and value, one of equal, less or greater

• int32 start_step = 4

  Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

* Early stopping algorithm settings.

Used in: EarlyStoppingSpec

• string name = 1

• string value = 2

* Early stopping algorithm specification.

Used in: ExperimentSpec, ValidateEarlyStoppingSettingsRequest

• string algorithm_name = 1

• repeated EarlyStoppingSetting algorithm_settings = 2

* Structure for a single Experiment.

Used in: GetEarlyStoppingRulesRequest, GetSuggestionsRequest, ValidateAlgorithmSettingsRequest

• string name = 1

  Name for the Experiment.

• optional ExperimentSpec spec = 2

  Experiment specification.

* Specification of an Experiment. Experiment represents a single optimization run over a feasible space. Each Experiment contains a configuration describing the feasible space, as well as a set of Trials. It is assumed that objective function f(x) does not change in the course of an Experiment.

Used in: Experiment

• optional ExperimentSpec.ParameterSpecs parameter_specs = 1

• optional ObjectiveSpec objective = 2

  Objective specification for the Experiment.

• optional AlgorithmSpec algorithm = 3

  HP or NAS algorithm specification for the Experiment.

• optional EarlyStoppingSpec early_stopping = 4

  Early stopping specification for the Experiment.

• int32 parallel_trial_count = 5

  How many Trials can be processed in parallel.

• int32 max_trial_count = 6

  Max completed Trials to mark Experiment as succeeded.

• optional NasConfig nas_config = 7

  NAS configuration for the Experiment.

* List of ParameterSpec.

Used in: ExperimentSpec

• repeated ParameterSpec parameters = 1

* Feasible space for optimization. Int and Double type use Max/Min. Discrete and Categorical type use List.

Used in: ParameterSpec

• string max = 1

  / Max Value

• string min = 2

  / Minimum Value

• repeated string list = 3

  / List of Values.

• string step = 4

  / Step for double or int parameter

• Distribution distribution = 5

  Distribution of the parameter.

Used in: GetSuggestionsReply

• repeated ParameterAssignment assignments = 1

• string trial_name = 2

  Optional field to override the trial name

• map<string, string> labels = 3

  Optional field to add labels to the generated Trials

* GraphConfig contains a config of DAG

Used in: NasConfig

• int32 num_layers = 1

  / Number of layers

• repeated int32 input_sizes = 2

  / Dimensions of input size

• repeated int32 output_sizes = 3

  / Dimensions of output size

Used in: MetricLog, Observation

• string name = 1

• string value = 2

Used in: ObservationLog

• string time_stamp = 1

  / RFC3339 format

• optional Metric metric = 2

* NasConfig contains a config of NAS job

Used in: ExperimentSpec

• optional GraphConfig graph_config = 1

  / Config of DAG

• optional NasConfig.Operations operations = 2

  / List of Operation

Used in: NasConfig

• repeated Operation operation = 1

* Objective specification.

Used in: ExperimentSpec, TrialSpec

• ObjectiveType type = 1

  Type of optimization.

• double goal = 2

  Goal of optimization, can be empty.

• string objective_metric_name = 3

  Primary metric name for the optimization.

• repeated string additional_metric_names = 4

  List of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

* Direction of optimization. Minimize or Maximize.

Used in: ObjectiveSpec

• UNKNOWN = 0

  / Undefined type and not used.

• MINIMIZE = 1

  / Minimize

• MAXIMIZE = 2

  / Maximize

Used in: TrialStatus

• repeated Metric metrics = 1

Used in: GetObservationLogReply, ReportObservationLogRequest

• repeated MetricLog metric_logs = 1

* Config for operations in DAG

Used in: NasConfig.Operations

• string operation_type = 1

  / Type of operation in DAG

• optional Operation.ParameterSpecs parameter_specs = 2

* List of ParameterSpec

Used in: Operation

• repeated ParameterSpec parameters = 1

Used in: GetSuggestionsReply.ParameterAssignments, TrialSpec.ParameterAssignments

• string name = 1

• string value = 2

* Config for a hyperparameter. Katib will create each Hyper parameter from this config.

Used in: ExperimentSpec.ParameterSpecs, Operation.ParameterSpecs

• string name = 1

  / Name of the parameter.

• ParameterType parameter_type = 2

  / Type of the parameter.

• optional FeasibleSpace feasible_space = 3

  / FeasibleSpace for the parameter.

* Types of value for HyperParameter.

Used in: ParameterSpec

• UNKNOWN_TYPE = 0

  / Undefined type and not used.

• DOUBLE = 1

  / Double float type. Use "Max/Min".

• INT = 2

  / Int type. Use "Max/Min".

• DISCRETE = 3

  / Discrete number type. Use "List" as float.

• CATEGORICAL = 4

  / Categorical type. Use "List" as string.

* Structure for a single Trial.

Used in: GetEarlyStoppingRulesRequest, GetSuggestionsRequest

• string name = 1

  Name for the Trial.

• optional TrialSpec spec = 2

  Trial specification.

• optional TrialStatus status = 3

  Trial status.

* Specification of a Trial. It represents Trial's parameter assignments and objective.

Used in: Trial

• optional ObjectiveSpec objective = 2

  Objective specification for the Trial.

• optional TrialSpec.ParameterAssignments parameter_assignments = 3

  List of assignments generated for the Trial.

• map<string, string> labels = 4

  Map of labels assigned to the Trial

* List of ParameterAssignment

Used in: TrialSpec

• repeated ParameterAssignment assignments = 1

* Current Trial status. It contains Trial's latest condition, start time, completion time, observation.

Used in: Trial

• string start_time = 1

  Trial start time in RFC3339 format

• string completion_time = 2

  Trial completion time in RFC3339 format

• TrialStatus.TrialConditionType condition = 3

  Trial current condition. It is equal to the latest Trial CR condition.

• optional Observation observation = 4

  The best Trial observation in logs.

Trial can be in one of 8 conditions. TODO (andreyvelich): Remove unused conditions.

Used in: TrialStatus

• CREATED = 0

• RUNNING = 1

• SUCCEEDED = 2

• KILLED = 3

• FAILED = 4

• METRICSUNAVAILABLE = 5

• EARLYSTOPPED = 6

• UNKNOWN = 7