package benchpress

Get desktop application:
View/edit binary Protocol Buffers messages

Activation function specs.

Used in: Layer

• optional string fn = 1

  Name of activation function.

Used in: ActiveSampler

• oneof model

  • QueryByCommittee query_by_committee = 1

  • ExpectedErrorReduction expected_error_reduction = 2

• optional string downstream_task = 3

• optional string training_corpus = 4

• optional string test_db = 5

• optional int32 num_train_steps = 6

• optional int32 random_seed = 7

• optional int32 top_k = 8

Active Sampling specifications message.

Used in: CorpusConfig

• optional int32 active_search_depth = 1

• optional int32 active_search_width = 2

• optional float active_dropout_prob = 3

• optional int32 batch_size_per_feed = 4

• optional string feature_space = 5

• oneof target_features

  • string target = 6

  • ActiveLearner active_learner = 7

The field name suffix '_micros' shows that the value contained in the field is converted at runtime to a floating point number by dividing it by 1e6. The reason for _micros fields is so that we can realiably encode and compare protos without having to worry about floating point rounding and comparisons.

Used in: TrainingOptions

• optional int32 initial_learning_rate_micros = 1

  The initial learning rate. Must be >= 0. A recommended starting value is 2000 (i.e. real value 0.002).

• optional int32 learning_rate_decay_per_epoch_micros = 2

  The ratio by which the learning rate decays per epoch of training. Must be >= 0. A recommended starting value is 5000 (i.e. real value 0.05).

• optional int32 beta_1_micros = 3

  Must be in real value range 0 < beta_1 < 1. A recommended starting value is 900000 (i.e. real value 0.9).

• optional int32 beta_2_micros = 4

  Must be in real value range 0 < beta_2 < 1. A recommended starting value is 999000 (i.e. real value 0.999).

• optional int32 normalized_gradient_clip_micros = 5

  The normalized gradient clip value. A recommended starting value is 5000000 (ie. real value 5.0).

21

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• repeated DatabasesGroups db_group = 3

• optional PlotterConfiguration plot_config = 4

4 Analyze the size of target bechmraks in number of instructions and token length.

Used in: evaluators

• repeated string target = 1

• optional PlotterConfiguration plot_config = 2

10 Map on 2-Dimensional space the number of memory vs computational instructions from Grewe's features.

Used in: evaluators

• optional string target = 1

• repeated DatabasesGroups db_group = 2

• optional PlotterConfiguration plot_config = 3

The specification of a training corpus.

Used in: CorpusMeta, Model, SampleCorpus

• oneof contentfiles

  The input contentfiles to the corpus. Shell variables are expanded. E.g. $HOME -> /home/<user>.

  • string content_id = 1

    The ID of an already-cached corpus.

  • string local_directory = 2

    The path to a directory on the local system containing content files.

  • string local_tar_archive = 3

    The path to a tarball archive file containing content files.

  • string bq_database = 4

    The path to a bigQuery database containing files.

  • string pre_encoded_corpus_url = 5

    The path to a database of encoded files.

• optional tokenizer tokenizer = 6

• repeated string preprocessor = 7

  A list of preprocessor passes to run on each contentfile in the corpus prior to training, in the order in which they are run.

• optional string contentfile_separator = 8

Used in: SampleCorpus

• oneof sampling_type

  Encoding and data masking configuration for sample corpus. sampling type can be 'normal', 'online', 'active' normal: corpus is pre-masked, then fed for predictions. online: A datapoint is requested from original corpus and masked on the fly. active: Same as online. Active learning is applied between sample and target features

  • bool normal = 1

  • bool online = 2

  • ActiveSampler active = 3

• optional int32 max_predictions_per_seq = 4

• optional float masked_lm_prob = 5

• oneof mask_technique

  • Mask mask = 6

  • Hole hole = 7

  • MaskSeq mask_seq = 8

The schema for a corpus metafile. An instance of this proto is created with name META.pbtxt in the cache directory of each corpus.

• optional Corpus config = 1

• optional int32 preprocess_time_ms = 2

Used in: TrainingOptions

• optional string datapoint_type = 1

  Represent single training instance as whole padded kernel, or arbitrary statement sequences. Valid options are "kernel" or "statement".

• optional string datapoint_time = 2

  When datapoints should be pre-processed for training/validation/sampling. 'pre': Raw corpus is masked and stored. Then used. 'online': Raw corpus is stored. During training/validation/sampling, datapoints are pre-processed on the fly and provided to the model.

• optional bool use_start_end = 3

  Use [START] and [END] meta tokens at the beginning and end of each sequence.

• optional bool truncate_large_kernels = 4

  If datapoints are 'kernel', kernels > seq len are discarded. If true, they are kept and truncated instead.

• optional int32 steps_per_epoch = 5

  Number of steps that constitute an epoch. Checkpoints and samples are taken once every epoch.

• optional int32 validation_split = 6

  Select a value between 0 and 100. This percentage will be used to split dataset into training and validation. Validation set will not be seen during training.

• oneof mask_technique

  single token masks are BERT's default. Alternatively, use a hole token to represent an arbitrary amount of hidden tokens.

  • Mask mask = 7

  • Hole hole = 8

  • MaskSeq mask_seq = 9

• repeated ValidationSet validation_set = 10

Define a group of Databases that will be considered jointly.

Used in: AnalyzeBeamSearch, CompMemGrewe, FeaturesDistribution, GenDistanceDistribution, GreweCSV, GreweTopKCSV, HumanLikeness, KAverageScore, LLVMInstCountDistribution, LogFile, MinScore, MutecVsBenchPress, PCASamplesFeatures, SRCIROR_IRVsBenchPress, SRCIROR_srcVsBenchPress, TokenSizeDistribution, TopKCLDrive

• optional string group_name = 1

• optional string db_type = 2

• repeated string database = 3

• optional int32 size_limit = 4

  Length of datapoint threshold.

• optional string tokenizer = 5

Dropout Specs.

Used in: Layer

• optional float p = 1

Input embeddings are usually very useful.

Used in: Layer

• optional int32 num_embeddings = 1

• optional int32 embedding_dim = 2

• optional int32 padding_idx = 3

• optional int64 id = 1

• optional string text = 3

• optional string contentfile_separator = 4

• optional bytes pickled_tokenizer = 5

Specification of app's evaluation pipeline.

• optional string workspace = 1

  Define workspace path

• optional string tokenizer = 2

  Define tokenizer path

• repeated evaluators evaluator = 3

  Define a list of different evaluators to run.

ExpectedErrorReduction is the predictive head and a few parameters.

Used in: ActiveLearner

• optional MLP head = 1

20

Used in: evaluators

• optional string grewe_baseline = 1

• repeated GreweCSVGroup csv = 2

• optional PlotterConfiguration plot_config = 3

19

Used in: evaluators

• optional string grewe_baseline = 1

• repeated GreweCSVGroup csv = 2

• optional PlotterConfiguration plot_config = 3

Used in: RLAgent

• optional int32 feature_max_value_token = 1

  Max value that can be tokenized. Anything above that is UNK.

• optional int32 feature_singular_token_thr = 2

  Threshold of values being singularly tokenized (not represented as range).

• optional int32 feature_token_range = 3

  Range width of tokenized values after singularity threshold

• optional int32 feature_sequence_length = 4

  Sequence length of tokenized feature vectors.

8 Calculate and plot the average score of the top-K best samples per target from each DB group.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• optional int32 top_k = 3

• repeated DatabasesGroups db_group = 4

• optional PlotterConfiguration plot_config = 5

22

Used in: evaluators

• optional string target = 1

• optional int32 generation_id = 2

• optional string feature_space = 3

• repeated DatabasesGroups db_group = 4

• optional PlotterConfiguration plot_config = 5

15 Generate CLSmith samples.

Used in: evaluators

• optional string clsmith_db = 1

17 Convert databases per db group to CSV for Grewe's predictive model.

Used in: evaluators

• optional string cldrive_cache = 1

• repeated DatabasesGroups db_group = 2

Define a message representing a CSV for Grewe's predictive model.

Used in: FeatureSpaceCovGroup, FeatureSpaceCovLabel, TrainGrewe

• optional string name = 1

• optional string path = 2

16 Calculate top samples to target per db group and store to csv for Grewe's predictive model.

Used in: evaluators

• optional string target = 1

• optional string cldrive_cache = 2

• optional int32 top_k = 3

• repeated DatabasesGroups db_group = 4

Used in: CorpusConfig, DataGenerator, ValidationSet

• oneof length

  In case sequences are hole-d, choose upper bound range of possible hole length (will be [0, hole_length]).

  • int32 absolute_length = 1

  • float relative_length = 2

• oneof length_distribution

  Select distribution from which each hole length will be sampled.

  • bool uniform_distribution = 3

  • Normal normal_distribution = 4

• optional bool stage_training = 5

  Learning holes is a difficult task. Stage training to start from many single-token holes (equivalent to masks) and slightly move to fewer and increasingly lengthier holes.

9 Per target benchmark, print the best candidates among each database group.

Used in: evaluators

• optional string target = 1

• optional int32 top_k = 3

• repeated DatabasesGroups db_group = 4

• optional PlotterConfiguration plot_config = 5

23

Used in: evaluators

• optional string human_or_ai_db = 1

• optional PlotterConfiguration plot_config = 2

A BenchPress instance encapsulates all of the settings for training and sampling a language model.

Used in: Instances

• optional string working_dir = 1

  The path to the benchpress working directory. This directory stores persistent caches files, including the corpus, model checkpoints, and sampler outputs. If not provided, the default value of $HOME/.cache/benchpress is used.

• optional github.GithubMiner github_miner = 2

  Optionally, a github miner to scrape files with a requested specification.

• oneof model_specification

  The language model specification. Either the full description of a model, or the path to a pretrained_model, as created by --export_tf_model option of benchpress.

  • Model language_model = 3

  • RLModel rl_model = 4

  • string pretrained_model = 5

• optional Sampler sampler = 6

  The sampler specification.

A collection of instances.

• repeated Instance instance = 1

• optional string file_path = 1

  The path of the source file.

• optional string src = 2

  The string source code.

• optional JavaRewriterJob.Status status = 3

• optional string status_msg = 4

  A string description of the status. Only set if status != OK.

Used in: JavaRewriterJob

• OK = 0

• FAIL = 1

2 Calculate and plot the average score of the top-K best samples per target from each DB group.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• optional int32 top_k = 3

• repeated DatabasesGroups db_group = 4

• optional PlotterConfiguration plot_config = 5

KMeans type architecture. See scitkit-learn specs for parameters below.

Used in: QueryByCommittee

• optional int32 n_clusters = 1

• optional string init = 2

• optional int32 n_init = 3

• optional int32 max_iter = 4

• optional float tol = 5

• optional string algorithm = 6

kNN architeture.

Used in: QueryByCommittee

• optional int32 n_neighbors = 1

• optional string weights = 2

• optional string algorithm = 3

• optional int32 leaf_size = 4

• optional float p = 5

6 Plot LLVM-IR Instruction count distribution of given database groups.

Used in: evaluators

• repeated DatabasesGroups db_group = 1

• optional PlotterConfiguration plot_config = 2

Layer wrapper to help with layer's ordering consistency.

Used in: MLP

• oneof layers

  • Embedding embedding = 1

  • Linear linear = 2

  • Dropout dropout = 3

  • LayerNorm layer_norm = 4

  • ACT_fn act_fn = 5

LayerNorm specs.

Used in: Layer

• optional int32 normalized_shape = 1

• optional float eps = 2

• repeated LexerJob input = 1

• repeated string candidate_token = 2

  The candidate vocabulary. This is the list of multicharacter tokens, including all those already in the vocabulary map below.

• map<string, int32> vocabulary = 3

  The derived vocabulary and its numerical mapping.

Used in: LexerBatchJob

• optional string string = 1

  The string to tokenize.

• repeated int32 token = 2

  The tokenized string.

Linear specs.

Used in: Layer

• optional int32 in_features = 1

• optional int32 out_features = 2

1 Dump log file for all databases provided.

Used in: evaluators

• optional string target = 1

• repeated DatabasesGroups db_group = 2

MLP-type architecture.

Used in: ExpectedErrorReduction, QueryByCommittee

• repeated Layer layer = 1

• optional float initial_learning_rate_micros = 2

• optional int32 batch_size = 3

• optional int32 num_warmup_steps = 5

Used in: CorpusConfig, DataGenerator, ValidationSet

• optional bool random_placed_mask = 1

  When selecting an index in the input tensor, the original BERT model gives 80% chance to replace it with a MASK, a 10% chance to replace it with another random token and another 10% to leave it be after all. Set True to enable this behavior. Otherwise, when selecting an index in the input, this will be replaced by a MASK.

Used in: CorpusConfig, DataGenerator

• oneof length

  Special category of a hole represented as a set of masks.

  • int32 absolute_length = 1

  • float relative_length = 2

• oneof length_distribution

  Select distribution from which each hole length will be sampled.

  • bool uniform_distribution = 3

  • Normal normal_distribution = 4

• optional bool stage_training = 5

  Learning holes is a difficult task. Stage training to start from many single-token holes (equivalent to masks) and slightly move to fewer and increasingly lengthier holes.

Used in: SampleTerminationCriterion

• optional int32 maximum_tokens_in_sample = 1

  The maximum length of a sample, as a number of tokens. The length of the Sampler.start_text counts towards this total.

3 Calculate and plot the minimum score from each DB group for every target benchmark.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• repeated DatabasesGroups db_group = 3

• optional PlotterConfiguration plot_config = 4

The specification of a benchpress model.

Used in: Instance, ModelMeta, RLModel

• optional PreTrainCorpus pre_train_corpus = 1

• optional Corpus corpus = 2

• optional NetworkArchitecture architecture = 3

• optional TrainingOptions training = 4

Records telemetry data about a single epoch of model training.

• optional string timestamp_unix_epoch_ms = 1

  The number of milliseconds since the epoch that epoch training completed.

• optional int32 epoch_num = 2

  The epoch which has just finished training, starting at one.

• optional int64 epoch_wall_time_ms = 3

  The wall time that it took to train the epoch.

• optional double loss = 4

  The model's loss.

The schema for a model metafile. An instance of this proto is created with name META.pbtxt in the cache directory of each model.

• optional Model config = 1

12 Calculate and plot the average score of the top-K best samples per target from each DB group.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• optional int32 top_k = 3

• optional int32 beam_width = 4

• optional string mutec_cache = 5

• optional DatabasesGroups seed = 6

• optional DatabasesGroups benchpress = 7

• optional PlotterConfiguration plot_config = 8

• optional DatabasesGroups db_group = 9

  Dummy selector, not used.

The specification of a benchpress language model.

Used in: Model

• optional NetworkArchitecture.Backend backend = 1

• optional int32 embedding_size = 2

  The size of the input embedding layer. Only required if backend == KERAS_SEQ. Must be > 0.

• optional NetworkArchitecture.NeuronType neuron_type = 3

  The type of neuron. Valid options are: {"lstm","rnn","gru"}.

• optional int32 neurons_per_layer = 4

  The number of neurons in each layer of the network.

• optional int32 num_layers = 5

  The total number of layers in the network.

• optional int32 post_layer_dropout_micros = 6

  If greater than zero, this adds a dropout layer after each layer of neurons with probability post_alyer_drop_micros / 1000000. E.g. a value of 2000 would insert a dropout with probability of 0.2.

• optional int32 hidden_size = 7

  Size of the encoder layers and the pooler layer.

• optional int32 num_hidden_layers = 8

  The messages below correspong to BERT parameters. Number of hidden layers in the Transformer encoder.

• optional int32 num_attention_heads = 9

  Number of attention heads for each attention layer in the Transformer encoder.

• optional int32 intermediate_size = 10

  The size of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder.

• optional string hidden_act = 11

  The non-linear activation function (function or string) in the encoder and pooler.

• optional float hidden_dropout_prob = 12

  The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.

• optional float attention_probs_dropout_prob = 13

  The dropout ratio for the attention probabilities.

• optional int32 max_position_embeddings = 14

  The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048).

• optional int32 type_vocab_size = 15

  The vocabulary size of the `token_type_ids` passed into `BertModel`.

• optional float initializer_range = 16

  The stdev of the truncated_normal_initializer for initializing all weight matrices.

• optional float layer_norm_eps = 17

  The epsilon used by the layer normalization layers.

• optional bool feature_encoder = 18

  Usage flag for feature vector encoding during training.

• optional int32 feature_sequence_length = 19

  Concatenated raw features vector input length.

• optional int32 feature_embedding_size = 20

  Embedding dimension of feature transformer input.

• optional float feature_dropout_prob = 21

  Dropout probs of feature transformer encoder.

• optional int32 feature_singular_token_thr = 22

  Threshold up to which every numerical value maps to a single token.

• optional int32 feature_max_value_token = 23

  Maximum value allowed for feature numerical encoding.

• optional int32 feature_token_range = 24

  Length of value range mapping to single token for feature tokenizer.

• optional int32 feature_num_attention_heads = 25

  Num of attention heads for feature transformer

• optional int32 feature_transformer_feedforward = 26

  Size of feature encoder's internal feedforward.

• optional float feature_layer_norm_eps = 27

  Layer norm epsilon for feature encoder.

• optional int32 feature_num_hidden_layers = 28

  Num of hidden layers for feature encoder.

Used in: NetworkArchitecture

• TENSORFLOW_SEQ = 0

• TENSORFLOW_BERT = 1

• TORCH_BERT = 2

• KERAS_SEQ = 3

• INCODER_1B = 4

• INCODER_6B = 5

Used in: NetworkArchitecture

• LSTM = 0

• RNN = 1

• GRU = 2

Used in: Hole, MaskSeq

• optional float mean = 1

• optional float variance = 2

7 Map the features of database groups on a PCA-2 reduced space.

Used in: evaluators

• repeated DatabasesGroups db_group = 1

• optional PlotterConfiguration plot_config = 2

• optional string feature_space = 3

Plot configurations

Used in: AnalyzeBeamSearch, AnalyzeTarget, CompMemGrewe, FeatureSpaceCovGroup, FeatureSpaceCovLabel, FeaturesDistribution, GenDistanceDistribution, HumanLikeness, HumanLikenessAnalysis, KAverageScore, LLVMInstCountDistribution, MinScore, MutecVsBenchPress, PCASamplesFeatures, SRCIROR_IRVsBenchPress, SRCIROR_srcVsBenchPress, TokenSizeDistribution, TopKCLDrive, TrainGrewe

• optional string title = 1

• optional string x_name = 2

• optional string y_name = 3

• optional int32 titlefont = 4

• optional int32 axisfont = 5

• optional int32 tickfont = 6

• optional bool showline_x = 7

• optional bool showline_y = 8

• optional string linecolor = 9

• optional string gridcolor_x = 10

• optional string gridcolor_y = 11

• optional bool mirror = 12

• optional bool showgrid_x = 13

• optional bool showgrid_y = 14

• optional int32 linewidth = 15

• optional int32 gridwidth = 16

• optional string margin = 17

• optional float legend_x = 18

• optional float legend_y = 19

• optional string traceorder = 20

• optional int32 legendfont = 21

• optional string bg_color = 22

• optional int32 height = 23

• optional int32 width = 24

• optional int32 x_tickangle = 25

• optional string violinmode = 26

• optional bool showticklabels_x = 27

• optional bool showticklabels_y = 28

The specification of a pre-training corpus.

Used in: Model

• oneof contentfiles

  The input contentfiles to the corpus. Shell variables are expanded. E.g. $HOME -> /home/<user>.

  • string content_id = 1

    The ID of an already-cached corpus.

  • string local_directory = 2

    The path to a directory on the local system containing content files.

  • string local_tar_archive = 3

    The path to a tarball archive file containing content files.

  • string bq_database = 4

    The path to a bigQuery database containing files.

  • string pre_encoded_corpus_url = 5

    The path to a database of encoded files.

• optional tokenizer tokenizer = 6

• repeated string preprocessor = 7

  A list of preprocessor passes to run on each contentfile in the corpus prior to training, in the order in which they are run.

• optional string contentfile_separator = 8

• optional string contentfile_root = 1

• optional string relpath = 2

• repeated string preprocessors = 3

A preprocessor worker input.

• optional string contentfile_id = 1

• optional string src = 2

• repeated string preprocessors = 3

The output of a preprocessor worker.

Used in: PreprocessorWorkerJobOutcomes

• optional string contentfile_id = 1

• optional string contents = 2

• optional PreprocessorWorkerJobOutcome.Status status = 3

Used in: PreprocessorWorkerJobOutcome

• OK = 0

• FAIL = 1

• DOES_NOT_COMPILE = 2

  More verbose failure causes:

• TOO_FEW_CHARACTERS = 3

• TOO_FEW_LINES = 4

• REWRITER_FAIL = 5

• repeated PreprocessorWorkerJobOutcome outcome = 1

• repeated int64 preprocess_time_ms = 2

  The elapsed time of each preprocessing job.

Committee is a list of models.

Used in: ActiveLearner

• repeated MLP mlp = 1

• repeated KMeans k_means = 2

• repeated KNN knn = 3

  repeated <ModelType> model_type = x;

Used in: RLModel

• optional FeatureTokenizer feature_tokenizer = 1

• optional int32 num_epochs = 2

• optional int32 num_episodes = 3

• optional int32 steps_per_episode = 4

• optional int32 num_updates = 5

• optional float gamma = 6

• optional float lam = 7

• optional float epsilon = 8

• optional float learning_rate_micros = 9

• optional float value_loss_coefficient = 10

• optional float entropy_coefficient = 11

• optional int32 batch_size = 12

• optional int32 action_temperature_micros = 13

• optional int32 token_temperature_micros = 14

Used in: Instance, RLModelMeta

• oneof target_features

  • bool train_set = 1

  • bool random = 2

• optional RLAgent agent = 3

• optional Model language_model = 4

The schema for a model metafile. An instance of this proto is created with name META.pbtxt in the cache directory of each model.

• optional RLModel config = 1

Used in: TrainingOptions

• optional int32 initial_learning_rate_micros = 1

  The initial learning rate. Must be >= 0. A recommended starting value is 1000 (i.e. real value 0.001).

• optional int32 learning_rate_decay_per_epoch_micros = 2

  The ratio by which the learning rate decays per epoch of training. Must be >= 0. A recommended starting value is 0.

14 Calculate and plot the average score of the top-K best samples per target from each DB group.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• optional int32 top_k = 3

• optional int32 beam_width = 4

• optional string srciror_ir_cache = 5

• optional DatabasesGroups seed = 6

• optional DatabasesGroups benchpress = 7

• optional PlotterConfiguration plot_config = 8

• optional DatabasesGroups db_group = 9

  Dummy selector, not used.

13 Calculate and plot the average score of the top-K best samples per target from each DB group.

Used in: evaluators

• optional string target = 1

• optional string feature_space = 2

• optional int32 top_k = 3

• optional int32 beam_width = 4

• optional string srciror_src_cache = 5

• optional DatabasesGroups seed = 6

• optional DatabasesGroups benchpress = 7

• optional PlotterConfiguration plot_config = 8

• optional DatabasesGroups db_group = 9

  Dummy selector, not used.

A generated sample. Instances of this proto are returned by a Model's Sample() method.

• optional string original_input = 1

• optional string sample_feed = 2

• optional string text = 3

• optional string encoded_text = 4

• optional int32 sample_time_ms = 5

• optional string sample_indices = 6

• optional string encoded_sample_indices = 7

• optional string feature_vector = 8

• optional int32 wall_time_ms = 9

  Sampling may be batches, so that the sum of sample_time_ms over a range of samples may be much higher than the actual amount of time required to sample the set. This field contains the number of milliseconds between the last sample completing and this sample completing, so that by summing wall_time_ms, it is possible to get an accurate idea of the actual time taken to produce a set of samples.

• optional int64 sample_start_epoch_ms_utc = 10

• optional int32 num_tokens = 11

• optional bool compile_status = 12

• optional bool categorical_sampling = 13

• optional int32 train_step = 14

• optional string date_added = 15

Specification of a new sample corpus to get feeds from.

Used in: Sampler

• optional CorpusConfig corpus_config = 1

• oneof input_corpus

  • Corpus corpus = 2

  • string start_text = 3

Criteria used for determining when to stop sampling.

Used in: Sampler

• oneof criterion

  • MaxTokenLength maxlen = 1

  • SymmetricalTokenDepth symtok = 2

The specification of a benchpress sampler.

Used in: Instance, SamplerMeta

• oneof sample_feed

  The initial text to the seed the language model with. Each sample will begin with this text.

  • string start_text = 1

    Simple string

  • bool train_set = 2

    Sample from training set

  • bool validation_set = 3

    Sample from validation set

  • ValidationSet sample_set = 4

    Create set with new specs from original corpus

  • SampleCorpus sample_corpus = 5

    Specify a whole new corpus to encode and sample with new specs.

  • bool live_sampling = 6

    Sample live by getting input() from user.

• optional int32 batch_size = 7

  The sampling batch size. TODO(cec): Always sample with max batch size.

• optional int32 sequence_length = 8

  The length of sampling sequences.

• optional int32 temperature_micros = 9

  The sampling temperature. Must be >= 0. A recommended starting value is 1000000 (i.e. 1.0 in real values).

• repeated SampleTerminationCriterion termination_criteria = 10

  The criteria that determine when to terminate a sample, in the order in which they will be executed. Duplicates are allowed, for example if you would like to have symmetrical token depth counters for two pairs of tokens.

• optional string description = 11

  A message describing the experiment of this sampler.

The schema for a sampler metafile. An instance of this proto is created with name META.pbtxt in the cache directory of each model that samples it.

• optional Sampler config = 1

Used in: SampleTerminationCriterion

• optional string depth_increase_token = 1

• optional string depth_decrease_token = 2

5 Plot token size relative distribution of given database groups.

Used in: evaluators

• repeated DatabasesGroups db_group = 1

• optional PlotterConfiguration plot_config = 2

11 Collect CPU vs GPU labels for

Used in: evaluators

• optional string target = 1

• optional string cldrive_cache = 2

• optional string feature_space = 3

• optional int32 top_k = 4

• repeated DatabasesGroups db_group = 5

• optional PlotterConfiguration plot_config = 6

18

Used in: evaluators

• optional string grewe_baseline = 1

• repeated GreweCSVGroup csv = 2

• optional PlotterConfiguration plot_config = 3

Options used for training a benchpress language model.

Used in: Model

• optional int32 num_epochs = 1

  The number of epochs to train the network for.

• optional int32 num_train_steps = 2

  The length of training sequences.

• optional int32 num_pretrain_steps = 3

  BERT only. Number of training steps.

• optional int32 num_warmup_steps = 4

  BERT only. Number of pre-training steps.

• optional int32 num_prewarmup_steps = 5

• optional int32 sequence_length = 6

  BERT only. Number of warmup steps.

• optional int32 max_predictions_per_seq = 7

  Maximum number of masked LM predictions per sequence.

• optional int32 dupe_factor = 8

  Number of times to duplicate the input data (with different masks).

• optional float masked_lm_prob = 9

  Masked LM probability.

• optional int32 random_seed = 10

  Random seed for data generation.

• optional bool shuffle_corpus_contentfiles_between_epochs = 11

  If true, shuffle the order of contentfiles in the corpus between each training epoch.

• optional int32 batch_size = 12

  The training batch size. Note that this is only a *requested* batch size, there may be cases where the runtime decides to modify this value. For example, when the corpus size is smaller than the batch size. Any changes to this value at runtime will be logged as errors.

• optional DataGenerator data_generator = 13

  In case of BERT model, a specific data generator is needed.

• oneof optimizer

  The optimizer configuration.

  • AdamOptimizer adam_optimizer = 14

  • RmsPropOptimizer rmsprop_optimizer = 15

Used in: DataGenerator, Sampler

• optional int32 max_predictions_per_seq = 1

• oneof option

  • Mask mask = 3

    Create new sets out of the original corpus, to use them for validation or sampling. Useful to test the model against different specs.

  • Hole hole = 4

Instance of a single evaluation pass.

Used in: Evaluation

• oneof evaluator

  Define all different evauators supported.

  • LogFile log_file = 1

  • KAverageScore k_average_score = 2

  • MinScore min_score = 3

  • AnalyzeTarget analyze_target = 4

  • TokenSizeDistribution token_size_distribution = 5

  • LLVMInstCountDistribution llvm_instcount_distribution = 6

  • PCASamplesFeatures pca_samples_features = 7

  • FeaturesDistribution features_distribution = 8

  • HumanLikeness human_likeness = 9

  • HumanLikenessAnalysis human_likeness_analysis = 10

  • CompMemGrewe comp_mem_grewe = 11

  • TopKCLDrive topk_cldrive = 12

  • MutecVsBenchPress mutec_vs_benchpress = 13

  • SRCIROR_srcVsBenchPress srciror_src_vs_benchpress = 14

  • SRCIROR_IRVsBenchPress srciror_ir_vs_benchpress = 15

  • GenerateCLSmith generate_clsmith = 16

  • GreweTopKCSV grewe_top_k_csv = 17

  • GreweCSV grewe_csv = 18

  • TrainGrewe train_grewe = 19

  • FeatureSpaceCovLabel feature_space_cov_label = 20

  • FeatureSpaceCovGroup feature_space_cov_group = 21

  • AnalyzeBeamSearch analyze_beam_search = 22

  • GenDistanceDistribution gen_distance_distribution = 23

The tokenizer to use to encode the corpus.

Used in: Corpus, PreTrainCorpus

• required string token_type = 1

• optional string token_list = 2

• optional bool mask_tokens = 3

• optional bool wordpiece_tokenization = 4