These 84 commits are when the Protocol Buffers files have changed:
| Commit: | b703eae | |
|---|---|---|
| Author: | jinhuang | |
| Committer: | tf-model-analysis-team | |
Add a note in eval model spec config for a b/399156775 PiperOrigin-RevId: 752473894
The documentation is generated from this commit.
| Commit: | 387c09f | |
|---|---|---|
| Author: | matrixzhou | |
| Committer: | tf-model-analysis-team | |
Add support for setting top_k and class_id at the same time. First set the non top_k index to -inf and then select the class_id See reference of the details at https://www.tensorflow.org/api_docs/python/tf/keras/metrics/Precision?version=nightly PiperOrigin-RevId: 542918228
| Commit: | 51c4f1e | |
|---|---|---|
| Author: | matrixzhou | |
| Committer: | tf-model-analysis-team | |
Add name for each plots PiperOrigin-RevId: 535075578
| Commit: | d73b11d | |
|---|---|---|
| Author: | jinhuang | |
| Committer: | tf-model-analysis-team | |
Add f1, false_positive_rate, false_omission_rate, and f1 score to the confusion matrices based plots. PiperOrigin-RevId: 527091917
| Commit: | 1c04726 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Internal change PiperOrigin-RevId: 501615454
| Commit: | f990553 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Internal change PiperOrigin-RevId: 501380535
| Commit: | 634092f | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Automated rollback of commit 7467554e2c326723850edea4f8f39e1a4b4f5565 PiperOrigin-RevId: 498445165
| Commit: | 7467554 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Internal change PiperOrigin-RevId: 498401818
| Commit: | fe43471 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Enable TFDV users to provide slicing specifications of StatsOptions via config. PiperOrigin-RevId: 477563489
| Commit: | 3fbc837 | |
|---|---|---|
| Author: | jinhuang | |
| Committer: | tf-model-analysis-team | |
Deprecate BoundedValue and TDistributionValue. PiperOrigin-RevId: 477071320
| Commit: | 9c90d06 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for specifying if a metric is weighted or not. PiperOrigin-RevId: 405974788
| Commit: | 37825bc | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Fix a typo. PiperOrigin-RevId: 401376125
| Commit: | b85d9d4 | |
|---|---|---|
| Author: | embr | |
| Committer: | tf-model-analysis-team | |
Change confidence interval output format to support bounds on structured metrics. PiperOrigin-RevId: 391784529
| Commit: | 380640d | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Updated comments for metric_specs.output_names to indicate that it is required for multi-output models. PiperOrigin-RevId: 390480764
| Commit: | 09fa445 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Update comment to indicate that 'scores' is an TF estimator output key that will automatically be searched for along with probabilities, etc. PiperOrigin-RevId: 388525293
| Commit: | 384a5cf | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Update comment for config field within MetricConfig to specify that it applies to the from_config method not the __init__ method. PiperOrigin-RevId: 386993492
| Commit: | 25bead3 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Adding extra comments for change threshold. PiperOrigin-RevId: 386491780
| Commit: | 628ae55 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
no-op PiperOrigin-RevId: 370539844
| Commit: | e4c6f89 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for output aggregation. PiperOrigin-RevId: 364370999
| Commit: | 86d26d3 | |
|---|---|---|
| Author: | srjindal | |
| Committer: | tf-model-analysis-team | |
Fixes following bugs related to CrossSlicingSpec based evaluation results. * metrics_plots_and_validations_writer was failing while writing cross slice comparison results to metrics file. * Fairness widget view was not compatible with cross slicing key type. PiperOrigin-RevId: 363122588
| Commit: | 5178b4d | |
|---|---|---|
| Author: | srjindal | |
| Committer: | tf-model-analysis-team | |
Don't add metric config (config.MetricsSpec) to baseline model by default PiperOrigin-RevId: 360372998
| Commit: | 67bab46 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Add rubberstamp in the validation result when the run is rubberstamped. PiperOrigin-RevId: 356650256
| Commit: | 47e6297 | |
|---|---|---|
| Author: | huanmingf | |
| Committer: | tf-model-analysis-team | |
Add Batched Slice Key extractor. PiperOrigin-RevId: 356611172
| Commit: | 9408ef2 | |
|---|---|---|
| Author: | zhuo | |
| Committer: | tf-model-analysis-team | |
Added an explicit config for padding the labels and predictions. PiperOrigin-RevId: 352625501
| Commit: | 3b82a47 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 345139336
| Commit: | 5182197 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for feature transformations using TFT and other preprocessing functions. PiperOrigin-RevId: 344270041
| Commit: | b9002af | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for aggregation of feature attributions. PiperOrigin-RevId: 343913060
| Commit: | 40eaaf2 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add missing_thresholds error support to validation results. PiperOrigin-RevId: 342193872
| Commit: | 56998b6 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Cross Slice results writer for the file system. PiperOrigin-RevId: 340492489
| Commit: | a9901a1 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Update to temporarily support calling keras models directly (since they support sparse and ragged tensors). PiperOrigin-RevId: 337547628
| Commit: | 20de494 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Remove support for models using multiple signature_names (re-add if requested). PiperOrigin-RevId: 337410944
| Commit: | 4ca35ed | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for macro/micro aggregation @k and cleanup AggregationOptions. PiperOrigin-RevId: 335778015
| Commit: | dd789c4 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Automated rollback of commit aa0da4c417becf9c77dd70d3d991e06d3e914bbb PiperOrigin-RevId: 335537831
| Commit: | aa0da4c | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 335466553
| Commit: | 1efe8e6 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 334261335
| Commit: | 8eb67f0 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for optional labels and updated comments for prediction_key. PiperOrigin-RevId: 333836059
| Commit: | fb1e826 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 330581668
| Commit: | 495cdab | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 329776811
| Commit: | 3506655 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
TFJS predict extractor. PiperOrigin-RevId: 327457230
| Commit: | 39065ce | |
|---|---|---|
| Author: | srjindal | |
| Committer: | tf-model-analysis-team | |
Adding multi class confusion matrix metrics PiperOrigin-RevId: 326116090
| Commit: | 666aa02 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
metrics_validator implementation for cross slicing. PiperOrigin-RevId: 320271232
| Commit: | cc2ad2f | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added errors for missing slices during metrics validation. PiperOrigin-RevId: 319790867
| Commit: | 355a83c | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
CrossSliceMetricThreshold updates for config. PiperOrigin-RevId: 317525264
| Commit: | e6db7b4 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for slicing to threshold configs. PiperOrigin-RevId: 315766041
| Commit: | 8593af1 | |
|---|---|---|
| Author: | embr | |
| Committer: | tf-model-analysis-team | |
Create confidence interval config message. PiperOrigin-RevId: 313884878
| Commit: | cf74180 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Removing deprecated metric type from TFMA proto. PiperOrigin-RevId: 312353248
| Commit: | 1007b18 | |
|---|---|---|
| Author: | embr | |
| Committer: | tf-model-analysis-team | |
Add Jackknife method for computing confidence intervals. PiperOrigin-RevId: 302085949
| Commit: | 87ef4a8 | |
|---|---|---|
| Author: | srjindal | |
| Committer: | tf-model-analysis-team | |
Renaming k_anonymization_count to min_slice_size PiperOrigin-RevId: 301470654
| Commit: | 23d30ec | |
|---|---|---|
| Author: | pachristopher | |
| Committer: | tf-model-analysis-team | |
Move TDistributionValue fields outside the oneof value. Deprecate the old field. PiperOrigin-RevId: 299394819
| Commit: | bfef8a3 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for copying candidate config to baseline when only one spec given. Also updated to compute default model_types based on model and tags. PiperOrigin-RevId: 298075347
| Commit: | 60c60eb | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Enable custom model types via config. PiperOrigin-RevId: 295444669
| Commit: | 2068794 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for inserting default model_names to metrics_specs. PiperOrigin-RevId: 294563779
| Commit: | 6d60320 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Add writers for validation result. PiperOrigin-RevId: 294352272
| Commit: | ce2916c | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for thresholds on metrics that were saved as part of the model. PiperOrigin-RevId: 294349592
| Commit: | 4f85a67 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Add validation results for TFMA. PiperOrigin-RevId: 293664569
| Commit: | e5ade89 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Add model validation related protos for the upcoming model validation implementations. PiperOrigin-RevId: 292640678
| Commit: | 81bad33 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Wrap binarize and disabled_outputs options into an additional proto message so that we can better add default values when no value is given. PiperOrigin-RevId: 292483034
| Commit: | 0bfe88e | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Add diff_metrics support for TFMA PiperOrigin-RevId: 292457959
| Commit: | 331d060 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Updates to V2 config: * Remove input/output/model locations in favor of passing them directly to the run_eval. * Updates to pass model locations, tags, and EvalSharedModel as either a single value (for single models) or a dict (if multiple models). PiperOrigin-RevId: 289343903
| Commit: | 331a316 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Handle functional large batch failures by executing examples serially. PiperOrigin-RevId: 283838094
| Commit: | ff6393b | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for micro_average, macro_average, and weighted_macro_average metrics. PiperOrigin-RevId: 280132811
| Commit: | 96ab7a3 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Automated rollback of commit adb04766b3830bbeb917d3dd7798dc8a713d508e PiperOrigin-RevId: 278888350
| Commit: | adb0476 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Updated comments regarding discouraged use of desired_batch_size. Removed custom config. PiperOrigin-RevId: 278745075
| Commit: | c6fc5b1 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add multi-class/multi-label plots based on new V2 infrastructure. PiperOrigin-RevId: 275400360
| Commit: | dac2771 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
noop PiperOrigin-RevId: 274289491
| Commit: | d2e0739 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add support for serializing with new MetricKey and PlotKey types. PiperOrigin-RevId: 273606812
| Commit: | bc1b6bf | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Restructured config to include input_data_specs, model_specs, output_data_specs, and metrics_and_plots_spec. Update serialization to use JSON instead of pickling. Also update SliceKeyType to use text type for slicing (already enforced by init) and update slicer to match on string froms of int/float. PiperOrigin-RevId: 271417366
| Commit: | c392763 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Change MultiClassKey to SubKey so that it can be generally usable for all metric sub-types. PiperOrigin-RevId: 268371969
| Commit: | c6ec120 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Update metric keys / values to be encapsulated in a message within in a single repeated field. PiperOrigin-RevId: 268299587
| Commit: | 93e11b1 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added support for ModelKey type. PiperOrigin-RevId: 267517156
| Commit: | 2aea434 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Add plot definition for multi-class confusion matrix. PiperOrigin-RevId: 267499586
| Commit: | 3967a25 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Updated to use structured multi-class key in metrics_for_slice proto. PiperOrigin-RevId: 266463055
| Commit: | e8eaccd | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
Added structured metric and plot keys to metrics_for_slice proto. PiperOrigin-RevId: 261574410
| Commit: | 545106a | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
To not compute metrics for the slices with example count less than 'k'. (where value of 'k' is configurable and set to 1 by default) PiperOrigin-RevId: 249854302
| Commit: | f407527 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Automated rollback of commit b12ba3492ba6587559472a7cb345aa955fea720f PiperOrigin-RevId: 239853115
| Commit: | 5c31591 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Automated rollback of commit b12ba3492ba6587559472a7cb345aa955fea720f PiperOrigin-RevId: 239247971
| Commit: | b12ba34 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Internal Refactoring. PiperOrigin-RevId: 239236099
| Commit: | d64a570 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Create TDistribution value. This gives more flexibility of calculating confidence intervals. PiperOrigin-RevId: 238489608
| Commit: | 03d0515 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | tf-model-analysis-team | |
Adding a new proto field in order to enable future support for multiple plots in a single evaluation. PiperOrigin-RevId: 234190697
| Commit: | 27f5c50 | |
|---|---|---|
| Author: | mdreves | |
| Committer: | tf-model-analysis-team | |
no-op PiperOrigin-RevId: 232056915
| Commit: | d321b37 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | chuanyu | |
Project import generated by Copybara. PiperOrigin-RevId: 231295457
| Commit: | 73760b2 | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | mdreves | |
Project import generated by Copybara. PiperOrigin-RevId: 222356971
| Commit: | 6682cbf | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | xinzha623 | |
Project import generated by Copybara. PiperOrigin-RevId: 207606532
| Commit: | 1fd9e1e | |
|---|---|---|
| Author: | tf-model-analysis-team | |
| Committer: | pfyang | |
Project import generated by Copybara. PiperOrigin-RevId: 198919171