Proto commits in tensorflow/metadata

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2025-04-14

Support squared pearson correlation metric PiperOrigin-RevId: 747427219

The documentation is generated from this commit.

2025-04-01

Proto changes for supporting content chunk semantic type in AI Flow PiperOrigin-RevId: 742870644

2025-02-20

Add Video as a domain to TFMD schema PiperOrigin-RevId: 729170148

2024-11-24

Support Audio as a domain in Schema PiperOrigin-RevId: 699716595

2024-09-03

Automated g4 rollback of changelist 669103619. *** Reason for rollback *** Broke TAP tests the LegoML project *** Original change description *** Mark message types as requiring the go/jspb object format methods. This CL marks types that use the JSPB object format accessors (see go/jspb-api-gotchas#objects) so that we do not remove them in the future. This is a backwards compatibility option and will represent no immediate change. The implementations of these methods are large and expensive for the JSCompiler to process; so we intend to limit their generation to improve compilation performance. See go/lsc-constrain-jspb-object-format-us... *** PiperOrigin-RevId: 670640538

2024-08-29

Internal only change. PiperOrigin-RevId: 669103619

2024-08-08

Updates schema proto documentation to clarify that top-level float/int domains are not supported in TFDV. PiperOrigin-RevId: 660899053

2024-05-07

For nested features with N nested levels (N > 1), the statistics counting the number of values in `CommonStatistics` and `WeightedCommonStatistics` will rely on the innermost level. PiperOrigin-RevId: 631265288

2024-04-04

Remove unused field NaturalLanguageDomain.location_constraint_regex. It was documented as "please do not use" and never implemented. PiperOrigin-RevId: 621881996

Comment fix for copy&paste glitch PiperOrigin-RevId: 621830437

2024-02-29

* Add `embedding_type` to `FloatDomain` to specify the semantic type of the embedding, which is useful for use cases where downstream tasks depend on knowing where the embedding came from. PiperOrigin-RevId: 611538841

[Tunelab Integration] Update PSW to support text_generation task type PiperOrigin-RevId: 611265010

2024-01-08

Clarify comment on tensorflow.metadata.v0.FixedShape: it matches tensorflow.TensorShapeProto only for fully defined shapes. PiperOrigin-RevId: 596643499

2023-12-13

Create prototype LoRA trainer in AutoTFX service This change implements the flow described in go/autotfx-pets-lora-backend PiperOrigin-RevId: 590643060

2023-10-20

Enable schema configuration to set default feature value for failed slicing sql when using SqlDeriver. PiperOrigin-RevId: 575254073

2023-09-30

[MTL] All primary final model selection with task weight. PiperOrigin-RevId: 569627723

2023-09-11

#MulticlassDistillation Update the distillation spec to include config for multiclass distillation. PiperOrigin-RevId: 564478592

2023-09-06

Remove optional from problem statement proto. PiperOrigin-RevId: 563160918

2023-08-23

Internal only PiperOrigin-RevId: 559467373

2023-08-08

* Add `embedding_dim` to `FloatDomain` to specify the embedding dimension, which is useful for use cases such as restoring shapes for flattened sequence of embeddings. * Add `sequence_truncation_limit` to `SequenceMetadata` to specify the maximum sequence length that should be processed. PiperOrigin-RevId: 554643195

2023-07-27

Add BOOL_TYPE_INVALID_CONFIG anomaly types. PiperOrigin-RevId: 551650214

2023-06-09

Internal only PiperOrigin-RevId: 538947625

2023-04-26

Add `joint_group` to `SequenceMetadata` to specify which group this sequence feature belongs to so that they can be modeled jointly. PiperOrigin-RevId: 527186422

2023-04-20

Internal only PiperOrigin-RevId: 525823318

2023-04-11

Add deriver classes and tests for image feature along with associated schema configuration support. PiperOrigin-RevId: 521859426

2023-04-04

Add deriver classes and tests for image feature along with associated schema configuration support. PiperOrigin-RevId: 521859426

2023-02-28

Adds currently unused HistogramSelection field to Schema PiperOrigin-RevId: 513062587

2023-02-07

Internal only PiperOrigin-RevId: 507908632

MVP of BooleanFlipRate metric for use as a secondary tuning objective PiperOrigin-RevId: 507862768

2023-01-10

Add documentation for sql-based derived features. https://g3doc.corp.google.com/third_party/py/tensorflow_data_validation/google/g3doc/sql_derived_features.md?cl=495457531 PiperOrigin-RevId: 501017930

2023-01-05

Adds a knob to TFMD schema to infer RaggedTensors for variable length features. By default they are inferred as ragged left-aligned SparseTensors. PiperOrigin-RevId: 499913930

2022-12-15

Add supporting schema configuration for sql-based derived features. PiperOrigin-RevId: 495452787

2022-11-15

Propagate `is_sorted`(`already_sorted`) field from schema's `SparseFeature` to `SparseTensor` TR. PiperOrigin-RevId: 488767868

2022-10-24

Supports normalized absolute difference validation in tfdv. This can be used to verify that the exact numeric values of counts is similar when normalized by the overall size of two datasets. PiperOrigin-RevId: 483486591

2022-10-20

Internal only PiperOrigin-RevId: 482544627

2022-10-19

Adds a new feature comparator NormalizedAbsoluteDifference for use in comparing datasets that are expected to have identical categorical value counts. A followup will implement the comparison. PiperOrigin-RevId: 482258142

2022-10-12

Add a CUSTOM_VALIDATION anomaly Type. PiperOrigin-RevId: 480641213

2022-09-27

NA PiperOrigin-RevId: 477099310

2022-09-23

Add the SequenceMetadata field to the schema to specify if this feature could be treated as a sequence feature. PiperOrigin-RevId: 476400441

2022-09-12

1) improved histogram accuracy Previously, QUANTILES histograms were generated with identical counts per bucket. When the elements selected as quantiles boundaries are far from the idealized boundary - e.g., you have a distribution that's far from continuous - this was very wrong. This is a common enough case that it probably affects histogram accuracy for many users. STANDARD histograms, being derived from the same quantiles source, were also affected. This CL propagates the cumulative weight sum from the underlying quantiles sketch, and uses those values to fill in sample counts. Since what we get from the sketch is a sequence of bin upper bounds and associated weights, this requires changing Bucket semantics to include their upper bound and not lower bound, except for the first bin which includes both, because the quantiles sketch always gives us the minimum element and its count separately. 2) simplified infinity handling The old handling of infinite values was complicated, and could result in bins that mixed finite and infinite values. For STANDARD histograms this is a problem, since we'd like to be able to align histograms using interpolation to calculate distance measures. I've updated STANDARD histogram generation to generate separate -inf and +inf bins, if applicable. 3) fixed nested list length custom stat It looks like we were computing custom stats for nested list length based on the count of elements up a level in the nested list hierarchy, which I've fixed. 4) handles float64 overflow by omitting the standard histogram PiperOrigin-RevId: 473787450

2022-09-02

Add is_auxiliary field to Task in problem_statement. PiperOrigin-RevId: 471911554

2022-08-31

internal PiperOrigin-RevId: 471150553

2022-08-22

Add a categorical indicator to the schema for StringDomain PiperOrigin-RevId: 469291771

2022-08-19

Mark task_weight and weight as deprecated PiperOrigin-RevId: 468596453

2022-08-15

Clarifies that num_non_missing statistics.proto includes examples that define a feature but contain an explicitly empty value list. PiperOrigin-RevId: 467706566

2022-07-20

Internal cleanup PiperOrigin-RevId: 462255959

2022-07-12

Internal only PiperOrigin-RevId: 460497139

2022-07-05

Add FPR and FNR as DifferenceAcrossSlice metrics We originally avoided these because we disprefer threshold-based metrics (thresholds don't have a priori meaning unless a model is calibrated). However, clients find these fairness metrics much more intuitive and they align better with PA / policy guidance. Also, equalizing FPR and FNR is more directly what MinDiff is able to achieve. Moreover, we've already added threshold-based metrics recently for multi-label. The new fairness metrics added here are for binary classification where the final model is always calibrated, and thus the threshold is only unstable in the AutoML loop, which we address in documentation. PiperOrigin-RevId: 458987803

2022-06-29

Make ThresholdConfig.threshold subfield into a oneof This is to ensure future compatibility with other kinds of thresholds. PiperOrigin-RevId: 457881611

2022-06-15

TFDV: fixes a bug in validation of derived features wherein we'd always produce a DERIVED_FEATURE_BAD_LIFECYCLE anomaly, and adds test coverage. PiperOrigin-RevId: 455209162

2022-05-25

internal PiperOrigin-RevId: 450971903

2022-05-23

internal PiperOrigin-RevId: 450458691

2022-05-19

Remove `option jspb_use_correct_proto2_semantics = false` from proto files where it has no effect, allowing the value to be the default `true`. This option only affects jspb gencode for singular primitive fields without default values in in proto2 files. All of these files either have no such fields or are proto3. More info: go/jspb-correctness-lsc Tested: TAP for global presubmit queue http://test/OCL:448077555:BASE:449009947:1652809126374:7c4f8811 PiperOrigin-RevId: 449856532

2022-05-13

Introduce a new metric that computes multilabel recall at a given score threshold. For quality experimentation with the Feedback team. PiperOrigin-RevId: 448441850

2022-05-02

Changes tfdv and tfx-bsl to use renamed derived source fields, and removes the old names from the proto. PiperOrigin-RevId: 446050596

Changes the names of several TFMD fields pertaining to derived features to avoid conflict with existing uses of "derived feature". The old names will be deleted in a followup. PiperOrigin-RevId: 445952456

2022-04-28

Marks derived stats protos as experimental in tfmd. PiperOrigin-RevId: 445184813

2022-04-22

Minor update to lifecycle stage documentation. PiperOrigin-RevId: 443753669

2022-04-21

Adds a second anomaly type for derived features covering the source being set incorrectly PiperOrigin-RevId: 443423121

2022-04-20

Adds DERIVED_FEATURE_BAD_LIFECYCLE anomaly to signal schema anomalies when a derived feature appears in the schema with an incompatible lifecycle (e.g., PRODUCTION). PiperOrigin-RevId: 443222946

2022-04-19

internal PiperOrigin-RevId: 442882736

2022-04-13

Handle derived source in tfx-bsl proto merger code. PiperOrigin-RevId: 441549608

2022-04-12

Introduce a new metric that computes multilabel precision at a given score threshold. For quality experimentation with feedback. PiperOrigin-RevId: 441069918

2022-03-31

Adds a DerivedFeatureSource message to TFMD to track metadata describing derived features. Derived features will be features that are computed from ordinary features during statistics generation, and which are available for exploratory analysis or validation, but not present in the raw inputs. PiperOrigin-RevId: 438584379

2022-03-23

Remove "do not use" from TensorRepresentationGroup docstring. The attribute is fairly mature and is already in use in a number of TFDV and TFT features PiperOrigin-RevId: 436783627

2022-02-17

Add new Jaccard score performance metric. PiperOrigin-RevId: 429182336

2022-01-14

Rolls back addition of utf8_encoded field to StringDomain. PiperOrigin-RevId: 421914463

rollback PiperOrigin-RevId: 421902202

Adds an anomaly type for unexpected non-utf8 strings. PiperOrigin-RevId: 421868016

2022-01-10

Introduces fields for unicode validation. PiperOrigin-RevId: 420781430

2021-12-22

Mark various unsupported metrics as deprecated These metrics aren't supported in AutoTFX (the only implementation of this proto) PiperOrigin-RevId: 417707332

2021-12-08

Fully deprecate objective_function and mark multi_objective for deprecation. The `multi_objective` bool isn't understood by our system currently and it's not at all clear that a single boolean like this is how we'd express a pareto search longer term. Since we're starting to support thresholded metrics and possibly soon weighted combinations, having this extra unused config adds confusion. PiperOrigin-RevId: 414920758

2021-11-04

Proposes API for thresholded metrics A more complete proposal of the behavior/contract can be found here: https://docs.google.com/document/d/1xT8Fq2fc2j3ZkrNamFEK9I2OQg5988X4xV8fQFXBUH0/edit?resourcekey=0-9uxb8KFe_JCOesZo4Z317Q#heading=h.da395nbulm5q (Ideally contractual information belongs in the proto, but AutoTFX-specific details don't seem appropriate here so I think the above will have to be translated into some kind of AutoTFX g3doc instead?) PiperOrigin-RevId: 407688935

2021-11-01

Add a Google-only Differential Privacy Performance Metric to Problem Statement. PiperOrigin-RevId: 406898733

2021-10-13

Proposal for experimental version of a Google-only API for a coverage safety metric. PiperOrigin-RevId: 402708855

2021-09-10

Updates tensorflow metadata schema to contain is_categorical field in FloatDomain. PiperOrigin-RevId: 396027954

2021-07-14

Add specification of Positive and Negative class to the problem statement and handle it in AutoTFX schema augmentation. Note, this does not actually plug in the schema augmentation at this time. PiperOrigin-RevId: 384726076

2021-05-14

Update warnings now that we have NLP support in TFDV. PiperOrigin-RevId: 373807037

2021-04-27

Adding probability prediction to problem_statement.proto PiperOrigin-RevId: 370727045

2021-04-15

Add IFTTT guard to update TFDV anomalies docs. PiperOrigin-RevId: 368731855

Add anomaly when natural language stats are not computed. PiperOrigin-RevId: 368680677

2021-04-13

Add new anomaly info types. PiperOrigin-RevId: 368249522

2021-03-10

Schema: Adding sequence length min / max and set of tokens to ignore when computing the sequence length. Statistics: Adding sequence length distribution, min, max PiperOrigin-RevId: 362120228

2021-02-19

Add (Google-only) API for fairness remediation problem specification This is the direction I'm leaning based on review. I think we're close enough to start talking about naming, etc. After getting feedback from you three, I'll run it by Kapla. Note the API does not support pre-split paths like Kapla has. They'd need to refactor their data materialization to align with this API. We might build them a little tool do to the data massaging but it would be outside of AutoTFX for now. PiperOrigin-RevId: 358469597

2021-02-18

Extend RaggedTensor representation to support more ragged tensor representations. PiperOrigin-RevId: 358190488

2021-02-08

Added a new type of anomaly for feature shape validation. PiperOrigin-RevId: 356351795

2021-02-04

add MAX_IMAGE_BYTE_SIZE_EXCEEDED to tfmd anomalies and max_num_bytes_int field to bytes_stats. PiperOrigin-RevId: 355641556

2021-02-01

Implements reported_sequences and token statistics in natural language stats generator. PiperOrigin-RevId: 355009342

2021-01-15

Add is_embedding to FloatDomain and a corresponding Anomaly type for invalid embeddings PiperOrigin-RevId: 352034922

2020-12-12

[TFDV for NLP] Extend anomalies proto to report back anomalies associated with NaturalLanguageDomain. PiperOrigin-RevId: 347106293

2020-12-03

Update schema to support differentiating between tokens to be excluded from the coverage calculation and those that represent oov tokens. e.g [PAD] represents the former while [UNK] represents the later. PiperOrigin-RevId: 345335922

2020-11-30

[TFDV for NLP] Update statistics.proto to define NaturalLanguageStatistics. PiperOrigin-RevId: 344852583

2020-11-24

[TFDV for NLP] Definition of NaturalLanguageDomain. PiperOrigin-RevId: 344132237

2020-11-02

LSC: opt-out of default proto2 semantic correctness, we're inverting the default. This is intended to be a no-op change to your codebase, please rollback and let us know if there are unexpected issues caused by this change. To learn more, please visit: go/jspb-correct-proto2 #jspb-correct-proto2-lsc PiperOrigin-RevId: 340256812

2020-10-27

Add a DISABLED lifecycle stage This can be used to denote features that are excluded from a model. Whereas the semantics of DEPRECATED indicate that the feature was previously used but should not be in the future, DISABLED is more temporally agnostic and simply indicates a feature that is not currently in use. PiperOrigin-RevId: 339273198

2020-10-13

Added a new proto DriftSkewInfo for keeping the raw measurements of skew/drift. Also added a new repeated field of that type in the Anomalies proto so that TFDV can report the raw measurements regardless of whether a drift/skew was detected. PiperOrigin-RevId: 336910440

2020-10-02

Fixing comment typos that caused me small confusion. PiperOrigin-RevId: 335090115

2020-09-23

Add new Anomaly type to describe when a domain is incompatible with the data type. PiperOrigin-RevId: 333393799

Document skew/drift detection support for numeric features and remove experimental warning from Jensen-Shannon divergence field in schema proto. PiperOrigin-RevId: 333191383

2020-09-02

Allow for specifying a label as a Path or a string PiperOrigin-RevId: 329725820