Proto commits in google/differential-privacy

These 28 commits are when the Protocol Buffers files have changed:

2025-02-26

Update PipelineDP4j examples to the new API PipelineDP4j: * Migrate examples to the new API and update documentation * Migrate to the latest versions of external dependencies * Internal changes to support VECTOR_SUM metric Go: * Implement differentially-private variance Change-Id: I9c1e47cddc305edd30a60fc249f43098c236b878 GitOrigin-RevId: a776f9f8c43ab415738bb7ebd2f8e93c9802f29a

The documentation is generated from this commit.

2025-01-29

Update serialization in C++ (still backwards compatible) C++: * Refactor code and introduced `BoundsProvider` making it easier to implement new bounding algorithms. * Add `ApproxBoundsAsBoundsProvider` wrapper for using `ApproxBounds` with the new interface. * Add new serialization for `BoundsProvider`. * Move reconstruction logic for clamped bounds into separate class. * Change serialization for sum, mean, and variance which is still fully backwards compatible (for now). PoB: * New sharded public partitions implementation is now by default on. Proto changes: * Deprecate `bounds_summary` field in all protos (still populated) * Add new message and field to support adding new `BoundsProvider` PipelineDP4j: * Make flatten work with local collections * Sort ranks in quantiles * Allow relative budget spec for group selection Python DP Accounting: * Add tests for `AdditiveNoisePrivacyLoss` classes Change-Id: Ie66e82fc222597a9a1c817bf118d014b7b144fdd GitOrigin-RevId: 6bcc4974328347eb2eaa3955795c7965a255a536

2024-10-29

Publish PipelineDP4j, JVM version of PipelineDP PipelineDP4j is an end-to-end differential privacy solution for JVM that supports various frameworks for distributed data processing such as Apache Spark and Apache Beam. It is the JVM implementation of PipelineDP (https://pipelinedp.io/), and is conceptually similar to Privacy on Beam. Change-Id: I84a0da8fbfd55bbbb37a7013e66c22e1f222e566 GitOrigin-RevId: f574c7e2a8601092d073377ef4afc07187697132

2024-10-23

Cleanup and documentation changes C++: * cleanup code, refine comments * update absl dependency Java: * Load java rules from @rules_java Change-Id: I3141a3389af98ba221c13c230b36e9afb2420187 GitOrigin-RevId: cb0c561fbd518ac22686e23e7833ddf2c75e9468

2024-03-06

AddMixturePrivacyLoss in PLD accountant, DP Auditorium notebook example. Core DP: - Remove deprecated Element::error proto field. - Remove deprecated ErrorReport::noise_confidence_interval proto field. DP Auditorium: - run_mean_mechanism_example in IPython notebook. Python DP PLD Accountant: - Adds MixtureGaussianPrivacyLoss to privacy_loss_mechanism. - Adds constructor method for discrete PLD of MixtureGaussianPrivacyLoss to privacy_loss_distribution. Privacy on Beam: - Remove remaining references to the oldPrivacySpec API. Change-Id: I10e18421ffe24f6a5ea9a12f692b011b24797e86 GitOrigin-RevId: c4a7436267b393015db2f98560d42c6a8aab3cb7

2023-05-25

Implement LongBoundedSum in Java DP library Java DP lib: - Introduce `LongBoundedSum` an equivalent of `BoundedSum` returning `int64` results. Python accounting library: - Use `typing.List` instead of `list` for Python 3.7 compatibility [Python DP Accounting] - Introduce mu_lower_log_cdf to fix numerical issue in get_delta_for_epsilon [Python PLD Accounting] PiperOrigin-RevId: 528544183 Change-Id: I9baef7232dabaed01c037637a2dc406f51b2cc42 GitOrigin-RevId: 027162b67b04393606408b2f9b4dbf1130385a7a

2023-03-29

Updates to dependencies; add pre-thresholding in C++ & Java Build - Update Protobuf to version 21.12 - Update bazel to version 6.1.1 Python PLD Accounting - Increment minor version of DP accounting library for PyPi release, reflecting improvements in PLD Accountant (Connect-the-Dots) C++ - Extract CalculateThreshold from stddev in GaussianPartitionSelection - Update dependencies - Return absl::Status in statistical tests Pre-thresholding - Release pre-thresholding in C++ and Java Go - Remove deprioritized TODO Change-Id: Ia1f72ef4cca16dcd3ff99e463dc19d7f02a7b0ca GitOrigin-RevId: efdf42fde42fb55e971c7ea38d38092c0fc02d56

2023-03-07

Unify Beam universal type usage in PoB, fix errors in C++ example Privacy on Beam: - Unify how Beam universal types are used, refactor aggregations.go & generated_functions.go C++: - Fix errors in the example Java: - Add MechanismType.MECHANISM_NONE. It can be used for implementing mechanisms for testing purposes Change-Id: I1aa1719c2c5201ef857e832387a163fd29ff2bbd GitOrigin-RevId: c1bdb81b90da8c74de1621ffd08b5fe97b8c9973

2022-04-20

Minor changes to the libraries and Python Accounting library: * Add statistical tests for approximate bounds for DP libraries. * Add a version of NoiseConfidenceInterval without privacy_budget in C++ DP library. * Cleanup typing annotations in Python Accounting library. PiperOrigin-RevId: 439689331 Change-Id: Ie74f116e61175a701095c1c9c0f48e572e2b758a GitOrigin-RevId: 108b2bf86c5da379f6a6f667b52893b7ca7360a1

2022-04-04

Output noise conf intervals to repeated field in C++ C++ library: - Output.noise_confidence_internal to a repeated field. privacy-on-beam: - Update Bazel instructions to v1.1.2 Py Accounting: - Add typing annotations in RDP accounting. PiperOrigin-RevId: 435703861 Change-Id: Iaeed672866014491dcf7f72ce26122a5f8de668b GitOrigin-RevId: 1970a21ac26b9dc5726d0c17f4e0c969f51dfaae

2022-03-17

Implement Discrete Laplace in Java, various minor improvements overall Python Accountant: - Remove dependence on `six` and use `super()` in rdp_privacy_accountant - Add Laplace DpEvent - Fix typos C++: - Enables support for partition selection when counts can be larger than max int32 Go & PoB: - Prepare for v2 release Java: - Add Discrete Laplace noise class which adds the discrete version of laplace noise (aka symmetric geometric noise) that is DP for integer-value function with given sensitivities. Change-Id: I5e61d3fa74a0b2071e9df0531c94d9419c67a7ad GitOrigin-RevId: 3d471c0db206a3df06b710a1cf7269f8207f6d0d

2022-02-24

Minor fixes/improvements/dependency updates. Global changes: - Update bazel version to 5.0.0 C++ DP library: - Modify BinarySearch to avoid use of the privacy_budget parameter in LaplaceMechanism.AddNoise Java DP library: - Move sampleGeometric and sampleTwoSidedGeometric into a separate class. Go DP library: - Rename BoundedMeanFloat64 to BoundedMean - Fix MinValue/MaxValue documentation for mean.go and quantiles.go Privacy on Beam: - Update Apache Beam dependency to v2.36.0 - Add Privacy on Beam to build instructions on README.md PipelineDP: - Add PipelineDP to README.md Python DP accountant: - fix from_privacy_parameters for the case epsilon = 0. PiperOrigin-RevId: 427022604 Change-Id: Ib54ae0518164ebbb32716da55f8a5574a4a1f731 GitOrigin-RevId: 23ce0d95c86b72c7b9b27052faa96ed677d444a1

2022-01-14

Fix bugs in Go, implement BoundedVariance in Java, Renyi DP in Python Go: - Fix a bug in ThresholdedResult for Count that may lead to higher than intended delta - Add checks for thresholdDelta in noise.Threshold() Privacy on Beam: - Fix small bugs with some tests, making them less flaky - Use RoundedLaplaceTolerance instead of LaplaceTolerance for integer aggregation in tests Java: - Implement BoundedVariance - Fail to build bounded quantiles, mean and variance if bounds are equal Accounting: - Introduce a new PrivacyAccountant implementation based on Renyi DP. This API is forked from TF Privacy, which will soon be switched to depend on the version here - Add accounting for tree aggregation as described in "Practical and Private (Deep) Learning without Sampling or Shuffling" PiperOrigin-RevId: 417627089 Change-Id: Id1f0ffbe607ae8f90b488053394eb577f2d48c6b GitOrigin-RevId: b02ccac42d590f72c2f20440721e4d0126a47f78

2021-12-20

List of changes: - DP clustering: refactored central privacy util parameters. - DP statistical test updated to more precise labels - Statistical closeness tests for Gaussian noise added - C++: Increased min success probability in approx bounds. PiperOrigin-RevId: 416048860 Change-Id: I1aa6ce94f36c35a9320a756f2a7054e4a42188a3 GitOrigin-RevId: fce3759f7d37f56fa12d96f534f5017f8b87bcb1

2021-12-01

Many fixes & cleanups overall C++: - Add empty checks for divide by 0 for mean. - Remove privacy budget parameter from PartialResult method. - Refactor LaplaceDistribution::Builder to return more errors. Statistical tests: - Remove long_value from DataRecord, double_value is enough. - Clean up of bounded_quantiles_dp_test_cases.textproto documentation Go/PoB: - Breaking change for the Go  Library - Big refactoring of how error reporting works in the Go Library.   Return errors in the Go Library instead of log.Fatal/Exit'ing. - We return errors on combineFns in Privacy on Beam as well. - Implement in-memory public partitions for MeanPerKey, SumPerKey, DistinctPrivacyId, QuantilesPerKey - "go mod tidy" on go.mod files. - Remove checks.CheckMaxPartitionsContributed. - Move getMaxContributionsPerPartition to checks package as ChecksMaxContributionsPerPartition. - Fix a bug in DistinctPrivacyID where errors are not propagated properly. - Update plot module version. Accounting library: - Introduce a new DpEvent and associated PrivacyAccountant API for differential privacy accounting. This API is forked from TF Privacy, which will soon be switched to depend on the version here. Misc: - Replace references to some non-inclusive terms. Change-Id: I83ac68219a84628770e1cdc15b2ee8a379f16628 GitOrigin-RevId: 1bd1061a39f6fdf9d33ae3b2f15aa501e5068492

2021-11-22

Fix special cases for sum; add in-memory public partitions in PoB C++: * Fix BoundedSum with ApproxBounds use case for lowest int input Go: * Allow lower == upper for sums * Change lower bound for normalizedSumOfSquares Java: * Improve utility of Count's computeThresholdedResult Privacy on Beam: * Implement in-memory public partitions for Count * Reword the Count's MaxValue documentation Change-Id: I8cc8a4009aa35e91e2c449da1e60d1ef58d027b4 GitOrigin-RevId: b29ba88e2ce7b66f967da9dcfff92bc0986f9f28

2021-05-10

Refactoring & small fixes in Java DP library, new features in C++ DP library & Accounting. DP libraries: - Use com.google.privacy.differentialprivacy.proto package name for Java protos - Add GaussianPartitionSelection as a PartitionSelectionStrategy to C++ library - Add documentation for quantiles algorithm (Differentially_Private_Quantile_Trees.pdf) Accounting: - Add serialization for Privacy Loss Distribution GitOrigin-RevId: 9bb13902faf0011001f79b538d02f9f9cf6dfa81 Change-Id: I33ac735b9854454f9f6c23cc77b2a792d983f171

2021-05-03

Confidence intervals for Quantiles in Java & optimizations in Accounting Java: - Implement confidence intervals for Quantiles - Refactor existing confidence interval implementations Privacy on Beam: - Modify Bazel test sizes (Also for Go DP Library) - Refactor DoFns to return errors instead of log.Exit'ing Accounting: - Optimize self convolution for both C++ and Python GitOrigin-RevId: f11113214c18a07f0ce29187e9e7945940f53b06 Change-Id: Id9c5ba65f3b47f84b5c3f2b9ba21ed4fefd41359

2021-03-17

Fix clamping for variance and stdev in C++, add quantiles in Java This change includes an important fix to bound the sensitivity for the variance and stdev implementation in C++. C++: - Fix clamping in BoundedVariance implementation. Java: - Implement BoundedQuantiles. Go && Privacy on Beam: - Small cleanup type modifications. GitOrigin-RevId: f71680e3e53195078855a662f9b107bd4b483040 Change-Id: Ie6e27cd134f54f70203b4f10a446e82528dd09f7

2021-02-23

Serialization for numerical mechanisms in C++, documentation updates C++ library: - Added Serialization for numerical mechanisms - Serialization of Algorithms is now `const` - Refactored `BoundedSum` (not visible externally) Java library: - Documentation updates to BoundedMean, BoundedVariance and order statistics Python DP accountant: - Add identity privacy loss distribution. GitOrigin-RevId: dcf9737b39b6d753411e81592d688141c1b8e9b8 Change-Id: I7671d0a99baeeeb53109c4235d740cc04a29b58b

2021-02-09

Many smallish improvements overall. C++: - Change count-related variables from uint64 to int64 & prevent non-positive count arguments. - Add tests for rounding integers to the granularity for Laplace. - Add documentation on the MaxContributionsPerPartition and MaxPartitionsContributedTo options. Java: - Library doesn't fail if it cannot recognize noise type and validate delta or sensitivity. This allows clients to plug in their own noise. Accounting: - Fix output missing output type for self_convolve_dictionary. - Separate out self_convolve and make the list size smaller by 1. - Move convolution-related functions to common. Privacy on Beam: - Small refactorings. - Fix a bug where parameter checks were ignored for DistinctPerKey. GitOrigin-RevId: 038419cdbd82b806a29e9fb221ec33d238742c46 Change-Id: I249a28d095dd895dea1c30879f50f4a36b3d47bf

2021-01-19

Update contact channels C++: - Added builder for distributions - Added error messages for sensitivities Java: - Added serialization for BoundedMean - Simplified testing of builders and improved test coverage GitOrigin-RevId: 68012e8f7d1dde6f82474a7012796090e2e23380 Change-Id: I06d049cad534062af63e4e231d4007e67a13d4b3

2020-11-04

Privacy on Beam adds DistinctPerKey / C++ Handles Overflows Privacy on Beam: - Implement DistinctPerKey, which allows for counting distinct things other than privacy IDs (user IDs). - Implement SelectPartitions, a primitive for doing only partition selection without any other aggregation. The selected partitions can then be used as "public partitions" to other aggregations to avoid reusing budget for partition selection. - Small clean-ups. GitOrigin-RevId: 8c461dec6b3bb086354f4bc5f5622b2a9a2a77c0 Change-Id: I081319dbdd8d1f7764fa2c855b7325d162daf805

2020-08-17

Stability improvements and refactoring for C++ C++: - stability improvements - refactoring for NumericalMechanismBuilder complete Accounting: - add a function that computes epsilon given delta GitOrigin-RevId: b69f5dccb05f90a66c6dea98eb71113e66ccbcb8 Change-Id: I77fcdd190e3338e2eee897ed2b329bb895f03df1

2020-06-24

Adding Go, Privacy on Beam, and DP Accounting Go: - Initial import of Go Differential Privacy Library. - Add Laplace & Gaussian noise. - Add Count, Sum, Mean aggregations. - Add thresholding primitives. - Add PreAggPartitionSelection. Java: - Add statistical tests for Laplace noise and Count. - Count.incrementBy() accepts long instead of int. Privacy on Beam: - Initial import of Privacy on Beam. - Add DistinctPrivacyKey, Count, SumPerKey, MeanPerKey. - Support for Laplace & Gaussian noise. - Uses partition selection & thresholding primitives. Initial import of Privacy Loss Distribution accounting Other: - Add noise generation documentation. GitOrigin-RevId: ff623c498820aebb400c767a1dae294ca0f43fe6 Change-Id: I44ba4d613ae321d6c640821ec74b1ea73b441d11

2020-06-02

Move C++ library to cc/ folder C++ - Move the library to the cc/ folder - Use absl::optional in builders Java - Make Count and BoundedSum Params public Change-Id: I99c81b324006190e435b80a56b03ee656686699b GitOrigin-RevId: 9715e6fcdd967668716668783a34726523c36960

2020-05-11

Import Java library. C++: - Update bazel external dependencies to follow recommendations. - Add the ability to specify the confidence interval returned. - Removed superfluous count stochastic test - testing small datasets with different contents isn't useful. - Make GenerateResult protected. This was a mistake, and clients should be calling Result or PartialResult. - Make ResetState protected. This was a mistake, and clients should be calling Reset. - Update SFINAE to be more readable using C++11. - Add a LaplaceDistribution constructor that accepts epsilon/sensitivity, rather than diversity. - Short circuit animals and carrots example if file can't be opened. - Stop stochastic tester when enough datasets fail to violate bounds, rather than continuing with all trials. Java: - Add Laplace Noise. - Add Gaussian Noise. - Add Count algorithm. - Add BoundedSum algorithm. Other: - Define key terms used in documentation. - Break out differential privacy description and key terms. - Add C++ Specific documentation. Change-Id: I793ec1f0d99641fa4297c427059679afb5082c3c GitOrigin-RevId: 889eeb232f65cb86e6e937924b61717a30d7101d

2019-09-05

Project import