Proto commits in google/mozc

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2025-04-24

Deprecated user_history_prediction_filter_redundant_candidates_mode and user_history_prediction_aggressive_bigram. These mode already launched to prod. PiperOrigin-RevId: 750827634

The documentation is generated from this commit.

2025-04-09

Add a feature flag to filter NWP suffix candidates based on their transition cost PiperOrigin-RevId: 745503591

2025-04-03

Add a comment about the alternative with the obsolete `mode`. Note, this `mode` was obsoleted by cl/16872841. #codehealth PiperOrigin-RevId: 743375728

2025-03-31

Add a clarification comment for id in commands.proto. #codehealth PiperOrigin-RevId: 742164769

Add 'is_incognito_mode' field to Request proto message. This is an additional option to Config::incognito_mode. Clients can use Request::is_incognito_mode instead of Config::incognito_mode so that the clients can skip calling SET_CONFIG commands to the server. #codehealth PiperOrigin-RevId: 742109093

2025-03-21

Update the comment about HIGHLIGHT_CANDIDATE #codehealth PiperOrigin-RevId: 739082082

2025-03-11

Clean up the experimental flag: realtime conversion candidate checker PiperOrigin-RevId: 735604374

Clean up the experimental flag: user_segment_history_rewriter_replace_proper_noun PiperOrigin-RevId: 735595946

2025-03-10

Deprecated enabled and auto_registered fields of UserDictioanryStorage. These fields is no longer used. PiperOrigin-RevId: 735289024

2025-03-09

Completely removed spelling correction features from Mozc. PiperOrigin-RevId: 735104457

2025-03-07

Remvoed dictionary sync features. They are no longer used and mantained. PiperOrigin-RevId: 734589342

2025-03-04

Add the experimental flag to filter NWP bigram entries. PiperOrigin-RevId: 733138670

2025-03-03

Add is_handwriting to Request PiperOrigin-RevId: 732786172

2025-02-25

Add a experimental flag to control the bugfix in cl/694010741 PiperOrigin-RevId: 730762929

2025-02-13

- Introduced Options::incognito_mode to make an on-the-fly incognito mode request. - Added ConversionRequest::incognito_mode() - Stop having incognito_config_ in EngineConverter. PiperOrigin-RevId: 726384302

2025-02-06

Removed usage_stats feature. UsageStats is no longer used. PiperOrigin-RevId: 723855520

2024-12-19

Deprecated install_path. install_path is no longer used after MDD. PiperOrigin-RevId: 707856864

2024-11-28

Fix typos. PiperOrigin-RevId: 700945320

Reranks suffix-dictionary-based zero query suggestions with neutral sentence classifier model. PiperOrigin-RevId: 700894055

2024-11-11

Add experimental param to relax the condition to remember candidates for the segment including proper noun words. Example: 1. Submit "崇が" for "たかしが". - The candidate may not have personal name POS, but some others should hae personal name POS. 2. Input "たかしの" 3. "崇の" will be promoted with this feature. PiperOrigin-RevId: 695144738

2024-11-05

Add logics to support Emoji 16.0. * This is a preparation to update the Emoji data to 16.0. #codehealth PiperOrigin-RevId: 693255328

Add a feature flag to change the cost offset for handwriting conversion candidates PiperOrigin-RevId: 693200996

Rename `java_outer_classname` of `candidate_window.proto` to `ProtoCandidateWindow`. PiperOrigin-RevId: 692825449

2024-10-30

Clean up the experimental param: user_history_prediction_min_selected_ratio PiperOrigin-RevId: 691275706

2024-10-23

Rename `candidates.proto` to `candidate_window.proto` for better readability. PiperOrigin-RevId: 688832889

2024-10-22

Rename `SessionState.candidates` to `SessionState.candidate_window` for better readability. PiperOrigin-RevId: 688396251

Rename `CandidateWindow.subcandidates` to `CandidateWindow.sub_candidate_window` for better readability. PiperOrigin-RevId: 688383891

Deprecates the T13n promotion based on the base LM score. PiperOrigin-RevId: 688376533

2024-10-21

Rename `Output.candidates` to `Output.candidate_window` for better readability. PiperOrigin-RevId: 688146644

2024-10-17

Rename `Candidates` to `CandidateWindow` for better readability. PiperOrigin-RevId: 686787899

2024-10-15

Suppress false-positive corrections with surrounding context. When TC is triggered, this module aggregates at most top 6 (3+3) literal and TC candidates with the same key. Then, run NSC reranker to decide literal-on-top. This feature has multiple mode 1. Only runs literal-on-top while keeping the original ranking. 2. Promote the predicted literal candidate that has the highest reranker score. 3. Promotes the predicted TC candidate too. PiperOrigin-RevId: 685995877

Add parameters for candidate checker - Cost max diff for first segment candidates - Charactor coverage based filtering for realtime conversion candidate checker. - We can provide full match realtime conversion result for short query. Example: query "のとき", candidates: "の時", "のとき" PiperOrigin-RevId: 685974932

2024-10-08

Update Next ID. PiperOrigin-RevId: 683512354

2024-10-03

Removed migration flags for typing corrections. These flags were introduced to make trivial modifications on the scoring functions. Confirmed that these migration has no quality impact through 10% rollout experiment. PiperOrigin-RevId: 681765823

2024-10-02

Remove typing_correction_enable_number_decoder flag. This flag is set true by default. PiperOrigin-RevId: 681269488

2024-09-17

Populates the typing corrector's penalty/bonus to the candidates containing auto-kana-modifier-expansions generated by composer or dictionary-based expansions. We can evaluate all typing corrections with uniform scoring. The actual populates are implemented inside the supplemental model. PiperOrigin-RevId: 675586853

2024-09-06

Introduced TypingCorrectionReranker module. Currently, literal-on-top decision and actual literal-on-top reranking are performed separately with different functions. We will merge them into one general reraking component. We will perform literl-on-top and literal-at-least-second reranking with various signals including surrounding context information. PiperOrigin-RevId: 671652661

Add a flag to disable the transformer rescoring for non-NWP results. In general, latency is more permissible for next word prediction. Since the transformer rescoring is a heavy operation, we may want to limit this feature for NWP in case where improving latency is difficult. PiperOrigin-RevId: 671608920

2024-09-02

Clean up experimental flag: apply_user_segment_history_rewriter_for_prediction, user_segment_history_rewriter_use_inner_segments #codehealth PiperOrigin-RevId: 670113068

2024-08-30

Added typing_correction_fix_bias param. penalty is used to determin the literal-on-top behavior, while bias is used to the base rank of the candidates. By this definition, penalty should not be populated to bias. PiperOrigin-RevId: 669207074

2024-08-26

Add comments to explain the key codes for "Katakana Hiragana Romaji" key. #codehealth PiperOrigin-RevId: 667453766

2024-08-21

Promotes Katakana transliteration based on the confidence of the literal candidate. It would be better to provide Katakana translation than un-grammatical Japanese sentences (just a combination of random nouns). We calculate the decoder's confidence with the per-char-length cost. This CL also includes the fix to the handling of legacy dictionary based typing corrections. They are now handled as literal candidate. PiperOrigin-RevId: 665644778

2024-08-15

Enables number decoder on the typing corrected compositions. PiperOrigin-RevId: 663170440

2024-08-13

Support long character completions with extra penalty. In order to predict "ざいません" from "もうしわけご", we have to change the max_completion_length to 5, but long character completions would have large side-effect if the completion fails. We would like to add them if and only if the confident score is large enough. The extra penalty is set as penalty = extended_per_char_penalty * (completed_length - max_char_length). When extend_per_char_penalty is zero, no penalty is added. PiperOrigin-RevId: 662379828

2024-07-18

Clean up user_segment_history_rewriter_new_replaceable from decoder experiment params #codehealth PiperOrigin-RevId: 653463000

2024-07-16

Added disabled_supplemental_model_ids mendel flag. This flag allows to disable some specific supplemental models for debugging purpose. PiperOrigin-RevId: 652677422

2024-07-01

Remove the fix literal on top flag. We've confirmed that this bug fix doesn't have any side-effects on quality. PiperOrigin-RevId: 648325104

2024-06-27

Enables more aggressive bigram-based zero-query suggestion. 1. Suggests "は" from "今日" when user has typed "今日は" before 2. Takes the longest context information to predict the next candidate, e.g., next token is predicted from "今日は" instead of "は" even when "今日" and "は" are typed separately. PiperOrigin-RevId: 647234111

2024-06-25

Clean up `enable_realtime_conversion_v2` flag The server side CL is cl/644637617 PiperOrigin-RevId: 646339602

2024-06-24

Support typing correction in the user history prediction predictor. PiperOrigin-RevId: 646005786

2024-06-21

Added new candidate filtering logic based on candidate redundancy. Candidates sharing the same prefix are filtered. This filter usually filters the candidates that end with Japanese function words. PiperOrigin-RevId: 645342721

Clean up `mozc_filter_noisy_number_candidate` Server side CL is cl/644366651. PiperOrigin-RevId: 645224022

2024-06-20

Add a feature flag to control Katakana prmotion in T13N promotion rewriter PiperOrigin-RevId: 644988052

2024-06-19

Changes the size of history candidate with accumulated character length. This feature is an approximation of the adaptive history candidates determined by the width of the area visible to the user. PiperOrigin-RevId: 644625474

2024-06-17

Removed unused flags for user history suggestions v2. We could not get any good results on this v2 feature. To keep the simple implementation, we remove v2 feature. PiperOrigin-RevId: 643962194

2024-06-13

Added post_correction_apply_user_history_mode. We will apply post correction to the user history candidate with retricted conditions. PiperOrigin-RevId: 642979777

Cleanup DecoderExperimentParams. PiperOrigin-RevId: 642856925

2024-06-12

Set wcost in N-gram predictor. Currently no cost is set in the N-gram predictor and they appear in generated order. However, by setting wcost properly, the dictionary predictor sets their costs accordingly and the transformer rescorer can effectively re-rank NWP candidates. Similarly to the other prediction sources like BIGRAM, wcost adjustment (bonus or penalty) can be given. Its value is controled by the flag. To implement this feature, the underlying Predictor is extended to return more detailed information of each prediction result. PiperOrigin-RevId: 642533910

2024-06-04

Support multiple (nbest) corrections in post corrector. When one query has only single span to correct, we apply multiple (n-best) corrections. PiperOrigin-RevId: 640177185

2024-05-24

Add a feature to remove the unselected user history prediction PiperOrigin-RevId: 636770877

Support surrounding context information in PostCorrector. Also added parameters to control the latency of post corrector. PiperOrigin-RevId: 636754464

2024-05-23

Threshold as penalty refactoring for typing correction. Before: score > threshold After: score - penalty > 0 We had been using threshold-based triggering control. i.e. trigger if score > threshold. However, we recently introduced a conditional threshold in which the threshold is dynamically updated. This conditional threshold is confusing as we can update both the raw score and threshold. This refactoring fixes the threshold to be zero and controls the triggering condition only with the penalty. i.e, score - penalty > 0.0. This CL migrates the thresholds to penalties. Since the scores are used in other ranking, this migration is flag guarded. PiperOrigin-RevId: 636428033

2024-05-20

Added typing correction penalty scorer with heuristics rules. We will introduce two penalties. 1. Penalty for the first character correction 2. Penalties for the kana modifier correction when intended kana modifiers exist. PiperOrigin-RevId: 635356368

Remove unused params for typing corrections. Literal-on-top parameters are not used. PiperOrigin-RevId: 635338916

2024-05-14

Remove deprecated parameters. PiperOrigin-RevId: 633492736

- Introduced typing_correction_promote_kana_modifier_insensitive_corrections flag. When the top correction is pure kana insensitive corrections, we do not allow other non-kana-insensitive corrections. - Refactoring on typing correction module PiperOrigin-RevId: 633471767

2024-04-30

- Introduced new Options structure that are shared by Reranker and StackDecoder. - Added typing_correction_reranker_max_score_diff_override flag so we can optimize the Parameter via Mendel flag. Currently this flag is only updated by the model update. PiperOrigin-RevId: 629312698

2024-04-25

Changes the default values of literal on top parameters. We confiremed that following two parameters have no positive but negative impact so set default values (0) to turn off these features. typing_correction_literal_on_top_conversion_cost_max_diff typing_correction_literal_on_top_correction_score_max_diff typing_correction_literal_at_least_second are enabled both in canary and production, so we change the default value. PiperOrigin-RevId: 627983164

2024-04-24

More kana insensitive corrections. After disabling legacy typing corrections, we see several feedback that the kana insensitive corrections are less aggressive. This CL allows to include kana insensitive corrections even when the score is smaller than identity score. We put an extra penalty to avoid false positives. PiperOrigin-RevId: 627645209

Fix the bugs of literal-on-top. 1) Checks the values of typing and non-typing correction to determine literal-on-top condition. 2) Apply literal-on-top just before immutable dictionary prediction module. Do not want to apply literal-on-top to the user histories. PiperOrigin-RevId: 627577185

2024-04-23

- Add a flag to limit the Hiragana reading estimation for performance - Set small cost for the top recognition result PiperOrigin-RevId: 627241857

2024-04-22

Enabled candidate-consistency based candidate ranking. Puts the previous candidate if the current and previous candidate are consent. This change will avoid candidate flickering. PiperOrigin-RevId: 626920089

2024-04-19

Update the candidate filtering rule for realtime conversion V2. - Apply cost based filtering for its boundary information instead of the fixed candidate size. PiperOrigin-RevId: 626276562

Add a Transformer LM based rescoring module. PiperOrigin-RevId: 626239318

2024-04-09

Remove typing_correction_mixer_v2 flag. Mixer v2 is already launched. PiperOrigin-RevId: 623045420

2024-03-29

Adding literal-on-top parameters based on the input length. We would like to demote typing corrections more aggressively on shorter queries. PiperOrigin-RevId: 620172961

2024-03-21

Add space insertion mechanism for transliteration. This CL introduces a mechanism into the EnglishVariantsRewriter, which expand variants with prefixed spaces. While many English words are supported in Mozc, currently there is no way to insert space between words. Therefore, 'ぐーぐるじゃぱん' always becomes 'GoogleJapan',not 'Google Japan'. This is not useful, especially for new words like じぇみにぷろ→'Gemini Pro' and 'ぴくせるふぉーるど'→'Pixel Fold'. By using this mechanism, when the user select 'Pixel' for 'ぴくせる', space-prefixed candidates like ` Fold` are inserted. By selecting that candidate, users can input 'Pixel Fold' as a compound. This will be learned in subsequent inputs. As there can be several ways to implement the same feature, I guarded this feature with a flag named "english_variation_space_insertion_mode". The current style is mode 1, and we can test mode 2 or later. PiperOrigin-RevId: 617748976

2024-03-05

Support commit text with mozc session so that we can log COMMIT_TEXT when a user committed the candidate by starting the next handwriting stroke. PiperOrigin-RevId: 612815746

2024-02-20

Get the data version from mozc server Now MOZC_DICTIONARY_VERSION is explicitly logged via MozcMetricsType. As far as I observed, onKeyboardActivated() is called before onStartInputView(), where we used to log the MOZC_DICTIONARY_VERSION event. PiperOrigin-RevId: 608481758

2024-02-15

Deprecates old typing correction. They had a negative impact on the quality. PiperOrigin-RevId: 607284972

2024-02-05

Add a feature flag for using inner segments info. PiperOrigin-RevId: 604208077

2024-01-31

Introduce a feature flag for the new replaceable condition PiperOrigin-RevId: 602966973

2024-01-26

Introduced history_prediction_triggering_v2 module. We've introduced several new parameters to control the aggressiveness of the history prediction. PiperOrigin-RevId: 601711838

2024-01-23

Set proper request mixed_conversion: prediction is the main interaction auto_partial_suggestion: candidate list includes prefix candidates - Handwriting request should be "mixed_conversion:true" and "auto_partial_suggestion:false" - Make sure that prefix candidates are not included for "auto_partial_suggestion:false" Offline diff: (not changed) https://<internal_url> PiperOrigin-RevId: 600659684

2023-12-20

Refactoring for the typing_correction_mixer_v2. Added new AggregateTypingCorrectedResults to the aggregator interface so we can make the triggering decision by comparing the base and typing corrected results. PiperOrigin-RevId: 592487425

2023-12-18

Add UPDATE_COMPOSITION command to support handwriting IME PiperOrigin-RevId: 591790628

2023-12-11

Update build rules for `dictionary/` PiperOrigin-RevId: 589817165

Fix a typo. #codehealth PiperOrigin-RevId: 589676094

2023-11-29

Automated g4 rollback of changelist 586135894. *** Reason for rollback *** Mozc's renderer_command.proto is shared between internal Bazel build and external Bazel build [1]. The problem is that there seems to be no simple way to specify '--experimental_editions' option in cc_proto_library in Bazel yet. Thus renderer_command.proto cannot be converted to Edition until we figure out a way to enable Edition in OSS Bazel build. [1]: https://github.com/google/mozc/blob/master/src/protocol/renderer_command.proto *** Original change description *** Migrate C++ protos to Editions (go/editions) WARNING: We have determined that these protos are only used in C++. If this is incorrect, or if you plan on using them in other languages within the next few months, a clean rollback should be safe. Please notify us if you hit any issues. This change has been verified to be behavior preserving both in the generated code and on the wire. The only potential changes are for code that uses syntax-dependent reflection. We've already migrated most of... *** PiperOrigin-RevId: 586334075

Migrate C++ protos to Editions (go/editions) WARNING: We have determined that these protos are only used in C++. If this is incorrect, or if you plan on using them in other languages within the next few months, a clean rollback should be safe. Please notify us if you hit any issues. PiperOrigin-RevId: 586135894

2023-11-27

Fix typos #codehealth PiperOrigin-RevId: 585552975

2023-11-22

Update build rules for `composer/` PiperOrigin-RevId: 584588764

2023-11-14

Add a feature to filter noisy number candidates PiperOrigin-RevId: 582222424

2023-11-10

Fix the cost for realtime conversion result based on the ranking. Background: User segment history rewriter reranks candidates without changing their cost, and mobile dictionary predictor sorts candidates based on their cost, so the reranking was not reflected. PiperOrigin-RevId: 581093367

2023-11-01

Add a feature flag to change top candidates number for findability oriented order. PiperOrigin-RevId: 578396483

2023-10-26

Add priority to EngineReloadRequest As commented in engine_builder.proto, this CL introduces `priority:int32` field to `EngineReloadRequest` message. The decoder will use the most prioritized LM. The client impl determine the priority policy. In this CL, from prioritized source, MDD -> Superpacks -> .apk embedded -> system preloaded. AI: Record metrics about the data source PiperOrigin-RevId: 576843937

Add a feature to enable findability oriented candidate order. PiperOrigin-RevId: 576789852

2023-10-24

Automated g4 rollback of changelist 575979626. *** Reason for rollback *** This rollback was for cherry-picking, and now it can be rollbacked again *** Original change description *** Automated g4 rollback of changelist 575072274. *** Reason for rollback *** This CL causes conflict when rollbacking cl/568386718. To resolve the conflict, we have to first rollback this CL. This rollback will be rollbacked again after issues are resolved. *** Original change description *** Fix problem that NO_POS is not working in Android Gboard - The cause of the bug was that while user_pos_*.data is in the dictionary side, Android Gboard did not have an updated version of dictionary dat... *** PiperOrigin-RevId: 576057664

Support Emoji 15.1 in logic side implementation. Emoji 15.1 is released in this Sep 2023, and we have to update implementation for this latest version. Note that this CL only updates logic implementation. In order to add new emojis, we have to re-generate emoji data for Mozc based on the latest unicode data. PiperOrigin-RevId: 576025120

Automated g4 rollback of changelist 575072274. *** Reason for rollback *** This CL causes conflict when rollbacking cl/568386718. To resolve the conflict, we have to first rollback this CL. This rollback will be rollbacked again after issues are resolved. *** Original change description *** Fix problem that NO_POS is not working in Android Gboard - The cause of the bug was that while user_pos_*.data is in the dictionary side, Android Gboard did not have an updated version of dictionary data. As dictionary is separate from engine, there is no guarantee that '品詞なし' user pos is available in the specific dictionary. To avoid such invalid behavior, I added special treatment for NO_POS in user_dictionary Load function. By using this implementation they will work as expected even... *** PiperOrigin-RevId: 575979626