Skip to content

Commit aca8f86

Browse files
john-prestonjohn-preston
committed
Score Cyrillic passcodes and bound the estimator input
Task: 2026/09/03/score-non-latin-passcodes-and-bound-the-candidate
1 parent f408ae3 commit aca8f86

10 files changed

Lines changed: 100615 additions & 10 deletions

File tree

‎.gitattributes‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,5 +5,8 @@
55
*.diff text eol=lf
66
*.bat text eol=crlf
77

8+
# Keep the zxcvbn word lists byte-identical to upstream so their digests hold
9+
Telegram/Resources/zxcvbn/*.txt text eol=lf
10+
811
# Ensure lottie animations are treated as binary files
912
*.lottie binary

‎Telegram/Resources/zxcvbn/README.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
# zxcvbn frequency lists
2+
3+
The two word lists in this directory are unmodified copies of third-party
4+
frequency lists. They are read only at build time: the rules in
5+
`Telegram/cmake/lib_zxcvbn.cmake` transliterate them and compile the result
6+
into the `zxcvbn-c` password dictionary behind
7+
`Telegram/SourceFiles/ui/passcode_strength.cpp`. No built artifact contains
8+
these files: they are build-time inputs to a generated header. They are
9+
tracked, so every source distribution of Telegram Desktop ships them
10+
verbatim, and the attribution below covers those copies as well as the
11+
dictionary compiled from them.
12+
13+
## Provenance
14+
15+
- Title: *FrequencyWords*
16+
- Creator: Hermit Dave
17+
- Repository: <https://github.com/hermitdave/FrequencyWords>
18+
- Upstream commit: `f8a65e6ddc17e0baa2e366a909986798d8dbe55b` (2019-02-15,
19+
"Additional languages including pl, pt and zh variants", the last upstream
20+
commit touching either file)
21+
- Derivation: generated by the upstream author from the OpenSubtitles2018
22+
corpus (<http://opus.nlpl.eu/OpenSubtitles2018.php>); one `word count`
23+
pair per line, separated by a single space, in descending order of count;
24+
50,000 lines each, all lowercase, LF line endings, no BOM.
25+
26+
| Tracked file | Upstream path | Bytes | Lines |
27+
| --- | --- | --- | --- |
28+
| `ru_50k.txt` | `content/2018/ru/ru_50k.txt` | 998,861 | 50,000 |
29+
| `uk_50k.txt` | `content/2018/uk/uk_50k.txt` | 887,259 | 50,000 |
30+
31+
sha256 digests, equal to the GitHub raw files at the pinned commit (verified
32+
2026-09-06):
33+
34+
```
35+
6095f507cc167488ec66ada5a85ac50433503a08ad24a07c6eabdf54352c4e7f ru_50k.txt
36+
881686fe66a168b79f6a9ee2f6cbbf944cfb12bfb7040913ef535cb29bd86ec9 uk_50k.txt
37+
```
38+
39+
`.gitattributes` pins `Telegram/Resources/zxcvbn/*.txt` to `text eol=lf`, so
40+
the digests hold on a Windows checkout as well.
41+
42+
## Licence
43+
44+
Upstream's `README.md` states, under "License", at the upstream head
45+
`525f9b560de45753a5ea01069454e72e9aa541c6` (2022-02-07) — head on
46+
2026-09-06, the day these copies were taken:
47+
48+
> MIT License for code.
49+
> CC-by-sa-4.0 for content.
50+
51+
The licence *version* in that statement has changed exactly once. At the
52+
pinned commit `f8a65e6ddc17e0baa2e366a909986798d8dbe55b` (2019-02-15, the
53+
only commit that ever touched either list) the section read
54+
`MIT License for codes.<br>` / `CC-by-sa-3.0 for contents.`; commit
55+
`0c6a7e40cfd8c645f9e91ff70cd94d7aa8c85c09` (2019-02-15) reworded it to
56+
`MIT License for code.<br>` / `CC-by-sa-3.0 for content.`, leaving the
57+
version at 3.0; and commit `34bb6447a96d4434dc2d4b24e0cf946c523090b3`
58+
(2020-08-11, "Update README.md") replaced `CC-by-sa-3.0` with
59+
`CC-by-sa-4.0` in that one line, since when the section has read as quoted
60+
above.
61+
62+
The word lists are content, not code, and the licence that governs these
63+
copies is the Creative Commons Attribution-ShareAlike 4.0 International
64+
licence (CC BY-SA 4.0), <https://creativecommons.org/licenses/by-sa/4.0/>,
65+
through the licensor's standing offer at head — not through the pin, and
66+
not merely because the bytes are unchanged. A public licence is an offer
67+
made when the material is made available, and a licensor may add a new offer
68+
over unchanged files without withdrawing the old one (CC BY-SA 3.0 §7(b),
69+
CC BY-SA 4.0 §6(c)). The pin fixes which bytes are tracked, and the digests
70+
above stay pinned to it; it does not fix which offer they were taken under.
71+
72+
The earlier 3.0 offer is perpetual and remains available, but it is not
73+
relied on here. Were the 4.0 offer ever disputed, 3.0 alone would still
74+
reach CC BY-SA 4.0, because 3.0 §4(b)(ii) permits distributing an Adaptation
75+
under "a later version of this License with the same License Elements"; it
76+
would not reach the GNU GPL directly, because Creative Commons records that
77+
"Currently, no non-CC licenses have been designated as compatible with BY-SA
78+
3.0".
79+
80+
The repository's `LICENSE` file (MIT, "Copyright (c) 2016 Hermit Dave")
81+
covers the generator code only. Upstream supplies no separate copyright or
82+
disclaimer notice for the content beyond the licence line quoted above.
83+
84+
## Changes made
85+
86+
The lists are used as build-time input only: the word column is kept, each
87+
word is transliterated to Latin letters by `zxcvbn_translit`, words that are
88+
still non-ASCII or shorter than two letters are dropped, and the result is
89+
compiled into the zxcvbn dictionary; the tracked files are unmodified copies.
90+
The count column only orders the words: the generated dictionary keeps a
91+
word's rank, not its count.
92+
93+
## Compatibility with the application licence
94+
95+
Telegram Desktop is distributed under the GNU General Public License, version
96+
3 or (at your option) any later version (`/LICENSE`, `/LEGAL`), with the
97+
OpenSSL linking exception. CC BY-SA 4.0 content may be adapted into a GPLv3
98+
project on this basis:
99+
100+
- Creative Commons, "Compatible Licenses",
101+
<https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/>:
102+
"GPLv3: The GNU General Public License version 3 was declared a
103+
'BY-SA–Compatible License' for version 4.0 …" and "Note that compatibility
104+
with the GPLv3 is one-way only, which means you may license your
105+
contributions to adaptations of BY-SA 4.0 materials under GPLv3, but you
106+
may not license your contributions to adaptations of GPLv3 projects under
107+
BY-SA 4.0." (declared 2015-10-08).
108+
- Creative Commons wiki, "ShareAlike compatibility: GPLv3",
109+
<https://wiki.creativecommons.org/wiki/ShareAlike_compatibility:_GPLv3>:
110+
"When someone creates an adaptation of a BY-SA licensed work and includes
111+
it in a GPLv3-licensed project, both licenses apply and downstream users
112+
must comply with both licenses. However, Section 2(a)(5)(B) of BY-SA 4.0
113+
allows anyone who receives the adapted material downstream to satisfy the
114+
conditions of both BY-SA and GPLv3 (i.e. attribution and ShareAlike) in
115+
the manner dictated by the GPLv3." and "this compatibility determination
116+
only allows movement one way, from BY-SA 4.0 to GPLv3."
117+
- Free Software Foundation, "Various Licenses and Comments about Them",
118+
<https://www.gnu.org/licenses/license-list.html#ccbysa>: "CC BY-SA 4.0 is
119+
one-way compatible with the GNU GPL version 3: this means you may license
120+
your modified versions of CC BY-SA 4.0 materials under GNU GPL version 3,
121+
but you may not relicense GPL 3 licensed works under CC BY-SA 4.0. Because
122+
Creative Commons lists only version 3 of the GNU GPL on its compatible
123+
licenses list, it means that you can not license your adapted CC BY-SA
124+
works under the terms of 'GNU GPL version 3, or (at your option) any later
125+
version.' However, Section 14 of the GNU GPL version 3 allows licensors to
126+
specify a proxy to determine whether future versions of the GNU GPL can be
127+
used. Therefore, if someone adapts a CC BY-SA 4.0 work and incorporates it
128+
into a GNU GPL version 3 licensed project, they can specify Creative
129+
Commons as their proxy …"
130+
131+
**Consequence for this repository — an owner decision this file records but
132+
cannot make.** Creative Commons declared only GPL version 3 a
133+
BY-SA-Compatible License, and the wiki page quoted above instructs adapters:
134+
"You must specify version 3 of the GPL when redistributing a GPL project
135+
into which BY-SA 4.0 content is adapted." Telegram Desktop's own offer is
136+
broader: `/LEGAL` — and every source file header that points at it — says
137+
"either version 3 of the License, or (at your option) any later version".
138+
Once the dictionary compiled from this corpus is linked in, that broader
139+
offer cannot be made for the combined work unless the Program itself names a
140+
proxy under GPL §14 ("If the Program specifies that a proxy can decide which
141+
future versions of the GNU General Public License can be used…"). This file
142+
is not the Program's licence statement and cannot make that designation. The
143+
repository owner has to pick one:
144+
145+
- (a) add a GPL §14 proxy designation naming Creative Commons
146+
(<https://creativecommons.org/compatiblelicenses>) to `/LEGAL`, after
147+
which "or any later version" works again for the combined work as and when
148+
Creative Commons declares a later GPL version compatible;
149+
- (b) accept that builds containing this dictionary are offered under GPL
150+
version 3 only, and qualify `/LEGAL` accordingly;
151+
- (c) drop this corpus — no BY-SA content, no constraint.
152+
153+
Until (a) or (b) is recorded in `/LEGAL`, this component and the tree's "or
154+
any later version" offer are in tension. The in-app statement,
155+
`lng_about_text2`, already reads "licensed under the GNU GPL version 3". The
156+
tracked lists themselves remain verbatim copies under CC BY-SA 4.0.
157+
158+
## Reproducing the dictionary
159+
160+
The build performs these steps itself; run them by hand from the checkout
161+
root after a Debug build to check that the generated header is reproducible.
162+
`<dir>` is any scratch directory outside the checkout.
163+
164+
```
165+
out/Telegram/Debug/zxcvbn_translit \
166+
Telegram/Resources/zxcvbn/ru_50k.txt <dir>/ru_50k-latin.txt
167+
out/Telegram/Debug/zxcvbn_translit \
168+
Telegram/Resources/zxcvbn/uk_50k.txt <dir>/uk_50k-latin.txt
169+
out/Telegram/Debug/zxcvbn_dictgen -o <dir>/dict-src.h \
170+
Telegram/ThirdParty/zxcvbn/words-eng_wiki.txt \
171+
Telegram/ThirdParty/zxcvbn/words-female.txt \
172+
Telegram/ThirdParty/zxcvbn/words-male.txt \
173+
Telegram/ThirdParty/zxcvbn/words-passwd.txt \
174+
Telegram/ThirdParty/zxcvbn/words-surname.txt \
175+
Telegram/ThirdParty/zxcvbn/words-tv_film.txt \
176+
<dir>/ru_50k-latin.txt \
177+
<dir>/uk_50k-latin.txt
178+
cmp <dir>/dict-src.h out/Telegram/zxcvbn_gen/dict-src.h
179+
```
180+
181+
The input order matters: the generator interleaves its inputs by rank, so
182+
the six upstream lists come first, in the order `lib_zxcvbn.cmake` lists
183+
them, followed by the Russian and then the Ukrainian intermediate. `cmp`
184+
must report no difference.

0 commit comments

Comments
 (0)