Hey Marclean,

I am retiring sorry. Thanks for all your support and kind words through the years!

I am proud to have accompanied you in these years, I can say: ‘"I saw all this happen !!"
Its sad u retimerant but i understand
good rest in your retiriment (I hope you are open to consultations)

Marcelo

Hello

I am trying to use the n-grams provided with the AZDecrypt package.
I want to use with another decrypter I made myself (in Python) specialized for homophones.
( want to do better than with the 5-grams file I got from the "practical cryptography" site : http://practicalcryptography.com/cryptanalysis/ )
It seems after decompressing the files, you get some kind of binary file. The format is adapted for the AZDecrypt software I reckon.

I need it in clear form :
AZERT 236764
QSDFF 236764
(or with floats instead of ints)

Any clue on how to do this ?
(I would prefer to avoid the trouble of hacking the AZDecrypt software et make it write the file after extraction)
Regards

Ok, sorry, I found by myself, from a help text file in AZDecrypt package.
So I made a program to extract them.
Frequencies logarithms are coded in 0-255 range. I apply exp and multiply by 1000.
(You have to be careful because of the greed of memory that implies for n get higher.)
All is fine

For the 6 grams, the top are :

BECAUS 2150
METHIN 2108
SOMETH 2100
OMETHI 2100
WITHTH 2084
THATTH 2084
CTUALL 2084

regards

AZdecrypt 1.21 released: https://drive.google.com/file/d/1OMgHA-bm06adGossu4bMu2tcdgzTe3N4/view?usp=sharing

I’m slowly working on creating a solver that can crack the Z340 automatically.

That would be rather easy to do if one targets the Z340’s transposition scheme directly, but Zodiac could have picked another transposition scheme entirely.

For that reason it is preferred to have a more general and universal approach. My idea was to create a periodic transposition solver. If you come to think of it is quite logical since almost every systematic transposition scheme you can think of can be summarized periodically! Though a big hurdle would be to figure out the exact distribution of periodicity itself.

This is exactly the problem we faced with the Z340. We were relatively sure it was a period 19 cipher because of the bigram repeats but we could not solve it as one. Because we weren’t sure how the period 19 was exactly distributed. It turns out that there are many minor additional periods as well because of some of the elements in the cipher.

This is the transposition matrix of the Z340:

1   10  19  28  37  46  55  64  73  82  91  100 109 118 127 136 145
137 146 2   11  20  29  38  47  56  65  74  83  92  101 110 119 128
120 129 138 147 3   12  21  30  39  48  57  66  75  84  93  102 111
103 112 121 130 139 148 4   13  22  31  40  49  58  67  76  85  94
86  95  104 113 122 131 140 149 5   14  23  32  41  50  59  68  77
69  78  87  96  105 114 123 132 141 150 6   15  24  33  42  51  60
52  61  70  79  88  97  106 115 124 133 142 151 7   16  25  34  43
35  44  53  62  71  80  89  98  107 116 125 134 143 152 8   17  26
18  27  36  45  54  63  72  81  90  99  108 117 126 135 144 153 9
154 163 172 181 190 199 208 217 226 235 244 301 302 303 304 305 306
285 293 155 164 173 182 191 200 209 218 227 236 245 253 261 269 277
270 278 286 294 156 165 174 183 192 201 210 219 228 237 246 254 262
255 263 271 279 287 295 157 166 175 184 193 202 211 220 229 238 247
239 248 256 264 272 280 288 296 158 167 176 185 194 203 212 221 230
222 231 240 257 265 273 281 289 297 159 168 177 186 195 204 213 249
205 214 223 232 241 250 258 266 274 282 290 298 160 169 178 187 196
188 197 206 215 224 233 242 251 259 267 275 283 291 299 161 170 179
171 180 189 198 207 216 225 234 243 252 260 268 276 284 292 300 162
310 309 308 307 311 312 313 314 316 315 318 317 319 320 321 322 325
324 323 327 326 335 334 333 332 331 330 329 328 336 337 338 339 340

We can use the new AZdecrypt 1.21 to extract the periodicity using the following steps:

1) Put the matrix in the Input window

2) Click on Format, Convert …, Into period map 1 to length

3) Click on Stats, Unigrams

Now we get the following periods:

19: 210
2: 31
-134: 24
1: 15
-1: 15
18: 13
20: 11
3: 4
-115: 4
-151: 2
-132: 2
4: 2
32: 1
6: 1
-140: 1
140: 1
9: 1
8: 1

Also note in the unigram stats that the Entropy is 2.089 and the normalized entropy which is 0.2486. The normalized entropy (what I called it) is simply the entropy expressed into a 0 to 1 range where 1 is maximal. A normalized entropy of 0.2486 simply means that the periodicity of the Z340 is about 75% redundant. A normalized entropy of 0.5 would mean the periodicity is about 50% redundant and so on.

So now we have an idea of how complicated the transposition of the Z340 is and what kind of search space we have to take on. And it basically could be any collection of periods (that make up the transposition scheme) that would equate to about a 75% redundancy.

In the previous post I explained that it would be a big hurdle to figure out the distribution of the periods itself. But possibly, also on top of that, the periods themselves have to figured out by a solver.

In AZdecrypt 1.21 the Periodic transposition (automatic) solver does exactly that. But for now only works on plaintext ciphers, moving up to homophonic substitution is planned but will be quite a challenge.

You can try it on the Z340 in untransposed plaintext form:

IRONCAOOIIERGRTML
ECHETTATNWNNIAABW
EITEOHSRTWTWGTAIS
DCCLOAPAOYCAHHOAM
BNOHALPLEVFIHSEIU
CPOOFAASALYIFNMNT
TVHAUTTMSERTONAGE
TTMSBPTAHBENAHUGN
LIOHEHROMFEEIDDEA
SAASOHOSHACLIFEIS
SHOUVLRENNECEROAA
MIEAOSEAVREONHSEF
ADNDITHEEVFEETTPO
ATFEOBVMEENEOELHH
IERRATENYRNOSRVSH
EENYAEATACONBOUTM
OERHGRDYIHFAWEEWG
HHWWYAWEIADIRUTWC
EFILWILLEBNAEASYE
NONIECIDARAPDEATH

Preferably load up 6-grams or better.

There’s a huge power factor the solver it took months to get it where it is now. A few notes:

1) Works better on smaller cipher since the search space is naturally smaller. 340 characters seems to be quite a stretch.

2) The Z340’s plaintext has issues and the solver may make trade-offs and the solution will not be perfect.

3) The Z340’s transposition matrix has issues and some of the minor periods have a very small periodic signature and the solver may make trade-offs as well.

4) A look-up table needs to be made for the solver to be performant, it may take a long while (minutes) and longer for longer ciphers.

5) The search space the solver works in naturally expands with the higher n-gram sizes. I’m figuring that up to about 60% periodic redundancy is easily possible (see previous post about that calculation).

6) Solver wants fast PC with lots of threads/cores.

7) Will solve ANY scheme as long as it is sufficiently periodic redundant!

Here’s a result from the cipher above with 7-grams:

Hill climber: 35.88M/832
Score: 23412.42 IOC: 0.0667 Multiplicity: 0.0617 Minutes: 36.51
Repeats: ESOMEONE NOTHING AFRAID CAUSE PLAC ETHE EALL ANDI

Periodic entropy: 2.83046

PAPOINT ABOUT ME I AM NOT AFRAID OF THEIR 
LIVES ID BE AFRAID AS MADE THE HAS NOTHING 
WHEN THEY REALLOWED TO PAY WHAT ARE THE PLACE 
SOMEONE EACH PLACE ALL THE AL BE AND I CAN 
CAUSE I HAVE EVERYONE ELSE NOW THAT TRADING 
AS CHAMBER BECAUSE OF IT WILL SEE SOMEONE 
IS TRYING LOTS OFF AND I WISH TO CATCH YOUR 
NEW SHOW WHICH BRING OUT THAT WASNT EVEN 
BECOME I HOPE YOU ARE A FAME WHERE FOR THEY 
ARE HAVING SO FAR AND IM NOTHING FOR VVE

Note that while it is imperfect it will get the message out and you will be able to work your way from here manually.

Transposition matrices are also output, in numeric and visual formats.

Great to see you in action again @Jarvle!

I imagine you are working around AZD , now , more for the technical challenge , designing the program to solve the transposed ciphers in various ways.

Always looking out for your work !!!
ps: could you put something in the program that we could test the keys applied on z408 and z340 ( symbols and the corresponding letter ) , on Z13 and Z32 ?
Going further, some ways to work on these letters ( vigenere, cesar cipher, rout 13, etc)
I usually do this manually, with your program, various sites)

congratulations , you are brilliant

Thanks Marclean,

Nice to see you around. Though there seems to be not much action here after the forum revamp?

I was thinking of doing something around Z32, testing the Z340 and Z408 keys as you say, in a solver, not sure if I’ll ever get there. For what it’s worth, if you download the program and go to the Ciphers/Zodiac ciphers directory there are Z32 with 340 and 408 key. I’ll post them here:

Z32_340:

YNSETAI1DOBP2AORT
RDFNOPTIYRGAIWN

Z32_408:

1IFELNIOWHDA2NGOA
OESNBX3T1ETDIEI

There has to be something to the Z32.

I personally think Z13 is solved as Alfred E. Neum(f)an.

It is encouraging to see you working on this!
z32 must have its meaning .
thanks for the keys !!!

AZdecrypt 1.22 release: https://drive.google.com/file/d/12ngl8-_hd7EvofHRKkNB7VtBNBceB2Tl/view?usp=share_link

New features:

– Ciphers up to 4000 characters long, up from 2000

– Non-substitution Nulls and skips solver

– Automatic word spacing and dedicated n-grams for the following latin alphabet languages: Danish, Dutch, English, Finnish, French, German, Italian, Portuguese, Spanish and Swedish

– Chi-squared test for the languages above as well, (under Statistics)

– N-gram tools to convert/output existing n-grams to text or binary format. Plus downscaling of existing n-grams, from letter 5-grams to letter 4-grams etc (under File, N-gram tools)

– “Substitution + crib grid” can now be used in junction with “Substitution + simple transposistion”. Have the pop-up windows of both solvers open and enable the “Use Substitution + crib grid” option in the Simple transposition solver window to have it work.

– Homophone weight setting (under Settings, Solver settings) that works for all solvers that output the number of homophones in the Output window, and the latter being a new addition as well.

– Added some error checking for loading n-grams, possible problems will show up as *** WARNINGS ***

– Lots of bugfixes, small additions and improvements

Posted by: @jarlve

↑

If you find any bugs or have suggestions I would greatly appreciate the feedback.

What development language is used for the decoder? Java? C+? C#? Python?

FreeBASIC: https://www.freebasic.net/

It’s almost fully Qbasic compatible with additional features. Internally it translates the BASIC to C code which is then optimized by GCC.