@smokie,
Your last table/image is for the cipher I gave you right? How does it looks for the 340?
Since smokie works with numbers in his spreadsheet I try to use the original mapping.
Also, when you add new symbols to the cipher alphabets, are you starting the numbering of new symbols at 64?
You have my vote on that.
Jarlve: The last image is for the 340; I edited the post to reflect that.
Doranchak: When using the 340, I always use the same numbers for symbols.
The first symbol for any message that I make is not necessarily a 1. The first word could be ZODIAC, represented by 3 12 63 14 21 30, where 3 maps to Z, 12 maps to O, 63 maps to D, etc.
Sometimes I make new symbols starting with 64-99 because my position formulas may require that. Sometimes I combine two or more pieces of information into one cell for sorting or ordering purposes, such as 185.12. That would mean position 185, symbol 12. Sometimes I make null symbols 100, or start numbering new symbols from 101, 201, if for a special purpose.
Mr. Lowe: You have mentioned rail cipher a few times. I spent a few minutes experimenting with that some time ago, and found it difficult to make a message with a period of 19. With rail ciphers, the period varies. Try it.
EDIT: Jarlve, I have a minor issue with my spreadsheet that adds symbols to test for whether the 340 is a period 20 route transposition with a skipped symbol at intervals. Currently the spreadsheet uses a random number from between 64 and 99 for added symbols. But that has created a few improbable repeats that skew the score. I will have to get that fixed.
smokie.. Hope all is good and you are running at 100% again. in answer to my mentioning of the rail cipher a few times I did a bit of work on it about 20 pages back but it was not as convincing as the scytale, but it was still pretty easy to write out. From memory I did get the period 19s back to a period 1? (to be confirmed) by writing it out and missing a line, then coming back to the second row it became an odds and evens type set up… I will revisit it in the next month and put some effort into seeing if it works and report back to all.. I am looking forward to doranchacs book arriving soon and hoping it holds a clue or two.
Keep up the good work guys n girls..
I C G M I U E C E E P F I T E N L S E T
N O L A P U K S O H K I L U L I B I I S
It reads I like killing people because killing people is so much fun it. This is in rail fence cipher. There are a few different permutations that one could throw up, as in starting point etc. It also fits the 19s nicely.and the 15s I think. Two rows needed then start again..
Edit..it did not come out evenly spaced when I posted it so it looks wrong but it’s ok..
Just realized I made it 20 periods wide.
Hope you can follow this one Blue turns to brown turns to yellow turns to green
Mr lowe I looked at your scheme. I like it because it somewhat randomizes rows in pairs of two which gives a good approximation to the general encoding randomness of the 340 (if applied after encoding). Haven’t tested all variations of your idea though I did make a test cipher using your transposition and while it looks good for the cycles the bigrams do not conform.
Test cipher with Mr lowe's railfence transposition: 1 2 3 1 4 5 6 7 8 9 10 11 12 13 14 5 15 16 17 18 19 20 21 22 23 24 25 22 26 6 27 28 29 30 31 3 32 1 9 16 33 34 4 28 35 36 12 37 38 24 39 40 41 33 22 8 15 37 18 27 20 42 29 43 11 26 44 45 4 8 21 11 46 45 2 21 30 12 5 34 10 42 9 37 32 19 24 35 39 17 25 31 3 13 43 33 26 23 36 47 38 7 44 25 45 17 47 29 10 1 27 14 36 20 23 40 28 4 16 39 19 30 22 14 31 40 15 33 3 7 26 18 44 35 11 46 48 16 7 35 21 20 46 8 8 49 32 17 38 19 18 26 11 10 13 13 37 43 5 39 30 31 29 42 6 12 1 24 47 28 24 4 46 5 30 20 23 21 2 25 44 33 42 47 17 41 13 32 40 48 35 38 22 3 50 37 10 19 9 8 33 43 32 42 12 42 24 23 31 33 30 41 44 43 17 19 47 26 36 4 46 38 31 43 25 2 17 45 29 7 38 16 1 6 3 15 46 32 34 6 47 3 13 47 2 45 41 34 32 15 14 5 1 31 2 8 23 12 45 24 34 16 36 40 28 29 4 37 42 18 30 35 19 5 8 48 38 39 41 7 48 11 28 41 22 21 2 18 14 18 23 12 10 34 19 36 30 29 1 13 50 15 4 16 14 24 28 24 27 41 25 36 50 15 20 26 45 46 48 50 8 50 50 40 27 37 5 39 16 43 9 35 14 12 30 11 32 13 10 2 ,X;,(:"?T%*YZR9:& B1J7G=)UK3)@"O2A C;>,%BN-(26FZWEKP [4N)T&WJOG<A!Y@+8 (T=Y58X=Z:-*<%W> 7K6P13C;R!N@UFME? +381MA*,O9FGU[2(B P7)9C[&N;?@J+6Y5 HB?6=G5TT$>1E7J@Y *RRW!:PCA<"Z,KM2 K(5:GU=X3+N<M14R >[H6E);.W*7%TN!>< Z<KUCN4+!17M@F(5 EC!3X18A?EB,";&5> -"M;RMX84->&9:,CX TUZ8K-BF[2A(W<J6 7:THEP4?HY24)=XJ9 JUZ*-7FA,R.&(B9K 2KO43F.&G@85H.T.. [OW:PB!%69ZY>R*X Mr lowe's railfence transposition matrix: 1 19 11 28 3 21 13 30 5 23 15 32 7 25 17 34 9 10 27 2 20 12 29 4 22 14 31 6 24 16 33 8 26 18 35 53 45 62 37 55 47 64 39 57 49 66 41 59 51 68 43 44 61 36 54 46 63 38 56 48 65 40 58 50 67 42 60 52 69 87 79 96 71 89 81 98 73 91 83 100 75 93 85 102 77 78 95 70 88 80 97 72 90 82 99 74 92 84 101 76 94 86 103 121 113 130 105 123 115 132 107 125 117 134 109 127 119 136 111 112 129 104 122 114 131 106 124 116 133 108 126 118 135 110 128 120 137 155 147 164 139 157 149 166 141 159 151 168 143 161 153 170 145 146 163 138 156 148 165 140 158 150 167 142 160 152 169 144 162 154 171 189 181 198 173 191 183 200 175 193 185 202 177 195 187 204 179 180 197 172 190 182 199 174 192 184 201 176 194 186 203 178 196 188 205 223 215 232 207 225 217 234 209 227 219 236 211 229 221 238 213 214 231 206 224 216 233 208 226 218 235 210 228 220 237 212 230 222 239 257 249 266 241 259 251 268 243 261 253 270 245 263 255 272 247 248 265 240 258 250 267 242 260 252 269 244 262 254 271 246 264 256 273 291 283 300 275 293 285 302 277 295 287 304 279 297 289 306 281 282 299 274 292 284 301 276 294 286 303 278 296 288 305 280 298 290 307 325 317 334 309 327 319 336 311 329 321 338 313 331 323 340 315 316 333 308 326 318 335 310 328 320 337 312 330 322 339 314 332 324
Jarlve & Smokie, thanks for clarifying about the numerical representation of your ciphers.
What I’m getting at is that we could use a stronger plaintext measurement (bigrams, repeating fragments). Smokie, I would like to test out your bigram measurement. Could you explain your method again so that I can add it to my program? Thanks.
I was recently thinking about ways to measure the repeating fragments. One problem is that some fragments are contained in others (a repeating trigram, for example, will appear as two sets of repeating bigrams). This means some fragments are over-counted in a measurement. Another problem is how to measure how much better one set of repeating fragments is than another.
I tried a different approach. For each of the 340 positions in the cipher text, I record the probability of the rarest repeating fragment that the symbol in that position appears in. This can act as a foundation for a heat map style plot, showing regions of the cipher text that tend to be involved with interesting repeating fragments.
But I want to compare manipulated cipher texts against the original 340, to see how well the manipulation yields more fragments. I start with the grid of probabilities mentioned above. It gets converted to a list, then gets sorted by probability from lowest to highest. This gives you a curve that you can directly compare to similarly sorted values for the original 340. For example, the 10th spot in the list represents the position that is associated with the 10th best repeating fragment probability. In other words, the first 9 spots have equal or lower probability, and the remaining 330 spots have equal or higher probability.
Finally, when comparing differences in probabilities, which usually involve very different orders of magnitude, I apply a variation of sigmoid function to the differences in probabilities, to force the values to remain in a range from 0 to 1. Then I can add up those values across all 340 positions.
Not sure how mathematically sound this all is, but it gives me a sense of whether there is a higher number of "better" repeating fragments distributed through the manipulated cipher texts, compared to the original 340.
In my spreadsheet of transpositions, the measurement appears in the "frag faster" column: https://docs.google.com/spreadsheets/d/ … 1394933925
the rail system can go three or more rows as well but it gets messy.
And obviously it could have starting points other than top left and could be columns.
And out of left field I give you this.. Currently the thinking is we are dealing with 63 unique symbols.. If by chance it was written in rail scytale or other and each other row was written upside down how many unique symbols…. + is a + upside down as is N or O or an I, but a, m is a w and a q is a backwards q , as a d is a p etc .. This would bring the unique 63 count down considerably..is it possible to dissect the 340 to see if any particular rows inherent upside down traits… Sorry if this sounds a bit ambiguous but it does have an element of potential pattern about it for a "rail"or "scytale" if you decided to write every other row upside down…
Just thinking out loud ..sorry if it bothers.
Finally, when comparing differences in probabilities, which usually involve very different orders of magnitude, I apply a variation of sigmoid function to the differences in probabilities, to force the values to remain in a range from 0 to 1. Then I can add up those values across all 340 positions.
So a positive frag faster value represents a better score relative to the original 340? Could you come up with a period 1 to 170 graph for the 340 using your frag faster measurement?
I was recently thinking about ways to measure the repeating fragments. One problem is that some fragments are contained in others (a repeating trigram, for example, will appear as two sets of repeating bigrams). This means some fragments are over-counted in a measurement. Another problem is how to measure how much better one set of repeating fragments is than another.
Very true. I have another broader approach, the idea is pretty rough at the moment. Measure how close one transposition is versus another.
What we would like to find in the 340 or in any transposition cipher that is sufficiently measureable is a landscape that is more or less equal to what we would expect from a left-to-right, top-to-bottom plaintext. Generate two lists, one for the plaintext and one for the cipher and correlate the likeliness with something like a chi-squared test. These lists should include a full period transposition and untransposition table, for example period 1 to 339. (that is my current prefered transposition measurement technique and recommendation).
The idea is that the full range of periods represent transposition quite well. To adjust for uneven measurement counts (plaintext versus cipher) the difference from the average expected measurement value could be used (and I think needed to filter out noise).
I have an early version of this running with bigrams and here is an example. Lower score is better correlation.
P1 plaintext with period 19 transposition – versus – P2 cipher with period 19 transposition: 114. (closer)
P1 plaintext with period 19 transposition – versus – P2 cipher with period 18 transposition and column period 2: 129.
In both cases it is compared versus a 50 symbol cipher with the same symbol distribution. Since period 18 + column period 2 creates a period 19 peak it’s interesting to see that it does identify the regular period 19 transposition as closer, as it is. I think at least it could give us a rough approximation as to close one transposition is versus another. You could use any measurement, bigrams, cycles, frag faster.
Finally, when comparing differences in probabilities, which usually involve very different orders of magnitude, I apply a variation of sigmoid function to the differences in probabilities, to force the values to remain in a range from 0 to 1. Then I can add up those values across all 340 positions.
So a positive frag faster value represents a better score relative to the original 340? Could you come up with a period 1 to 170 graph for the 340 using your frag faster measurement?
Yes, positive = better than the original 340.
Here’s a spreadsheet of the results of the measurement across all periods, sorted from best improvement to worst:
https://docs.google.com/spreadsheets/d/ … sp=sharing
Interestingly, only a few result in net improvement (include our friend, Period 19). I’m a bit surprised that period 28 takes the top position.
You may want to look at the top results there and confirm if they really do seem to produce more repeating fragments overall than the original Z340. I’m not yet sure how effective my measurement is.
UPDATE: I aded a tab "Z408 (first 340 characters) periods" which runs the same measurements for all periods of the first 340 characters of Z408. It confirms that period 1 has the best result, which I suppose is what we would expect with a normally-enciphered substitution cipher.
Here are dumps of the fragments found during my measurement’s executions for periods 1, 19 and 28. For each, they are sorted by probability (lowest to highest):
Position, Min probability, Pattern, Start positions of occurrences Period 1: 125 2.589325E-11 #O???Y [22, 125] 126 2.589325E-11 #O???Y [22, 125] 127 2.589325E-11 #O???Y [22, 125] 128 2.589325E-11 #O???Y [22, 125] 129 2.589325E-11 #O???Y [22, 125] 130 2.589325E-11 #O???Y [22, 125] 22 2.589325E-11 #O???Y [22, 125] 23 2.589325E-11 #O???Y [22, 125] 24 2.589325E-11 #O???Y [22, 125] 25 2.589325E-11 #O???Y [22, 125] 26 2.589325E-11 #O???Y [22, 125] 27 2.589325E-11 #O???Y [22, 125] 195 4.1108127E-11 5?4?. [227, 195] 196 4.1108127E-11 5?4?. [227, 195] 197 4.1108127E-11 5?4?. [227, 195] 198 4.1108127E-11 5?4?. [227, 195] 199 4.1108127E-11 5?4?. [227, 195] 227 4.1108127E-11 5?4?. [227, 195] 228 4.1108127E-11 5?4?. [227, 195] 229 4.1108127E-11 5?4?. [227, 195] 230 4.1108127E-11 5?4?. [227, 195] 231 4.1108127E-11 5?4?. [227, 195] 317 5.8259814E-11 O?*?C [317, 91] 318 5.8259814E-11 O?*?C [317, 91] 319 5.8259814E-11 O?*?C [317, 91] 320 5.8259814E-11 O?*?C [317, 91] 321 5.8259814E-11 O?*?C [317, 91] 91 5.8259814E-11 O?*?C [317, 91] 92 5.8259814E-11 O?*?C [317, 91] 93 5.8259814E-11 O?*?C [317, 91] 94 5.8259814E-11 O?*?C [317, 91] 95 5.8259814E-11 O?*?C [317, 91] 120 1.0193137E-10 G2 [270, 14, 120] 121 1.0193137E-10 G2 [270, 14, 120] 14 1.0193137E-10 G2 [270, 14, 120] 15 1.0193137E-10 G2 [270, 14, 120] 270 1.0193137E-10 G2 [270, 14, 120] 271 1.0193137E-10 G2 [270, 14, 120] 184 2.5692576E-10 M??2?c [184, 268] 185 2.5692576E-10 M??2?c [184, 268] 186 2.5692576E-10 M??2?c [184, 268] 187 2.5692576E-10 M??2?c [184, 268] 188 2.5692576E-10 M??2?c [184, 268] 189 2.5692576E-10 M??2?c [184, 268] 268 2.5692576E-10 M??2?c [184, 268] 269 2.5692576E-10 M??2?c [184, 268] 272 2.5692576E-10 M??2?c [184, 268] 273 2.5692576E-10 M??2?c [184, 268] 212 3.1719227E-10 |5F [212, 76] 213 3.1719227E-10 |5F [212, 76] 214 3.1719227E-10 |5F [212, 76] 76 3.1719227E-10 |5F [212, 76] 77 3.1719227E-10 |5F [212, 76] 78 3.1719227E-10 |5F [212, 76] 145 9.32157E-10 FBc [145, 214] 146 9.32157E-10 FBc [145, 214] 147 9.32157E-10 FBc [145, 214] 215 9.32157E-10 FBc [145, 214] 216 9.32157E-10 FBc [145, 214] 16 1.087268E-9 G2???+ [270, 14] 17 1.087268E-9 G2???+ [270, 14] 18 1.087268E-9 G2???+ [270, 14] 19 1.087268E-9 G2???+ [270, 14] 274 1.087268E-9 G2???+ [270, 14] 275 1.087268E-9 G2???+ [270, 14] 166 1.1185886E-9 FB [145, 214, 166] 167 1.1185886E-9 FB [145, 214, 166] 200 1.9329212E-9 4??+?B [234, 197] 201 1.9329212E-9 4??+?B [234, 197] 202 1.9329212E-9 4??+?B [234, 197] 234 1.9329212E-9 4??+?B [234, 197] 235 1.9329212E-9 4??+?B [234, 197] 236 1.9329212E-9 4??+?B [234, 197] 237 1.9329212E-9 4??+?B [234, 197] 238 1.9329212E-9 4??+?B [234, 197] 239 1.9329212E-9 4??+?B [234, 197] 138 3.0694067E-9 M+ [70, 138, 38] 139 3.0694067E-9 M+ [70, 138, 38] 38 3.0694067E-9 M+ [70, 138, 38] 39 3.0694067E-9 M+ [70, 138, 38] 70 3.0694067E-9 M+ [70, 138, 38] 71 3.0694067E-9 M+ [70, 138, 38] 141 5.3692255E-9 +???FB [141, 210] 142 5.3692255E-9 +???FB [141, 210] 143 5.3692255E-9 +???FB [141, 210] 144 5.3692255E-9 +???FB [141, 210] 210 5.3692255E-9 +???FB [141, 210] 211 5.3692255E-9 +???FB [141, 210] 250 2.9932593E-8 UZ [250, 40] 251 2.9932593E-8 UZ [250, 40] 40 2.9932593E-8 UZ [250, 40] 41 2.9932593E-8 UZ [250, 40] 287 6.7348346E-8 )L [287, 45] 288 6.7348346E-8 )L [287, 45] 45 6.7348346E-8 )L [287, 45] 46 6.7348346E-8 )L [287, 45] 219 7.6627444E-8 8R [86, 219] 220 7.6627444E-8 8R [86, 219] 86 7.6627444E-8 8R [86, 219] 87 7.6627444E-8 8R [86, 219] 163 8.14915E-8 p7 [53, 163] 164 8.14915E-8 p7 [53, 163] 53 8.14915E-8 p7 [53, 163] 54 8.14915E-8 p7 [53, 163] 101 9.166856E-8 (# [101, 21] 102 9.166856E-8 (# [101, 21] 21 9.166856E-8 (# [101, 21] 303 9.166856E-8 () [44, 303] 304 9.166856E-8 () [44, 303] 44 9.166856E-8 () [44, 303] 289 1.2370695E-7 ++ [289, 236, 63] 290 1.2370695E-7 ++ [289, 236, 63] 63 1.2370695E-7 ++ [289, 236, 63] 64 1.2370695E-7 ++ [289, 236, 63] 80 1.7241175E-7 +& [200, 80] 81 1.7241175E-7 +& [200, 80] 327 2.2636524E-7 Np [17, 327] 328 2.2636524E-7 Np [17, 327] 168 2.6939338E-7 By [34, 167] 34 2.6939338E-7 By [34, 167] 35 2.6939338E-7 By [34, 167] 282 3.6667424E-7 Fl [282, 89] 283 3.6667424E-7 Fl [282, 89] 89 3.6667424E-7 Fl [282, 89] 90 3.6667424E-7 Fl [282, 89] 162 9.05461E-7 Op [263, 162] 263 9.05461E-7 Op [263, 162] 264 9.05461E-7 Op [263, 162] 171 2.758588E-6 +R [171, 64] 172 2.758588E-6 +R [171, 64] 65 2.758588E-6 +R [171, 64] 62 4.310294E-6 O+ [126, 62] Period 19: 219 3.2254564E-14 9^?OF [219, 298] 220 3.2254564E-14 9^?OF [219, 298] 221 3.2254564E-14 9^?OF [219, 298] 222 3.2254564E-14 9^?OF [219, 298] 223 3.2254564E-14 9^?OF [219, 298] 298 3.2254564E-14 9^?OF [219, 298] 299 3.2254564E-14 9^?OF [219, 298] 300 3.2254564E-14 9^?OF [219, 298] 301 3.2254564E-14 9^?OF [219, 298] 302 3.2254564E-14 9^?OF [219, 298] 228 1.887618E-12 Xz6 [311, 228] 229 1.887618E-12 Xz6 [311, 228] 230 1.887618E-12 Xz6 [311, 228] 311 1.887618E-12 Xz6 [311, 228] 312 1.887618E-12 Xz6 [311, 228] 313 1.887618E-12 Xz6 [311, 228] 142 8.948706E-12 <S [142, 266, 315] 143 8.948706E-12 <S [142, 266, 315] 266 8.948706E-12 <S [142, 266, 315] 267 8.948706E-12 <S [142, 266, 315] 315 8.948706E-12 <S [142, 266, 315] 316 8.948706E-12 <S [142, 266, 315] 144 1.0277032E-11 P?L??5 [144, 58] 145 1.0277032E-11 P?L??5 [144, 58] 146 1.0277032E-11 P?L??5 [144, 58] 147 1.0277032E-11 P?L??5 [144, 58] 148 1.0277032E-11 P?L??5 [144, 58] 149 1.0277032E-11 P?L??5 [144, 58] 58 1.0277032E-11 P?L??5 [144, 58] 59 1.0277032E-11 P?L??5 [144, 58] 60 1.0277032E-11 P?L??5 [144, 58] 61 1.0277032E-11 P?L??5 [144, 58] 62 1.0277032E-11 P?L??5 [144, 58] 63 1.0277032E-11 P?L??5 [144, 58] 166 2.7200981E-11 p+ [254, 189, 166, 75] 167 2.7200981E-11 p+ [254, 189, 166, 75] 189 2.7200981E-11 p+ [254, 189, 166, 75] 190 2.7200981E-11 p+ [254, 189, 166, 75] 254 2.7200981E-11 p+ [254, 189, 166, 75] 255 2.7200981E-11 p+ [254, 189, 166, 75] 75 2.7200981E-11 p+ [254, 189, 166, 75] 76 2.7200981E-11 p+ [254, 189, 166, 75] 11 5.898806E-11 #2 [204, 131, 11] 12 5.898806E-11 #2 [204, 131, 11] 131 5.898806E-11 #2 [204, 131, 11] 132 5.898806E-11 #2 [204, 131, 11] 204 5.898806E-11 #2 [204, 131, 11] 205 5.898806E-11 #2 [204, 131, 11] 10 3.355766E-10 V??+?# [6, 126] 126 3.355766E-10 V??+?# [6, 126] 127 3.355766E-10 V??+?# [6, 126] 128 3.355766E-10 V??+?# [6, 126] 129 3.355766E-10 V??+?# [6, 126] 130 3.355766E-10 V??+?# [6, 126] 6 3.355766E-10 V??+?# [6, 126] 7 3.355766E-10 V??+?# [6, 126] 8 3.355766E-10 V??+?# [6, 126] 9 3.355766E-10 V??+?# [6, 126] 116 9.32157E-10 +|T [116, 156] 117 9.32157E-10 +|T [116, 156] 118 9.32157E-10 +|T [116, 156] 156 9.32157E-10 +|T [116, 156] 157 9.32157E-10 +|T [116, 156] 158 9.32157E-10 +|T [116, 156] 111 1.9329212E-9 +4 [111, 29, 329] 112 1.9329212E-9 +4 [111, 29, 329] 114 1.9329212E-9 G+ [86, 114, 328] 115 1.9329212E-9 G+ [86, 114, 328] 29 1.9329212E-9 +4 [111, 29, 329] 30 1.9329212E-9 +4 [111, 29, 329] 328 1.9329212E-9 G+ [86, 114, 328] 329 1.9329212E-9 +4 [111, 29, 329] 330 1.9329212E-9 +4 [111, 29, 329] 86 1.9329212E-9 G+ [86, 114, 328] 87 1.9329212E-9 G+ [86, 114, 328] 110 3.0694067E-9 (+ [155, 283, 110] 155 3.0694067E-9 (+ [155, 283, 110] 283 3.0694067E-9 (+ [155, 283, 110] 284 3.0694067E-9 (+ [155, 283, 110] 232 4.7892152E-9 YA [294, 232] 233 4.7892152E-9 YA [294, 232] 294 4.7892152E-9 YA [294, 232] 295 4.7892152E-9 YA [294, 232] 281 7.483148E-9 N: [306, 281] 282 7.483148E-9 N: [306, 281] 306 7.483148E-9 N: [306, 281] 307 7.483148E-9 N: [306, 281] 231 1.0775734E-8 PY [231, 144] 108 4.3102936E-8 ^D [108, 134] 109 4.3102936E-8 ^D [108, 134] 134 4.3102936E-8 ^D [108, 134] 135 4.3102936E-8 ^D [108, 134] 95 5.8667887E-8 D( [95, 109] 96 5.8667887E-8 D( [95, 109] 162 6.7348346E-8 k. [23, 162] 163 6.7348346E-8 k. [23, 162] 23 6.7348346E-8 k. [23, 162] 24 6.7348346E-8 k. [23, 162] 193 9.698162E-8 .L [24, 193] 194 9.698162E-8 .L [24, 193] 25 9.698162E-8 .L [24, 193] 121 1.1973037E-7 -R [26, 121] 122 1.1973037E-7 -R [26, 121] 26 1.1973037E-7 -R [26, 121] 27 1.1973037E-7 -R [26, 121] 21 1.3200275E-7 *5 [21, 82] 22 1.3200275E-7 *5 [21, 82] 82 1.3200275E-7 *5 [21, 82] 83 1.3200275E-7 *5 [21, 82] 119 2.6939338E-7 TB [118, 225] 180 2.6939338E-7 |< [78, 180] 181 2.6939338E-7 |< [78, 180] 225 2.6939338E-7 TB [118, 225] 226 2.6939338E-7 TB [118, 225] 78 2.6939338E-7 |< [78, 180] 79 2.6939338E-7 |< [78, 180] 237 3.6667424E-7 MF [237, 243] 238 3.6667424E-7 MF [237, 243] 243 3.6667424E-7 MF [237, 243] 244 3.6667424E-7 MF [237, 243] 173 3.879265E-7 ;+ [239, 173] 174 3.879265E-7 ;+ [239, 173] 239 3.879265E-7 ;+ [239, 173] 240 3.879265E-7 ;+ [239, 173] 196 1.0775735E-6 BO [278, 196] 197 1.0775735E-6 BO [278, 196] 278 1.0775735E-6 BO [278, 196] 279 1.0775735E-6 BO [278, 196] 88 1.0775735E-6 +k [87, 129] 256 2.112044E-6 +l [255, 9] 139 4.310294E-6 +c [139, 174] 140 4.310294E-6 +c [139, 174] 175 4.310294E-6 +c [139, 174] 170 6.206824E-6 +B [170, 240] 171 6.206824E-6 +B [170, 240] 241 6.206824E-6 +B [170, 240] Period 28: 118 3.5574893E-15 |?V??O [118, 86, 310] 119 3.5574893E-15 |?V??O [118, 86, 310] 120 3.5574893E-15 |?V??O [118, 86, 310] 121 3.5574893E-15 |?V??O [118, 86, 310] 122 3.5574893E-15 |?V??O [118, 86, 310] 123 3.5574893E-15 |?V??O [118, 86, 310] 310 3.5574893E-15 |?V??O [118, 86, 310] 311 3.5574893E-15 |?V??O [118, 86, 310] 312 3.5574893E-15 |?V??O [118, 86, 310] 313 3.5574893E-15 |?V??O [118, 86, 310] 314 3.5574893E-15 |?V??O [118, 86, 310] 315 3.5574893E-15 |?V??O [118, 86, 310] 86 3.5574893E-15 |?V??O [118, 86, 310] 87 3.5574893E-15 |?V??O [118, 86, 310] 88 3.5574893E-15 |?V??O [118, 86, 310] 89 3.5574893E-15 |?V??O [118, 86, 310] 90 3.5574893E-15 |?V??O [118, 86, 310] 91 3.5574893E-15 |?V??O [118, 86, 310] 316 2.0973537E-11 W7???F [316, 62] 317 2.0973537E-11 W7???F [316, 62] 318 2.0973537E-11 W7???F [316, 62] 319 2.0973537E-11 W7???F [316, 62] 320 2.0973537E-11 W7???F [316, 62] 321 2.0973537E-11 W7???F [316, 62] 62 2.0973537E-11 W7???F [316, 62] 63 2.0973537E-11 W7???F [316, 62] 64 2.0973537E-11 W7???F [316, 62] 65 2.0973537E-11 W7???F [316, 62] 66 2.0973537E-11 W7???F [316, 62] 67 2.0973537E-11 W7???F [316, 62] 201 2.7754323E-11 UM [32, 201, 319] 202 2.7754323E-11 UM [32, 201, 319] 32 2.7754323E-11 UM [32, 201, 319] 33 2.7754323E-11 UM [32, 201, 319] 227 8.389415E-11 F*G [321, 227] 228 8.389415E-11 F*G [321, 227] 229 8.389415E-11 F*G [321, 227] 322 8.389415E-11 F*G [321, 227] 323 8.389415E-11 F*G [321, 227] 105 1.3108456E-10 y?zc [180, 105] 106 1.3108456E-10 y?zc [180, 105] 107 1.3108456E-10 y?zc [180, 105] 108 1.3108456E-10 y?zc [180, 105] 180 1.3108456E-10 y?zc [180, 105] 181 1.3108456E-10 y?zc [180, 105] 182 1.3108456E-10 y?zc [180, 105] 183 1.3108456E-10 y?zc [180, 105] 12 2.8197752E-10 |?<?p [48, 12] 13 2.8197752E-10 |?<?p [48, 12] 14 2.8197752E-10 |?<?p [48, 12] 15 2.8197752E-10 |?<?p [48, 12] 16 2.8197752E-10 |?<?p [48, 12] 48 2.8197752E-10 |?<?p [48, 12] 49 2.8197752E-10 |?<?p [48, 12] 50 2.8197752E-10 |?<?p [48, 12] 51 2.8197752E-10 |?<?p [48, 12] 52 2.8197752E-10 |?<?p [48, 12] 184 3.4401834E-10 z?2??2 [236, 182] 185 3.4401834E-10 z?2??2 [236, 182] 186 3.4401834E-10 z?2??2 [236, 182] 187 3.4401834E-10 z?2??2 [236, 182] 236 3.4401834E-10 z?2??2 [236, 182] 237 3.4401834E-10 z?2??2 [236, 182] 238 3.4401834E-10 z?2??2 [236, 182] 239 3.4401834E-10 z?2??2 [236, 182] 240 3.4401834E-10 z?2??2 [236, 182] 241 3.4401834E-10 z?2??2 [236, 182] 134 1.3423064E-9 +B???C [332, 134] 135 1.3423064E-9 +B???C [332, 134] 136 1.3423064E-9 +B???C [332, 134] 137 1.3423064E-9 +B???C [332, 134] 138 1.3423064E-9 +B???C [332, 134] 139 1.3423064E-9 +B???C [332, 134] 332 1.3423064E-9 +B???C [332, 134] 333 1.3423064E-9 +B???C [332, 134] 334 1.3423064E-9 +B???C [332, 134] 335 1.3423064E-9 +B???C [332, 134] 336 1.3423064E-9 +B???C [332, 134] 337 1.3423064E-9 +B???C [332, 134] 57 2.4463527E-9 2?+2 [184, 57] 58 2.4463527E-9 2?+2 [184, 57] 59 2.4463527E-9 2?+2 [184, 57] 60 2.4463527E-9 2?+2 [184, 57] 225 8.948708E-9 +O [122, 225, 90] 226 8.948708E-9 +O [122, 225, 90] 190 4.6769678E-8 TN [190, 7] 191 4.6769678E-8 TN [190, 7] 7 4.6769678E-8 TN [190, 7] 8 4.6769678E-8 TN [190, 7] 213 1.3200275E-7 VK [213, 120] 214 1.3200275E-7 VK [213, 120] 219 1.4487377E-7 Dp [304, 219] 220 1.4487377E-7 Dp [304, 219] 304 1.4487377E-7 Dp [304, 219] 305 1.4487377E-7 Dp [304, 219] 114 1.7241175E-7 R* [114, 26] 115 1.7241175E-7 R* [114, 26] 26 1.7241175E-7 R* [114, 26] 27 1.7241175E-7 R* [114, 26] 112 1.8707871E-7 kc [112, 330] 113 1.8707871E-7 kc [112, 330] 330 1.8707871E-7 kc [112, 330] 331 1.8707871E-7 kc [112, 330] 96 2.1820863E-7 .z [96, 181] 97 2.1820863E-7 .z [96, 181] 192 2.6939338E-7 NB [191, 243] 243 2.6939338E-7 NB [191, 243] 244 2.6939338E-7 NB [191, 243] 197 2.9700618E-7 (z [290, 197] 198 2.9700618E-7 (z [290, 197] 290 2.9700618E-7 (z [290, 197] 291 2.9700618E-7 (z [290, 197] 273 3.6667424E-7 Ol [273, 78] 274 3.6667424E-7 Ol [273, 78] 78 3.6667424E-7 Ol [273, 78] 79 3.6667424E-7 Ol [273, 78] 81 5.28011E-7 B5 [81, 333] 82 5.28011E-7 B5 [81, 333] 102 1.0775735E-6 +- [102, 18] 103 1.0775735E-6 +- [102, 18] 155 1.0775735E-6 y+ [155, 89] 156 1.0775735E-6 y+ [155, 89] 18 1.0775735E-6 +- [102, 18] 19 1.0775735E-6 +- [102, 18] 80 1.0775735E-6 |B [86, 80] 44 2.758588E-6 R+ [58, 44] 45 2.758588E-6 R+ [58, 44] 157 5.215456E-6 +p [51, 156]
You may want to look at the top results there and confirm if they really do seem to produce more repeating fragments overall than the original Z340. I’m not yet sure how effective my measurement is.
Overvaluing longer fragments and/or undervaluing shorter fragments? I don’t know anything about probability. I wonder, if a fragment appears 2 times, or 5 times or 10 times. Is the probability (decrease?) linear? That should not be the case I think. Something I noticed a while ago is that when the n-gram length of the measurement increases – from 2 to 3, 4 grams etc – it appears to pick up more and more random stuff/noise. And thus becomes increasingly more prone to outliers.
To illustrate my noise argument, image graphs: 340 character plaintext bigrams c*(c-1). You can see strong diminishing returns kick in from period 3 and onwards. Couple that with homophonic substitution and the relevance of fragments > 3 is greatly diminished.
Overvaluing longer fragments and/or undervaluing shorter fragments? I don’t know anything about probability. I wonder, if a fragment appears 2 times, or 5 times or 10 times. Is the probability (decrease?) linear?
Here’s the way I do it: It is simply based on symbol frequency. For example, if A appears 5 times, and B appears 10 times, then:
p(AB) = ((5/340)*(10/340))^1 = 1/2312
p(AB AB) = ((5/340)*(10/340))^2 = 1/78608 (decreased by factor of 34)
p(AB AB AB) = ((5/340)*(10/340))^3 = 1/2672672 (also decreased by factor of 34)
So, the probability does decrease linearly.
Let’s say symbol C occurs 8 times in the cipher. Then p(AC AC AC) will be lower than p(AB AB AB), which gives us a way to say that AC AC AC is a "better" repeating sequence than AB AB AB (because AC AC AC is a little bit less likely to be happening by random chance than AB AB AB).
I’ve been interested in calculating the exact probability of, say, the repeating bigrams AB AB AB appearing anywhere in the cipher text. This can be calculated by first determining how many different ways AB AB AB can appear (i.e., how many ways the AB bigram can appear in 3 locations of the cipher text). Then you divide that total by (340!), the total number of possible ways to arrange all 340 symbols. Then you will know the exact proportion of all rearrangements that contain 3 appearances of the AB bigram.
It can be computed using nesting sums (since you have to iterate through all the different ways to place the bigrams). But I’m not aware of a more direct calculation (i.e., I think you have to compute the sums as loops rather than use some formula). Completing this analysis is on my long term todo list! (And it gets more complex when you consider fragments that have wildcards, such as A??B?C A??B?C A??B?C).
But at any rate, the simpler probability calculation is cheaper to calculate, and is more of a relative measurement — it gives you a way to directly compare different sequences to determine which ones are rarer than others.
Couple that with homophonic substitution and the relevance of fragments > 3 is greatly diminished.
Interesting. Maybe we need more statistics on test ciphers. Could try to figure out what the expected fragment distribution should look like for many normal substitution ciphers (at the same multiplicity as the original 340), then try to compare the untransposed ciphers to that.
