I am wondering if there is a way to detect changes in a key by making X = 2 to 339. Then taking a measurement of cycles to the left of X and to the right of X. Find a place where there is a maximum score of some sort.
This seemed easy to do so I ran my perfect cycle score for X = 1 to 338 (my cipher positions start at zero), for L=2,3,4. The left chunk is defined by [0,X-1] and the right chunk is defined by [X, 339]. I did not maintain a length of 170 (did you want that?).
Here’s the raw data:
https://docs.google.com/spreadsheets/d/ … sp=sharing
Graphs:
Z408, L=2:
Z408, L=3:
Z408, L=4:
X-axis represents the split point (position from which left and right sections are considered). Y-axis represents the perfect cycle score. Red line represents perfect cycle score for cipher segment to the left of the split point. Green line represents perfect cycle core for cipher segment to the right of the split point.
Notice how the red lines have much higher peaks than the green lines. This is because the section of cipher text that "ruins" the perfect cycles appears at the end of Z408, so excluding them from consideration causes the perfect cycle score to go up even higher than the score for the entire Z408.
Z340, L=2:
Z340, L=3:
Z340, L=4:
The plots for Z340 look similar, but are on a much smaller scale due to its general tendency towards lower cycle scores. If Z340 uses multiple keys, then we’d probably have to look at a greater variety of ciphertext partitioning functions, since they could involve transpositions. But, also, it is still unknown when cycling was applied. Before or after a possible transposition scheme?
So far I am not convinced that Zodiac changed his key throughout the message.
The plots for Z340 look similar, but are on a much smaller scale due to its general tendency towards lower cycle scores. If Z340 uses multiple keys, then we’d probably have to look at a greater variety of ciphertext partitioning functions, since they could involve transpositions. But, also, it is still unknown when cycling was applied. Before or after a possible transposition scheme?
I don’t believe the 340 uses multiple keys, at least not to any significant degree (in any way that I understand). Because polyalphabetism in general messes with the spread of the symbols, and the 340 scores well on that. I believe something may be acting upon the cipher that is decreasing the flatness of the key (which is observed relative to the 408), but that may be a simple as a more random encoding process.
There are several measurements that indicate left-to-right encoding, and since there are also more unique symbols per cipher length at the top of the cipher, top-to-bottom looks the most plausible option. I believe no transposition is acting upon the 340 after encoding.
These are just my 2 cents and current thoughts, by all means proceed with the work on multiple keys, it is interesting!
I’m still not satisfied that we’ve ruled out something as simple as applying columnar transposition as part of encipherment. What do you think?
I have been watching your great work guys.. I have been quiet as I did not want to interrupt you. i think doranchac is right and hopefully it will be a simple transposition shift. I have been working on the pivots and the relationships with each other using doranchacs great period calculator.. An excellent tool. An observation is at period 101 it brings some of the yellow pivots adjacent to the red pivots. Some others i have noted as well, period 39 brings interestingly every one of the Red and blue pivots adjacent to each other. Still trying to figure out what would make these happen using substitution ciphers. It could be just an anomaly of the amount of turns applied using the period calculator. Eventually stuff happens.. Also in its original format is it unusualy high to have 30/11 combinatation in three of the pivots. See below my period repeats observation on the pivots.
Cheers
Notice the real high repeats in the red and blue pivot boxes adjacent to each other….is that just another weird anomaly on top of them all being at period 1 to each other. .i apologies in advance if this has been observed before. It seems real interesting to me. Repeats 11/36, 41/28, 3/61.. That’s a high stat, out off seven bigrams they repeat in three lots. that only leaves the 30/8 on its own .. So bringing red and blue pivots together at period 39 is strange …Statistics are not my good point so feel free to point me in the right direction..but be polite
Edit.. I now see why the repeats come together.. Same same..
Thanks, doranchak. The graphs pretty much show about what I was thinking.
I think that he must have transposed the plaintext before encoding. Otherwise the transposition would have mixed up the cycles. I recall Jarlve experimenting with this idea somewhere in this thread.
I am still interested in why the L=3 cycle scores are so low. He either just randomized his symbol selection or he did something else during or after encoding.
Jarlve made four transposition messages for me with different periods. I was able to re-connect the repeat symbols, even without figuring out the exact inscription rectangle or transcription direction, and get solutions that are better than what we are getting now.
Anyway, I am just throwing mud at the wall to see if anything sticks. I think that it may be very difficult to show that Zodiac changed his key at certain positions in the message. But my next idea is to shuffle the message a lot of times. And look at the start positions of L=2 cycles with at least eight consecutive alternations. Then compare to the 340. If we can find examples of two symbols that start to cycle together closer to the end of the message, but don’t cycle together before the cycle start position, and where shuffles show that such a cycle at that location closer to the end of the message is improbable. That may be evidence of a key change. Or some similar analysis.
And I have been thinking about a test message. But I would want it to replicate what I find with the 340 stats. I would want it to look like the 340 as much as possible. And I have considered making some test messages by changing the key dramatically at some position to see what would happen.
I hate to change the subject, but I was looking at different types of ciphers online. The grille cipher is a classical cipher, and I was wondering about the period 15 repeats and period 29 repeats. Can a grille be constructed to replicate 340 stats, and what size would it be? Would it rotate? Do the diagonal rows of period 19 repeats eliminate the grille cipher?
Here is an interesting site. Alexander Hamilton used a grille cipher, which I find interesting.
And interestingly enough when changing it to 20 wide and 17 high we get them cojoined in the columns using of all things period 19 with repeats of 37…
I’m still not satisfied that we’ve ruled out something as simple as applying columnar transposition as part of encipherment. What do you think?
I think that transposition was applied before the homophonic substitution layer and not after.
340 encoding hypothesis:
Layer 1: transposition
Layer 2: homophonic substitution
Ruled out: regular and irregular columnar transposition.
Not ruled out: regular and irregular columnar transposition with transposition deviations, keyed regular and irregular columnar transposition with or without transposition deviations.
While especially keyed columnar transposition has not been ruled out, smokie and I have somewhat concluded it to be unlikely because it does not seem to produce the bigram stats we are looking for.
The question is: Did Zodiac transpose the plaintext before encoding? To re-frame the question: Is it possible to make a message that is not transposed which has +/- 70 period 19 bigram repeats? If you have all 1:1 substitutes, then the answer is yes. But can it be done with 63 symbols?
Smokie, this question you asked long ago was on my to do list for a long time. I finally have an answer: Yes, it can be done, and quite easily.
I modified my cipher generator to create ciphers that match these qualities of the original Z340:
- Same 1-gram distribution
- Same 2-gram distribution at period 1
- Same 2-gram distribution at period 19
- Presence of a pair of symmetrical pivots
- Presence of encoding errors
- Presence of filler section
- Similar appearances of "psuedo-words" in the raw ciphertext
- Spelling errors and missing words in the plaintext
[/list:u:3cl5u3vb]
I did not optimize for things such as cycles and higher-order ngrams/fragments.
By my calculations, the original Z340 has 67 repeating bigrams at period 19 if you count AB AB AB CD CD as 5 repeats. It has 37 repeats at period 19 if you count AB AB AB CD CD as 3 repeats. Adding the two numbers together, you get 67+37 = 104. I made the cipher generator search for ciphers that had the same sum of 104. Here are the results:
16OjYU_ykp+BB+^WU (KX<Mbpc^4|5FBcfj kkqlHpbY5p^1GG(dH ERJ|*8YzZ)d;7Clj9 @WqB|z+WUB;-2U<J/ T25+J8D(+cc31l*F2 T9W^RLN<pYDN)p5M6 zklF%+b.cV2lX|OWB 4<F3+lkBFcFP-*8yp -f+(WMDJ_.U1V((T- 42>SkFB;qbN+%+P25 &NZ^+^/FBctO4#++E JD2A/+&LO*cGC7+Z+ CC...py3#pD+j_DZB F2S:DzM<p8|5FUlS_ LtVT)pGzKM<*-12MA Lc1bDd(|+B1|pqYNz +6&jKDPA|Vyf++|SV #)5V2z+1K)VTft1LG +D<:#G>1f464*%+Tz 01 02 03 04 05 06 07 08 09 10 11 12 12 11 13 14 06 15 16 17 18 19 20 10 21 13 22 23 24 25 12 21 26 04 09 09 27 28 29 10 20 05 24 10 13 01 30 30 15 31 29 32 33 34 23 35 36 05 37 38 39 31 40 41 42 28 04 43 44 14 27 12 23 37 11 14 06 12 40 45 46 06 18 34 47 48 46 24 11 34 36 49 15 11 21 21 50 01 28 35 25 46 48 43 14 13 33 51 52 18 10 05 49 52 39 10 24 19 02 37 09 28 25 53 11 20 54 21 55 46 28 17 23 03 14 12 22 18 25 50 11 28 09 12 25 21 25 56 45 35 36 08 10 45 26 11 15 14 19 49 34 07 54 06 01 55 15 15 48 45 22 46 57 58 09 25 12 40 27 20 52 11 53 11 56 46 24 59 52 38 13 11 13 47 25 12 21 60 03 22 61 11 11 32 34 49 46 62 47 11 59 51 03 35 21 30 42 41 11 38 11 42 42 54 54 54 10 08 50 61 10 49 11 04 07 49 38 12 25 46 58 63 49 37 19 18 10 36 23 24 25 06 28 58 07 51 60 55 48 39 10 30 37 16 19 18 35 45 01 46 19 62 51 21 01 20 49 31 15 23 11 12 01 23 10 27 05 52 37 11 02 59 04 16 49 56 62 23 55 08 26 11 11 23 58 55 61 39 24 55 46 37 11 01 16 39 55 48 26 60 01 51 30 11 49 18 63 61 30 57 01 26 22 02 22 35 53 11 48 37 --- &7t/Vc.bP4()TOM+p /^#Cc(yOBKf#8d5-G >|5F.kO8_F&9KzctG zO-6bk+fX;cZ+^N5k 2+MzUX2+d|71y;d44 .|5FBcj)+21y/2+9y (KPHER+*8++WB9MV& :t+UNFdTllt^.%&Nq yO7zZ3D|^C;PF#+VE AK:EGGEEppR+FC9YK -5*<Fcc3+5C|*-Ek- AU+lO+U<zS-*|H&Oc )qMT^L*S+LMCD*|JR (l|_4Z4DY+JR_|52) .)z2zZ&Rj&VNLJ#BS @W+@*+5cFBcR+RB>W BUC5&G*TbPBbUJFBN F+SB2G(dO>5(#3TK8 DlB>&*M+V<W&B#kz6 4f|<G.@+8(lL<&O1l 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 04 18 19 20 06 11 21 14 22 23 24 19 25 26 27 28 29 30 31 27 32 07 33 14 25 34 32 01 35 23 36 06 03 29 36 14 28 37 08 33 16 24 38 39 06 40 16 18 41 27 33 42 16 15 36 43 38 42 16 26 31 02 44 21 39 26 10 10 07 31 27 32 22 06 45 12 16 42 44 21 04 42 16 35 21 11 23 09 46 47 48 16 49 25 16 16 50 22 35 15 05 01 51 03 16 43 41 32 26 13 52 52 03 18 07 53 01 41 54 21 14 02 36 40 55 56 31 18 20 39 09 32 19 16 05 47 57 23 51 47 29 29 47 47 17 17 48 16 32 20 35 58 23 28 27 49 59 32 06 06 55 16 27 20 31 49 28 47 33 28 57 43 16 52 14 16 43 59 36 60 28 49 31 46 01 14 06 12 54 15 13 18 61 49 60 16 61 15 20 56 49 31 62 48 11 52 31 34 10 40 10 56 58 16 62 48 34 31 27 42 12 07 12 36 42 36 40 01 48 45 01 05 41 61 62 19 22 60 63 50 16 63 49 16 27 06 32 22 06 48 16 48 22 30 50 22 43 20 27 01 29 49 13 08 09 22 08 43 62 32 22 41 32 16 60 22 42 29 11 26 14 30 27 11 19 55 13 23 25 56 52 22 30 01 49 15 16 05 59 50 01 22 19 33 36 37 10 24 31 59 29 07 63 16 25 11 52 61 59 01 14 44 52 --- %7zJL.b|5FBcfW&7D p62+)p.cMzz+)52Wk J^q-Bpcyz1pycOTD_ +Yqb.cVSZLZF4|+%f 4UOBz7PY&XR249%#9 ^3%E#&%_TyHt|2Ok) 8+A+pCz|5Fp*>j2&+ c8zT^6FBc2+pO7|FB 7O+RKO8LULtMTdy+3 RPO+dKRFP<+.;8TkM -pkyG@DL/l8B5:4<- FK(lzZ#d;9C8&V:pC N.lR>cySD6+1+7U4+ tB<JkpZ5<NjTG+XdN 2+G|B)MYRl(WUUfHE RJ|*V<(pd1F+524+| +._/c<W(F<+c1PN-* P>O&O+>Cz5l3p)YGK kB/VJWA*-+CNF+P5B .BS|LWj|MO>-j2yLf 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 02 16 17 18 19 20 21 17 06 12 22 03 03 20 21 09 19 14 23 04 24 25 26 11 17 12 27 03 28 17 27 12 29 30 16 31 20 32 25 07 06 12 33 34 35 05 35 10 36 08 20 01 13 36 37 29 11 03 02 38 32 15 39 40 19 36 41 01 42 41 24 43 01 44 42 15 01 31 30 27 45 46 08 19 29 23 21 47 20 48 20 17 49 03 08 09 10 17 50 51 52 19 15 20 12 47 03 30 24 18 10 11 12 19 20 17 29 02 08 10 11 02 29 20 40 53 29 47 05 37 05 46 22 30 54 27 20 43 40 38 29 20 54 53 40 10 38 55 20 06 56 47 30 23 22 26 17 23 27 57 58 16 05 59 60 47 11 09 61 36 55 26 10 53 62 60 03 35 42 54 56 41 49 47 15 33 61 17 49 63 06 60 40 51 12 27 34 16 18 20 28 20 02 37 36 20 46 11 55 04 23 17 35 09 55 63 52 30 57 20 39 54 63 19 20 57 08 11 21 22 32 40 60 62 14 37 37 13 45 44 40 04 08 50 33 55 62 17 54 28 10 20 09 19 36 20 08 20 06 31 59 12 55 14 62 10 55 20 12 28 38 63 26 50 38 51 29 15 29 20 51 49 03 09 60 43 17 21 32 57 53 23 11 59 33 04 14 48 50 26 20 49 63 10 20 38 09 11 06 11 34 08 05 14 52 08 22 29 51 26 52 19 27 05 13 --- #)k>-O|Mb^M)b)cdA qKUWb)*-)+<HTB)j3 ):Kf.L^3SyO)2pb6) YE(pcOALS+)zR4zV; ^b.cVkq)Uy%T9CN+C yC^ZS#7yqVq#GDGjD ZDjjHG+G*DN<BK&9d lR%zY;pb)>S1qMpF( Nc&pL|qFkzT)KlcXc )<|5F|8N_9WU^8^T: -C<E#HER66M%FUB2- B8%@1*7J_#/pUBz*6 O)-lNq2|lt+7(4q9^ FBckRJ|*OBYfL%)-7 kt^/-*W;p)()l6pTM JB2^/|l)VRc6()fS4 l9|5FBcO%y3Wz^)R| bKd9Y7zZ#d8|c/pz) lcPTLpFlWfLBFKNqK SLBRP1)tFBf)dqXtP 01 02 03 04 05 06 07 08 09 10 08 02 09 02 11 12 13 14 15 16 17 09 02 18 05 02 19 20 21 22 23 02 24 25 02 26 15 27 28 29 10 25 30 31 06 02 32 33 09 34 02 35 36 37 33 11 06 13 29 30 19 02 38 39 40 38 41 42 10 09 28 11 41 03 14 02 16 31 43 22 44 45 46 19 45 31 45 10 47 30 01 48 31 14 41 14 01 49 50 49 24 50 47 50 24 24 21 49 19 49 18 50 46 20 23 15 51 44 12 52 39 43 38 35 42 33 09 02 04 30 53 14 08 33 54 37 46 11 51 33 29 07 14 54 03 38 22 02 15 52 11 55 11 02 20 07 56 54 07 57 46 58 44 17 16 10 57 10 22 26 05 45 20 36 01 21 36 39 34 34 08 43 54 16 23 32 05 23 57 43 59 53 18 48 60 58 01 61 33 16 23 38 18 34 06 02 05 52 46 14 32 07 52 62 19 48 37 40 14 44 10 54 23 11 03 39 60 07 18 06 23 35 27 29 43 02 05 48 03 62 10 61 05 18 17 42 33 02 37 02 52 34 33 22 08 60 23 32 10 61 07 52 02 41 39 11 34 37 02 27 30 40 52 44 07 56 54 23 11 06 43 31 25 17 38 10 02 39 07 09 15 12 44 35 48 38 47 01 12 57 07 11 61 33 38 02 52 11 63 22 29 33 54 52 17 27 29 23 54 15 46 14 15 30 29 23 39 63 53 02 62 54 23 27 02 12 14 55 62 63 --- #&c*_Zb.cVPRJ|*9l &Xt6)ES7Kc&3Ly|F& Dq%U&TbXU.j_1&J#B 5<&T<plcpbp6HERM% G2DLRYEGjFBcdl413 RjVUWMNzX)-*|5FPt ^^NzB>ztckdFY)j+c VyRX&(|6V&<BOUFyz #CT&CU8R@X6|c/#&f -N6&HqN:&J>b:8*-- RDYplqHZ<|&6G<|/F d6UB3F1zX42B|5FBc Rf9kY|-&(J2M8FNR; &lW@-*&4B/+5&(fZ/ 4q6)CNBZ.22Sq<..D DE2yqD7ySE(&8W9YK fWfzb/:&6WWzZ3DA/ c|M;Fp8zLz6#b&3WB 33GA2lL3&6#9&3tXP p7B4(N)3:)^X6&kpk 01 02 03 04 05 06 07 08 03 09 10 11 12 13 04 14 15 02 16 17 18 19 20 21 22 23 03 02 24 25 26 13 27 02 28 29 30 31 02 32 07 16 31 08 33 05 34 02 12 01 35 36 37 02 32 37 38 15 03 38 07 38 18 39 20 11 40 30 41 42 28 25 11 43 20 41 33 27 35 03 44 15 45 34 24 11 33 09 31 46 40 47 48 16 19 49 04 13 36 27 10 17 50 50 47 48 35 51 48 17 03 52 44 27 43 19 33 53 03 09 26 11 16 02 54 13 18 09 02 37 35 55 31 27 26 48 01 56 32 02 56 31 57 11 58 16 18 13 03 59 01 02 60 49 47 18 02 39 29 47 61 02 12 51 07 61 57 04 49 49 11 28 43 38 15 29 39 06 37 13 02 18 41 37 13 59 27 44 18 31 35 24 27 34 48 16 45 42 35 13 36 27 35 03 11 60 14 52 43 13 49 02 54 12 42 40 57 27 47 11 62 02 15 46 58 49 04 02 45 35 59 53 36 02 54 60 06 59 45 29 18 19 56 47 35 06 08 42 42 21 29 37 08 08 28 28 20 42 26 29 28 22 26 21 20 54 02 57 46 14 43 23 60 46 60 48 07 59 61 02 18 46 46 48 06 24 28 63 59 03 13 40 62 27 38 57 48 25 48 18 01 07 02 24 46 35 24 24 41 63 42 15 25 24 02 18 01 14 02 24 17 16 10 38 22 35 45 54 47 19 24 61 19 50 16 18 02 52 38 52 --- #l1>_UC*ttFlyR_VJ p2+z6|K:zZ%7|8kZK jf+zqPddk77^+kdXl 2+L%1B|5FBc<pCA)c 45M*+T*-BpPR+5+6+ P^Y<FzWO-*BfU5.XE FBcyU;fO/|#9^lN+z K5K_+O>MN6-V^4C+l 2%Ay#)(p4SU@py+b/ -t(/L++W.pyCP<pL- B+*4%R|5FRzDK/&G3 D+l-TMc&BAORz|b.N 63+9_fVXb.cVyYl>^ 4;RJ|*3Nc|.c^)+dc _t9M+|+DKc6.+29pW zNOO/J%;WHpb|Hc%6 LpOTLRDM1dRlRB+<P FBDpZUO8pqF2SMF)+ Gz2TMA6jFL>WKF)Oc HEROY%<:d12S#S+C/ 01 02 03 04 05 06 07 08 09 09 10 02 11 12 05 13 14 15 16 17 18 19 20 21 22 18 23 24 25 20 26 27 23 21 28 29 17 18 30 31 32 32 27 25 25 33 17 27 32 34 02 16 17 35 24 03 36 20 37 10 36 38 39 15 07 40 41 38 42 37 43 08 17 44 08 45 36 15 31 12 17 37 17 19 17 31 33 46 39 10 18 47 48 45 08 36 29 06 37 49 34 50 10 36 38 11 06 51 29 48 52 20 01 53 33 02 54 17 18 21 37 21 05 17 48 04 43 54 19 45 13 33 42 07 17 02 16 24 40 11 01 41 55 15 42 56 06 57 15 11 17 58 52 45 09 55 52 35 17 17 47 49 15 11 07 31 39 15 35 45 36 17 08 42 24 12 20 37 10 12 18 59 21 52 60 61 62 59 17 02 45 44 43 38 60 36 40 48 12 18 20 58 49 54 19 62 17 53 05 29 13 34 58 49 38 13 11 46 02 04 33 42 51 12 14 20 08 62 54 38 20 49 38 33 41 17 32 38 05 09 53 43 17 20 17 59 21 38 19 49 17 16 53 15 47 18 54 48 48 52 14 24 51 47 63 15 58 20 63 38 24 19 35 15 48 44 35 12 59 43 03 32 12 02 12 36 17 39 31 10 36 59 15 23 06 48 26 15 30 10 16 56 43 10 41 17 61 18 16 44 43 40 19 28 10 35 04 47 21 10 41 48 38 63 50 12 48 46 24 39 22 32 03 16 56 01 56 17 07 52 --- |#P-z4_Xl>(HjlDXk +lj;ScLMGM:z;MVTF X2CRqMEEjEd9EM7.% )&X(kWWK8@V8M&|6G H^Syq-76./t1T-*bF U(|5FBc12fG3l7p2; Oc|5F%;kt2&(%#z2b V^M;t)WXUKGYD#A<| M)L8MOMl)MHz^STql 7c./b.cV6tF;M<5GX -jM+6FBcyzpN42#&- *-;&.qy.Mz;CY+3YX |A(4C|Lb;8*LMU(z1 f5RJ|*^cfqtXMBt<| 5Y(N7|1V*WP/St3)M BzMXDJX;MFM:L#HER TN^P(RFz*>W(#MbWt 4ZpJBDH)MMcG%2jAl ;86|FG4dA.D+yBH5f cJEMFzZ_9|7cB5B^t 01 02 03 04 05 06 07 08 09 10 11 12 13 09 14 08 15 16 09 13 17 18 19 20 21 22 21 23 05 17 21 24 25 26 08 27 28 29 30 21 31 31 13 31 32 33 31 21 34 35 36 37 38 08 11 15 39 39 40 41 42 24 41 21 38 01 43 22 12 44 18 45 30 04 34 43 35 46 47 48 25 04 49 50 26 51 11 01 52 26 53 19 48 27 54 22 55 09 34 56 27 17 57 19 01 52 26 36 17 15 47 27 38 11 36 02 05 27 50 24 44 21 17 47 37 39 08 51 40 22 58 14 02 59 60 01 21 37 20 41 21 57 21 09 37 21 12 05 44 18 25 30 09 34 19 35 46 50 35 19 24 43 47 26 17 21 60 52 22 08 04 13 21 16 43 26 53 19 45 05 56 61 06 27 02 38 04 49 04 17 38 35 30 45 35 21 05 17 28 58 16 55 58 08 01 59 11 06 28 01 20 50 17 41 49 20 21 51 11 05 48 54 52 29 62 01 49 44 19 54 30 47 08 21 53 47 60 01 52 58 11 61 34 01 48 24 49 39 03 46 18 47 55 37 21 53 05 21 08 14 62 08 17 21 26 21 23 20 02 12 31 29 25 61 44 03 11 29 26 05 49 10 39 11 02 21 50 39 47 06 63 56 62 53 14 12 37 21 21 19 22 36 27 13 59 09 17 41 43 01 26 22 06 32 59 35 14 16 45 53 12 52 54 19 62 31 21 26 05 63 07 33 01 34 19 53 52 53 44 47
They should all be easy to solve.
(Note: We may need to compare notes, since you count 70 repeats for Z340, whereas I only count 67. Can you share your period 19 untransposition of Z340?)
I actually have 69. I have a spreadsheet that finds the repeats in a vertical array, ranks them by score, and then re-lists them by rank. But there is a bug that adds one extra repeat at the end of the list. I have another spreadsheet that finds the repeats in a horizontal array, but doesn’t rank them. That one seems to be more accurate, and finds 69.
Here is my 340 untransposed at period 19:
1 19 37 44 11 56 8 60 31 19 6 22 16 2 28 20 33
4 2 20 19 30 50 10 28 13 55 3 3 19 53 4 32 11
5 37 3 21 38 8 51 51 40 47 40 54 41 18 61 8 37
33 11 25 4 22 39 45 9 4 13 17 6 50 11 11 28 38
57 13 51 1 5 23 15 5 19 16 11 29 38 48 30 50 36
39 15 19 10 18 6 24 26 23 52 25 21 37 8 2 50 51
8 50 16 19 17 5 7 25 21 19 53 21 15 19 19 11 14
20 53 55 3 33 26 10 8 26 33 19 10 22 16 61 7 25
53 36 48 19 36 26 29 42 9 27 13 3 54 50 41 19 41
27 37 21 19 11 14 56 43 40 10 28 22 31 5 19 32 39
19 20 28 58 19 36 19 40 48 39 11 29 40 16 44 31 49
3 23 5 19 44 34 28 13 26 20 23 12 30 1 46 3 57
22 16 5 61 52 3 20 45 12 36 46 44 13 31 41 47 7
24 23 51 43 14 20 6 59 40 63 9 27 62 14 32 42 37
51 58 19 20 29 37 51 15 12 20 56 23 23 11 15 33 5
19 6 16 46 36 51 31 40 51 30 31 29 42 20 31 16 20
5 40 23 38 18 34 20 23 63 18 35 21 19 1 30 58 17
34 43 48 55 36 27 62 34 16 47 7 53 23 51 14 55 19
18 35 7 49 30 59 40 63 55 29 42 32 47 5 6 54 56
5 36 6 17 17 15 19 53 38 36 34 50 56 7 26 21 36
I have maybe one of the coolest spreadsheets in the world. Here is a screenshot with period 1 untransposed side by side with period 19 "transposed". I can change the period in red outlined boxes and the spreadsheet will automatically show the new repeat period, with colored conditional formatting for different symbols, etc. On the right side list, below, you can see the last repeat at the bottom. Symbols 1 and 19 with a count of 1. That’s the bug.
I will take a look at your messages. Did you hill-climb the key? I mean, did you start with a key, and make little changes until you had the correct period 1 and period 19 stats? With smokie18e, I had to find the plaintext that were heavily represented in the plaintext period 19 repeats, and diffuse then less. And then find the plaintext that were represented less in the plaintext period 1 repeats, and diffuse them more. I didn’t think that it was very easy, though.
I will take a look at your messages. Did you hill-climb the key? I mean, did you start with a key, and make little changes until you had the correct period 1 and period 19 stats?
Yes, that’s pretty much how it works. First, a random plaintext is found that supports the pair of pivots (not many of them do, but they show up more easily when enough perturbations are done by applying random misspellings or word omissions). Then an initial key is assigned that is very close to the Z340 unigram distribution. From there the hillclimber takes over, making random small adjustments to the key until the measurements match Z340.
I will compare our untransposed period 19 ciphers soon. My guess is there is a slight difference in how we perform the untransposition.
Meanwhile, my generator is still running on a loop, producing a few hundred more test ciphers. I think I will gradually expand it to capture the other measurements again (cycles, higher-order ngrams, prime phobia, etc). Then I can work towards producing some test ciphers that incorporate transpositions. I think I’ll be able to make, say, keyed columnar transposition ciphers that have similar measurements compared to Z340.
OK our untranspositions match exactly. But I still count only 67 repeats. Here’s the breakdown:
4: 5 19
3: 29 42
3: 22 16
3: 21 19
3: 19 53
3: 15 19
2: 9 27
2: 7 25
2: 59 40
2: 58 19
2: 55 3
2: 47 7
2: 40 63
2: 37 51
2: 30 50
2: 28 13
2: 27 62
2: 25 21
2: 23 51
2: 20 23
2: 19 6
2: 19 36
2: 19 20
2: 19 11
2: 19 10
2: 18 35
2: 14 20
2: 11 29
2: 11 14
2: 10 28
Can you show me which ones I might have missed?
You have all of them. Apparently I have another strange bug. I concatenate the two symbols, with a ":" between them. Then I count the list to look for repeats. My spreadsheet counts that there are two 18:61, but there is only one. I have no idea why.
I looked at your first message. There are about 209 period 19 repeats in the plaintext to start with. I made smokie18e with Jarlve’s message 44, which had 204 and the highest of all in his library. I don’t have much to work with as far as corpus goes, but it seems as though you picked a part of the story that has similar plaintext stats.
But wait. I updated my pivot detection spreadsheet so that I can look at multiple periods all on the same grid. That part of the story has a pivot. I wonder how many pivots there are in War and Peace, and how many 340 plaintext sets have such high period 19 plaintext stats. Or perhaps 209 isn’t particularly high? Excellent message.
A C C U S I N G P I E R R E O F I
L L U M I N I S M O T H E R S S U
P P O R T I N G H I M A T T H A T
M E E T I N G H E W A S S T R U C
K F O R T H E F I R S T T I M E B
Y T H E E N D L E S S V A R I E T
Y O F M E N S M I N D S W H I C H
P R E V E N T S A T R U T H F R O
M E V E R P R E S E N T I N G I T
S E L F I D E N T I C A L L Y T O
T W O P E R S O N S E V E N T H O
S E M E M B E R S W H O S E E M E
D T O B E O N H I S S I D E U N D
E R S T O O D H I M I N T H E I R
O W N W A Y W I T H L I M I T A T
I O N S A N D A L T E R A T I O N
S H E C O U L D N O T A G R E E T
O A S W H A T H E A L W A Y S W A
N T E D M O S T W A S T O C O N V
E Y H I S T H O U G H T T O O T H
A 1 31
B 47 62
C 2 3 43
D 49
E 11 25 33 34 38 53 61
F 14
G 8
H 24 37 60
I 6 10 19 35
J
K 44
L 15 16
M 18 32
N 7 20 36 51
O 13 22 27 57
P 9 58
Q
R 12 28
S 5 21 26 40 41 52 54 59
T 23 29 30 42 45 46 55 56
U 4 17
V 50
W 39 63
X
Y 48
Z
01 02 03 04 05 06 07 08 09 10 11 12 12 11 13 14 06 15 16 17 18 19 20 10 21 13 22 23 24 25 12 21 26 04 09 09 27 28 29 10 20 05 24 10 13 01 30 30 15 31 29 32 33 34 23 35 36 05 37 38 39 31 40 41 42 28 04 43 44 14 27 12 23 37 11 14 06 12 40 45 46 06 18 34 47 48 46 24 11 34 36 49 15 11 21 21 50 01 28 35 25 46 48 43 14 13 33 51 52 18 10 05 49 52 39 10 24 19 02 37 09 28 25 53 11 20 54 21 55 46 28 17 23 03 14 12 22 18 25 50 11 28 09 12 25 21 25 56 45 35 36 08 10 45 26 11 15 14 19 49 34 07 54 06 01 55 15 15 48 45 22 46 57 58 09 25 12 40 27 20 52 11 53 11 56 46 24 59 52 38 13 11 13 47 25 12 21 60 03 22 61 11 11 32 34 49 46 62 47 11 59 51 03 35 21 30 42 41 11 38 11 42 42 54 54 54 10 08 50 61 10 49 11 04 07 49 38 12 25 46 58 63 49 37 19 18 10 36 23 24 25 06 28 58 07 51 60 55 48 39 10 30 37 16 19 18 35 45 01 46 19 62 51 21 01 20 49 31 15 23 11 12 01 23 10 27 05 52 37 11 02 59 04 16 49 56 62 23 55 08 26 11 11 23 58 55 61 39 24 55 46 37 11 01 16 39 55 48 26 60 01 51 30 11 49 18 63 61 30 57 01 26 22 02 22 35 53 11 48 37
Doranchak, I was also thinking that looking at stats for period 2 / 38, period 3 / 57, etc, is a way to compare route ciphers that transpose large chunks of a message, route or other ciphers that transpose small chunks of a message, and non-transposition messages like the ones that you are making.
A route cipher that transposes large chunks of message should generally have higher period 38, period 57, etc. repeat counts because the plaintext should have high period 2, period 3, etc. repeat counts. Something to think about.
