Thanks, are these the final renditions?
My plan is to gradually add more objectives (prime phobia, for example). I’m not sure how important they are. And if I add too many objectives, the search tends to get slow or stuck. Also, since I’m still rather curious about columnar transposition, I might try to generate some ciphers that have transposition applied before encipherment. It may be more fruitful to work towards strongly excluding columnar transposition as a hypothesis. Then I can gradually create more test ciphers under more schemes. I think the focus should be on gathering stronger evidence for or against specific schemes.
Okay, thanks for doing this and if you don’t mind I’ll wait until you have the final versions. I’m very interested in comparing your (keyed) columnar transpositions. Currently my hunch says no but hey.
OK.
Which measurements do you think are most important to mimic from Z340?
My list would include: unigrams, bigrams (period1 and period19, and maybe flipped period 15), cycles, and pivots. I’m on the fence about prime phobia, other bigram biases (top vs bottom half, even vs odd positions), and repeating fragments.
I want to settle on a reasonably small list so the hillclimber for the generator doesn’t have too many objectives to optimize for.
Smokie and doranchak. I made a message with some difference in the encoding that also makes it unsolveable. I didn’t try to match 340 stats but I wonder how it compares.
Below is my breakdown. Your cipher is a bit more cyclic than Z340. It also has very flat bigram repeats across all periods.
Multiplicity:
Z340: 0.185
p105_jarlve_e1.txt: 0.176
IOC:
Z340: 0.019
p105_jarlve_e1.txt: 0.015
unigrams:
Z340: 24 12 11 10(4) 9(2) 8 7(5) 6(9) 5(10) 4(11) 3(9) 2(8) 1 [277 repeats]
p105_jarlve_e1.txt: 11 9(2) 8(4) 7(8) 6(19) 5(14) 4(6) 3(4) 2 1 [280 repeats]
bigrams:
Z340: 3(4) 2(17) 1(293) [25 repeats]
p105_jarlve_e1.txt: 3(2) 2(15) 1(303) [19 repeats]
trigrams:
Z340: 2(2) 1(334) [2 repeats]
p105_jarlve_e1.txt: 1(338) [0 repeats]
period with highest # of bigrams:
Z340: 19 (37 repeats)
p105_jarlve_e1.txt: 4 (23 repeats)
repeated bigram distribution for all periods:
Z340: 37 30(4) 29(4) 28(2) 27(5) 26(7) 25(12) 24(10) 23(10) 22(16) 21(16) 20(10) 19(20) 18(20) 17(11) 16(3) 15(8) 14(3) 13(2) 12(3) 11 9
p105_jarlve_e1.txt: 23 21(3) 20(3) 19(3) 18(13) 17(12) 16(9) 15(10) 14(15) 13(23) 12(25) 11(16) 10(14) 9(13) 8(4) 7(4) 6
unique per-line symbol counts for Olson lines:
Z340: 102
p105_jarlve_e1.txt: 93
jarlve non-repeat score 1:
Z340: 4462
p105_jarlve_e1.txt: 3473
jarlve non-repeat score 2:
Z340: 1599
p105_jarlve_e1.txt: 1692
perfect cycle score (L=2):
Z340: 247.85
p105_jarlve_e1.txt: 359.25
perfect cycle score (L=3):
Z340: 62.36
p105_jarlve_e1.txt: 150.01
jarlve m_2s_cycles:
Z340: 2150.72
p105_jarlve_e1.txt: 2480.19
pivots:
Z340: 2 pivots
p105_jarlve_e1.txt: 0 pivots
prime phobia:
Z340: 23, 11
p105_jarlve_e1.txt: 10, 7
Manipulation operation(s): Flip(17,20,1), Diagonal(16,22,1,1,2,1)
Could you remind me how to interpret the arguments to your manipulation functions?
Smokie, a while back ago I made a mathematical cipher that somewhat cycles of its own. I think the algorithm is not a good hypothesis but perhaps it might be interesting to look at its cycles. The cipher is very polyalphabetic and it’s interesting to see that its bigrams are unaffected (did not shift because of polyalphabetism). Strong returns from period 1, 2 and 3. The plaintext is the 408.
I ran my comparisons on this one as well. Below are the results.
Multiplicity:
Z340: 0.185
jarlve_mimic: 0.188
IOC:
Z340: 0.019
jarlve_mimic: 0.016
unigrams:
Z340: 24 12 11 10(4) 9(2) 8 7(5) 6(9) 5(10) 4(11) 3(9) 2(8) 1 [277 repeats]
jarlve_mimic: 10(4) 9(2) 8(4) 7(11) 6(6) 5(13) 4(10) 3(6) 2(6) 1(2) [276 repeats]
bigrams:
Z340: 3(4) 2(17) 1(293) [25 repeats]
jarlve_mimic: 3(3) 2(33) 1(264) [39 repeats]
trigrams:
Z340: 2(2) 1(334) [2 repeats]
jarlve_mimic: 2(3) 1(332) [3 repeats]
period with highest # of bigrams:
Z340: 19 (37 repeats)
jarlve_mimic: 1 (39 repeats)
repeated bigram distribution for all periods:
Z340: 37 30(4) 29(4) 28(2) 27(5) 26(7) 25(12) 24(10) 23(10) 22(16) 21(16) 20(10) 19(20) 18(20) 17(11) 16(3) 15(8) 14(3) 13(2) 12(3) 11 9
jarlve_mimic: 39 30 23 22(2) 21 20(7) 19(5) 18(7) 17(13) 16(15) 15(20) 14(23) 13(21) 12(12) 11(7) 10(17) 9(8) 8(4) 6(2) 5(2)
unique per-line symbol counts for Olson lines:
Z340: 102
jarlve_mimic: 96
jarlve non-repeat score 1:
Z340: 4462
jarlve_mimic: 4353
jarlve non-repeat score 2:
Z340: 1599
jarlve_mimic: 2321
perfect cycle score (L=2):
Z340: 247.85
jarlve_mimic: 216.70
perfect cycle score (L=3):
Z340: 62.36
jarlve_mimic: 40.99
jarlve m_2s_cycles:
Z340: 2150.72
jarlve_mimic: 1715.02
pivots:
Z340: 2 pivots
jarlve_mimic: 0 pivots
prime phobia:
Z340: 23, 11
jarlve_mimic: 9, 6
I’ve let AZdecrypt run on the 340 for over an hour with the possibility of either keeping the normal orientation, mirroring, flipping or reversing and diagonal transposition. Here’s the best result, it is interesting because it has a high score and a normal index of coincidence.
poadderssensonded
tortwascsingemasi
thislydidthiswils
oheldinutagainere
useonlasanontarof
thesthelongboyand
eediocetonhsmithn
icmultancedfroemb
arrestinlaruesith
asbontastheatfilt
oflassiesaidrient
heedaneofulaginat
etsthatccoupiedfr
omthethindaystsfr
ryourfiddleoanddw
hiteemranomingara
wacrewsshayoopull
spaulateytoldinfr
ienetorerournwiso
nastacinguidthama
There are a couple of contextual words, and I have gotten a similar solution before when I tried different transcription directions. This one must be the same as one of the ones that I tried.
Wow, you guys are working hard on this.
What I meant by rotation of the key is something like this:
Encode with this key…
A 1 2 3
B 4 5 6
C 7 8 9
D 10 11 12
E 13 14 15
F 16 17 18
G 19 20 21
H 22 23 24
I 25 26 27
J
K 28 29 30
L 31 32 33
M 34 35 36
N 37 38 39
O 40 41 42
P 43 44 45
Q
R 46 47 48
S 49 50 51
T 52 53 54
U 55 56 57
V
W 58 59 60
X
Y 61 62 63
Z
And at some interval start encoding with this key:
A 28 2 3
B 31 5 6
C 34 8 9
D 37 11 12
E 40 14 15
F 43 17 18
G 46 20 21
H 49 23 24
I 52 26 27
J
K 55 29 30
L 58 32 33
M 61 35 36
N 1 38 39
O 4 41 42
P 7 44 45
Q
R 10 47 48
S 13 50 51
T 16 53 54
U 19 56 57
V
W 22 59 60
X
Y 25 62 63
Z
And then at another interval make a similar rotation. It is just a very rough idea at this point. But there are situations where you have ABABAB and then later ACACAC. What I am saying is that A and B map to the same plaintext in the beginning of the message, and A is changed so that A and C map to another plaintext later in the message. EDIT: Maybe that would explain why there aren’t very many L=3 cycles across the entire message.
I started making my charts. Below is a screenshot, albeit very tiny so that I could make it fit, of the cycle start and stop positions. On the left vertical axis is the symbol number. On the top horizontal axis is the position. Green means that the symbols begins cycling with another symbol, even though that symbol and the other symbol may appear at that position. Red means that the cycle ends at that position. Yellow is where a symbol both stops with a prior cycle and begins with a new cycle.
I marked a few interesting areas with red boxes. Look at the left, and see a diagonal green row of positions. That is where those symbols begin a cycle with other symbols. But notice the void. There are several symbols all together that do not start cycling at the beginning of the message. Other boxes show areas where there are interesting clusters of beginnings or endings of cycles. Minimum number of alternations is 7.
So what I want to do is count where a symbol starts cycling with one symbol after stopping a cycle with another symbol. And then experiment with shuffles and messages. Maybe if we have ABABAB and later ACACAC, we can make a new symbol for A toward the end of the message so that we get ABABAB and later DCDCDC.
EDIT: I am also on the fence for prime phobia.
OK.
Which measurements do you think are most important to mimic from Z340?
My list would include: unigrams, bigrams (period1 and period19, and maybe flipped period 15), cycles, and pivots. I’m on the fence about prime phobia, other bigram biases (top vs bottom half, even vs odd positions), and repeating fragments.
I want to settle on a reasonably small list so the hillclimber for the generator doesn’t have too many objectives to optimize for.
I don’t know really. I’m currently focusing on bigrams and smokie is on the encoding, these are to me the big two. How exactly did Zodiac encode the 340? And what was the state of the plaintext prior to it. The pivots and all of the other strange stuff in the 340 are really cool and out there but I can’t work with them, it’s to strange and I’m hoping they are isologs. For my upcoming test it would be useful to me if you could create a few ciphers that match the 340’s flipped bigram count of 41 at period 15. Thanks.
Thanks for running through these ciphers. I was mainly interested to see their cycling behaviour. Perhaps the mathematical cipher that I created way back is not so uninteresting after all as it shows similar stats "for perfect cycle score (L=3)". That’s allot to consider though.
But this:
jarlve non-repeat score 2:
Z340: 1599
jarlve_mimic: 2321
The measurement that sums the average distances from the middle, the spread of the symbols, always seems to be the problem with any thorough polyalphabetism.
Manipulation operation(s): Flip(17,20,1), Diagonal(16,22,1,1,2,1)
Could you remind me how to interpret the arguments to your manipulation functions?
Flip(dimension_x,dimension_y,snaking option)
Diagonal(dimension_x,dimension_y,(un)transpose,shift,diagonal direction,snaking option)
A 1 means no snaking and a diagonal direction of 2 should be top-left corner, reading upwards. I have 8 possible directions. Thinking of it I may have confused transposition with untransposition again. I’m not sure wether it is. But a 1 should be transposition. The shift is what makes it diagonal. Suppose I want to diagonally transform the following string in the codebox.
123 456 789
Then I just do a shift of +1.
123 456 789
And read it in from top-to-bottom. Or possibly shift downwards and read differently, whatever you prefer.
I recently discovered that my diagonal routine is not able to do all transpositions. For instance, if Zodiac transposed into a 16 by 22 rectangle then he would have had 12 nulls. My routine only considers these nulls to be at the bottom-right position of the rectangle. While that could be a start point and then the nulls would from a triangle in the top-left corner of the rectangle.
There are a couple of contextual words, and I have gotten a similar solution before when I tried different transcription directions. This one must be the same as one of the ones that I tried.
Wow, you guys are working hard on this.
What I meant by rotation of the key is something like this:
Encode with this key…
Oh, I see. I’d like to see such a cipher.
This gap, or void in the diagonal row of positions where symbols begin to cycle at the beginning of the message is intriguing. I updated my spreadsheet so that it can show the start and end positions of 150 cycles and I can specify a maximum and minimum number of consecutive alternations. After fixing some conditional formatting problems and setting the number of alternations to 6 or more, here is the diagonal row, with the void ( first 85 positions only showed ). Those symbols may be cyclic, but they do not start cycling in any long cycles at the beginning of the message.
Now I shuffle the message. No diagonal row, and no void.
So that is another way to see the intentional cycling. You can see where the cycles start by symbol and by position. The vertical axis on the left is the symbol, and the horizontal axis on the top is the position. Why is there a group of symbol positions all clustered together where a cycle does not begin, but where their neighbors in the first two rows are mostly positions where a cycle does begin?
Smokie, I had trouble seeing your charts so here are direct links to the image, so I can use my browser to zoom in an out and scroll around:
http://zodiackillersite.com/download/file.php?id=5351
http://zodiackillersite.com/download/file.php?id=5352
Which cipher is used for those charts? Z340 or your cycled key cipher?
The 340 and 340 shuffled, only to position 85. I am going to keep working on the charts. Tonight I will put in the first 340 of the 408 to see if there is a diagonal row.
For my upcoming test it would be useful to me if you could create a few ciphers that match the 340’s flipped bigram count of 41 at period 15. Thanks.
OK – the generator is running now.
I expanded the number of cycles allowed for my spreadsheet chart so that it could accommodate all of the 408 cycles with at least 6 consecutive alternations. And I shaded very lightly in yellow and white alternately 2 rows, or 34 positions. So that a white area is 34 positions, and the next lightly yellow shaded area is 34 positions. Just to give some perspective.
Sorry about the size of the picture. Below is the 340, and I also lightly shaded in grey the symbols where the void is. Most of those symbols start cycling eventually in the message, but not in the beginning. I also marked with a blue box, scroll way to the right, the positions 205-272. Despite the fact that the 340 is not as cyclic as the 408, shown below, there are 19 cycles starting at 9 positions in that region with a minimum of 6 consecutive alternations.
Here is the 408, where you can see the characteristic diagonal row, but also that a lot more cycles start in the beginning of the message. Scroll way to the right to positions 205-272, marked by a blue box. Only 6 cycles start in the same four rows, and all at position 243.
This is gnarly complicated. But I am wondering about some type of subtle double cycling of the key. One that wouldn’t destroy the period 19 repeats ( EDIT or the L=2 cycles ), but which would ( EDIT: destroy the L=3 cycles, and ) make the message polyalphabetic enough so that it is difficult to solve.
Some other thoughts. Does the presence of multiple true cycles in the beginning of a message cause false cycles at the beginning of a message? But as the true cycles progress into the middle and end area of a message, do they not overlap as much so as to cause false cycles? Because there are so few cycles starting in the last eight rows of the 408 ( the last four shaded areas ).
That is it for tonight.
