I am still working on it, but not as much. I have had some other stuff happening and couldn’t devote a lot of mental energy lately. The daily posts can be exhaustive and feel more like a grind that fun sometimes ( but more often fun). I may post less frequently in the foreseeable future but maybe with more comprehensive work chunks. I am going to continue working on route transposition, including ( trying ) to detect inscription shape, inscription route, and transcription route. I actually have what may be some really interesting / cool spreadsheets and stats, but am still working on them at this point. If nothing else maybe I can get a job making exotic spreadsheets. That’s what I really like doing.
Keep up the good work, Smokie! And by all means, don’t burn yourself out. It’s too easy to do so on this case.
Here is my summary of test ciphers, posted in its own thread to try to help de-clutter this one.
Wow, doranchak, it looks like you have been very busy. Thanks for all of the work. Are your multiobjective evolution ciphers transposition ciphers?
I am still slowly working on it. I spent a week working on a new spreadsheet that compares the lists of period 1 – 40 repeats with each other, for a total of 1600 comparisons. With the original message compared to the original message and the original message compared to the mirrored vertically flipped message. I wanted to see if I could definitively detect any incomplete inscription rectangle transposition or possibly a rail fence transposition. But I didn’t find anything.
I am working on trying to narrow down possible geometric inscription shapes, and have an experiment all set up to run. Because of the period 15/19 stats, it is likely a rectangle, but I want to know if there could be multiple inscription rectangles of the same period, and if so, how many could there be.
There is also something that has been bothering me for a while. Why there are so many period 15 bigram matches with the mirrored version as compared to the number of period 19 bigram matches for the original. I have to take a closer look.
I also want to pursue skipped or added nulls some more.
And I want to pursue the idea that some of the symbols are polyalphabetic. Especially high count non-cyclic symbols that map to high frequency letters. I have an idea for using ZKD to do that. When a transposition cipher is not polyalphabetic, ZKD scores do not vary much. I was thinking about making some messages with increasing counts of polyalphabetic symbols, using ZKD on them, and then compiling result stats. Perhaps there could be a correlation between variance of ZKD scores and count of polyalphabetic symbols. The fewer polyalphabetic symbols, the less "confused" ZKD is and the lower the variance in scores. The more polyalphabetic symbols, the more "confused" ZKD is and the higher variance in scores. Make a lot of messages with increased counts of polyalphabetic symbols and compare score variances to actual 340 score variances. See what happens.
I will post my multiple inscription rectangle experiments soon, which should help to get a rough approximation of just how many plaintext in any transposition scheme must be adjacent to their original plaintext message adjacent plaintext to get similar counts of period 15/19 matches. To help rule out a lot of possible inscription shapes. I already tried rail fence, with two rows, and will include that with my experiment. It is the same as multiple two column x 15 or 19 row inscription rectangles.
I have also been looking at a lot of different cipher types and trying to find one that is not a route transposition but which also can create a lot of period x repeats. That’s the real question, and I don’t have an answer. So far, nothing.
Looks like you have been very busy yourself!
No, my evolved ciphers are simple substitution for now. I want to generate a few more of them before moving on to transposition ciphers. I feel like I am going in too many directions, so I want to stay focused and gradually get there, one scheme at a time.
I look forward to hearing more about the results of your experiments on geometric inscription. Sounds like hard work going through all those possibilities!
The period 15/19 thing bothers me too. And those damned pivots — what scheme would cause both the pivots AND the 15/19 thing?
Keep up the good work!
I am only working with 11 different multi inscription rectangle schemes to try to get a rough idea about whether any suggested shape of scheme could produce so many period 15/19 repeats.
This shape, of the 11 schemes, has the highest count of adjacent plaintext. The plaintext in the leftmost and rightmost columns have only one adjacent plaintext each, for a total of 1 x 15 x 2 = 30. And the plaintext in columns 2 – 21 each have two adjacent plaintext, for a total of 20 x 15 x 2 = 600. 600 + 30 = 630 adjacent plaintext sides that, when transcribed, could produce period 15 repeats.
This is the other extreme. There are 2 x 15 x 11 = 330 adjacent plaintext sides that, when transcribed, could produce period 15 repeats.
I have nine other configurations, with one to 9 gaps. I am going to make a lot of random messages, track some simple stats, and get an idea about how many adjacent plaintext sides it takes to produce so many period 15 repeats. So when thinking about possible inscription shapes, I can decide whether any particular shape could practically create so many repeats.
That’s quite interesting! It does seem like that kind of scheme could account for the periodic bigram stats of the Z340. I still wonder how the pair of pivots might be emerging from such schemes.
As far as the pivots are concerned, the approach that I know how to use is to make a lot of messages and see how easy or difficult it is to create them by random coincidence. Because there are 63 symbols and a certain distribution of those symbol counts, I suspect that it may not be that difficult for the pivots to appear at random. But it would have to be tested.
I only have a couple of hours a day to work on this, so here are all of my inscription schemes for experimentation purposes:
Each scheme has a decreasing number of adjacent plaintext sides. So I am going to make 100 random messages from the plaintext library, with random symbol selection within the homophonic cycles that map to each letter to closely match 340 cycle stats, for each scheme. Then make a graph showing how many of the 100 messages for each scheme have more than… probably 60 period 15 bigram matches. That way if I am wondering if a trapezoid ( or whatever shape ) of a certain width and height, for example, is a practical idea. I can compare the number of adjacent plaintext sides in that trapezoid ( or whatever shape ) to the values on the graph.
Some of the old cryptography books show transposition schemes used for military purposes, and they show simple inscription squares of 5 x 5 which make for fewer transcription mistakes in the heat of battle. That is how I got the idea.
Mr. Lowe, your were right about rail fence. But the number of columns must be exactly right to make period 15 ( or period 19 ) repeats, considering the number of rows that there are. A two row rail fence could create of a lot of period 15 repeats, if the number of columns is 29 or 30. As the number of rows gets bigger, so must the number of columns to achieve that. Best case scenario is the scheme at the bottom of the picture above. But you will soon see that even with that, it is probably not likely that the scheme could have been used to create the 340.
Here is a typical key for all of the messages ( left ). It will vary depending on letter counts, but does not diffuse symbol frequencies as much as possible. That is why there are only six symbols that map to E instead of eight, similarly with the other high frequency plaintext, and oppositely with lower frequency plaintext such as P.
The most common bigrams are TH, HE, IN, ER, AN, and RE. So I will map symbol 1 to A, I and N ( blue ), and map symbol 14 to T, H, E and R ( red ). That will approximately mimic the B and + symbols with count, cycle activity, and count representation on the list of period 15 repeats for each message. In other words, the + symbol especially, is involved with a lot of the period 15/19 repeats, and this scheme is intended to mimic that. Without such polyalphabetic mapping, the number of period 15/19 repeats would be lower. See what happens when I run the experiment on the first scheme, one simple rectangle.
On the right is a typical symbol distribution as compared to the 340. It will be different each time, but roughly close to the 340.
Random symbol selection within ciphertext cycle groups will be 10% in rows 1-5, 20% in rows 6-10, 30% in rows 11-15, and 40% in rows 16-20. The average random symbol selection will be 25%, which is close to the 340. The difference in cycle scores between the first half and second half may be a bit higher than with the 340, but that shouldn’t affect the period 15 bigram match counts.
I will work on it more throughout the week.
hi smokie.. I see you have put a lot of effort in.. I will try to follow your works as best i can.. yes the rail fence has its problems for the longer diagonal repeats. and we nearly exhausted skytale, hope we did not miss anything because that suits the (15) (19) so well but not the pivots..
hope you get a result soon.
ps.. crashed the new Lenovo third week..total seizure.. got it back yesterday.
cheers
As far as the pivots are concerned, the approach that I know how to use is to make a lot of messages and see how easy or difficult it is to create them by random coincidence. Because there are 63 symbols and a certain distribution of those symbol counts, I suspect that it may not be that difficult for the pivots to appear at random. But it would have to be tested.
A long time ago, I did a random shuffle test of the 340 and found that a pair of pivots, oriented in the same directions, occurs in about one per million random shuffles.
And when I studied pivots I found that if you create normal substitution ciphers, the chance of pivots forming goes up a lot if there are many repeated strings or phrases in the plain text. We could suspect then that maybe Z340’s plaintext might have such repeated phrases, but so far the evidence seems to point away from normal homophonic substitution in the normal reading direction.
Looking at your inscription schemes, I have a few questions: What do the orange boxes (GGESTSOMET) represent? Also, how do the gaps affect (if at all) the transcription of the rectangles into the final cipher text layout?
I should probably get a copy of your reference books to catch up to the inscription schemes you’ve been exploring.
I should have shown the inscription sequence instead of the plaintext for a message.
Shown by the original plaintext position numbers below, I inscribe left to right top bottom, filling the rectangles one at a time. Then transcribe vertically.
It is a way to make period 15 repeats. But if someone un-transposes assuming one big inscription rectangle, the period 15 bigram repeat symbols will be re-connected into period 1 repeats but the message will not be solvable because the maximum word fragment that can be recovered is only the width of the rectangles.
The orange boxes are just the last ten plaintext that don’t make an entire column. I left those out of the inscription.
I don’t have the book titles at hand. I took all of the books from the library shelf and read through the transposition chapters in each. One of the books was by John Laffin, and that I think is the one that showed a row of 5 x 5 inscription squares and described the use of smaller transcription shapes for military purposes. Let me see if I can find a bit more info and edit this post.
EDIT: I am almost positive that it was Cryptography: The Science of Secret Writing by Laurence D. Smith 1955. It is a very good book.
I will take a closer look at the pivots that you are talking about.
EDIT: The article about pivots was a very good read. I wonder if there was some type of scheme that could have caused that. What about a grille cipher? I will take a closer look at grille ciphers and learn about them soon.
What about a herringbone pattern? Bricklayers have used them for centuries, and they are commonly found in patios and walkways. Maybe Zodiac had some herringbone linoleum flooring on his bathroom floor and he thought that it looked cool.
See: https://en.wikipedia.org/wiki/Herringbone_pattern
I may have to temporarily suspend my multi inscription rectangle experiment to take a closer look at the pivots. I already have a cool cross period spreadsheet, and after plugging in period 2 and period 34 I started finding a lot of them. It would probably be a good idea to compare period 1 with period 17, period 2 with period 34, period 3 with period 51, etc. See how many pivots there are and if they are statistically significant as a whole.
It looks like I missed one at column 11 row 12. There are probably more.
Could a herringbone transposition scheme cause a lot of period 19 repeats?
Thanks for the clarification of your approach. It makes a lot more sense to me now. I worry about all the little variations in transposition schemes, which make these attacks very difficult. I think I will gradually work my way from one scheme to the next only when I’m sure all the test ciphers under a particular scheme are definitively solvable with zkdecrypto or azdecrypt or similar. Or hopefully you will stumble on the correct transposition scheme which will save us all a lot of time!! 
I went ahead and ordered a copy of the Laurence D. Smith book.
Speaking of grille ciphers, this caught my eye:
https://en.wikipedia.org/wiki/Grille_(cryptography)#A_Third-Party_Grille:_the_crossword_puzzle
Some people have suggested crossword or word-search layouts could explain the appearance of the pivot patterns.
I finally completed my experiment to determine the likelihood for Zodiac using multi inscription rectangles for a transposition scheme. I made 3,300 random messages as described a few posts above.
Here are the period 15 matches ( message mirrored horizontally ). Note that there are 71 count.
And here are the simple statistics. On the x axis is the number of plaintext "adjacent sides" for a multi inscription rectangle scheme. For instance, with one 22 x 15 rectangle, there are 630 adjacent sides. Each plaintext has two sides, left and right, for which to create a possible period 15 match. But the plaintext in the leftmost and rightmost column only have one side. ( 20 columns x 15 rows ) + ( 2 columns x 15 rows ) = 630. And each time I increase the number of rectangles, the fewer adjacent sides there are. If I use two inscription rectangles, there will be 30 fewer adjacent sides. And with three rectangles, 30 fewer adjacent sides again. And so on.
On the y axis is the count of messages that had a threshold minimum number of period 15 matches.
For the first trial, the threshold was 60 matches. It was relatively easy to make a message with 60 period 15 matches and one big rectangle. Out of 100 trials, there were 25 such messages. As the number of rectangles increase and count of adjacent sides decrease, making such messages became more and more difficult. The second trial threshold was 65 matches, and the third trial threshold was 70 matches. To be in the ballpark of 70 matches, there were only 6 messages out of 100 using one big inscription rectangle. The graph lines are not very smooth because I only made 100 messages for each rectangle scheme and threshold. They would be much smoother if I made more messages. But the basic idea is that to make messages with 340 stats, it is not particularly easy to use multiple inscription rectangles. See the green line.
The somewhat imprecise conclusion is that the inscription scheme was more likely one big inscription rectangle instead of multiple rectangles or any other geometric shape that does not have a high count of plaintext "adjacent sides." There could have been more than one inscription rectangle or some other shape, but that is less likely.
.
