smokie62a (from 59 to 38 symbols):
Score: 8758.62 Ioc: 0.03723 19 36 32 41 40 47 40 52 40 32 36 13 16 54 50 16 16 47 5 32 2 13 55 9 6 52 36 13 13 43 44 23 17 16 32 54 32 16 40 40 10 52 16 13 50 27 32 38 16 2 54 9 23 20 27 13 25 10 5 32 18 45 26 16 20 38 4 23 26 39 32 55 22 10 5 52 38 40 36 20 44 54 20 39 10 45 55 4 25 9 10 38 39 13 25 13 41 56 39 39 36 10 29 27 52 20 16 38 27 32 27 38 25 52 19 43 40 29 52 25 27 10 34 13 39 25 16 10 13 2 38 36 39 19 10 10 39 54 10 36 39 16 23 27 55 39 39 25 9 13 36 25 36 4 9 36 27 54 40 29 26 55 39 4 2 27 10 35 10 25 38 27 10 25 9 9 16 10 17 40 18 41 47 13 17 39 40 32 20 16 26 52 7 13 9 26 16 10 2 52 32 13 32 18 40 32 25 27 44 2 16 13 57 50 36 38 9 17 52 54 25 16 4 38 20 26 47 45 18 10 13 25 29 9 27 31 16 40 56 32 25 55 39 10 22 9 13 9 55 2 41 40 36 26 7 34 36 5 2 41 4 13 2 38 27 36 19 10 54 54 2 4 39 4 2 50 4 38 25 47 23 43 27 10 9 9 39 27 13 52 45 52 2 26 36 32 36 54 27 38 41 35 38 20 27 13 52 31 27 27 20 7 9 27 27 4 9 17 2 10 13 27 40 47 17 56 27 4 10 39 10 2 34 47 22 17 38 35 9 9
smokie62b (from 60 to 38 symbols):
Score: 8244.06 Ioc: 0.03123 41 14 35 47 44 43 56 21 41 57 51 14 51 3 58 23 19 45 19 35 34 17 51 54 1 44 43 24 2 29 35 23 23 51 14 31 3 10 17 34 58 27 43 17 58 29 38 19 11 7 24 14 23 10 32 3 24 3 1 29 17 24 10 51 58 29 22 44 40 17 12 3 38 50 54 51 21 47 29 58 24 3 33 44 12 12 14 58 44 10 19 47 12 11 29 35 28 23 44 27 17 54 31 2 58 7 51 54 50 31 33 58 45 22 29 38 14 31 22 43 29 3 11 51 47 47 41 54 10 21 21 8 14 23 50 38 12 32 8 14 21 11 23 33 51 10 47 45 41 45 54 38 57 3 10 32 32 41 24 12 45 24 29 17 41 33 3 7 51 19 38 23 31 45 51 54 11 51 57 22 29 51 26 24 14 14 1 28 41 54 29 58 56 38 29 23 3 38 57 1 1 11 14 32 24 43 23 29 34 21 51 12 33 17 24 12 12 24 14 30 44 11 56 51 3 51 43 3 30 23 8 45 26 58 50 29 14 58 51 14 24 44 10 31 10 17 2 24 3 10 12 24 43 38 57 7 7 29 24 19 58 41 45 47 41 2 2 58 32 31 26 22 27 12 1 54 29 51 14 43 11 54 31 41 21 38 14 32 12 33 12 21 24 35 2 34 10 1 12 43 54 7 31 54 38 22 29 29 32 50 24 50 23 2 29 29 21 56 41 21 1 30 22 50 29 24 30 30 14 56 58 43 10 41 38 29 50 24 3 38
I haven not taken a thorough look yet at smokie61ab and smokie62ab but noticed that they all have a high degree of encoding randomness. What did you do?
AZdecrypt 1.08 BETA: https://drive.google.com/open?id=0B5r0r … kp0QTZWeGs
The collapse sequential homophones functionality can be found at the bottom of the solver list. The alphabet szie setting can be found under solver options with a minimum of 3 and maximum of 255. If it is set higher than the amount of symbols in the cipher then it will automatically downscale.
It is not optimized yet and you may want to run it like 5 to 10 minutes per cipher.
All have about 25% randomization, and similar cycle scores to the 340.
61A is 17×20 period 20 transposition + homophonic and there are only 20 different letters of the alphabet in the message. I wanted to see what would happen. 61B is the same exact plaintext, digraph + 17×20 period 20 transposition ( making 26 letters of the alphabet ) + homophonic.
62A is 20×17 inscription rectangle with diagonal inscription starting upper left corner LRBT + redrafting at 17 columns + homophonic and there are 21 different letters of the alphabet. 62B has the same plaintext, inscription and redraft + digraph ( making 26 letters of the alphabet ) + homophonic. Un-transposed correctly there is a strong odd even bias, but un-transposed at period 19 which is the spike there is not an odd even bias. 62AB were inspired by the Indianapolis Speech transposition scheme.
Very cool new feature, Jarlve. It reminds me of the algorithm "REMOVE_HOMOPHONES" described in the King/Bahler paper.
Very cool new feature, Jarlve. It reminds me of the algorithm "REMOVE_HOMOPHONES" described in the King/Bahler paper.
Thanks. It crossed my mind too.
I have optimized the algorithm to be 20 times faster, improved its output, fixed a bug and changed the name to merge sequential homophones: https://drive.google.com/open?id=0B5r0r … zJxbjlRZms If you have been doing any testing with the previous shared beta then I highly recommend you run them again with this new version.
AZdecrypt merge sequential homophones update, fixed 2 bugs and improved its output: https://drive.google.com/open?id=0B5r0r … 0laV01NOGs
I am totally loving it.
Awesome, Jarlve! Does it work in batch mode? I’m eager to run it on the thousands of transposition candidates I’ve run through the regular solver before.
Awesome, Jarlve! Does it work in batch mode? I’m eager to run it on the thousands of transposition candidates I’ve run through the regular solver before.
https://drive.google.com/open?id=0B5r0r … 0laV01NOGs
I have added it for you. File —> Batch ciphers (merge sequential homophones). Not sure how many iterations and restarts you may need to consider though it seems to be less demanding than the substitution solver.
If you set the alphabet size to 255 then the solver will always try to use the maximum amount of symbols per cipher, often producing better results for ciphers with allot of encoding randomness while using more symbols.
If you do no want the following string/information "Ngrams: 1234 P:C cycles: 12345" to appear in the solution then it can be disabled in the solver options menu by setting output additional stats to 0.
This is great – thank you so much!
What does the alphabet size mean in relation to merging of symbols? Does it mean the max number of symbols that are allowed to merge together per plaintext letter?
The alphabet size equals the maximum number of symbols allowed in the solution. Setting it equal to the number of cipher symbols is safer, especially for ciphers with encoding randomness. As it can then use more cycles that have fewer symbols or 1:1 substitutions for symbols that do not cycle well.
Oh – so it splits symbols as well? (Converting 1:1 polyphones to unique symbols?)
No splits. If the alphabet size is 26 then the solution cannot exceed 26 symbols.
Example: alphabet size 26 and the 340 as input. The hill-climber has to merge 63 cipher symbols down to 26 solution symbols or less. And with the encoding randomness of the 340 the hill-climber probably has to do merges that it does not like to meet the alphabet size. Thus, setting the alphabet size equal to the amount of cipher symbols is generally safer because the hill-climber will only assume merges/cycles it likes. Setting the alphabet size greater than the amount of cipher symbols will not cause splits, instead the alphabet size caps to the amount of cipher symbols at runtime.
Would it be clearer if I rename alphabet size to maximum solution symbols or similar?
That makes sense to me now, thanks. Maybe "target alphabet max size" or similar would be a clearer name.
Hi,
long time no see =)
I am very busy at the moment and so I have not much time for z340. Nevertheless I still read this forum to be up to date. Jarlve, I really like the new features in AZDecrypt, nice work!
When I read about the strange behavior of row 14 I asked myself: "How is this related to the bigram peaks at p15/p19"? So I ran a test and checked how many bigrams would be found if row x was removed and period n was applied (first removed the row, then applied period n) Here are are the results:
z340 unmodified:
Removed row 1 -> Highest bigram count: 36 at period 19 Removed row 2 -> Highest bigram count: 32 at period 19 Removed row 3 -> Highest bigram count: 37 at period 19 Removed row 4 -> Highest bigram count: 35 at period 19 Removed row 5 -> Highest bigram count: 31 at period 19 Removed row 6 -> Highest bigram count: 33 at period 19 Removed row 7 -> Highest bigram count: 30 at period 19 Removed row 8 -> Highest bigram count: 29 at period 65 (24 at period 19) Removed row 9 -> Highest bigram count: 30 at period 19 Removed row 10 -> Highest bigram count: 35 at period 19 Removed row 11 -> Highest bigram count: 31 at period 19 Removed row 12 -> Highest bigram count: 31 at period 19 Removed row 13 -> Highest bigram count: 31 at period 19 Removed row 14 -> Highest bigram count: 31 at period 19 Removed row 15 -> Highest bigram count: 35 at period 19 Removed row 16 -> Highest bigram count: 30 at period 19 Removed row 17 -> Highest bigram count: 30 at period 19 Removed row 18 -> Highest bigram count: 34 at period 19 Removed row 19 -> Highest bigram count: 35 at period 19 Removed row 20 -> Highest bigram count: 35 at period 19
z340 flipped left to right:
Removed row 1 -> Highest bigram count: 38 at period 15 Removed row 2 -> Highest bigram count: 34 at period 15 Removed row 3 -> Highest bigram count: 38 at period 15 Removed row 4 -> Highest bigram count: 35 at period 15 Removed row 5 -> Highest bigram count: 34 at period 15 Removed row 6 -> Highest bigram count: 36 at period 15 Removed row 7 -> Highest bigram count: 35 at period 15 Removed row 8 -> Highest bigram count: 32 at period 261 (28 at period 15) Removed row 9 -> Highest bigram count: 32 at period 29 (30 at period 15) Removed row 10 -> Highest bigram count: 34 at period 15 Removed row 11 -> Highest bigram count: 34 at period 15 Removed row 12 -> Highest bigram count: 33 at period 15 Removed row 13 -> Highest bigram count: 32 at period 15 Removed row 14 -> Highest bigram count: 34 at period 261 (33 at period 15) Removed row 15 -> Highest bigram count: 38 at period 15 Removed row 16 -> Highest bigram count: 33 at period 15 Removed row 17 -> Highest bigram count: 32 at period 15 Removed row 18 -> Highest bigram count: 35 at period 15 Removed row 19 -> Highest bigram count: 36 at period 15 Removed row 20 -> Highest bigram count: 37 at period 15
Removing row 14 has a big impact on the bigram counts at p15/p19. If it is removed, the amount of bigrams is decreased significantly. To me it seems that row 14 is important for the cipher and it is no filler.
Row 8 looks interesting, decreasing bigram count a lot.
What do you think?
