Do you guys want the key and the solution?

Yes I do!! And to the 340 as well.

Well done so far doranchak, how did you go about it?

I cheated.

Smokie identified which of your expanded symbols were correct (six of them so far). I combined that information with all his posts regarding which of your plaintext positions were correct. Then in zkdecrypto, I loaded the ciphertext with the six symbols expanded, and locked all the positions corresponding to the correct plaintext, and let zkdecrypto explore the rest of the key.

At this point, I think an exhaustive search is within reach (all selections of the two remaining polyalphabetic symbols). But it is not a good test, since it relies on cheating!

Without cheating, it is so easy for hillclimbers to get stuck in local optima since the multiplicity gets so high. I am wondering if a more sophisticated language model (such as markov chain or expectation maximization) would be more effective than simple ngram models.

Well done anway.

I could try to crack the smokie33 with correct expansion of all symbols but it will be something for later. Just to see if multiplicity is the problem or not.

What exactly is a markov chain? I’ve seen that name before in relation to the range problem of n-grams, is that correct? And I’m not sure how EM works either. The main design goal of AZdecrypt has always been speed so I feel happy with what I have now. Which seems to be an excellent combination of solving strength and speed.

The new version is running right now on a new project on my solving machine, which is a dual Xeon X5650 (Dell T5500). Two sessions, 10 threads each, the first session is doing 14.1 million iterations per second and the second session is doing 12.5 million iterations per second. That’s possibly 53 ciphers per second at 500k iterations, at which the new version has a more than 90% chance of solving the 408 (poor example).

Do you guys want the key and the solution?

I don’t need it right away but maybe later on.

What exactly is a markov chain? I’ve seen that name before in relation to the range problem of n-grams, is that correct?

I am not too knowledgeable about them. My basic idea of them is that they are used to represent transitions from one event to another. For example, given the ngram "SPEA", there is a probability of transitioning from that to the next ngrams. Moving from "SPEA" to "KING" would have a relatively high probability. A Markov chain model of English would consist of a large network of those "events" and all the transitions that can come from them. So, the goal of a hillclimber can be to match the candidate plaintext to the Markov chain model built from an English corpus.
Here’s a paper about the use of Markov models to solve ciphers: http://scholarworks.sjsu.edu/cgi/viewco … d_projects

And I’m not sure how EM works either.

Same here. I’d love to experiment with it one day though.

The main design goal of AZdecrypt has always been speed so I feel happy with what I have now. Which seems to be an excellent combination of solving strength and speed. The new version is running right now on a new project on my solving machine, which is a dual Xeon X5650 (Dell T5500). Two sessions, 10 threads each, the first session is doing 14.1 million iterations per second and the second session is doing 12.5 million iterations per second. That’s possibly 53 ciphers per second at 500k iterations, at which the new version has a more than 90% chance of solving the 408 (poor example).

Wow, that is quite impressive!

Here’s a paper about the use of Markov models to solve ciphers: http://scholarworks.sjsu.edu/cgi/viewco … d_projects

Here’s the version of that paper that was updated for publication in Cryptologia this year: http://zodiackillerciphers.com/images/v … ty2016.pdf