I can’t even solve your "unwildcarded" cipher with AZdecrypt, it probably suffers from the same problem as daikon3 (word entropy, same words repeating over and over again). This troubles me a bit.

I would guess the reason for this cipher being harder to solve is that it’s the song lyrics. It doesn’t have proper sentences. It has words next to each other that usually wouldn’t pair, like "blowing blowing". It has contractions that are usually not present in a written text ([actin’] and [‘scuse]). Basically, it is likely to be underrepresented in the corpus that was used to create the N-gram stats.

With ZKDecrypto or your own solver? I can’t seem to solve it with AZdecrypt096.

My own. It takes a few restarts, but it converges on the correct solution about every 5th restart. The "hard cipher" benchmarks, tonyb 1 and 2, both require more restarts for my solver, so it’s not the hardest.

I don’t exactly understand some of the things that you are talking about, but I started out usually masking the bigrams near the top of the message in general, until I realized that I needed to distribute them a bit more evenly. I also generally substituted the first symbol in the bigram and not the second symbol, if that helps.

Maybe I can do another one without song lyrics later. Purple Haze has 339 letters, so it was a good fit, and I couldn’t think of anything else at the time.

I started with symbol 37 and kept working with that for a while until trying to mask a bigram was easy with 37. Then I switched to 49, then to 51, then to 59. The first several of each wildcard symbol is really easy. Most of it is pretty easy.

So it was mostly random. I was hoping your process would create a cycle of 37, 49, 51, 59, 37, 49, 51, 59, etc. throughout the cipher, which could be detected. But now that I think about it, it is rather unlikely as it would mean each wildcard would be present roughly the same number of times.

Jarlve, there were three high count 1:1.

13 maps to M, count 18;
17 maps to R, count 15;
24 maps to Y, count 14.

Yes, it was random. It would be fairly difficult to select the wildcards in the masking process so that they would show up in the message as a cycle because the bigrams just appear in different places. But, the +, the B, and the F don’t cycle well with other symbols. This process would explain why.

If I am right about Zodiac using the wildcards to mask the bigram repeats (Jarlve thought of it before I did), then it may be possible to verify. A + where a cycle symbol is missing will be sitting between two adjacent symbols. We may be able to trace to the other bigram that was not masked using those adjacent symbols. We may be able to reconstruct the cycles with some sort of a tracing process.

EDIT: Here would be an example. Cycle 6 30 37, which starts in row 1, ends in row 20, and is missing only one 30. Maybe the most reliable cycle that we have. Identify the possible wildcards for 30 where 30 is missing. Then check where 30 ( the * ) is elsewhere in the message and compare adjacent symbols. Was the 51 here a 30 before Zodiac masked the bigram 29 30? What are the odds of finding two adjacent 29’s like that (I don’t know)?

What about comparing all trigrams where + is in the middle to all of the trigrams where the B’s and F’s are in the middle. See if there is any evidence that he could have shifted wildcards to avoid creating a new bigram.

I made another post that didn’t post, but it was short.

Here is another prospective sister bigram that I found right away. It’s for cycle 16 14. EDIT 16 40 – a typo; I only found one of two.

I wonder if further study of prospective sister bigrams can show a statistical significance?

Smokie

Here, two out of three prospective sister bigrams for cycle 11 36, which starts in Row 1 and ends in Row 20:

You guys, I want to get this solved.

I decided to take a break for a while. Everything is okay, just proctecting myself from overextension. May the doge be with you.

Wow. Such exile. So sad. Much sleeping.

Wow. Such exile. So sad. Much sleeping.

Hehehe

No problem, Jarlve. I was actually thinking about taking a break myself. Regroup and refresh. Sometimes a break has a really good positive effect because you have the time to quietly mull things over for a while and come up with good new ideas. I should probably hold my horses on the "prospective sister bigram cycle reconstruction" idea. There are just so many uncertainties. Thanks for making the suite, there is plenty for me to do when you are gone if I feel like working.

Regroup and refresh. Sometimes a break has a really good positive effect because you have the time to quietly mull things over for a while and come up with good new ideas.

Totally agree!

This morning I realized why I over did it on the masking. I was trying to get close to 340 numbers, where the list of bigram repeats is count 46. I started with a list of 104 count bigram repeats, and masked 42 bigrams. But since the new list didn’t show the unmasked sister bigram repeats, I reduced the list by twice as much as I had intended.

The question is, if I had only used a wildcard count necessary to get the bigram repeat list down to about count 46, would the message still have been unsolvable? To get from 104 to 46, I would have only had to use count 29 wildcards.

I wonder how many bigram repeats Zodiac initially had to start with before alleged masking. Note that a lot of Zodiac’s bigram repeats after alleged masking include several suspected wildcards. So perhaps Zodiac did more masking than the minimum necessary to get to count 46, but because he didn’t carefully track how many bigram repeats he was actually creating when masking, the list of repeated bigrams after masking was longer than he intended. On the other hand, the alleged wildcards in the 340 bigram repeat list are high count, making them more likely to cause bigram repeats.

Final bigram repeat list count = initial bigram repeat list count – ( wildcard count used in bigram masking step * 2 ) + bigram repeats created by bigram masking step.

Makes me think about the + with count of 24 and the count of other alleged wildcards. So this is probably obvious to everyone in the room except me, but I have to write it down at least for myself.

340 factors include:

1. The message;
2. The key, which distributes 63 symbols among high, medium and low frequency letters in a way that causes a certain number of bigram repeats;
3. The randomization of the cycles, including which cycles, how much, and where;
4. The bigram masking step, including how many unique wildcard symbols and how many of each unique wildcard symbol; and
5. Mistakes

Smokie

I decided to take a break for a while. Everything is okay, just proctecting myself from overextension. May the doge be with you.

Sorry to hear! Don’t leave us for too long, it won’t be the same without you. 🙂

I should probably hold my horses on the "prospective sister bigram cycle reconstruction" idea. There are just so many uncertainties.

I think you idea about sister bigrams is good. But I’m not sure how far you can take it, as you said, there are too many uncertainties. Another thing that occurred to me. The ‘+’ symbol in Z340 is the prime suspect of being a wildcard, correct? Why does it appear right next to itself (i.e. double ‘+’) three times?! If the wildcards were used to reduce bigram repeats, here he creates a very obvious bigram repeat. He could’ve easily used one of the other wildcards. It would’ve reduced the overall number of ‘+’s as well. I can’t explain this.

The question is, if I had only used a wildcard count necessary to get the bigram repeat list down to about count 46, would the message still have been unsolvable? To get from 104 to 46, I would have only had to use count 29 wildcards.

That is definitely worth exploring! The more tests we have/run, the more reliable the empirical conclusion we can make based on the results. I would also use a different plaintext, as your un-wildcarded cipher is already pretty hard to solve, so adding even a small number of wildcards will likely render it unsolvable. Something from Z’s letters?