Zodiac Discussion Forum

Homophonic substitu…
 
Notifications
Clear all

Homophonic substitution

1,434 Posts
21 Users
0 Reactions
304.3 K Views
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

I’m going to read your approach after you get the results, and we can go from there.

A preliminary analysis of M3 shows that it is much more cyclic than the 340. There are a few missing symbols in some of the higher scoring cycles, but only a few. Not sure why. Top half is comparable to bottom half.

The scattergraph of count versus total score shows basically the same thing. Most of the dots are bunched up really close together in the respective count columns, showing the strong cycling. Symbols 5 and 13 are on the far right because of their high count and they cycle together. But there are a few missing symbols in that cycle too. Symbols 1, 3 and 7 have count 13, but they do not cycle together well, even though I thought that they did at first. I had to check manually to see what was going on. Their scores are high and similar, I am guessing, because they have such high count. With 13 count, there are going to be some short to medium false cycles with some of the other 50+ symbols just because of probabilities.

I am a little bit burned out tonight, so I lazily marked the latest cycle hillclimber with colored boxes showing symbol groups that need further investigation as possible cycles. Because of the strong cycling overall, I am guessing that maybe at least half of them are grouped together correctly, if not more. There are 91 bigram repeats on my list. I haven’t tried to solve it yet.

 
Posted : August 11, 2015 5:30 am
daikon
(@daikon)
Posts: 179
Estimable Member
 

Nothing to report regarding wildcard solves in Z340. I’ll let it run overnight, just to be sure, and then move on to the next set of possible wildcards.

Didn’t notice that there is a new Mystery Cipher — #3. I’ll check it out tomorrow too.

 
Posted : August 11, 2015 10:01 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

Thanks for introducing me to the world of 340 bigram repeats. We have way fewer of them than we should, and look at all of those that have suspected wildcards in them. Why? Maybe it is because the suspected wildcards are high in count. I think that the relationship between symbol count, cycling, randomness, bigram repeats, and whatever else is fascinating. The fact that a large proportion of the bigram repeats include symbols that don’t cycle with other symbols. Is that significant?

I marked the symbols that I strongly suspect as being wildcards in green, the weaker candidates in grey. So nineteen out of 46 repeats include symbols 19 or 20. Let’s say, hypothetically, that Zodiac created these 19 repeats during the masking process. So then take these out for a minute, and we have 46 – 19 = 27 bigram repeats that were created by the cipher’s cycle step and not masked. Total count of symbols 19 and 20 = 36. With 36 symbols, and masking bigram repeats with only one symbol per repeat, we can mask 72 repeats. 72 + 27 = 99 bigrams to start with. Isn’t that about what we should have without the mystery step?

So those are interesting numbers. And I am also thinking about randomization of cycles and how that can increase or decrease the number of bigram repeats. I did just a little bit of experimentation, and it seemed like randomization will lower the number of repeats. But not every time. When I make the Tolkien message, I actually randomized several times until I had a high number of repeats to work with. So that is something that can be studied further also.

Daikon, I read your post. I have privately tried to expand many of the suspected wildcard symbols and used combinations of them. I have a huge spreadsheet of messages with that happening. It didn’t work for me, but I wasn’t as patient as you, and am not as good at solving these messages. Someday I would like to do that and then merge suspected cycle symbols to decrease multiplicity. If you make a cyclic message and randomly plop… I think about 40 but can’t remember exactly… wildcards so that ZKD cannot solve. And then you expand those wildcards and merge the known cycle symbols, the program will solve.

EDIT: I am going to have to find ways to study the cycle behavior or each individual symbol in a different way so that I can compare them in different ways. I think that it may be interesting the compare the cycle activities of the symbols in the 340 bigram repeat list, and make the same comparison with practice messages. Later.

 
Posted : August 11, 2015 6:15 pm
(@mr-lowe)
Posts: 1197
Noble Member
 

Not sure if this has been performed yet.. Can you take out the top two, three or four top symbol counts and replace them with a space. Making sure the space is not a part of the code! Maybe show the space as an underscore. I know it’s reducing the size of the code but I’m wondering if it may give a partial break that could lead you to what he used as a format for a substitution system. I’m in NY and along way from a real computer so I can’t run that myself for a few weeks. The Reasoning behind this is that the human brain only needs partial words to read. It’s ea-y to re-d a s-nte-ce ev-n if -t is mis–ng bi-s and piec-s. Wlel not so Esay but you get the piutcre.

 
Posted : August 11, 2015 8:05 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

I tried to take out the alleged wildcards, and it didn’t solve. I think that replacing with a space is similar to what daikon was talking about. Your are welcome to try. I just have a little laptop with a spreadsheet and ZKD. And intermittent OCD.

 
Posted : August 11, 2015 9:59 pm
daikon
(@daikon)
Posts: 179
Estimable Member
 

Tried to post earlier, but my internet is super flaky today, so I couldn’t until now. I’ve now tested the following sets of wildcards:
{‘+’,’B’,’p’}
{‘+’,’B’}
{‘+’,’p’}
{‘B’,’p’}
{‘+’,’B’,’W’}
{‘+’,’W’}
{‘+’,’B’,’F’}
{‘+’,’F’}
No obvious solutions. The top scores were also nowhere near the expected level for a good solve.
I believe this exhausts all possible combinations of the top 3 symbols: ‘+’, ‘B’ and ‘p’. The next 4: ‘|’, ‘c’, ‘F’ and ‘O’, all repeat 10 times, but the number of combinations of 2-3 symbols out of a set of 7 (42+210) is kind of large, considering it takes several hours to get a reliable indication that it doesn’t solve to anything. I guess I’ll keep going through all of them, since I don’t have any other better ideas to test at the moment. I think Smokie Treats likes ‘F’ as another possible candidate for a wildcard, so I’ll prioritize it.

 
Posted : August 11, 2015 11:39 pm
 Soze
(@soze)
Posts: 810
Prominent Member
 

OCD was obvious yesterday when you said you were going to bed but struck at it again. Sucks doesn’t it. To get something on your mind and can’t shake it cause you think if I just tried this or maybe if I did that. Keep up the work though. You will figure it out.

Soze

 
Posted : August 11, 2015 11:40 pm
daikon
(@daikon)
Posts: 179
Estimable Member
 

Not sure if this has been performed yet.. Can you take out the top two, three or four top symbol counts and replace them with a space. Making sure the space is not a part of the code! Maybe show the space as an underscore. I know it’s reducing the size of the code but I’m wondering if it may give a partial break that could lead you to what he used as a format for a substitution system. I’m in NY and along way from a real computer so I can’t run that myself for a few weeks. The Reasoning behind this is that the human brain only needs partial words to read. It’s ea-y to re-d a s-nte-ce ev-n if -t is mis–ng bi-s and piec-s. Wlel not so Esay but you get the piutcre.

Yep, that’s exactly the idea of "wildcards" we are working on in this message thread. Or was it another thread? I lost track. 🙂

 
Posted : August 11, 2015 11:42 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

OCD was obvious yesterday when you said you were going to bed but struck at it again. Sucks doesn’t it. To get something on your mind and can’t shake it cause you think if I just tried this or maybe if I did that. Keep up the work though. You will figure it out.

Funny, I started joining Zodiac sites about 3 years ago so that I could have something else Zodiac-like to do without working on the 340, which doesn’t take as much time and mental energy. That lasts only for a while. When the Ross Sullivan thread on this site started showing no immediate prospects for new information, I drifted over to this thread. And here I am. Somebody find out some new info about Ross Sullivan, please.

So, with the suspected wildcards being so prominent on the bigram repeat list. They could be 1:1 substitutes, but if they are, then it would seem to me that they would also have to map to high frequency letters so that they would appear on the bigram repeat list. But the high count could have something to do with that also. So I will look at the other messages in the suite and draw some comparisons to see of the high count 1:1’s appear prominently on the respective bigram repeat lists.

Thanks for doing all of this work daikon. I understand that you are expanding the suspected wildcards to multiple symbols, correct? After you are done we will have to check to see if Haze or Tolkien have lower multiplicities; I don’t know off of the top of my head. There is a certain point where the solvers won’t solve with expanded wildcards. I played around with this in the previous thread. See: viewtopic.php?f=81&t=267&start=130.

 
Posted : August 12, 2015 1:49 am
daikon
(@daikon)
Posts: 179
Estimable Member
 

Thanks for doing all of this work daikon. I understand that you are expanding the suspected wildcards to multiple symbols, correct?

Yep.

After you are done we will have to check to see if Haze or Tolkien have lower multiplicities; I don’t know off of the top of my head.

For Tolkien one, I expanded to 115 symbols (top 3 symbols are wildcards) and got a pretty good solve. It’s actually solvable if you only expand to 102 symbols (top 2). For Purple Haze, which is a much harder cipher even without wildcards, I expanded to 97 symbols. It ended up being fewer than for Tolkien because 2 out of 3 wildcards are not the most frequent symbols in the cipher. Expanding 2 wildcards wouldn’t solve, it only worked with 3, unlike with Tolkien cipher. So my solver can work with multiplicities up to about 3 characters per unique symbol (i.e. length of cipher divided by number of unique symbols, which is not the classic definition of multiplicity, but I find it easier to work with and remember "5" or "4" instead of "0.185" or "0.238"). But it takes a few hours and many-many restarts to get the correct solution, instead of seconds for lower multiplicities. Expanding the top 3 symbols in Z340 gives you 107 total symbols, and even less with other symbols, so if it’s anything like Tolkien/Purple Haze, it should be solvable. We just need to hit the 3 wildcards exactly. Assuming Z used wildcards of course. 🙂

 
Posted : August 12, 2015 4:00 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

Well I thank you for all of that work. It sounds to me like the wildcard hypothesis, at least with these symbols, may be resolved. And I am glad to some extent, and will think about it and possibly pursue it some more.

I performed an analysis to answer the question about whether the 340 wildcard suspects could have been used to mask bigrams and that is why they appear in the bigram repeat list, or whether they simply occur in bigram repeats because of their high count. It looks like it is because of high count. Look at the results below, where I compared the suite messages that have high count 1:1 substitutes. All in all, high count 1:1 substitutes account for about half of the bigram repeats, across all of the examples. Note that although the 340 has only 46 bigram repeats, C_S4_P3 has only 52 bigram repeats, and R3_S4_P3 has only 51 bigram repeats. So the 340 bigram repeat list is not necessarily extraordinarily short. It has to do with the message and the key, and maybe with a second step.

I probably should have done this first before making the Purple Haze and Tolkien messages, but that’s o.k., I learned a great deal. Has anyone ever written a computer program that randomly creates thousands of messages based on different variables, including key, cycling and randomization, and tallies up statistics to compare with the 340? Maybe that could establish some parameters that will give some perspective. I don’t know. Instead of hillclimbing the solution, what about hillclimbing the cipher to match 34o stats?

There are cycles in the 340, but they are fractured somehow. I will be thinking of another way that Zodiac could have used a second step, after encoding the cycles, to make the message unsolvable. Some other way of examining the pieces of cycles that remain. There are a lot of symbols. I thought that maybe Zodiac could have used a lot of low count symbols to mask bigram repeats, but that’s as far as my thoughts have gone. I see other possibilities.

Thanks again.

 
Posted : August 12, 2015 4:52 am
doranchak
(@doranchak)
Posts: 2614
Member Admin
 

Instead of hillclimbing the solution, what about hillclimbing the cipher to match 34o stats?

This is exactly what I’ve been working on. Will let you know how it goes once I have anything to report.

http://zodiackillerciphers.com

 
Posted : August 12, 2015 5:30 am
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

I tried daikon’s symbol expansion idea on suspected wildcard symbols "+", "F" and "B" and came up with something that just might be a 1st stage solve or an artifact of high multiplicity. Some things can be made out "A good man", could possibly also be "A dead man", "by the barn", could possibly also be "by the farm" and some other smaller fragments. Having seen so many solves I found this one interesting enough to share, the consonant/vowel structure of the message is also quite dominant for my solver.

Edit: found a 1955 book "A Good Man Is Hard to Find" which interestingly enough is about serial killers and features escaped prisoners. https://en.wikipedia.org/wiki/A_Good_Ma … rd_to_Find

I’m still taking somewhat of a break and will catch up with the rest of the thread when I’m fully rested.

A good man?

agoodmancessalera
ndbythebarnstopso
ftwithatertoahean
dddiamcontdepropr
inthentasurchfiel
dcoalmeroadeforat
humphbyrailenseas
ternsthewinnewman
atthestorywitheip
remandangeditistr
avolungsanddeltpe
rtimonsoulitsofth
eendthinsullityco
meinasuriesiningi
talsowindgonature
sistemptedassther
oilbasedthemresoa
ffordericedalltod
sheartinginecordi
otaandedetechspyk

HER>ÐÌ^VPËI²LTG±Ä
NÐ!"¢·OºDWy•<»KÆ£
#Ÿ„ÃM$uZGW¢£L·¤HJ
SÐн^̾»V´ÐO%&RK±
¸¼M'¤ÔÊÄIµ(P)ˆ³Ë/
оR^*ÌO-»ÄCË,>±D¢
·µ.KѺƒ±uÃXGV•¤LI
¢G±JÆÊ·O0¸Ny¤1¹L¼
Ä<M2Â3ZR±45ßA°³K
-¤ÌuV6^J7Oн<89Ÿ-
u:R/µÔEIDy;ÐÂTMKO
±<ÃÌRJI»µT³M•=ˆ?@
¤°¼SŸ·QNIµUY⃾R
ÌG[N^Ƶ±³Â•ÃV³Ô]
Ÿ_X²»„³¼CE>VuZµ-`
IÕ´¤aK¢OÐ^•ÆMÑG±
RÃTbL²°C<cdÌWeI£L
fg£WC¤WÃPOSHT/¢£Ð
IhËÄW<½Ôi¸yOj»-CÃ
>MDHNÐËS¤ZO¾AIKƒk

AZdecrypt

 
Posted : August 12, 2015 2:26 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

I have a question. Has anyone ever tracked to see if there are "hot spots" in the 340 where ZKD seems to find more, longer words? Has anyone ever kept track to see if long words or short phrases seem to appear on certain rows more than on other rows? Has anyone ever added a subroutine that made a list of words found and the positions of those words? I cannot imagine nobody ever having thought of or tried that. Just wondering because of the masked bigrams idea.

 
Posted : August 12, 2015 3:53 pm
(@masootz)
Posts: 415
Reputable Member
 

from jarlve’s "a good man", broken down by recognizable words where applicable, kept in the same order as his "solve". not sure if this is helpful but it was interesting to me –

a good man
cessaler
and
by the barn
stop
soft with a
ter
to a
he and
dd
i am con
tdepropr
in the
nta
surch field
coal
me
road
e
for at
humph
by rail en
sea sterns the win new man at the story with
eip
remand an
g
edit is
travo
lungs
and
deltpertimon
soul its of the end thins
ullity
come in as
urie
sining it also wind go nature
sis
tempted ass the roil based them
reso
afford
e
rice
d
all
tod
she art in gin
e
cord iota and
ede
tech
spy
k

 
Posted : August 12, 2015 4:39 pm
Page 7 / 96
Share: