AABAAB spikes

ABAAABA spikes ( not quite as interesting )

I made a 340 SYMBOL CHART, and put it online. I sorted the symbols by count, and included P15, P29, P78, pivots, ABABABAB+, ABAABAAB+, AABAABAA+, ABAAABAA+ ( which I picked because they actually look like patterns ), and the regional bias symbols that only appear in the top and bottom 4 and 5 rows. I tried to keep it as simple as possible and left out P19 and P39.

Some observations:

* symbols with count 8-12 are largely responsible for P78, but are not included in any perfect cycles
* almost all symbols with count 6-7 are in perfect cycles, but most of them are not in P78
* four out of five of the regional bias symbols are not in perfect cycles
* when sorted by count, some groups appear to be biased to certain cycle types, especially those count 4 biased to ABABABAB

It is very interesting to me that the regional bias symbols are not in any perfect cycles. I have done more and will show this weekend but I have to go to work now.

You can look at it the spreadsheet here: https://drive.google.com/file/d/1LI7TbF … sp=sharing

So a couple of days ago, I had this idea. The regional bias symbols, what if they are nulls? They are not in any of the cycle patterns. I have my shape sliding spreadsheet, and decided to slide all 26 combinations of 2, 3, 4, and 5 of the symbols from position 1 to position 340, deleting those symbols with the configuration, or shape, at that position, and see what happens to P15, P19, P29, and P39.

If it is a transposition, and some of the symbols are nulls, then there should be a spike when I delete the symbols at the positions where they are located in the message, but probably decrease the repeats at all other positions.

Some observations:

* symbols with count 8-12 are largely responsible for P78, but are not included in any perfect cycles
* almost all symbols with count 6-7 are in perfect cycles, but most of them are not in P78
* four out of five of the regional bias symbols are not in perfect cycles
* when sorted by count, some groups appear to be biased to certain cycle types, especially those count 4 biased to ABABABAB

Good observations. It all seems related.

Here are my 2 most likely hypotheses for the high unigram distance in the 340:
1. The high unigram distance of the 340 is related to the group of symbols that do not appear in the middle 7 rows (as marie said).
2. A long/efficient key (as Largo said) is used in the homophonic substitution process and some of the most frequently occuring symbols are for whatever reason not part it or are wildcards. Likely hypotheses for the symbols that were not taken up in the homophonic substitution or are wildcards are that these are 1:1 substitutes, plaintext nulls or wildcards (as smokie said). I would like to go as wide as possible with the interpretation of wildcards.

Compared to the 408 the 340 has a high ioc (inefficient key) which can mainly be attributed by the symbols that occur more than 7 times.

Suppose that all symbols occur equally frequently then we could say that frequency distribution is perfectly flat. In measuring flatness in a way the 408 is 87% flat while the 340 is only 66% flat. But after removing all the symbols that occur more than 7 times the 340 is 84% flat. Thus, the symbols that occur more than 7 times in the 340 could account for the increased key inefficieny over the 408.

My way of calculating the flatness of a frequency distribution is as follows. It returns a value between 0 and 1.

flatness_by_ioc=(((cipher_length/cipher_unique_symbols)*((cipher_length/cipher_unique_symbols)-1))*cipher_unique_symbols)/raw_ioc

Yes, it does all seem related. If symbols are separated into groups by count, then those groups behave differently and contribute to different things.

Anyway, of the 26 different combinations of 2,3,4, and 5 of the 5 symbols, and checking P15, P19, P29 and P39, only one combination when sliding resulted in increasing the repeats at the position where the symbols actually are, as compared to all other positions. It is the W and the Theta symbol at P19. There are 11 of them total. The heatmap below.

They are not in any of the cycle patterns, but, when deleted at their positions and only their positions, make an increase in P19 repeats as compared to all other positions.

340 on the left, and without the W and theta symbols on the right, P19. There are 118 cells highlighted on the left, and 123 on the right. The first thing that I noticed was a new <S P19 at the top of the message, it is my favorite P19, but one near the bottom is destroyed. It is the most improbable of all P19, and it remains the most improbable, but one is destroyed and one created.

It also looks like there are more cells highlighted at the top, and the void in lower right corner is more filled in or some cells shifted so it doesn’t look like there is a void anymore.

Here are the P16, P32 and P48 unigram repeats. Some minor changes in the top and bottom part of the message, but slight increase in highlighted positions from 56 to 57. I do not notice anything remarkable.

340 period 19 bigram heatmap while removing one character at a time.

340 mirrored period 15 AZdecrypt solve substitution heatmap while adding one character at a time. Scores are as high as 21068 @ position 200 which is the start of the "RJ|*" pivot. Though the PC-cycles stat is low which means that the substitution does not follow the cycles which is a bad indication for a partial solve.

Score: 21068.44 Ioc: 0.07457
Ngrams: 383 PC-cycles: 193

Add character(200), Period(UTP,15)

SDEHERELITOODREAP
ULINESNGHOLADOFST
IMEFORNOHIMSETTOA
SSEHASARAROSEARDO
IREDIEWEDINPINCAL
ITIONTHISCUOMPOLL
UPANDCOOTATNEREGE
TISIDUESHAHAREATH
STATINTHANDOFSIAA
NDLETHEDELASHOFTH
ETOFOURDENTOFGALI
ANCEISFORETHECTAN
DANONTHEDATANTCON
CEASFRPHOENTHATLS
WASTHANDITHGETRAI
NSONTHESHELLIFEST
HESTSETHNARCHCLOT
HEREDSSARASPARIAN
CENATHADINALLATIO
FFICEHHAROMHEPROD
L

Well, there are a lot of different transposition possibilities, even if the W and theta are nulls. My first thought, since they are 11 positions, is to take out the last 9 symbols also, the signature, starting with the circle cross hairs. That would give us a clean 320 symbol message, or 20 x 16 diagonal inscription. It could be something like that, and he planned 17 x 20 ahead of time, and planned the signature ahead of time, and realized he needed 11 nulls to make a perfect rectangle and that is what he did. I may try to solve the 20 x 16 idea, or think of other transposition of 320 that could give us P19. Maybe one of these days we will get lucky. Feel free to suggest or whatever.

How about a periodic transposition + null solver for AZdecrypt. The user would be able to determine the period and the amount of null characters to add or remove.

Well, use your time wisely. Today I am going to work on diagonal inscription, redrafting messages into whatever number of columns, reading by diagonal rows, drafting into horizontal rows, and solve by row bound. You have tried before and I may have, but I don’t think that I have a spreadsheet that does everything automatic. I have one, but it is for making matrices. I want one for messages. Easy task for me today.

EDIT: I already tried to solve by expanding the +, deleting the W and theta, then redrafting into 19 columns, transposing at 90 degrees, and also mirroring, flipping and flipping and mirroring that, all rows bound. I didn’t delete the last 9 symbols. I got a few interesting words and very short phrases, but… not a great solve. I did it on Wednesday. You might want to try that and see. Maybe instead of writing a program, just deleting those symbols and playing around with it.

Another memory of a long ago conversation with my poi who was explaining something about programming, specifically COBAL, computer grid paper and time-sharing. He had been working on something and was halfway through or something along those lines (or may have finished) when he realized he’d made a mistake and then thought it was too much trouble to go back and fix it so he did something else instead (used a null?).

My first thought, since they are 11 positions, is to take out the last 9 symbols also, the signature, starting with the circle cross hairs. That would give us a clean 320 symbol message, or 20 x 16 diagonal inscription. It could be something like that, and he planned 17 x 20 ahead of time, and planned the signature ahead of time, and realized he needed 11 nulls to make a perfect rectangle and that is what he did. I may try to solve the 20 x 16 idea.

I tried this and it didn’t work, + symbol expanded. I also tried the mirror of this and it didn’t work either. I was pretty amazed though about deleting the W and theta in the top half of the message and creating another <S P19. I will keep working on the 340 at my own pace, probably looking at the isomorphic patterns in closer detail and the suspected nulls as well. I guess programming is the most efficient use of time. Thanks for what you have done but peace and freedom with whatever you do.

Score: 23750.59 Ioc: 0.05758
Ngram size: 4.63888
Ngrams: 238 PC-cycles: 21

G (0)
WH (0)
EOI (29)
KOHL (70)
DIAYU (105)
OUGHSI (310)
DGETOFO (434)
THEINGPE (579)
BRYSOFATL (552)
MIWHERMYIS (615)
OGRANTINUES (909)
ECYWHENYOUGH (1020)
TOFTHEPANDBLA (1201)
UFLYINCOMEDOWA (1188)
WORTHANTHECCIHE (1267)
ISTHTHINGPERSENT (1426)
BALLTHIINATSTEAC (1234)
DAMEREADOFORREME (1452)
WMENTSSTICALLOFU (1501)
NRELATEDNISTRUNK (1380)
POTTERTHEMOFMYL (1398)
YAINTWASTHEOTH (1327)
WLANDATFORTHS (1122)
EYASCOMPARAT (1010)
NYHAVETWILL (932)
RGATHEGUYI (767)
AAHOWIMRU (500)
LDCOMIWA (450)
MORWHER (440)
FUIGER (179)
GSDON (209)
ERRY (94)
USI (36)
SC (0)
Y (0)

How about a periodic transposition + null solver for AZdecrypt. The user would be able to determine the period and the amount of null characters to add or remove.

I am working on it and hope to have something by the end of next weekend. If so, I will share a build of AZdecrypt here. AZdecrypt has over 30000 lines of code now and there is allot of progress.