O.k. I am working on it and will soon post a lot in a gmail file.
I have tried different shapes the same size as a pivot with transposed plaintext, and it is dead on accurate. Probably because of the size.
Really liking pivot sliding right now.
I went with 3 symbols horizontal and vertical and 4 symbols because of the axes, periods 19, 38, 57 and 76. It’s a little fuzzy but close look is self explanatory. The only spike was period 38 with vertical 3 symbols deleted. Period 38 four symbols deleted did not have a spike.
Tomorrow or soon I will iterate through periods 1 to undetermined high period sliding both pivots to see if there is a spike higher than the spike at P38.
I was a bit challenged about how to present the data, but came up with a simple way.
The idea was to slide one pivot, and two pivots LRTB.
1. I iterated through periods 1-85 four rows.
2. For each period, I made a heatmap which shows bigram repeat values at each position where a pivot, or both pivots, were deleted. Axes symbols included.
3. I put the heatmap values into a horizontal array.
4. I marked the positions, 141 for the left pivot, 180 for the right pivot, and 141 again for both pivots.
5. Then I sorted the heatmap bigram repeat values high to low in the horizontal array.
6. Then I sorted all of the arrays by the leftmost column, cells of which hold the highest count of bigram repeats.
I only show 20 columns for efficiency.
Here are the results. Left pivot only, looking at position 141.
An explanation about how to read the data. O.k., so the top row is P19, because the highest count of bigram repeats are at period 19 in the leftmost 20 columns. But, you don’t see a green value, meaning that deleting the left pivot did not result in a high repeat count at P19 at position 141. Go to the fourth row down. At P4 ( just a coincidence ), at position 141 there were 55 repeats. But, there were seventeen other positions in the heatmap with higher repeats other than at position 141. Got it?
Scroll down. At period 80, position 141 had the highest count of repeats, marked green. BUT, the repeat count isn’t very high, only 36. So that’s why it is close to the bottom of the list. No interesting periods show up at position 141, the left pivot.
Here is the right pivot, looking at position 180. At period 57 ( 19 x 3 ), there were 42 repeats when the right pivot was deleted at position 180. Even though a lot of periods show higher repeat counts above period 57 in the table, none of the high counts appeared at position 180 unless there were a lot of other positions with higher spikes. At period 81, the repeat count was highest at position 180, but there were fewer repeats than at period 57.
Both pivots, looking at position 141. Period 8 does the best, with 51 repeats when both pivots are deleted at position 141 ( where they actually appear in the message ). A close second is period 38 ( 19 x 2 ), with 50 repeats. No other period from 1 – 85 has similar results as position 141 in concerned.
Hopefully that makes sense. I will work on the mirrored this week.
Both pivots, looking at position 141. Period 8 does the best, with 51 repeats when both pivots are deleted at position 141 ( where they actually appear in the message ). A close second is period 38 ( 19 x 2 ), with 50 repeats. No other period from 1 – 85 has similar results as position 141 in concerned.
How are you counting repeats? Your numbers are not intuitive to me and my results seem to differ.
A few bigram results while removing both pivots + axes:
– period 8 jumps from 17 to 24,
– period 19 jumps from 37 to 22,
– period 38 jumps from 24 to 29,
– period 41 jumps from 17 to 28,
– period 117 jumps from 15 to 28.
Sorry about the slow progress, I have been under the weather a little bit.
I count AB AB as two, AB AB AB as three and AB AB AB AB as four. If you only count the actual repeats, for AB AB you would be counting 50%, for AB AB AB you would be counting 66.7%, and for AB AB AB AB you would be counting 75%. Probably the best way to count is the number of symbols involved in total across the entire message. I agree, however, that our stats are different, and the analysis is less than perfect. It was difficult for me to think of a way to distill the heatmap data.
I am on mirrored period 30 right now. For example, period 30 maxes out at period 138 and period 139, just a few positions away from position 141. However, position 141 will show up high in the final data.
Sorry about the slow progress, I have been under the weather a little bit.
The heat right?
Left pivot one pivot slide: Periods 8 ( again ), 13, 30, 47 and 57 ( again ) look interesting.
Right pivot one pivot slide: Period 35 looks interesting.
Both pivots two pivot slide: Period 30 ( mirror image of period 38 ) looks interesting. Period 57 shows up.
Here is that solve for smokie58, it has one wildcard. With a new transposition solver, the current one needed a rewrite and I had been dreading this for a while. It basicly is a totally fresh and very optimized rewrite (fully hard coded and whatnot) of the current one with more stuff. Saturday I spent allot of time thinking how to implement nulls and decided not to go with it because of nonaffinity with my design model. The new stats Ngrams and PC-cycles show their full effect here and will help in deciding if a solution is fruitful exclusive from its AZdecrypt score.
Score: 21255.01 Ioc: 0.07536 Ngrams: 872 PC-cycles: 2256 Reverse None Columnar(20*17,TP,C:2) Snake(17*20) ELIKEBELLINDPERPL EBECAUSEIAISSOMUC HFUNITISMORECURTH ARBIELINGMIESGAME INTHEFORRESABECAU SEMARISTHEMOSTDAR GERRUSANIMALOCALL TOKILLSOMETHINGDI VESMETHEMOSTAHREL EINGEXPERENCEIAIS EVERBEATERTHANDET AINGDRURROCKSRCOM EAHADERLTHEWESTPA RARFETISAHATTHENI TIEETILEBEREBORNI NPARADICEANDALETH EIHAVEBELEESMILEB ECOMEMOSLAVESIMIL LROADIVEDRUMONAME BECAUSEORUNILETRO
O.k.
Does anybody know if you can upload an animation to this site? I have never made one, but have been thinking about it.
Yes:
In that example I simply used image tags to embed the link to the image.
I think only GIF is supported, however. Some other animation formats might not embed well here.
O.k.
Does anybody know if you can upload an animation to this site? I have never made one, but have been thinking about it.
If you want to share allot of numbers you could use google spreadsheet.
No, I had just thought to animate some of my pictures at some point. I work on the 340 every day, if only for a few minutes, but haven’t had time to look into it further.
I did some experimenting with adding and removing two pivots to a message. Adding that many nulls in one area of the message typically does not create a shift of 1 period spacing. It is interesting that we have always wondered why period 29 and not 30, or why 39 and not 38. Then to find out if we remove the pivots, yes, you get a shift from 30/39 to 29/38. But P15/19 goes way down. I don’t know what to think but will keep it in mind.
I am looking forward to trying your new transposition solver.
I don’t know what to think but will keep it in mind.
Perhaps a hard to understand mechanic related to the positions and repeats of the pivots.
The new transposition solver is going well.
Basicly I ask questions such as: is one initial temperature setting better than another one? And then wait for about 7 to 8 hours to get a reasonable answer. In that time frame the solver has done about 100 restarts with 10.000 transposition iterations on one of my systems. I have experimented with a tabu list ( https://en.wikipedia.org/wiki/Tabu_search ) as a long-term memory which proved not so good, perhaps it will do better as a short-term memory. It may also just be a thing with simulated annealing ( https://en.wikipedia.org/wiki/Simulated_annealing ) which seems to prefer freedom and a smooth searchspace above anything else.
The old and new tranposition solvers are both using a bigram beam search ( https://en.wikipedia.org/wiki/Beam_search ) which then pass a single best bigram result to the substitution solver. That is considered one iteration. If the substitution solver scores it higher than the previous best score then the key is kept and iterated on. Essentially the substitution solver is deciding if one bigram result is better than another and simulated annealing is applied at this level to fix an ideal ratio between random walk and greedy hill climbing. This works really well for most ciphers but if there is no obvious bigram peak then it does not. I am now trying to minimize the bigram beam search so that it is still effective but increases the weight of the substitution solver to get better results for ciphers with no obvious bigram peaks.
The reason why I went with a bigram beam search was because of problems with ciphers that have stacked transpositions such as: period 15 + mirror. The aforementioned also has a peak at period 19 that serves as a strong local maximum without some form of looking ahead. Instead of using bigrams in the beam search the substitution solver could be used but that would slow down the search immensely. Something for another time. While I have mentioned beam search, simulated annealing and tabu search the interpretations thereof are my own and are most likely not exact.
Inspiring and interesting video: https://www.youtube.com/watch?v=4bM3Gut1hIk
Their idea is to take a computationally difficult task and translate it to a problem our brains do naturally. From what I understand they take information (in this case programs to reverse engineer) and output it in any amount of dimensions (digraphs for example) for visual analysis. Then n-gram and statistic models are applied to colorize and define the visual object. The user then learns how things look, in their case they are able to differentiate between compressed data, image data, functions, algorithms and whatnot. Their takeaway is that new ways to analyze and understand information are needed to overcome stagnation.
To translate the concept over to the cipher world, I think we could take and combine a bunch of statistics into a visual representation thereof and identify cipher types as such.
