This is really great, Jarlve. I am so happy that you did that. Even though the solve above isn’t perfect, it is still a pretty good solve. You can read a lot of it. I say when you are ready we test it on various different types of transpositions, a variety of messages, plines and transpositions with various disturbances. Compare with untranspositions and the regular solver, solve independent chunks of messages. Before attacking the 340. Thanks.

You are welcome and thanks. I will finish the row solver and then update AZdecrypt so that you can give it a try. For the moment being and individually, I would prefer to do as little as possible though.

O.k., that is fine.

That pline idea looks very interesting. I have to catch up and read the previous posts since I worked on some own ideas the last few weeks but for now I have an interposed question:

Load the following plaintext into peek-a-boo:

ASIWASGOINOVERTHE
CORKANDKERRYMOUNT
AINSISAWCAPTAINFA
RRELLANDHISMONEYH
EWASCOUNTINIFIRST
PRODUCEDMYPISTOLA
NDTHENPRODUCEDMYR
APIERISAIDSTANDAN
DDELIVEROHORTHEDE
VILHEMAYTAKEYAITO
OKALLOFHISMONEYAN
DITWASAPRETTYPENN
YITOOKALLOFHISMON
EYYEAHANDIBROUGHT
ITHOMETOMOLLYSHES
WORETHATSHEDLOVEM
ENONEVERWOULDSHEL
EAVEMEBUTTHEDEVIL
TAKETHATWOMANYEAH
FORYOUKNOWSHETRIC

Now resize it to 20 x 17 and apply a diagonal transposition (the default one in the diagonal transposition dialog). What you get is a transposed text that shows bigram peaks at period 19 and flipped + period 15. Nothing new so far.

AISNTKRIPLEICTANEONN
SAIRRRAAENTUDRADTANO
WOEOETCRONDNYDEIYEMU
GVCKNWRMUOAMNHAEPSOL
OEDUAASORLDATYNYIROA
HNOSFICPOETREOTHBMHN
AMINHSTTCSOKMTFIOTED
YSIDASSUDHASEODTEMLE
NANWRIDIOTIRLNEREUMV
TAEIPOARYHPLAMOVOEEH
LHFYRSEAFAAHOWOWVDTN
YIMPIVMOSKAHSLRAEEAO
NDNRIELAOETEDEEHKMFK
EEELHLWOYIHEVLTAOHUS
HIELATTYTSHEETTWAOWT
PDIKIIEHYSNHULTEYOEI
DVODYNGLTOSBIAYRNHRC

Now resize it back to 17×20 and encode it with the default key that comes with peek-a-boo. The bigram peaks are now at period 38 and flipped + period 15. The „original“ bigram peaks vanish.

dhoALjncY:aCbwDqx
X7R1i5GVtZd3A8rkn
DBsiq=lXy=+NeGX7H
RTk0hTP;vEQbjAlV;
S=Z;qFdfYzXu=gBrD
iKXtUHZLT7Tcn=d2R
XMICeY=awGxX8p9;F
AD;5q2osNb1=j;LIh
Xw3kTzcBiKMvHpZoy
=k8+;:07dRlVCB5Xs
htuAPnfS;QNDgcY=i
GTFYUZ;XQ=ax2:pIT
V13dIDiFXl=lQHLqT
C;Y5Q;XzjZ2Kutdy+
D=7kRnh0UiXPwfBga
pj;Ijx3y:Ful=Tc2+
QU8ZXprMFC0:dsNTL
o2Pfw8lD=lsYH5jhc
gpT1AFvuNaTXxCkQ=
BTqJULXz45iTG72Ve

This does not happen with all plaintexts tough. If you take the first 340 letters of z408 and do the transposition described above, the p19/15 peaks are preserved after the homophonic substitution. Obviously this depends on the plaintext and the key.
To me it looks possible that the p19/15 peaks in z340 could be a false positive and the „real“ period is a different one. AZDecrypt solves the cipher shown above within a minute (mode „Solve + transpose“). So a „false“ period peak could not be the only problem with z340.
Has this been discussed before?

This does not happen with all plaintexts tough. If you take the first 340 letters of z408 and do the transposition described above, the p19/15 peaks are preserved after the homophonic substitution. Obviously this depends on the plaintext and the key.
To me it looks possible that the p19/15 peaks in z340 could be a false positive and the „real“ period is a different one. AZDecrypt solves the cipher shown above within a minute (mode „Solve + transpose“). So a „false“ period peak could not be the only problem with z340.
Has this been discussed before?

Largo: Some thoughts. The 340 has a spike at p 15/19, but the bigram repeat count at period 1 is low. Evidence of transposition. If you do not transpose the plaintext, it is difficult to match these statistics. You can start with a plaintext that has a lower than average p1 and higher than average p19, then manipulate the key so that the p 1 bigrams are diffused more and the p 19 bigrams are diffused less. See this:

viewtopic.php?f=81&t=2617&p=43811&hilit=smokie18e#p43811

If you do not transpose the plaintext, then it is very difficult to match 340 p 15/19 stats. If you do transpose the plaintext, then it is much easier to match p 15/19 stats… and you can get phantom spikes at other periods depending on the plaintext and the key. I have been thinking about a small project, make one of these messages and then dissect it. If the 340 isn’t a transposition at periods 15 or 19, then how do we know it is a transposition at all? If it was then Jarlve’s program would have solved it anyway.

Pretty much everything you just said. A handful of null symbols, or skipped plaintext or symbols caused by transcription errors would cause misalignments in the untransposed message. I find it very plausible that someone could easily skip a few.

Check here for a list of possible transposition issues, and an attack plan that maybe could be used if there are disruptions or misalignments:

viewtopic.php?f=81&t=3196&start=330

That is why I am so excited about the independent row solver. I am hoping that the message can be solved with "sliding areas" plan and the independent row solver.

EDIT: Also consider scoring the bigram repeats according to probability. Here is my formula:

LN ( 1 / ( ( ( COUNT OF A / 340 ) * ( COUNT OF B / 340 ) ) ^ 2 ) )

Let’s say you have only four of symbol A and only six of symbol B, and there are three occurrences of AB at p 19. That is pretty good evidence. Make a list of all bigram repeats, score them, sort them by score, and then graph the distribution. It is easier, by far, to match these stats at the correct period, but not at a phantom period, or with a randomly shuffled message.

I just checked p15 and 19 for the 340 and the best result came from p15 but does not seem to be a solve comparing it to 16671 score of the 408 piece.

Ignore these results, there was a significant bug. Will redo it at a later stage.

Here’s another preview of your row solver concept smokie with added ngram scores in brackets.

Adjusted multiplicity: 0.15525 Average ngram size: 4.8
Score: 23387.92 Ioc: 0.06542

ILIKEKILLINGPEOPL (1060)
EBECAUSEITISSOMUC (1129)
HFUNITIAMOREFUNTH (928)
ANKILLINGWILDGAME (988)
INTHEFORRESTBECAU (1080)
SEMANISTHEMOATDAN (915)
GERTUEANAMALOFALL (828)
TOKILLSOMETHINGGI (1103)
VESMETHEMOATTHRIL (933)
LINGEXPERENCEITIS (1015)
EVENBETTERTHANGET (1115)
TINGYOURROCKSOFFW (980)
ITHAGIRLTHEBESTPA (1001)
RTOFITIATHAEWHENI (865)
DIEIWILLBEREBORNI (944)
NPARADICEANDALLTH (1020)
EIHAVEKILLEDWILLB (1036)
ECOMEMYSLAVESIWIL (886)
LNOTGIVEYOUMYNAME (960)
BECAUSEYOUWILLTRY (1126)
TOSLOIDOWNORATOPM (788)
YCOLLECTINGOFSLAV (1002)
ESFORMYAFTERLIFEE (976)
BEORIETEMETHHPITI (698)

Okay smokie, try this AZdecrypt demo update please: https://drive.google.com/open?id=0B5r0r … FZSVlF0ZEE

To use the 5-gram solver of the substitution + by rows solver simply select it and click solve. And to use the 6-gram solver variant, go to file and then click load ngrams, select the reddit 6-grams file. After it has loaded select the solver and click solve. On first use it will load the smaller ngram sizes in memory so wait that out. Also, it is not a performance solver and you may want to wait a while longer with solutions to appear.

Let me know if there is anything wrong, or if you have any suggestions or questions. If it works well then I suggest you use it as you please because I might not make this update formal for a while.

Hi Jarlve. Thanks a lot for all of your work. I just tried the row solver, but maybe doing something wrong. I made a message all homophonic, no transposition 17 x 20, then pasted only 15 x 20 into the solver cutting off two columns. The message seems to solve, but the output is one long row. It doesn’t look like your output. Thanks.

Hi Jarlve. Thanks a lot for all of your work. I just tried the row solver, but maybe doing something wrong. I made a message all homophonic, no transposition 17 x 20, then pasted only 15 x 20 into the solver cutting off two columns. The message seems to solve, but the output is one long row. It doesn’t look like your output. Thanks.

Nevermind, Jarlve. I put a period . at the end of each row and it worked awesome. You have contributed so much to this. Thanks a lot. This is a very useful tool.

Hi Jarlve. Thanks a lot for all of your work. I just tried the row solver, but maybe doing something wrong. I made a message all homophonic, no transposition 17 x 20, then pasted only 15 x 20 into the solver cutting off two columns. The message seems to solve, but the output is one long row. It doesn’t look like your output. Thanks.

Nevermind, Jarlve. I put a period . at the end of each row and it worked awesome. You have contributed so much to this. Thanks a lot. This is a very useful tool.

It worked to stop the rows and show a score at the end of each row, but it added symbols, the rows have more symbols. It must be a paste from Excel issue, not sure. How should I manipulate the message or use the period . ? Thanks

Hi Jarlve. Thanks a lot for all of your work. I just tried the row solver, but maybe doing something wrong. I made a message all homophonic, no transposition 17 x 20, then pasted only 15 x 20 into the solver cutting off two columns. The message seems to solve, but the output is one long row. It doesn’t look like your output. Thanks.

Nevermind, Jarlve. I put a period . at the end of each row and it worked awesome. You have contributed so much to this. Thanks a lot. This is a very useful tool.

It worked to stop the rows and show a score at the end of each row, but it added symbols, the rows have more symbols. It must be a paste from Excel issue, not sure. How should I manipulate the message or use the period . ? Thanks

O.k., I converted to ascii, the period is not the row stop symbol. I pasted into Word first, then removed the spaces. By the way, two of the messages in the library are the same. I know about it for a long time but never bothered to mention it. I think message 23 and 24. I made of them this, I like the plaintext a lot. Scroll down for solution row solve mode:

T	H	E	Y	H	A	D	A	H	O	U	S	E	O	F	C	R
Y	S	T	A	L	P	I	L	L	A	R	S	O	N	T	H	E
P	L	A	N	E	T	M	A	R	S	B	Y	T	H	E	E	D
G	E	O	F	A	N	E	M	P	T	Y	S	E	A	A	N	D
E	V	E	R	Y	M	O	R	N	I	N	G	Y	O	U	C	O
U	L	D	S	E	E	M	R	S	K	E	A	T	I	N	G	T
H	E	G	O	L	D	E	N	F	R	U	I	T	S	T	H	A
T	G	R	E	W	F	R	O	M	T	H	E	C	R	Y	S	T
A	L	W	A	L	L	S	O	R	C	L	E	A	N	I	N	G
T	H	E	H	O	U	S	E	W	I	T	H	H	A	N	D	F
U	L	S	O	F	M	A	G	N	E	T	I	C	D	U	S	T
W	H	I	C	H	T	A	K	I	N	G	A	L	L	D	I	R
T	W	I	T	H	I	T	B	L	E	W	A	W	A	Y	O	N
T	H	E	H	O	T	W	I	N	D	A	F	T	E	R	N	O
O	N	S	W	H	E	N	T	H	E	F	O	S	S	I	L	S
E	A	W	A	S	W	A	R	M	A	N	D	M	O	T	I	O
N	L	E	S	S	A	N	D	T	H	E	W	I	N	E	T	R
E	E	S	S	T	O	O	D	S	T	I	F	F	I	N	T	H
E	Y	A	R	D	A	N	D	T	H	E	L	I	T	T	L	E
D	I	S	T	A	N	T	M	A	R	T	I	A	N	B	O	N

e	D	<	p	E	/	9	0	F	X	i	a	;	Y	>	4	]
p	b	f	1	M		H	N	O	2	^	c	Z	T	g	G	:
	O	/	V	=	h	Q	0	^	d	3	r	e	D	:	;	8
B	<	X	?	1	V	<	R		f	q	a	:	2	/	W	9
;	l	<	`	q	R	X	^	T	I	U	B	r	Y	j	5	Z
k	M	7	a	=	<	Q	^	c	L	;	0	g	J	V	C	h
E	<	A	[	N	8	=	W	>	_	i	K	e	d	f	D	1
g	B	`	:	m	?	`	X	R	g	G	;	6	^	p	b	e
2	P	o	/	P	M	b	Y	]	4	N	<	0	T	I	U	C
f	D	=	E	Z	i	c	;	o	I	g	F	G	/	V	9	?
k	O	d	Y	@	R	2	A	W	;	h	K	5	7	j	a	e
m	D	K	6	E	f	/	L	H	T	B	0	P	M	8	I	`
g	n	I	h	F	H	e	3	P	<	o	1	m	2	q	X	U
f	G	=	D	Y	g	n	H	W	7	/	>	g	:	`	W	Z
[	T	b	o	G	=	U	e	E	<	?	X	c	d	I	O	a
=	1	m	1	c	n	2	^	Q	/	V	7	R	Y	e	J	Z
W	P	:	c	d	1	T	8	g	G	;	o	K	U	=	h	_
=	:	a	b	e	[	X	7	c	f	H	?	>	I	V	g	D
;	r	1	^	7	2	W	7	h	E	<	M	J	e	f	N	=
9	K	d	g	/	W	h	S	0	]	e	H	2	U	3	Y	V

THEYHADAHOUSEOFCR (1194)
YSTALPILLARSONTHE (1163)
PLANETMARSBYTHEED (1100)
GEOFANEMPTYSEAAND (1066)
EVERYMORNINGYOUCO (1287)
ULDSEEMRSKEATINGT (1032)
HEGOLDENFRUITSTHA (1137)
TGREWFROMTHECRYST (1152)
ALWALLSORCLEANING (1164)
THEHOUSEWITHHANDF (1244)
ULSOFMAGNETICDUST (1025)
WHICHTAKINGALLDIR (1207)
TWITHITBLEWAWAYON (1116)
THEHOTWINDAFTERNO (1157)
ONSWHENTHEFOSSILS (1203)
EAWASWARMANDMOTIO (1146)
NLESSANDTHEWINETR (1277)
EESSTOODSTIFFINTH (1087)
EYARDANDTHELITTLE (1316)
DISTANTPARTIANBON (1161)

Would it be possible to make it so I can paste numbers directly from Excel? Thanks.

Each new row is a row stop, is that okay?

Would it be possible to make it so I can paste numbers directly from Excel? Thanks.

Can you share a paste from Excel?

I just noticed that it doesn’t work with numbers yet. Let me get a fix on that.

Okay, try this one: https://drive.google.com/open?id=0B5r0r … Tcya004bE0

Here is an Excel paste, if you still need it. I will try the new version…!

EDIT: I was so excited I forgot the Excel paste.

55 22 14 66 23 1 11 2 24 42 59 51 13 43 16 6 47
66 52 56 3 31 46 26 32 33 4 48 53 44 38 57 25 12
46 33 1 40 15 58 35 2 48 54 5 68 55 22 12 13 10
20 14 42 17 3 40 14 36 46 56 67 51 12 4 1 41 11
13 62 14 50 67 36 42 48 38 27 39 20 68 43 60 7 44
61 31 9 51 15 14 35 48 53 30 13 2 57 28 40 21 58
23 14 19 45 32 10 15 41 16 49 59 29 55 54 56 22 3
57 20 50 12 63 17 50 42 36 57 25 13 8 48 66 52 55
4 34 65 1 34 31 52 43 47 6 32 14 2 38 27 39 21
56 22 15 23 44 59 53 13 65 27 57 24 25 1 40 11 17
61 33 54 43 18 36 4 19 41 13 58 29 7 9 60 51 55
63 22 29 8 23 56 1 30 26 38 20 2 34 31 10 27 50
57 64 27 58 24 26 55 5 34 14 65 3 63 4 67 42 39
56 25 15 22 43 57 64 26 41 9 1 16 57 12 50 41 44
45 38 52 65 25 15 39 55 23 14 17 42 53 54 27 33 51
15 3 63 3 53 64 4 48 35 1 40 9 36 43 55 28 44
41 34 12 53 54 3 38 10 57 25 13 65 29 39 15 58 49
15 12 51 52 55 45 42 9 53 56 26 17 16 27 40 57 22
13 68 3 48 9 4 41 9 58 23 14 31 28 55 56 32 15
11 29 54 57 1 41 58 37 2 47 55 26 4 39 5 43 40

This is the most significant achievement, tool, so far in the work for a solution to the 340, in my opinion. Check this out, I made a message 17 x 20, all homophonic no transposition, then pasted a 15 x 15 chunk of the message into the solver:

ILIKEKILLINGPEO (1665)
ABECAUSEITISSOM (1816)
HFUNITISMOREFUN (1604)
ANWILLINGWILDGA (1590)
INTHAFORRECTBAC (1509)
SEMANISTHEMOSTP (1749)
GAROUSANIMALOFA (1533)
TOKILLSOMETHING (1854)
VESMATHEMOSTTHR (1643)
LINGAPPERENCEIT (1614)
ECANBETTERTHANG (1803)
TINGYOURROCKSOF (1626)
ITHAGIRLTHEBEST (1632)
RTOFITISTHATWHE (1776)
DIEIWILLBEREBOR (1522)

You realize that we can find out now if the pivots are causing a distortion? We can paste individual rows, but avoid the pivots. This is really versatile.