… the best way to defeat this weakness is to do a columnar transposition. This way you can separate the double letters that created the bigram, and create new “false” bigrams to throw people off.

Whilst doing the washing-up, the idea of columnar transposition did grow on me, for the following reasons.

1) We know the z340 has interesting row and column repeat counts and IoC values, implying it was coded horizontally. That’s obviously invariant under columnar transposition, so we would still see that.
2) "Row spacing appears more regular than column spacing, suggesting the cipher was written out by rows". (Source: Jarlve) viewtopic.php?p=62476#p62476 That sounds like the second step having written horizontally.
3) Column transposition ciphers are a known standard approach and would be in the same knowledge domain as homophonic ciphers. So as a step up from homophonic, it’s not a huge step.
4) As beldenge said in thread "Period 19" viewtopic.php?f=81&t=4512 What else has p19 got going for it? perhaps it is just a red herring

With over 355 trillion column orders available, is it possible it the correct solution is just in there yet to be highlighted by the tools?

I suppose I’m asking, has this area been extensively mined for solutions or is it worthy of more focus?

CK

I can’t say how extensively it’s been tested, but I can say that it’s my current focus, and I’ll share a couple of my thoughts here.

Right now I’m testing for post-substitution transposition using a hill climber to find columnar transposition keys that result in high bigram and cycle counts. I understand that others have done research for which there is evidence that transposition was done pre-substitution.

(I’m not sure if I’m using the correct terms, so what I mean by pre- or post- substitution is simply to clarify whether the transposition was performed before or after substitution by ciphertext.)

I’m focusing on post-substitution transposition for the moment, because it’s actually a feasible thing to do with a hill climber. If transposition was done prior to substitution, however, then I think the problem becomes more or less intractable. The only way I can presently envision testing something like that is to have a hill-climber within a hill-climber. So the hill climber which tests transposition keys would use a scoring mechanism which itself uses a hill climber to test the substitution keys (essentially what the current solver tools do). This would take massive amounts of CPU time, and by the nature of hill climbing would need to be run several times over (random restarts).

I hope that makes some sense.

I wonder if we are talking about columnar transposition or columnar rearrangement?

I wonder if we are talking about columnar transposition or columnar rearrangement?

(I think) I am thinking about columnar rearrangement. With 17! permutations.

So I envisaged rearrangement of columns after homophonic enciphering.

I was going to say "using a key phrase of 17 characters to re-order alphabetically", but since this isn’t (presumably) a cipher intended to be deciphered by a trusted party with the key, any random rearrangement would do. It doesn’t necessarily need a sensible key. He would have picked a column, copied it down vertically in the grid, crossed it off, picked another one, x17.

Oops, I was referring to columnar transposition in my post, so maybe I misunderstood. Thanks for getting us straight Jarlve.

beldenge there is a big problem with post substitution columnar transposition. The plain text is encoded with sequential homophonic substitution, and then the keyed columnar transposition is applied. The keyed columnar transposition randomizes the column order but randomizing the column order does not randomize the cycle scores so much, so you end up with a big peak in the cycle scores at the correct period.

Step by step explanation and evaluation 17 by 20.

Plain text:

WHOSOLVESHARDPROB
LEMSANDRECEIVESAS
COREFORITWILLNORM
ALLYNOTWANTTOHELP
OTHERSOFLESSERABI
LITYTOUNDESERVEDL
YSCOREATTHATLEVEL
ORABOVETHEIRTRUEL
EVELASTHATWOULDRE
DUCETHEMEANINGOFT
HEFIRSTPERSONSOWN
SCORETHISCONSTITU
TESTHEMOTIVATIONF
ORSECRECYBUTWHORE
CEIVESTHESOLUTION
EFFORTLESSLYWITHO
UTHAVINGFOUNDITON
ONESOWNWILLALMOST
CERTAINLYABUSETHA
TINFORMATIONANDPA

Into sequential homophonic substitution:

3+^.R_5-FX'"2)1(Z
89P.*S%@B4WO-FT.
#EI9Y^$JN!_8HR:&
L_8=M(G3C[KUEXB_)
^N+W"FR;8-..910D
_JG6UR>S?BFW@5-Q8
7.A(I9/NGX'U_B5W8
E$*Z^O-G+9:K:>B_
-59_TFUXLG!R>8%"-
?>49K+B]WCHJ[,(<U
X9YI.N)W$FES.^3H
F#R:-G+JFA(MFKU>
N9.GX-PENJO0G^[;
RIF94$B#=D>K3+(:W
AWJ59.KXBFE_>G^S
W<YR"K8-.F_6!JU+(
>NX/OMV;E>[QJG^S
RH9.^3M!8_'8&RFK
AB1U'J[_7*Z>.WN+T
GJS<(@]LKEH0M2)/

Period versus cycle score inspection reveals period 1:

Periodic: (transposition, untransposition)
---------------------------------------------------------
Period 1: 2526, 2526 (13.57, 13.57) <---
Period 2: 1451, 1374 (3.69, 2.98)
Period 3: 1214, 1263 (1.51, 1.96)
Period 4: 1318, 911 (2.46, -1.27)
Period 5: 994, 892 (-0.51, -1.44)
Period 6: 1105, 1084 (0.50, 0.31)
Period 7: 1060, 961 (0.09, -0.81)
Period 8: 1217, 947 (1.53, -0.94)
Period 9: 1147, 1079 (0.89, 0.27)
Period 10: 913, 825 (-1.25, -2.06)
Period 11: 1037, 1139 (-0.11, 0.82)
Period 12: 978, 1040 (-0.65, -0.08)
Period 13: 1103, 1145 (0.49, 0.87)
Period 14: 1045, 897 (-0.04, -1.40)
Period 15: 1053, 876 (0.03, -1.59)
Period 16: 934, 1088 (-1.06, 0.35)
Period 17: 943, 925 (-0.97, -1.14)
Period 18: 1160, 930 (1.01, -1.09)
Period 19: 915, 1147 (-1.23, 0.89)
Period 20: 925, 943 (-1.14, -0.97)

Randomize column order:

F("3.2_1+5)'^ZX-R
BT8.OSF9%-WP.4@*
N:_#98^RE$HI&!JY
C_UL=E(B_GXK8)[3M
8D.^W9F0NR1.+-;"
?QW_6@R-J>5FG8BSU
GWU7(_95./B'A8XNI
+B:EZKO>$-:*_9G^
L"R-_>F%5U8!9-GXT
W<J?9[+(>B,H4UC]K
W3EXS.^9N.FYH$)I
FUMF:FG#+K(R>AJ-
N[0NGG-^9PO.;JEX
=:KR93$(IB+>FWD#4
B^_A5>.WKGEJSFX9
.+6WR!KU<8J_Y(F-"
;^[>/QGNMJ>XSEVO
F'R.83RHM&_9K8!^
7+>AU.JNB[WZ1T*_'
K)HG<0@2J]MES/L(

Period versus cycle score inspection reveals period 1 again since randomization over short distances does not affect the cycle score as much:

Periodic: (transposition, untransposition)
---------------------------------------------------------
Period 1: 2391, 2391 (12.02, 12.02) <---
Period 2: 1380, 1530 (2.97, 4.31)
Period 3: 1193, 1294 (1.30, 2.20)
Period 4: 1165, 1055 (1.05, 0.06)
Period 5: 1039, 1039 (-0.07, -0.07)
Period 6: 1068, 1120 (0.18, 0.64)
Period 7: 1037, 1039 (-0.09, -0.07)
Period 8: 1080, 911 (0.29, -1.22)
Period 9: 1053, 1198 (0.04, 1.34)
Period 10: 1028, 1026 (-0.17, -0.19)
Period 11: 1119, 1061 (0.63, 0.12)
Period 12: 955, 919 (-0.82, -1.15)
Period 13: 1100, 1124 (0.46, 0.68)
Period 14: 981, 922 (-0.59, -1.12)
Period 15: 1092, 955 (0.39, -0.82)
Period 16: 978, 1098 (-0.62, 0.45)
Period 17: 1153, 998 (0.94, -0.44)
Period 18: 1063, 1096 (0.13, 0.43)
Period 19: 1133, 995 (0.76, -0.47)
Period 20: 998, 1153 (-0.44, 0.94)

Apply columnar transposition:

FBNC8?G+LWWFN=B.;
7K(T:_DQWB"<3U[:
^+^F+)"_U.WU:RJE
M0K_6['>H38#L^_7E
-?XFNRAW>RAG..9=W
6(Z_9:G95R/.U<2O
8E9@_K>[SFG3>!Q8.
0_S^(FR9OF+.G-$.K
3J@1FRB0-5>%(^^
(UGRN2+9E_NJ.$5>
9#9IW<NHBJ5%$GR>/
-UBN+PBK8MM[])-HX
15B:8,.K+GJJ&WM'
WK.F'!HF(O>E_>_
ZE^PI8+GA*94YR.FJ
YX91SZ.&)88_-UH>
;WS(SKT/X4![-BX9G
C$AJDFFE8*-@J3;S
NGX])JE#X-V!_LR*Y
M"UI^TKI-X49"O^'(

Period versus cycle score inspection reveals period 20:

Periodic: (transposition, untransposition)
---------------------------------------------------------
Period 1: 998, 998 (-0.31, -0.31)
Period 2: 1144, 1028 (1.07, -0.03)
Period 3: 1329, 1122 (2.83, 0.86)
Period 4: 876, 1039 (-1.48, 0.07)
Period 5: 1115, 1165 (0.79, 1.27)
Period 6: 927, 929 (-0.99, -0.97)
Period 7: 1049, 969 (0.16, -0.59)
Period 8: 952, 1164 (-0.75, 1.26)
Period 9: 1056, 1118 (0.23, 0.82)
Period 10: 1008, 1380 (-0.22, 3.32)
Period 11: 925, 866 (-1.01, -1.57)
Period 12: 770, 970 (-2.49, -0.58)
Period 13: 991, 918 (-0.38, -1.08)
Period 14: 1130, 1028 (0.93, -0.03)
Period 15: 942, 961 (-0.85, -0.67)
Period 16: 1003, 1025 (-0.27, -0.06)
Period 17: 2391, 807 (12.95, -2.13) <---
Period 18: 1229, 907 (1.88, -1.18)
Period 19: 927, 1235 (-0.99, 1.94)
Period 20: 807, 2391 (-2.13, 12.95) <---

The 340 has its cycle score peak at period 1 so it cannot have been post substitution.

Periodic: (transposition, untransposition)
---------------------------------------------------------
Period 1: 2137, 2137 (7.39, 7.39) <---
Period 2: 1855, 1590 (5.17, 3.08)
Period 3: 1251, 1324 (0.41, 0.98)
Period 4: 1278, 1239 (0.62, 0.31)
Period 5: 1278, 1110 (0.62, -0.69)
Period 6: 1175, 975 (-0.18, -1.76)
Period 7: 1290, 1047 (0.72, -1.19)
Period 8: 1261, 1155 (0.49, -0.34)
Period 9: 1028, 1331 (-1.34, 1.04)
Period 10: 1323, 959 (0.98, -1.88)
Period 11: 1367, 1227 (1.32, 0.22)
Period 12: 1281, 1008 (0.65, -1.50)
Period 13: 1130, 1190 (-0.53, -0.06)
Period 14: 965, 1151 (-1.83, -0.37)
Period 15: 1194, 1082 (-0.03, -0.91)
Period 16: 1321, 1133 (0.96, -0.51)
Period 17: 1112, 1335 (-0.68, 1.07)
Period 18: 1111, 1181 (-0.68, -0.13)
Period 19: 1174, 1065 (-0.19, -1.05)
Period 20: 1335, 1112 (1.07, -0.68)

So I envisaged rearrangement of columns after homophonic enciphering.

I wrote a test for this a while ago, it is available in AZdecrypt as "Find post sequential homophonic row or columnar rearrangement".

I took the cipher in the post above before the columnar transposition step. It finds a 26.9% improvement rate at 17 by 20, the smaller column sizes like 3*114 have good improvement rates to but are unreliable since the permutations are very low (3!).

AZdecrypt find rearrangement stats for: p131.txt
---------------------------------------------------------
- Attempts to find a set of dimensions in which a rearrangement 
of rows or columns was applied after sequential encoding.
- A higher percentage (improvement rate) may be 
more indicative of a rearrangement.

Columnar rearrangement:
---------------------------------------------------------
2*170: 0%
3*114: 32.2%
4*85: 8.30%
5*68: 26.3%
6*57: 27%
7*49: 21.2%
8*43: 8.80%
9*38: 22.5%
10*34: 0.5%
11*31: 15.6%
12*29: 0.9%
13*27: 3.9%
14*25: 19%
15*23: 0.3%
16*22: 1.9%
17*20: 26.9%
18*19: 0.2%
19*18: 0%
20*17: 0.3%
21*17: 0%
22*16: 0.3%
23*15: 0%
24*15: 0%
25*14: 0%
26*14: 0%
27*13: 0%
28*13: 0%
29*12: 0%
30*12: 0%
31*11: 0%
32*11: 0%
33*11: 0.2%
34*10: 0%

For the 340:

Columnar rearrangement:
---------------------------------------------------------
2*170: 49%
3*114: 49.6%
4*85: 20.7%
5*68: 17.5%
6*57: 21%
7*49: 26.1%
8*43: 24.4%
9*38: 0.5%
10*34: 3.9%
11*31: 11.5%
12*29: 4.5%
13*27: 6.7%
14*25: 0.70%
15*23: 0.2%
16*22: 3%
17*20: 6.9%

The test is not perfect but it will do the job with careful interpretation.

not sure if this is of any interest red ryder victory patrol using a column type decoder

not sure if this is of any interest red ryder victory patrol using a column type decoder

Very interesting! Thanks for posting that.
Here’s a high resolution version I found:

http://zodiackillerciphers.com/images/r … ecoder.jpg

It interestingly fits in with other possible Zodiac connections in Red Ryder material, such as the cattle brand that resembles the Halloween card symbol in this illustration of Fred Harman’s "Red Ryder Ranch":

And the Tim Holt comic by the same guy, with its eerie similarities to the Halloween card:

using the victory patrol secret decoder in a slightly more complex way use all five columns

9,10,5,5,23
11,7,8,6,1
14,5,12,23,7
16,10,1, etc
ilike killi ngpeo ple

just attach a symbol or two to each column letter for its substitution
i did this using the 408 and it is interesting in column shifting sequence.
going on holidays to a winery region. will be foggy going for a while
cheers

using the victory patrol secret decoder in a slightly more complex way use all five columns

9,10,5,5,23
11,7,8,6,1
14,5,12,23,7
16,10,1, etc
ilike killi ngpeo ple

just attach a symbol or two to each column letter for its substitution
i did this using the 408 and it is interesting in column shifting sequence.
going on holidays to a winery region. will be foggy going for a while
cheers

That’s vigenère with a keyword length 5 right?

using the victory patrol secret decoder in a slightly more complex way use all five columns

9,10,5,5,23
11,7,8,6,1
14,5,12,23,7
16,10,1, etc
ilike killi ngpeo ple

just attach a symbol or two to each column letter for its substitution
i did this using the 408 and it is interesting in column shifting sequence.
going on holidays to a winery region. will be foggy going for a while
cheers

That’s vigenère with a keyword length 5 right?

Yes Jarlve it would be ACEGI in this case, each column has been shifted down by two. You could make it up with any word and any key length, good words could be KNIFE ROPE GUN even ZODIAC