Zodiac Discussion Forum

Homophonic substitu…
 
Notifications
Clear all

Homophonic substitution

1,434 Posts
21 Users
0 Reactions
304.3 K Views
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

That would be great! For my search to resume, it will need to detect when azdecrypt has returned to wait mode, perhaps by detecting a message in one of the output files, or the presence of a file made specifically when azdecrypt is completely done with the current batch of inputs.

I just completed the changes, can you test if this works for you? https://drive.google.com/open?id=0B5r0r … 3ZmdVd0NDQ

Change Solver mode in the upper right corner to Test mode. The number of ciphers to process fields will determine the range for which the program periodically scans. For instance From: 1.txt, To: 10.txt will scan from file 1.txt to 10.txt. When one of these files is found it will be processed with the settings you started the program with (random restarts, iterations, etc). When all threads have finished processing the input file will be deleted in the Ciphers directory and an empty file named "finished.txt" will be created in the Results directory.

When the program is running the files "stats.txt" and "log.txt" in the Results directory will be unavailable or in use. But the "group_list.txt" file which is generated when the program detects "Cipher_information=your stuff" in the cipher file will be available after the "finished.txt" file is generated.

I recommend you put everything in one big cipher file "1.txt" instead of several single cipher files. Let me know if anything.

AZdecrypt

 
Posted : December 12, 2015 7:02 pm
doranchak
(@doranchak)
Posts: 2614
Member Admin
 

This is great; thanks so much for taking the time to implement it!

So I’m wondering how few iterations and restarts I can get by with, since I want to try to run through each cipher as quickly as possible after each iteration of my search. Do you have any suggestions on how low I can set iterations and restarts to get quality decryptions? I guess it’s a subjective tradeoff, since more iterations and restarts are usually needed to find plaintexts from "messy" ciphers.

Really wishing I had funding for a high speed cluster of thousands of nodes of processors. :) Something like this: https://aws.amazon.com/hpc/ (Here’s an interesting article about using a record breaking 156,000 processor cores using Amazon web services: http://www.cnet.com/news/supercomputing … sor-cores/ It cost them $33,000 for those 18 hours of massively parallel compute time!)

http://zodiackillerciphers.com

 
Posted : December 12, 2015 11:24 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

Jarlve,

Here is smokie15:

* Simple transposition where the plaintext at position 81 also occurs ( added ) at position 82, and the symbol at position 243 is skipped during transposition. Row 20 is not transposed.
* The distortion areas are not close to each other; it is a good cipher for using experimental methods for detecting skipped and added symbols.
* The cycles are not perfect. About 75% of the time I selected ciphertext in cyclical order, and about 25% of the time I chose ciphertext within a cycle at random.
* There is one ciphertext, 19, that is high count and polyalphabetically maps to plaintext heavily represented in untransposed high count period 1 bigram matches ( which helped to increase period 19 matches after transposition ) and therefore doesn’t cycle well with the other ciphertext.
* The plaintext is a quote from Herman Melville.

T H E R I B S A N D T E R R O R S
I N T H E W H A L E A R C H E D O
V E R M E A D I S M A L G L O O M
W H I L E A L L G O D S S U N L I
T W A V E S R O L L E D B Y A N D
L I F T M E D E E P E N I N G D O
W N T O D O O M I S A W T H E O P
E N I N G M A W O F H E L L W I T
H E N D L E S S P A I N S A N D S
O R R O W S T H E R E W H I C H N
O N E B U T T H E Y T H A T F E E
L C A N T E L L O H I W A S P L U
N G I N G T O D E S P A I R I N B
L A C K D I S T R E S S I C A L L
E D M Y G O D W H E N I C O U L D
S C A R C E B E L I E V E H I M M
I N E H E B O W E D H I S E A R T
O M Y C O M P L A I N T S N O M O
R E T H E W H A L E D I D M E C O
N F I N E W I T H S P E E D H E F

A 1 24 44 59
B 2 25 45
C 3 26 46 60
D 4 27 47
E 5 28 48 19
F 6
G 7 29
H 8 30 49 19
I 9 31 50 19
J
K 10
L 11 32 51 61
M 12 33 52
N 13 34 53
O 14 35 54
P 15 36 55
Q
R 16 37 56 62
S 17 38 57 63
T 18 39 58 19
U 20 40
V 21 41
W 22 42
X
Y 23 43
Z

18 9 21 22 58 11 42 5 19 14 35 32 13 11 28 17 31
35 16 30 34 48 49 22 50 53 13 19 37 34 3 7 1 4
26 53 12 5 28 58 56 19 24 6 19 9 13 62 48 44 31
46 33 59 19 23 18 16 19 52 61 41 39 14 34 27 35 2
53 13 10 43 37 8 60 49 50 5 28 28 19 12 47 29 11
42 20 58 29 4 3 5 54 28 25 22 1 24 38 48 14 33
19 57 19 5 58 19 35 28 45 52 42 63 49 27 61 56 47
54 44 17 39 58 51 54 38 4 2 35 15 19 59 1 9 61
54 5 12 22 57 8 30 11 27 19 42 19 22 32 24 34 51
63 7 61 28 31 14 36 28 48 35 19 62 49 11 5 44 32
47 28 33 54 51 55 57 6 59 16 23 19 38 48 19 50 4
19 5 18 1 24 4 28 48 44 8 9 19 19 31 15 57 53
5 30 53 47 28 62 61 63 4 34 22 48 53 22 49 42 59
17 50 21 19 58 50 16 26 29 38 25 25 31 19 11 57 19
59 24 50 9 46 19 63 17 27 62 8 32 40 43 13 30 51
44 9 18 57 37 60 14 49 5 34 52 35 28 54 13 59 29
48 22 13 26 6 36 19 1 20 50 24 14 48 37 4 54 61
34 4 54 19 27 8 5 11 53 32 51 12 56 33 26 57 14
52 9 47 35 55 18 57 13 28 40 2 61 4 12 58 35 35
34 6 31 53 48 42 50 19 8 17 15 19 5 27 30 28 6

.

 
Posted : December 13, 2015 2:26 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
This week I worked on trying to make a message with a high number of period 19 repeats, but which is not a transposition cipher.

Question: Is is possible to make a message that has a count of period 19 bigram repeats comparable to the 340, but which is not a transposition of plaintext?

Short Answer: Yes, if you start with a plaintext message that already has a high count of period 19 bigram repeats before transposition. But that isn’t the problem. The problem is making a message that also has a count of period 1 bigram repeats comparable to the 340.

It is possible to get sort of close ( a scientific term ) to 340 period 1 and period 19 bigram repeat stats by constructing the key so that:

1) plaintext heavily represented in untransposed period 1 repeats but not so heavily represented in untransposed period 19 repeats are diffused more; and

2) plaintext heavily represented in untransposed period 19 repeats but not so heavily represented in untransposed period 1 repeats are diffused less.

I started by pasting all 100 of Jalrve’s plaintext library into a spreadsheet and calculating how many period 1 and period 19 matches there were in each one. Number 44 had the highest number of period 19 matches at 205 and also had the highest ratio of period 19 matches to period 1 matches, with a count of 223.

EDIT: I call AB AB two period 19 matches, but which is technically only one period 19 repeat. My spreadsheet counts them that way. The 340 has 47 period 1 matches, and 71 period 19 matches.

The top chart is a distribution of the count of period 19 matches across the 100 messages. The x-axis is the count of period 19 matches, and the y-axis is the number of messages that have that count. Real simple. The bottom chart is a distribution of the ratio of period 19 matches to period 1 matches. The highest on both charts is message number 44.

Here is message 44:

A L L T H E T R A C E S O F A F F
I N I T Y W I T H O R C O N S C I
O U S N E S S O F T H E B E O W U
L F T H A T W E C A N D I S C O V
E R A N D T H E Y A R E V E R Y F
E W A R E S U C H A S T O F A V O
U R T H I S D A T E T H E O N L Y
C O M P L E T E P A R A L L E L T
O T H E F A B L E I S F O U N D I
N T H E I C E L A N D I C S A G A
O F G R E T T I R W H O I S A K I
N D O F N O R T H E R N H E R C U
L E S T H I S H E R O P E R F O R
M S M A N Y G R E A T F E A T S B
U T T H E R E A R E T H R E E W H
I C H B E L O N G T O T H E S U P
E R N A T U R A L I N O N E O F T
H E S E H E W R E S T L E S W I T
H A F I E N D C A L L E D G L A M
A N D K I L L S H I M A N D T H O
.

 
Posted : December 13, 2015 3:19 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
Here is one extreme, smokie18a. Note that plaintext E, H, O, R, S and T are diffused not at all or as little as possible because they are heavily represented in the period 19 matches.

Period 1 matches: 133
Period 19 matches: 110

minimum diffusion of period 19 plaintext
perfect cycles
A 1
B 2 26
C 3 27 40 48 56
D 4 28 41 49 57
E 6
F 7 29 42 50 58
G 8 30
H 9
I 10 31 43 51 59 5
J
K 11
L 12 32 44 52 60 19
M 13 33
N 14 34 45 53 61 20
O 15
P 16 35
Q
R 17
S 18
T 21 36
U 22 37 46 54 62
V 23
W 24 38 47 55 63
X
Y 25 39
Z

1 12 32 21 9 6 36 17 1 3 6 18 15 7 1 29 42
10 14 31 21 25 24 43 36 9 15 17 27 15 34 18 40 51
15 22 18 45 6 18 18 15 50 21 9 6 2 6 15 38 37
44 58 36 9 1 21 47 6 48 1 53 4 59 18 56 15 23
6 17 1 61 28 36 9 6 39 1 17 6 23 6 17 25 7
6 55 1 17 6 18 46 3 9 1 18 21 15 29 1 23 15
54 17 36 9 5 18 41 1 21 6 36 9 6 15 20 52 39
27 15 13 16 60 6 21 6 35 1 17 1 19 12 6 32 36
15 21 9 6 42 1 26 44 6 10 18 50 15 62 14 49 31
34 36 9 6 43 40 6 52 1 45 57 51 48 18 1 8 1
15 58 30 17 6 21 36 59 17 63 9 15 5 18 1 11 10
53 4 15 7 61 15 17 21 9 6 17 20 9 6 17 56 22
60 6 18 36 9 31 18 9 6 17 15 16 6 17 29 15 17
33 18 13 1 14 25 8 17 6 1 21 42 6 1 36 18 2
37 21 36 9 6 17 6 1 17 6 21 9 17 6 6 24 9
43 3 9 26 6 19 15 34 30 36 15 21 9 6 18 46 35
6 17 45 1 36 54 17 1 12 51 53 15 61 6 15 50 21
9 6 18 6 9 6 38 17 6 18 36 32 6 18 47 59 21
9 1 58 5 6 20 28 27 1 44 52 6 41 8 60 1 33
1 14 49 11 10 19 12 18 9 31 13 1 34 57 36 9 15

This is the symbol count distribution as compared to the 340. The top is the 340, the bottom is smokie18a. X-axis is symbol count, and y-axis is number of symbols with that count.

And here is the other extreme, smokie18b. Note that plaintext E, H, I, O, R, S and T are diffused more efficiently because they are heavily represented in period 1 matches.

Period 1 matches: 37
Period 19 matches: 31

maximum diffusion of period 1 plaintext
perfect cycles
A 1 2 3 4
B 6
C 7 8
D 9 10
E 11 12 13 14 15 16 17 18
F 21 22 23
G 24
H 25 26 27 28 5
I 29 30 31 32
J
K 33
L 34 35 36 37
M 38
N 39 40 41 42
O 43 44 45 46 19
P 47
Q
R 48 49 50 51
S 52 53 54 55
T 56 57 58 59 20
U 60
V 61
W 62
X
Y 63
Z

1 34 35 56 25 11 57 48 2 7 12 52 43 21 3 22 23
29 39 30 58 63 62 31 59 26 44 49 8 45 40 53 7 32
46 60 54 41 13 55 52 19 21 20 27 14 6 15 43 62 60
36 22 56 28 4 57 62 16 8 1 42 9 29 53 7 44 61
17 50 2 39 10 58 5 18 63 3 51 11 61 12 48 63 23
13 62 4 49 14 54 60 8 25 1 55 59 45 21 2 61 46
60 50 20 26 30 52 9 3 56 15 57 27 16 19 40 37 63
7 43 38 47 34 17 58 18 47 4 51 1 35 36 11 37 59
44 20 28 12 22 2 6 34 13 31 53 23 45 60 41 10 32
42 56 5 14 29 8 15 35 3 39 9 30 7 54 4 24 1
46 21 24 48 16 57 58 31 49 62 25 19 32 55 2 33 29
40 10 43 22 41 44 50 59 26 17 51 42 27 18 48 8 60
36 11 52 20 28 30 53 5 12 49 45 47 13 50 23 46 51
38 54 38 3 39 63 24 48 14 4 56 21 15 1 57 55 6
60 58 59 25 16 49 17 2 50 18 20 26 51 11 12 62 27
31 7 28 6 13 37 19 40 24 56 43 57 5 14 52 60 47
15 48 41 3 58 60 49 4 34 32 42 44 39 16 45 22 59
25 17 53 18 26 11 62 50 12 54 20 35 13 55 62 29 56
27 1 23 30 14 40 9 8 2 36 37 15 10 24 34 3 38
4 41 9 33 31 35 36 52 28 32 38 1 42 10 57 5 46

You can see that the key makes a huge difference in period 1 and period 19 stats! But the trick is to make a key so that the period 1 matches are diffused more and the period 19 matches are diffused less. Smokie18c coming soon.
.

 
Posted : December 13, 2015 3:57 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
I am still working on this, but here are the period 1 and 19 stats for message 44:

Because H, D, F and A are more heavily represented with period 1 but not with period 19, I am working on diffusing those plaintext more. Because R, O, and S are more heavily represented with period 19 but not with period 1, I am working on diffusing those plaintext less.

Not a particularly technical approach, but this key with perfect cycles results in:

Period 1 matches: 81
Period 19 matches: 62

A 1 26 43 56 54
B 2 27
C 3 28
D 4 29 44 45 47
E 6 5
F 7 30 46 57 61
G 8 31
H 9 32 48 58 62 63
I 10 33 49
J
K 11
L 12 34 50 59
M 13 35
N 14 36 51
O 15 37 52
P 16 38
Q
R 17 19
S 18 20
T 21 39 53 60
U 22 40
V 23
W 24 41 55
X
Y 25 42
Z

And if 25% of the time I select symbols from cycles at random instead of using perfect cycles ( still completely homophonic ), I can make a lot of different efforts and get closer to 340 stats. Here is one good example, smokie18c:

Period 1 matches: 65
Period 19 matches: 63

1 12 34 21 9 6 39 17 26 3 5 18 15 7 43 61 46
49 14 33 53 25 55 49 60 32 37 19 28 37 36 20 3 10
15 22 18 51 6 18 18 52 57 21 48 5 2 6 52 41 40
50 61 53 58 54 53 55 5 3 56 14 47 33 20 3 15 23
6 19 1 36 44 60 62 5 42 26 19 6 23 5 17 25 30
5 41 1 17 5 18 22 28 63 54 20 21 37 30 54 23 52
40 17 39 9 49 18 44 1 53 6 60 32 5 15 51 59 42
3 37 13 16 50 6 21 5 38 26 19 43 34 50 6 59 39
52 53 48 5 46 56 27 12 6 10 20 57 37 22 36 45 33
51 60 63 5 49 28 6 34 54 51 47 10 3 18 1 8 26
52 61 31 19 5 53 39 33 19 41 62 52 49 20 43 11 49
14 4 15 7 36 37 17 53 63 6 19 51 9 5 17 28 40
50 6 20 60 32 10 20 48 5 19 52 38 5 17 30 15 19
35 18 13 56 14 25 8 17 5 56 21 46 6 1 39 20 2
22 53 60 58 5 19 6 26 17 5 21 62 19 6 5 55 63
49 3 9 27 6 59 52 36 31 39 52 53 32 5 18 40 38
6 17 51 43 60 22 19 56 12 10 36 15 36 5 37 57 21
48 6 18 5 9 6 24 19 5 18 21 34 6 20 41 10 53
62 54 61 10 5 51 29 28 1 50 34 6 44 8 12 26 35
43 14 45 11 10 12 50 18 9 33 13 56 36 47 60 9 52

And here is the distribution of symbol counts that is very similar to the 340:

EDIT: I am going to keep messing around with the key, random symbol selection within homophonic cycles, and perhaps explore making one or a few symbols polyalphabetic.
.

 
Posted : December 13, 2015 4:38 am
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

This is great; thanks so much for taking the time to implement it!

So I’m wondering how few iterations and restarts I can get by with, since I want to try to run through each cipher as quickly as possible after each iteration of my search. Do you have any suggestions on how low I can set iterations and restarts to get quality decryptions? I guess it’s a subjective tradeoff, since more iterations and restarts are usually needed to find plaintexts from "messy" ciphers.

It depends on the similarity of the ciphers you are processing, I have previously talked about overlapping in this regard. For instance, if you are processing many ciphers that only differ a bit here and there then you can go as low as 1 random restart. The default settings (10 random restarts and 500.000 iterations per cipher) are great for ciphers around the 340 multiplicity/expected difficulty. You are entirely right about "messy" ciphers.

I suggest you experiment the settings with some test ciphers.

AZdecrypt

 
Posted : December 13, 2015 11:54 am
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

Question: Is is possible to make a message that has a count of period 19 bigram repeats comparable to the 340, but which is not a transposition of plaintext?

Short Answer: Yes, if you start with a plaintext message that already has a high count of period 19 bigram repeats before transposition. But that isn’t the problem. The problem is making a message that also has a count of period 1 bigram repeats comparable to the 340.

I have reviewed smokie18a and smokie18b. It shows how hard it is to induce period 19 bigram repeats without transposition. In both ciphers there exist no substantial peak at period 19 with any of the following measurements:

Bigrams, bigrams c*(c-1), 5-gram rep. fragments:

Smokie18a, period 1: 97, 288, 1879.
Smokie18a, period 19: 76, 141, 1256.

Smokie18b, period 1: 18, 18, 147.
Smokie18b, period 19: 15, 15, 108.

AZdecrypt

 
Posted : December 13, 2015 12:41 pm
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

Thanks. And some test ciphers would be great. Not sure how good the algorithm will be on them but it will be a good exercise to improve its ability to find the correct transpositions. I worry about the vastness of the search space.

Here are 10 different period 19 transposition ciphers (schemes). They are all clean transpositions in that there are no filler, or interruptions like added or removed symbols etc. They all use the 408 plaintext and have 50 symbols so that bigram analysis is actual. Sometimes the cycles have added randomization, and sometimes the randomization is caused by the transposition etc.

https://drive.google.com/open?id=0B5r0r … HZCNklLaGc

AZdecrypt

 
Posted : December 13, 2015 6:46 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
I manually hill-climbed a key by systematically deleting and adding symbols to get a maximum difference between period 1 and maximum period 19. I got to the point where I had four more period 19 matches than period 1 matches with perfect cycles. Then I selected cycle symbols at random within cycles 25% of the time and tried about 20 times before getting this:

Period 1 matches: 53
Period 19 matches: 59

Still a ways off from 340 stats, where the difference is 24, but it was the best that I could do with purely homophonic substitution. I may try to make a few symbols polyalphabetic to see if I can improve on this.

smokie18d
minimum diffusion of period 19 plaintext
maximum diffusion of period 1 plaintext
25% random symbol selection
A 1 2 3
B 4 5
C 6 7 8
D 9 10 11
E 12 13 14
F 15 16 17
G 18 19 20
H 21 22 23
I 24 25 26 27
J
K 28
L 29 30 31 32
M 33 34 35
N 36 37 38
O 39 40 41 42 43
P 44
Q
R 45 46 47 48 49
S 50 51 52
T 53 54 55
U 56 57 58
V 59 60
W 61 62
X
Y 63
Z

1 29 30 53 21 12 54 45 2 6 13 52 39 15 3 16 16
26 36 25 55 63 61 26 53 22 40 46 7 41 36 51 7 27
42 56 52 38 12 50 52 43 15 54 22 12 4 13 39 61 57
31 16 55 21 1 54 61 14 6 2 36 9 27 52 8 40 59
12 48 3 37 10 54 22 13 63 1 48 14 60 12 47 63 17
13 62 2 45 14 50 58 8 23 1 50 55 41 15 1 59 42
56 46 53 22 25 52 11 2 54 14 55 21 13 43 38 32 63
6 39 35 44 29 14 53 12 44 3 47 1 30 31 13 29 54
40 55 23 14 16 2 5 29 12 26 50 17 39 57 36 9 24
37 53 21 13 24 7 14 32 3 36 10 25 8 51 3 18 2
42 15 18 48 14 53 55 26 49 61 22 43 27 51 3 28 24
36 11 41 16 37 41 45 53 23 13 46 38 21 14 47 6 56
32 12 50 54 22 25 51 22 13 48 41 44 14 49 17 42 45
35 52 35 3 36 63 20 49 12 2 55 17 13 3 53 50 4
56 54 54 21 13 47 12 1 49 13 53 22 48 14 12 62 23
26 8 21 5 12 32 43 37 18 55 39 55 22 14 51 57 44
12 45 38 2 53 58 46 3 29 27 36 40 37 13 41 17 54
23 14 52 12 22 13 61 47 14 50 55 30 12 52 62 24 53
22 1 17 25 13 37 10 8 2 29 29 14 10 19 29 3 33
1 36 10 28 26 30 31 52 23 27 34 2 37 9 54 21 43

The cycle scores should also be somewhat comparable to the 340.

 
Posted : December 14, 2015 1:32 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
Smokie18e

Smokie18e was the result of a lot of manual labor, but the stats in some ways are very comparable to the 340 even though it is not a transposition cipher.

I used message 44 from Jarlve’s plaintext message library.

After making symbols 14, 23, 52 and 55 polyalphabetic, causing more period 1 matches to be diffused and fewer period 19 matches to be diffused, Here is the key:

A 1 2 3
B 4 5
C 6 7 8
D 9 10 11
E 12 13 14 23
F 15 16 17
G 18 19 20
H 21 22 23 14
I 24 25 26 27
J
K 28
L 29 30 31 32 55
M 33 34 35
N 36 37 38
O 39 40 41 42 43
P 44
Q
R 45 46 47 48 49 23 14 55
S 50 51 52
T 53 54 55 52
U 56 57 58
V 59 60
W 61 62
X
Y 63
Z

With perfect cycles, There were 52 period 1 matches and 66 period 19 matches. But the cycle scores were very high and it also occurred to me that rearranging the order of symbols that map to each plaintext in the key would also affect the count of period 1 and period 19 matches. But I just kept going and randomized my symbol selection within the cycles 25% to get close to 340 cycle stats. I did that 100 times, and there were a few messages that had a big difference between period 1 and period 19 counts. Below is a distribution chart showing the difference in count. The x-axis is period 1 count – period 19 count. The y-axis is the number of messages with that difference between period 1 and period 19 counts. I chose the leftmost result for smokie18e, which I will compare with the 340:

Smokie18e has 19 more period 19 matches than it does period 1 matches. The 340 has a difference of 24.

Period 1 matches: 58
Period 19 matches: 77

2 29 30 52 23 12 54 45 2 6 13 50 43 15 3 16 16
24 37 25 55 63 61 26 52 22 40 46 7 41 36 51 8 27
42 56 52 38 14 50 51 43 15 53 22 23 4 12 39 62 57
55 16 54 14 1 54 61 13 6 2 37 11 24 52 6 40 59
23 45 3 37 11 52 21 23 63 1 48 12 60 14 49 63 17
14 62 2 23 23 50 58 8 23 3 51 53 41 15 1 59 42
56 14 54 14 25 52 10 3 55 12 52 14 13 43 38 32 63
6 39 33 44 55 14 53 23 44 3 55 3 29 30 12 30 53
40 55 21 13 16 3 5 29 14 26 50 17 41 56 36 9 27
37 52 22 23 24 7 12 55 3 38 9 25 8 51 1 19 2
42 17 19 45 13 53 54 25 46 61 23 41 27 52 3 28 24
36 11 39 16 37 40 47 55 14 14 48 38 21 23 49 8 56
31 12 50 52 22 27 51 23 13 23 41 44 14 14 17 39 14
34 52 33 2 36 63 20 45 23 2 53 16 13 3 54 50 4
56 55 52 14 13 46 14 1 47 23 53 21 45 12 13 61 22
26 7 23 5 14 30 43 37 18 54 43 54 22 23 51 57 44
12 49 37 2 52 58 55 3 31 27 36 43 37 23 41 16 53
21 13 52 12 22 12 61 14 14 50 54 32 14 52 62 24 55
23 1 15 24 23 37 9 6 2 55 29 12 10 19 30 3 33
1 37 11 28 26 30 32 52 14 27 34 2 37 11 52 21 42

And here is the distribution of symbol count as compared to the 340. The 340 is on top, and smokie18e is on the bottom:

.

 
Posted : December 14, 2015 4:02 am
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

.
Here are the cycle stats for smokie18e:

smokie18e 51224.
340: 63400.

The left column shows the cycle score, the middle column shows how many cycles in the 340 have that score, and the right column shows how many cycles in smokie18e have that score. Smokie18e is comparable to the 340.

Here are the period 19 bigram matches. Smokie18e is on the left, and the 340 is on the right. I marked some of the diagonal rows of period 19 matches to show that smokie18e also has a lot of diagonal rows. EDIT: Prior test messages that really are transposition schemes also have these diagonal rows.

But what about period 19 repeat ( matches ) probability scores? On the left is smokie18e, on the right is the 340. The scores are the natural logarithm of the probability of occurrence. The 340 has scores that are across the board higher than smokie18e, and statistically less probable of occurring:

Tomorrow or the next day I am going to try to find circumstances where you have A19B A19B, A19C A19C, and B and C cycle together. I can find those in transposition messages that I have already made because I encoded after transposing. And I can also find them in the 340 and they are astronomically statistically improbable. Not sure about smokie18e yet. That is it for tonight!
.

 
Posted : December 14, 2015 5:12 am
(@mr-lowe)
Posts: 1197
Noble Member
 

Not sure if you can follow this but here goes.. It was this corner block of wrap around bigram period 19 that I thought I should concentrate on.. Anyways what I have done is odds evens spacings and line miss. I started just a few rows up and made up some junk plaintext just so you can follow what I have done.. It brings the 19 periods for this block back in nicely. I deliberately did not fill it all out so it did not get to cluttered .. I have not proof read or checked the workings and have not tested it to see if it brings other period 19s into line. So it’s odds/evens miss a row.

 
Posted : December 14, 2015 12:57 pm
Jarlve
(@jarlve)
Posts: 2547
Famed Member
Topic starter
 

Okay smokie, here are my measurements for your latest ciphers with the 340 for comparison:

Bigrams, bigrams c*(c-1), 5-gram reps. (higher is better)

340:
Period 1: 25, 58, 253.
Period 1 (mirrored): 29, 66, 267.
Period 15 (mirrored): 41, 106, 322. <—
Period 19: 37, 90, 302.

Smokie18a:
Period 1: 97, 576, 1879. <—
Period 1 (mirrored): 96, 554, 1859.
Period 15 (mirrored): 81, 308, 1311.
Period 19: 76, 282, 1256.

Smokie18b:
Period 1: 18, 36, 147. <—
Period 1 (mirrored): 16, 32, 144.
Period 15 (mirrored): 14, 28, 110.
Period 19: 15, 30, 108.

Smokie18c:
Period 1: 36, 90, 398. <—
Period 1 (mirrored): 33, 84, 347.
Period 15 (mirrored): 36, 80, 328.
Period 19: 33, 74, 293.

Smokie18d:
Period 1: 29, 75, 303. <—
Period 1 (mirrored): 28, 66, 283.
Period 15 (mirrored): 31, 62, 234.
Period 19: 27, 54, 214.

Smokie18e:
Period 1: 32, 82, 310.
Period 1 (mirrored): 34, 86, 312.
Period 15 (mirrored): 43, 98, 341. <—
Period 19: 41, 92, 303.

I say smokie18e is succesful, though not as pronounced as the 340. Well done. It can be noted that it has allot more repeating fragment potential than the 340 with more high frequency symbols.

AZdecrypt

 
Posted : December 14, 2015 2:45 pm
smokie treats
(@smokie-treats)
Posts: 1626
Noble Member
 

It can be noted that it has allot more repeating fragment potential than the 340 with more high frequency symbols.

That is correct. Smokie18e has more higher count symbols than the 340. There are 17 count of symbol 52, 20 count of symbol 23, and 21 count of symbol 14, and I couldn’t quite reach 340 stats. It seems difficult to have as many period 19 repeats and so few period 1 repeats as the 340 because the values are juxtaposed. I am not quite done with my analysis, but you can see that with both messages the high count symbols are responsible for many of the period 19 repeats. That is how I accomplished it with smokie18e.

I think that what I have learned is that the period 19 repeats involving the + symbol may not be quite as significant. They may not be the ones to study more closely.

Black shows the top ten repeat scores, natural logarithm of probability score. The 340 is still higher, and if you don’t use a natural logarithm formula, they are much higher.

.

 
Posted : December 14, 2015 4:35 pm
Page 46 / 96
Share: