I sort of have restarts figured out. Except that [General] Restarts is set to 1 but the program restarts continuously. So our restart field isn’t what the program option is set to, but how many times we as testers allow the program to restart. ? Call it something like "restarts allowed"?

What do you say, after each trial, we copy and rename our output folder "trial x", and then put that into Google?

What about [General] Iterations factor at 1.05? Should we leave that?

Leave it. It only affects the basic substitution solver.

What about [General] Restarts? What is that?

Leave it at 1. It are the restarts of the substitution solver part.

Let’s say I have [General] Hill climber iterations set to 10000. It restarts after 10000 every time. When do I know when to make it stop? How did you restart so many times with your experiments?

You decide for yourself when to stop it. Let it run overnight and stop it in the morning for instance. Just let it run for a long time and you will gather more restarts. Figure out how many restarts and iterations it takes to solve ciphers with 4, 6, 8, 10 and 12 nulls/skips. Your 3/3 has about a 28.57% solve rate with 20000 iterations per restart. This tells me that if 20 to 40 restarts are done a 3/3 or 6 nulls/skips cipher would very likely solve. We could also add more restarts to work that has already been done if it is ran with the same settings. Add up over time and update the date.

What is PC-cycles?

It is short for plaintext to ciphertext cycles. It tells you how much the plaintext letters are following the cycles. It works really well actually. Even with ciphers that have 20% cycle randomization. Keep an eye on it with some of your own ciphers. If you ever see this over 2000 on the 340 be sure to let us know!

Say the letter "E" in the plaintext. It will check the symbols that make it up and how well this combination of symbols cycles and add that to the PC-cycles score. And this for all letters.

Here are the fields so far:

Looks good.

EDIT: Should we have a field for restarts to show thoroughness?

You mean (General) Restarts right? No, the hill climber will simulate many such restarts by itself since it attempts many slightly different variations of the key.

I sort of have restarts figured out. Except that [General] Restarts is set to 1 but the program restarts continuously. So our restart field isn’t what the program option is set to, but how many times we as testers allow the program to restart. ? Call it something like "restarts allowed"?

This setting is typically used in conjunction with Batch ciphers (substitution). This runs through many ciphers piled up in a single file and the user may want to have more than one restart per cipher. doranchak, Largo and I use or have used it. Yes, the basic substitution solver restarts automatically and applies the (General) Iterations factor after every single restart. Like this it will be able to solve ciphers with high multiplicity if it is just left running.

What do you say, after each trial, we copy and rename our output folder "trial x", and then put that into Google?

If you want. But I prefer to have everything contained in the spreadsheet. For me just the best solve will more than do.

I posted the solution to the second 4/4 above.

You decide for yourself when to stop it. Let it run overnight and stop it in the morning for instance. Just let it run for a long time and you will gather more restarts. Figure out how many restarts and iterations it takes to solve ciphers with 4, 6, 8, 10 and 12 nulls/skips. Your 3/3 has about a 28.57% solve rate with 20000 iterations per restart. This tells me that if 20 to 40 restarts are done a 3/3 or 6 nulls/skips cipher would very likely solve. We could also add more restarts to work that has already been done if it is ran with the same settings. Add up over time and update the date.

O.k., I understand.

What is PC-cycles?

It is short for plaintext to ciphertext cycles. It tells you how much the plaintext letters are following the cycles. It works really well actually. Even with ciphers that have 20% cycle randomization. Keep an eye on it with some of your own ciphers. If you ever see this over 2000 on the 340 be sure to let us know!

Say the letter "E" in the plaintext. It will check the symbols that make it up and how well this combination of symbols cycles and add that to the PC-cycles score. And this for all letters.

I understand.

I will do something like this.

[General] Hill climber iterations
Restarts allowed
future AZD options field
future AZD options field

I prefer to have everything contained in the spreadsheet. For me just the best solve will more than do.

O.k. that is what we will do. Thanks.

Second 4/4; symbol 63 is the null

53 20 49 13 24 43 1 31 61 21 55 14 15 50 33 16 17
26 44 13 27 44 45 51 14 39 52 18 2 28 23 9 30 15
3 34 10 17 5 11 20 43 58 40 49 63 17 12 52 62 41
13 29 4 1 63 54 16 56 9 6 2 42 39 40 55 53 7
5 46 55 17 61 60 37 8 23 7 25 56 26 6 27 22 10
22 28 45 58 38 24 51 16 17 5 25 19 57 13 46 41 43
42 15 56 15 11 58 56 57 3 44 39 51 16 17 49 49 21
18 20 59 21 19 22 54 13 35 36 22 55 46 57 22 12 9
46 28 43 25 57 55 44 14 3 61 12 29 15 34 37 27 1
11 38 8 45 51 15 53 22 35 60 40 2 7 53 12 17 19
41 9 46 23 57 36 43 20 3 48 13 33 14 10 53 34 55
4 11 44 2 45 46 15 24 2 37 16 11 3 9 43 54 44
23 28 17 52 49 42 48 50 16 38 4 57 2 51 29 61 54
39 26 37 52 32 45 2 10 14 61 15 45 47 39 22 41 31
60 14 15 55 21 6 54 3 56 36 53 42 23 11 57 48 12
27 9 35 13 10 34 25 24 43 22 35 23 36 4 14 15 20
24 28 63 25 54 33 26 49 25 53 16 1 39 50 35 17 55
61 11 40 13 14 15 14 56 44 41 63 57 26 54 45 42 5
2 17 13 25 55 56 24 3 51 52 57 25 46 21 43 38 55
57 11 44 39 62 40 4 22 23 14 19 53 8 34 57 56 35

EDIT: Here is the solution

The nulls are at positions 46, 56, 275 and 300. The skips are at positions 139, 156, 211 and 216.

S I G N I N G T H E D I V E R T O
G O O N H E F O L L O W E D H I M
R O U N D T H E V E S S E L S S T
E R N T H E S A N D O N T H E O T
H E R S I D E W A S H I G H A N D
O N E C O U L D C L I M B O N B O
A R D B U T L I S T E R S H R A N
K F R O M T H E D A R K A L L E Y
W A Y T H A T L E D T O T H E E N
G I N E R O O M F O R A L L T H A
T H E W E N T I N A N D S A W T H
E D I V E R H A D O P E N E D T H
E J A M B E D D O O R W H E N H E
R E A C H E D T H E L E D G E A F
L A S H F R O M T H E O T H E R S
E L E C T R I C L A M P P I E R C
E D T H E G L O O M A N D H E T R
I E D T O F O R G E T H I S T H R
O B B I N G H E A D A N D L O O K
E D A B O U T S P A R K L I N G B

I allowed 58 restarts of 10000 HC and this was the best scoring result:

Score: 22621.70 Ioc: 0.07041
Ngrams: 438 PC-cycles: 1934

Key: Period(20) Nulls(45,57,281,298), Skips(144,159,212,213)

SIGNINGOHEDIVEITO
GOONHESALLOWEDFIT
FOUNDTHEVESSELSST
ERNTHESANDONTREOT
HESSIDEWASHIGHAND
ONECOULDCLIMBONBO
ARDBUTLISTERSHMAN
YPROTTHEDARCALLEY
WAYTHATLEDFOTHEEN
GINEROOMSONALLTRA
THEWENTINANDSAWTH
EDIVERHADOPENEDTH
ECAMBEDDOOMWHENHE
MEARREDTHEREDGEAS
LASHSFORTHEOTHERS
ELECTFIOLATINIERR
EDTREGOROMANDSETI
EDTOPROGETHIRTROB
BINGTEADANDSNOEDA
BOUTSNARYLLSG

There is clearly a partial solve. The found positions for the nulls and skips were really close but not perfect. When I try to maximize Period X repeats with the heat maps that is also what happens. I get a range of possible positions that all rank about the same, usually in an area covering +/- one row.

What about, if partial solve and / or higher score, start adjusting the positions of the nulls and skips just a little bit to the left and a little bit to the right to fine tune? Test which positions, if adjusted left and right, make the score higher than compared to other positions when adjusted left and right. Then the positions that make a higher score keep, and the positions that don’t make a higher score chose a more random new position from 1-340. Forgive if the program already does that.

I will do something like this.

[General] Hill climber iterations
Restarts allowed
future AZD options field
future AZD options field

I would call it substitution restarts or substitution restarts per hill climber iteration.

What about, if partial solve and / or higher score, start adjusting the positions of the nulls and skips just a little bit to the left and a little bit to the right to fine tune?

It is a good idea. It is something that the hill climber could start doing when it gets nearer to its current iteration limit. Thank you for thinking with me on this one.

I will do something like this.

[General] Hill climber iterations
Restarts allowed
future AZD options field
future AZD options field

I would call it substitution restarts or substitution restarts per hill climber iteration.

And then I would also add (General) Iterations as substitution iterations or substitution iterations per substitution restart.

I have been looking into compiler options. FreeBASIC can compile with GCC and options can be passed through directly to the GCC compiler. This means I can specify to speed optimize the program for specific CPU architectures which can then further increase the speed of AZdecrypt. That said, the GCC that comes with the latest version of FreeBASIC seems to only supports up to the Intel Broadwell architecture. On my computer AZdecrypt runs about 2% faster when using such a compilation.

In that regard, smokie, check out these AZdecrypt versions if you please: https://drive.google.com/open?id=1Oe27d … oTg9XQvmH3

There are a bunch of versions in there, could you test only these two on your i5-8250U:

AZdecrypt 1.11 (64-bit)
AZdecrypt 1.11 (64-bit) Broadwell

To test, execute the version, enable performance mode and go to Options –> Benchmark (new feature) and please get back to me with how much MIPS the benchmark test returns with both versions. It is possible that the Broadwell version may lock up if some of the instructions are not supported by your CPU but I doubt it. The benchmark test return looks like this:

AZdecrypt benchmark completed:
--------------------------------------------------------
MIPS: 47.55402

Tyvm.

O.k. I will test them. I have a really long work day started now so probably tonight or tomorrow. Thanks.

What about, if partial solve and / or higher score, start adjusting the positions of the nulls and skips just a little bit to the left and a little bit to the right to fine tune?

It is a good idea. It is something that the hill climber could start doing when it gets nearer to its current iteration limit. Thank you for thinking with me on this one.

I am testing the first implementation of your idea. It is now a 10% flat chance to left or right shift one of the positions in the key. Currently the shift amount is a random number between 1 and half the length of a period 20 column. A run is under way to test how it affects the solve rate on your 3/3 cipher with 10000 HC iterations. If that is positive I will test 20000 and 40000 as well.

I have been looking into compiler options. FreeBASIC can compile with GCC and options can be passed through directly to the GCC compiler. This means I can specify to speed optimize the program for specific CPU architectures which can then further increase the speed of AZdecrypt. That said, the GCC that comes with the latest version of FreeBASIC seems to only supports up to the Intel Broadwell architecture. On my computer AZdecrypt runs about 2% faster when using such a compilation.

In that regard, smokie, check out these AZdecrypt versions if you please: https://drive.google.com/open?id=1Oe27d … oTg9XQvmH3

There are a bunch of versions in there, could you test only these two on your i5-8250U:

AZdecrypt 1.11 (64-bit)
AZdecrypt 1.11 (64-bit) Broadwell

To test, execute the version, enable performance mode and go to Options –> Benchmark (new feature) and please get back to me with how much MIPS the benchmark test returns with both versions. It is possible that the Broadwell version may lock up if some of the instructions are not supported by your CPU but I doubt it. The benchmark test return looks like this:

AZdecrypt benchmark completed:
--------------------------------------------------------
MIPS: 47.55402

Tyvm.

AZD 1.11 = 16.26 MIPS
Broadwell = 16.51 MIPS

EDIT: 6 threads

What about, if partial solve and / or higher score, start adjusting the positions of the nulls and skips just a little bit to the left and a little bit to the right to fine tune?

It is a good idea. It is something that the hill climber could start doing when it gets nearer to its current iteration limit. Thank you for thinking with me on this one.

I am testing the first implementation of your idea. It is now a 10% flat chance to left or right shift one of the positions in the key. Currently the shift amount is a random number between 1 and half the length of a period 20 column. A run is under way to test how it affects the solve rate on your 3/3 cipher with 10000 HC iterations. If that is positive I will test 20000 and 40000 as well.

It is tested (remember to keep breathing).

Previous build, smokie 3/3:

10000 HC iterations: 15.90% solve rate
20000 HC iterations: 28.57% solve rate
40000 HC iterations: 43.75% solve rate

New build, smokie 3/3:

10000 HC iterations: 27.79% solve rate
20000 HC iterations: 52.84% solve rate
40000 HC iterations: untested, but probably around 70%

It is working really well, the change nearly doubled the solve rates. A solve is seen as over 22000 in the case of smokie 3/3. What we need now is a cipher with a total of 8 nulls/skips that is harder than smokie 3/3 but easier than smokie 5/5 (unsolved so far). Both of your 4/4 ciphers are actually easier than the 3/3. Perhaps we need to try something like 3/5 or 5/3 etc. Please make a few more ciphers like this.

New build download: https://drive.google.com/open?id=1x7Tq- … 4BWIxEXbeh

AZD 1.11 = 16.26 MIPS
Broadwell = 16.51 MIPS

I will include a Broadwell version from now on.