The solution true grid was showing the last iteration instead of the high scoring iteration. It is easy to get close to 300 trues, but detection is perfect and only +1. Maybe +1, +2, +3, +4 and +5 would be a more accurate simulation.

I do not have Excel on my PC but it looks very cool. Have fun with it, hill climbers can be addictive.

I have worked on my matrix +1 to +5 hillclimber. I can go with +1 to +5, and up to 17 nulls / skips. I have to get ready for work, but will show you soon. Now that the programming is mostly done, I have to put it on a more powerful computer and add iteration rows. No position fine tuning rules yet.

smokie, what do you mean with +5?

I mean with +5, there would have to be five consecutive sequential position numbers in a row of an un-transposed hill climbed matrix to get a "true," maybe be a closer model to 5-grams and what we are dealing with.

Check out these screen shots or a 4/4, +3.

Initial position of randomly placed skips and nulls.

Transposed and un-transposed matrices.

True matrices, original on left and un-transposed after 1000 iterations on right. Big improvement, but that is also with only +3. With +5 much harder.

Graph showing increase in score ( y axis ) and iterations ( x axis ).

It’s nothing compared to your program, but it’s a hobby. I like doing it for some reason. My thinking is work with +5 to optimize the null skip hill climber operation EDIT: If it needs it. The matrix isn’t an exact model because the computer knows for a fact when position numbers are in the right place, but the homophonic hill climber doesn’t unless there are enough symbols in the right place. Probably a lot of them. But it may be possible to simulate that as well.

Here is another one, 5/5 matrix +5.

Hypothesis: Use matrix model, maybe +5, to experiment with null skip transposition to find optimum rules for hill-climbing. Although not exactly the same situation, close enough so that optimum rule applied to the computer program result in better solve rates.

I haven’t worked on any rules, except for change one position for each iteration yet, but I will work on it this week.

Nice work smokie. The score versus iterations graph is cool!

I prefer to leave the matrix work to you. And test some of your ideas later on.

Smokie here is a new build that allows you to change the settings: https://drive.google.com/open?id=1uzFIA … F1DNexJz9r

Also use the new .ini file please. I have changed substitution iterations to 498000 instead of 500000 which proved to be a small optimization.

Under solver settings you will find the following fields:

1. Temp
2. Shift %
3. Shift div
4. Restarts

Relate to:

1. The initial temperature of the nulls & skips hc is at 60. Test 50, 60 and 70.
2. The hc has a 10% chance of shifting a random null or skip a few positions. Test 5%, 10% and 15%.
3. The position shift (of the 10%) is a random number between 1 and the column length divided by 2. Test divided by 1.5, 2 and 2.5.
4. The amount of restarts to complete. After completion the program will stop the nulls & skips hill climber. It is important to let the program reach this number to keep the results unbiased. Use multiples of 10.

Also, the output window has additional information:

Over 22222: 0 (0%) Score: 20223.23 Ioc: 0.07607

Over 22222 counts the amount of restarts that achieved a score higher than 22222 and indicates that the cipher solved.

So the idea is then to do many restarts @ 40k hc iterations for each proposed test setting and to divide the number of times it reached over 22222 by the number of total restarts. Say that we did 1000 restarts for Temp 60 and it went over 22222 a 100 times then the solve rate is 100/1000=0.1 or 10%. Test each setting like that and see what works best.

If you want I can make the optimization spreadsheet.

I am going to try another approach. I did a simple little test with matrix 5/5 +5, which seems to indicate that it may be more important that the corrective skips and wildcards should jump around from 1 to 340 less often, and that they should walk around for shorter distances more often. Testing now on the spreadsheet. If I go with shift div at 4, then the skips and wildcards should walk around from -5 to + 5 positions at period 20, is this correct?

I am going to try another approach. I did a simple little test with matrix 5/5 +5, which seems to indicate that it may be more important that the corrective skips and wildcards should jump around from 1 to 340 less often, and that they should walk around for shorter distances more often. Testing now on the spreadsheet. If I go with shift div at 4, then the skips and wildcards should walk around from -5 to + 5 positions at period 20, is this correct?

Yes, with shift div at 5 it will range from -5 to +5 at period 20.

I suspect that with smokie_p20_5nulls_5skips the solver’s ability to distinct between good or bad is much more the issue than the working of the algorithm. Therefore you may want to test your hypothesis on easier ciphers first.

You mean at shift div 4, the positions change with be -5 to +5, right ( 20 / 4 = 5 )? I should have started with another message, but I already started with the 5/5. I will keep working though.

You mean at shift div 4, the positions change with be -5 to +5, right ( 20 / 4 = 5 )? I should have started with another message, but I already started with the 5/5. I will keep working though.

Yes, that range.

Here is a better solution for smokie 5/5. It seems to solve only once per 150 restarts with 80k hc iterations.

Score: 24499.71 Ioc: 0.06890
Ngrams: 955 PC-cycles: 2866

Period(20) Nulls(48,53,70,290,314), Skips(71,97,138,194,271)

BEASEEMEDSPECIALL
YGOODTHATNIGHTFOR
IWASVERYTIREDHOWL
ONGIHADBEENASLEEP
ICOULDNOTTELLBUTS
OMATIMEINTHENIGHT
IWESAWARENEDBYSOU
NDSOUTSIDEMYTENTA
SOFSOMEONEORSOMET
HINGWALKINGABOUTA
TFIRSTITHOUGHTITW
ASONEOFTHEMENTATP
RESENTLYDECIDEDIT
WASNOTANDBECAMEVE
RYWIDEAWAREITHOUG
HTABOUTTHEBEARTRI
LBUTDIDNOTQUITEEL
IEVEITWASTHEBERMI
THEIRRESENTLYODET
HINGSHOOKTHE

smokie 5/5:

5  62 26 40 27 41 28 36 49 25 53 1  47 60 48 23 29
13 47 42 14 20 61 37 7  35 60 9  43 26 18 50 15 2
62 53 32 33 35 9  63 40 3  21 41 63 17 4  49 19 38
27 63 42 52 51 61 1  6  27 29 13 50 43 51 26 57 53
52 40 50 21 47 39 14 36 27 5  58 15 56 53 16 11 59
24 32 28 2  57 41 61 50 17 37 40 43 54 59 25 22 13
55 14 3  12 34 61 56 35 4  53 43 54 55 15 58 10 16
17 29 34 23 47 11 38 13 2  51 14 33 26 18 32 2  3
56 27 28 39 15 4  62 5  41 28 30 29 42 31 53 54 62
36 60 56 12 56 44 21 9  53 56 16 55 39 17 10 38 28
24 24 9  12 1  23 39 61 47 52 49 37 28 13 54 53 37
14 15 38 43 16 17 30 15 40 2  14 23 45 29 48 39 15
24 15 34 43 20 58 34 8  17 16 6  54 51 51 40 14 21
15 3  33 16 9  62 47 4  20 17 26 7  27 13 46 55 13
43 8  22 10 52 32 38 5  56 49 6  22 37 11 1  53 2
58 23 38 31 28 54 24 33 5  29 62 15 40 40 53 16 35
22 48 26 17 56 53 2  18 42 13 57 21 50 39 34 58 27
57 63 12 14 43 56 55 5  32 22 33 41 60 15 56 23 41
56 16 54 55 54 28 59 63 57 47 15 13 62 14 32 47 33
45 51 53 57 1  54 1  61 45 54 15 20 [color=#FF0000][b]17 12 48 6  6[/b][/color]

viewtopic.php?f=81&t=3196&start=960

Jarlve, I think that I may have found a small problem with how we are working with smokie 5/5. Below left is your solution, with 335, and the actual message, with 340. The yellow letters are missing in the solution. The coded message, last 5 symbols are the symbols relocated to the end of the message because they were skipped in transcription. Have you been deleting the last five symbols, because I have not been.

EDIT:

If you take out the last five symbols and pour the message vertically into a 17 x 20 at P20, then you will create missing letters like what your solution shows. Is not the better thing to do to change the last 5 symbols into wildcards like this?

5 62 26 40 27 41 28 36 49 25 53 1 47 60 48 23 29
13 47 42 14 20 61 37 7 35 60 9 43 26 18 50 15 2
62 53 32 33 35 9 63 40 3 21 41 63 17 4 49 19 38
27 63 42 52 51 61 1 6 27 29 13 50 43 51 26 57 53
52 40 50 21 47 39 14 36 27 5 58 15 56 53 16 11 59
24 32 28 2 57 41 61 50 17 37 40 43 54 59 25 22 13
55 14 3 12 34 61 56 35 4 53 43 54 55 15 58 10 16
17 29 34 23 47 11 38 13 2 51 14 33 26 18 32 2 3
56 27 28 39 15 4 62 5 41 28 30 29 42 31 53 54 62
36 60 56 12 56 44 21 9 53 56 16 55 39 17 10 38 28
24 24 9 12 1 23 39 61 47 52 49 37 28 13 54 53 37
14 15 38 43 16 17 30 15 40 2 14 23 45 29 48 39 15
24 15 34 43 20 58 34 8 17 16 6 54 51 51 40 14 21
15 3 33 16 9 62 47 4 20 17 26 7 27 13 46 55 13
43 8 22 10 52 32 38 5 56 49 6 22 37 11 1 53 2
58 23 38 31 28 54 24 33 5 29 62 15 40 40 53 16 35
22 48 26 17 56 53 2 18 42 13 57 21 50 39 34 58 27
57 63 12 14 43 56 55 5 32 22 33 41 60 15 56 23 41
56 16 54 55 54 28 59 63 57 47 15 13 62 14 32 47 33
45 51 53 57 1 54 1 61 45 54 15 20 101 102 103 104 105

When I made smokie 5/5, first, I inserted 5 nulls. Then, the message was 345 long, but I only worked with 340 and left off the last 5 letters. Then, I skipped 5 letters, and moved them to the end to make 340. That is the problem. The last 5 still need to be in there so that the message is untransposed correctly, but they need to be new symbols, or wildcards, because otherwise they are just letters in the wrong place in the middle of a word.

Is not the better thing to do to change the last 5 symbols into wildcards like this?

That may be. At least, I have noted a better solve% after removing the last 5 instead of keeping them. I am in the middle of a large test and will not be able to test your idea for quite a while.

Score: 24930.90 Ioc: 0.06651 Multiplicity: 0.20298
Ngrams: 1223 PC-cycles: 4135

Period(20) Nulls(17,46,53,291,318) Skips(17,97,140,194,271)

BEUSEEMEDSPECIALL
YGOODTHATNIGHTFOR
IWASVERYTIREDHOWL
ONGIHADBEENASLEEP
ICOULDNOTTELLBUTS
OMETIMEINTHENIGHT
IWASAWAKENEDBYSOU
NDSOUTSIDEMYTENTA
SOFSOMEONEORSOMET
HINGWALKINGABOUTA
TFIRSTITHOUGHTITW
ASONEOFTHEMENBUTP
RESENTLYDECIDEDIT
WASNOTANDBECAMEVE
RYWIDEAWAKEITHOUG
HTABOUTTHEBEARTUA
LBUTDIDNOTQUITREL
IEVEITWASTHEBERMI
THERPRESENTLYODET
HINGSHOOKTHE