O.k. I am working on it now. I added a few columns to the spreadsheet for options. I will keep working on it for a long time. Let me know if I need to change anything. Thanks for your work.
Here is one of the top scoring results of my current run which is 76 restarts in to show that nulls can get high too:
Score: 21990.17 Ioc: 0.07448 Ngrams: 363 PC-cycles: 498 Period(15) Nulls(42,45,91,108,158,163,167,171,295) HTEADHEALLTHESMAP STHEIRSENTAGUESTE CHOFSTANCERALITHI SSTATINORTEROSSTH OSEIRDINEHEHINABE COLLERISINTESTILL ANDERFATOBEERCLAR ISSIPAINEDBLESTOR OFMORGONTHEPUTHER INGHINGTONONATSTA SSANCHROLENPAISIN NIESANATHLOCALLSH ITISCASTTHEREALLG AIAINSEMANDFISCAL LONHIDEADORREALRO ONPROTINGSISNATTH EROUNDISTHELATERI NESCANPHREATSCALA CTIOALOTHERFULTOC OMIDFUNL
Alternate to a random bias is to lock the hill climber into a certain amount of nulls and skips. Say 10 nulls & skips and let the hill climber use the following table:
Restart 1: 10 nulls
Restart 2: 9 nulls & 1 skip
Restart 3: 8 nulls & 2 skips
Restart 4: 7 nulls & 3 skips
Restart 5: 6 nulls & 4 skips
Restart 6: 5 nulls & 5 skips
Restart 7: 4 nulls & 6 skips
Restart 8: 3 nulls & 7 skips
Restart 9: 2 nulls & 8 skips
Restart 10: 1 null & 9 skips
Restart 11: 10 skips
Restart 12: Repeat from 10 nulls…
I could make it so that each nulls & skips division is placed in a different directory.
Thinking about it I really like the idea of locking in the hill climber into a certain amount of nulls & skips. Working on it.
Thinking about it I really like the idea of locking in the hill climber into a certain amount of nulls & skips. Working on it.
Here it is: https://drive.google.com/open?id=1wpO_i … EahlT3TncK
Gonna do some testing with it in the optimization thread.
What about a variable, X, and reduce the score by X * number of wildcards while keeping the random bias option. Then use trial and error to find best X. We could always leave X = 0 to try other methods. Or solve a message with a bunch of nulls but no skips, and figure out some way to analyse all of the scores, depending on nulls and skips "found," after 100 restarts or whatever to determine what X should be.
What about a variable, X, and reduce the score by X * number of wildcards while keeping the random bias option. Then use trial and error to find best X. We could always leave X = 0 to try other methods. Or solve a message with a bunch of nulls but no skips, and figure out some way to analyse all of the scores, depending on nulls and skips "found," after 100 restarts or whatever to determine what X should be.
It must be some normalization that acts upon changes in multiplicity. That may be different from cipher to cipher.
I much prefer the approach of looping through a table like this:
Restart 1: 8 nulls
Restart 2: 7 nulls & 1 skip
Restart 3: 6 nulls & 2 skips
Restart 4: 5 nulls & 3 skips
Restart 5: 4 nulls & 4 skips
Restart 6: 3 nulls & 5 skips
Restart 7: 2 nulls & 6 skips
Restart 8: 1 null & 7 skip
Restart 9: 8 skips
Restart 10: repeat from 8 nulls
Restart 11: etc …
It totally eliminates any bias problem and gives us much more feedback. The only con is that we will need to run more restarts. I am running through jarlve_p20_5nulls_3skips with this update and so far the solve rate percentage at 5 nulls & 3 skips is 21.49%. Though the overall solve rate percentage is only 2.38% because the cipher can only solve 1 out of 9 restarts. When the hill climber is at the correct number of nulls & skips the solve rate percentage is better than the old method because it does not need to hill climb the distribution of nulls & skips in the cipher.
I like your idea. I wonder, however, how the scores for restart 4 compare with the scores for restart 9. If scores were put on a graph, would there be a spike at restart 4? Perhaps after a certain number of restart loops, hone in on the one with the spike?
EDIT: I understand about how X described a few posts above would be different for different ciphers, but I wonder, how much different. There would be a variance, and I wonder how much the variance would be. Another approach maybe loop through the variance for X. I understand it would be a lot of work to figure it out. I would be willing to experiment on that at some point if you wanted me to. I could figure out what X should be. Otherwise I like your idea that is fine too.
I like your idea. I wonder, however, how the scores for restart 4 compare with the scores for restart 9. If scores were put on a graph, would there be a spike at restart 4? Perhaps after a certain number of restart loops, hone in on the one with the spike?
I thought the same. The question is, would you put your money on it that the spike is the correct one? Also, this can be done manually if we log the best scoring result for each unique nulls & skips distribution. It could be tested on your cipher with 17 nulls.
EDIT: I understand about how X described a few posts above would be different for different ciphers, but I wonder, how much different. There would be a variance, and I wonder how much the variance would be. Another approach maybe loop through the variance for X. I understand it would be a lot of work to figure it out. I would be willing to experiment on that at some point if you wanted me to. I could figure out what X should be. Otherwise I like your idea that is fine too.
It will never be perfect. And my feelings are that a small imperfection can go a long way with a large number of nulls & skips.
The test run with the new system has finished. Overall solve rate percentage is 3.6% and at the correct number of nulls & skips it is 32.14%.
The high score, multiplicity table:
8 nulls: 21400, 0.186
7 nulls & 1 skip: 21476, 0.188
6 nulls & 2 skips: 21920, 0.190
5 nulls & 3 skips: 24083, 0.192
4 nulls & 4 skips: 21761, 0.194
3 nulls & 5 skips: 21674, 0.195
2 nulls & 6 skips: 21723, 0.197
1 null & 7 skips: 21650, 0.199
8 skips: 21643, 0.201
smokie, you are testing 5.20.18. It is quite a bit better than the previous versions. I am testing 5.24.18 further in the optimization spreadsheet.
The question is, would you put your money on it that the spike is the correct one? Also, this can be done manually if we log the best scoring result for each unique nulls & skips distribution. It could be tested on your cipher with 17 nulls.
I started doing this last night when I got home from work because I had not saved the output from testing the 17 null cipher with 5.20.2018 before. Let me know what you want me to do and I will do it. I saw the new 5.24.2018 null skip loop table data and it looks good.
I would like the spreadsheet to be adjusted so that for every division of nulls & skips the best score, pc-cycles, multiplicity and ioc can be logged. Not sure if that is practical though. See what you can do. We may need to start things over on the 340 if we go with the table loop approach. In the meanwhile I will further test the approach and check if the settings are still somewhat optimal.
See the 340 spreadsheet rows 59-67, and example of what you may be looking for. Starting over is fine, and we can do that with the same spreadsheet. Note that I moved the TOTAL NULLS & SKIPS column way over to the left.
That looks fine. I wonder if we should keep all the information (score, ioc, multiplicity, pc-cycles and etc) in the solve log and make the field longer. And ditch the individual fields for score, multiplicity and pc-cycles. What do you think smokie?
