Yes and I can tell that it is a partial solve without looking at the score. Getting close to P15 territory for sure. What settings are you testing? Any new developments with the algorithm?
I tried to optimize several settings but every change just seems worse. Testing the initial temperature value and settings that relate to your idea of sometimes shifting a null or skip to the left or right a few positions. My thinking now is to test how well smokie 5/5 solves with 80k hill climber iterations and to test how well the solver handles extremes such as only nulls or only skips. Probably also need to test 6-grams. Want to help? Could need some spreadsheet organization for this.
That is fine. Let me know what you want.
I wonder about the position fine tuning and exactly what you are doing. You had been making random changes over a range of positions. What else have you tried?
EDIT: Let’s double check what message you are using for smokie 5/5 for a minute. Show me where you got that from.
That is fine. Let me know what you want.
I wonder about the position fine tuning and exactly what you are doing. You had been making random changes over a range of positions. What else have you tried?
EDIT: Let’s double check what message you are using for smokie 5/5 for a minute. Show me where you got that from.
Here is some of the things we could do.
– The initial temperature of the nulls & skips hc is at 60. Test 50, 60 and 70.
– The hc has a 10% chance of shifting a random null or skip a few positions. Test 5%, 10% and 15%.
– The position shift (of the 10%) is a random number between 1 and the column length divided by 2. Test divided by 1.5, 2 and 2.5.
– Test 5-grams versus 6-grams.
Each test needs at least 1000 restarts with 40k hc iterations, I guess you see the problem there. Not sure if it is worth the time. The restarts need to be done in a continuous run or in fixed chunks. I could make changes to the program to help with that of course.
This one needs to be done: try extremes such as a cipher with 8 nulls and a cipher with 8 skips. Just to see if they solve and if there is a problem with the inherent bias towards skips since these increase the multiplicity more.
Here is a better solution for smokie 5/5. It seems to solve only once per 150 restarts with 80k hc iterations.
Score: 24499.71 Ioc: 0.06890 Ngrams: 955 PC-cycles: 2866 Period(20) Nulls(48,53,70,290,314), Skips(71,97,138,194,271) BEASEEMEDSPECIALL YGOODTHATNIGHTFOR IWASVERYTIREDHOWL ONGIHADBEENASLEEP ICOULDNOTTELLBUTS OMATIMEINTHENIGHT IWESAWARENEDBYSOU NDSOUTSIDEMYTENTA SOFSOMEONEORSOMET HINGWALKINGABOUTA TFIRSTITHOUGHTITW ASONEOFTHEMENTATP RESENTLYDECIDEDIT WASNOTANDBECAMEVE RYWIDEAWAREITHOUG HTABOUTTHEBEARTRI LBUTDIDNOTQUITEEL IEVEITWASTHEBERMI THEIRRESENTLYODET HINGSHOOKTHE
smokie 5/5:
5 62 26 40 27 41 28 36 49 25 53 1 47 60 48 23 29 13 47 42 14 20 61 37 7 35 60 9 43 26 18 50 15 2 62 53 32 33 35 9 63 40 3 21 41 63 17 4 49 19 38 27 63 42 52 51 61 1 6 27 29 13 50 43 51 26 57 53 52 40 50 21 47 39 14 36 27 5 58 15 56 53 16 11 59 24 32 28 2 57 41 61 50 17 37 40 43 54 59 25 22 13 55 14 3 12 34 61 56 35 4 53 43 54 55 15 58 10 16 17 29 34 23 47 11 38 13 2 51 14 33 26 18 32 2 3 56 27 28 39 15 4 62 5 41 28 30 29 42 31 53 54 62 36 60 56 12 56 44 21 9 53 56 16 55 39 17 10 38 28 24 24 9 12 1 23 39 61 47 52 49 37 28 13 54 53 37 14 15 38 43 16 17 30 15 40 2 14 23 45 29 48 39 15 24 15 34 43 20 58 34 8 17 16 6 54 51 51 40 14 21 15 3 33 16 9 62 47 4 20 17 26 7 27 13 46 55 13 43 8 22 10 52 32 38 5 56 49 6 22 37 11 1 53 2 58 23 38 31 28 54 24 33 5 29 62 15 40 40 53 16 35 22 48 26 17 56 53 2 18 42 13 57 21 50 39 34 58 27 57 63 12 14 43 56 55 5 32 22 33 41 60 15 56 23 41 56 16 54 55 54 28 59 63 57 47 15 13 62 14 32 47 33 45 51 53 57 1 54 1 61 45 54 15 20 17 12 48 6 6
Here is some of the things we could do.
– The initial temperature of the nulls & skips hc is at 60. Test 50, 60 and 70.
– The hc has a 10% chance of shifting a random null or skip a few positions. Test 5%, 10% and 15%.
– The position shift (of the 10%) is a random number between 1 and the column length divided by 2. Test divided by 1.5, 2 and 2.5.
– Test 5-grams versus 6-grams.Each test needs at least 1000 restarts with 40k hc iterations, I guess you see the problem there. Not sure if it is worth the time. The restarts need to be done in a continuous run or in fixed chunks. I could make changes to the program to help with that of course.
This one needs to be done: try extremes such as a cipher with 8 nulls and a cipher with 8 skips. Just to see if they solve and if there is a problem with the inherent bias towards skips since these increase the multiplicity more.
The program, on 10% of the null skip detecting hill climbing iterations, makes a random position change for one of the skips or nulls, and the position is not between 1 and 340, but more regional. I am just writing this out so I understand, and the shift is random between 1 position and half of the column length. So, at P20, is the random shift between -17/2 and +17/2 ( except for 0)?
Would it be any more or less efficient to choose 10 regional position changes 1% of the time and compare them to each other, and then keeping the best one? Or what about, 1% of the time, compare the all positions -( 340 / P / 4 ) to +( 340 / P / 4 ), and keep the best one. If you don’t get the formula, then, basically, a smaller percentage of the time, but a more complete and smaller range, instead of random. Maybe experiment with different parameters on this idea.
As far as bias and multiplicity increased by skips, I think it would have to do with initial multiplicity. Maybe we don’t need to worry about that so much with the 340? I made a message, with 24 + symbol polyphones, and the program partial solved in the blink of an eye on regular mode. Then I expanded the polyphone, and it solved better in the blink of an eye. Still, both were plenty good enough. It seems like a few extra symbols may not hurt as much, depending on original multiplicity.
I vote to experiment with the fine tuning EDIT: of the null skip detection fine tuning algorithm in different ways, maybe that will leverage your time and efforts to the maximum.
The program, on 10% of the null skip detecting hill climbing iterations, makes a random position change for one of the skips or nulls, and the position is not between 1 and 340, but more regional. I am just writing this out so I understand, and the shift is random between 1 position and half of the column length. So, at P20, is the random shift between -17/2 and +17/2 ( except for 0)?
Indeed, except for 0 or values that would make it go out of bound.
Would it be any more or less efficient to choose 10 regional position changes 1% of the time and compare them to each other, and then keeping the best one? Or what about, 1% of the time, compare the all positions -( 340 / P / 4 ) to +( 340 / P / 4 ), and keep the best one. If you don’t get the formula, then, basically, a smaller percentage of the time, but a more complete and smaller range, instead of random. Maybe experiment with different parameters on this idea.
It probably would not work so well but it can be tried.
As far as bias and multiplicity increased by skips, I think it would have to do with initial multiplicity. Maybe we don’t need to worry about that so much with the 340? I made a message, with 24 + symbol polyphones, and the program partial solved in the blink of an eye on regular mode. Then I expanded the polyphone, and it solved better in the blink of an eye. Still, both were plenty good enough. It seems like a few extra symbols may not hurt as much, depending on original multiplicity.
Not sure if you understood what I meant. The nulls & skips hill climber has a bias towards using skips as that increases the score through the increased multiplicity. That may become more and more problematic as nulls & skips go up.
I vote to experiment with the fine tuning EDIT: of the null skip detection fine tuning algorithm in different ways, maybe that will leverage your time and efforts to the maximum.
Okay good.
Here is some of the things we could do.
– The initial temperature of the nulls & skips hc is at 60. Test 50, 60 and 70.
– The hc has a 10% chance of shifting a random null or skip a few positions. Test 5%, 10% and 15%.
– The position shift (of the 10%) is a random number between 1 and the column length divided by 2. Test divided by 1.5, 2 and 2.5.
Please throw these options in a simple spreadsheet, temp 50, 60 and 70, shift 5%, 10% and 15%, shift divider 1.5, 2 and 2.5. 9 things to test initially, see how that goes. I will build a version of AZdecrypt where you can change these settings and also record the solve rate. We will use my cipher jarlve_p20_5nulls_3skips, there are no nulls at the end. 40k hc iterations in chunks of multiples of 10 depending on how you long you want it to run, overnight or etc.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 6 19 20 21 22 18 23 18 24 25 17 26 27 28 22 15 4 15 29 6 8 30 14 19 31 13 2 20 32 33 34 35 17 36 32 37 38 39 24 28 40 41 42 11 25 43 44 25 45 10 29 46 47 2 48 32 31 21 9 17 27 49 23 29 37 27 34 12 30 47 50 51 52 39 6 20 17 53 6 18 11 8 4 43 45 38 36 40 21 10 2 44 13 18 42 6 12 17 19 30 43 54 14 38 21 24 3 18 55 35 47 49 56 11 5 30 17 7 45 37 25 57 38 6 41 44 28 34 29 15 20 21 58 10 20 38 27 42 12 21 8 17 4 36 53 33 43 36 6 13 36 14 20 6 32 23 29 48 35 24 22 38 47 30 16 35 24 49 21 22 47 42 25 8 27 4 13 44 59 19 31 2 9 23 29 11 18 8 38 19 30 14 20 10 53 21 50 60 17 4 22 38 35 21 18 12 15 32 35 6 3 38 37 20 30 17 13 26 42 47 39 58 32 45 6 2 24 54 20 7 49 30 17 30 34 61 47 51 25 37 14 11 28 6 21 29 43 44 20 4 30 10 7 5 13 53 16 17 3 28 18 52 24 49 58 38 40 6 29 62 61 44 55 51 21 45 18 20 8 44 2 18 10 25 38 9 21 37 30 35 11 17 36 45 2 56 22 11 6 20 38 3 49 1 17 30 15 47 7 42 21 18 27 60 30 32 4 14 35 33 58 53 2,Z>J@NH.BT)K74U9 Q@WEF0QDQS&9^;04 >4:@H67W]K,EY5=*9 IYCA(S;+-"T&#R&8B :/3,LY]F.9^<D:C^= )63V'?(@E9O@QTH># 8AI+FB,RKQ"@)9W6# !7AFSZQ[*3<XTJ69N 8C&$A@-R;=:4EFGBE A^")FH9>IO5#I@KI7 E@YD:L*S0A36U*S<F 03"&H^>KRMW],.D:T QHAW67EBOFV%9>0A* FQ)4Y*@ZACE69K"3 (GY8@,S!EN<696=13 '&C7T;@F:#RE>6BNJ KOU9Z;Q?S<GA+@:P1 R['F8QEHR,QB&A.FC 6*T9I8,X0T@EAZ<29 643N"FQ^%6Y>7*5GO
Have you been checking other sites for recent attempts at decryption? There are some interesting ones that remind me of your work re. pivots….one has to do with dice and pinholes. IIRC they are on Tom Voight’s site. Just wondering if perhaps any of those would help you with yours.D
One of interest is at zodiackiller.com in the discussion forum "Aeneas Tacticus & The Dice" by Mobyelectra 12-11-17.
Have you been checking other sites for recent attempts at decryption? There are some interesting ones that remind me of your work re. pivots….one has to do with dice and pinholes. IIRC they are on Tom Voight’s site. Just wondering if perhaps any of those would help you with yours.D
One of interest is at zodiackiller.com in the discussion forum "Aeneas Tacticus & The Dice" by Mobyelectra 12-11-17.
https://www.tapatalk.com/groups/zodiack … t7873.html
I have looked at it briefly but do not immediately understand what he is getting at.
Smokie here is a new build that allows you to change the settings: https://drive.google.com/open?id=1uzFIA … F1DNexJz9r
Also use the new .ini file please. I have changed substitution iterations to 498000 instead of 500000 which proved to be a small optimization.
Under solver settings you will find the following fields:
1. Temp
2. Shift %
3. Shift div
4. Restarts
Relate to:
1. The initial temperature of the nulls & skips hc is at 60. Test 50, 60 and 70.
2. The hc has a 10% chance of shifting a random null or skip a few positions. Test 5%, 10% and 15%.
3. The position shift (of the 10%) is a random number between 1 and the column length divided by 2. Test divided by 1.5, 2 and 2.5.
4. The amount of restarts to complete. After completion the program will stop the nulls & skips hill climber. It is important to let the program reach this number to keep the results unbiased. Use multiples of 10.
Also, the output window has additional information:
Over 22222: 0 (0%) Score: 20223.23 Ioc: 0.07607
Over 22222 counts the amount of restarts that achieved a score higher than 22222 and indicates that the cipher solved.
So the idea is then to do many restarts @ 40k hc iterations for each proposed test setting and to divide the number of times it reached over 22222 by the number of total restarts. Say that we did 1000 restarts for Temp 60 and it went over 22222 a 100 times then the solve rate is 100/1000=0.1 or 10%. Test each setting like that and see what works best.
If you want I can make the optimization spreadsheet.
I made the spreadsheet already, but you can modify:
EDIT GOOGLE DOCS VERSION: https://docs.google.com/spreadsheets/d/ … sp=sharing
Try the second link that I sent you.
Thanks.
Looks good.
We primarily want to record the solve rate of each setting (over 22222/restarts) so we need a field for the restarts "over 22222".
Perhaps we will test more settings later. I would rather use the note field to list the settings that were used. That way no new columns need to be added for each new test and we can just add horizontally. So I would put the NOTES "AA" column after the AZD VERSION "F" column and remove the columns "P" to "X". And we can enter in the NOTES columns something like "Temp 60, shift % 10, shift div 2".
O.k., it looks like you are in. Feel free to modify the format or whatever if you see that I don’t understand something. Have to go to work now. Thanks.
O.k., it looks like you are in. Feel free to modify the format or whatever if you see that I don’t understand something. Have to go to work now. Thanks.
Okay. I added 9 tests, the idea is to stack many restarts for each test to get increasingly more accurate solve rate percentages. This way we will know what settings are better and by how much.
Also use the new .ini file please.
Help me out. What is the new .ini file? It wasn’t in the download, but there was a new settings file.
I am working on the top row of the list. Check my settings, please. I changed to 40000 General Hill climber iterations, 1 General Hill climber iterations factor, 8 nulls & skips, Temp 50, Shift % 10, Shift div 2, Restarts 100. Is this all good?
Jep, it is good. The settings file is the .ini file. CPU threads could be changed to at least 4 but that is your choice.
What about 6?
