You don’t necessarily need to include a Broadwell version just for me if you don’t want to, you made a huge difference with the new programming versus the new compiler.
First 5/3
T H E D I S C O V E R Y O F T H E
C O I N P E R P L E X E D G I L E
S I T W A S C E R T A I N L Y T H
E T R I N K E T A T T A C H E D T
O T H E B A N G L E W H I C H H E
H A D G I V E N A N N E A N D H E
R E H E F O U N D I T I N T H E G
R O U N D S O F T H E P R I O R Y
T H I S W O U L D A R G U E T H A
T S H E W A S I N T H E N E I G H
B O R H O O D I N T H E H O U S E
I T M I G H T B E S H E H A D N E
V E R B E E N T O T H E P R I O R
Y W H E N L I V I N G A T T H E E
L M S C E R T A I N L Y N O T A F
T E R T H E N E W Y E A R W H E N
S H E F I R S T B E C A M E P O S
S E S S E D O F T H E C O I N H E
D E C I D E D T H E R E F O R E T
H A T A T S O M E L A T E P E R I
Null positions: 57, 60, 111, 181, 280
Skip positions: 108, 217, 275
51 7 47 11 37 20 43 44 52 53 5 24 57 62 29 54 48
49 9 21 23 38 25 51 52 1 12 40 23 47 40 53 13 59
31 14 23 15 11 3 15 26 51 45 20 10 22 55 27 20 43
32 43 20 48 44 63 15 48 63 8 51 10 36 60 24 14 18
15 34 49 11 21 26 6 11 7 53 16 50 26 1 27 41 4
35 5 24 17 10 59 59 37 19 13 36 14 21 27 14 9 53
48 12 48 28 1 38 49 39 63 2 40 21 12 30 45 13 46
10 14 50 8 43 7 15 33 14 56 37 55 47 9 54 34 25
51 35 50 38 10 39 40 42 12 52 18 35 33 16 29 27 24
6 53 57 3 13 54 17 51 31 58 29 44 4 29 1 9 52
10 34 35 14 37 25 26 59 5 52 63 20 15 11 13 54 53
13 36 24 21 3 52 53 47 52 33 34 62 14 22 15 30 45
61 3 53 60 35 52 11 46 23 20 21 19 30 15 8 13 43
4 62 14 24 1 23 15 25 41 18 15 12 13 14 1 62 3
4 7 15 53 38 9 36 8 25 3 33 45 56 34 21 21 42
54 35 45 32 40 16 11 17 18 30 23 7 36 51 27 12 13
2 46 52 37 59 14 24 63 38 41 53 26 62 15 21 10 20
39 51 24 55 9 27 21 51 22 42 33 43 11 23 29 52 10
20 21 12 44 22 18 50 35 40 13 3 14 38 23 15 45 11
12 20 53 13 14 19 62 4 21 15 11 45 13 17 57 22 37
EDIT: This is the correct message. I accidentally pasted the wrong one, and it was online for a minute or two. I deleted it.
That is really quite incredible.
I like to compound gains.
1. Speed up AZdecrypt.
2. Improve the substitution solver so that it does more with less.
3. Improve the period + nulls & skips solver so that it does more with less.
When improvements on all these come together very good things happen.
Thank you for the cipher. I am trying some myself. The following one seems to be of good difficulty. It is currently at 34 restarts of 10000 HC iterations and there has been only one partial solve so far and it scores under 22000. The message are some quotes from twitter.
jarlve_p20_5nulls_3skips:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 6 19 20 21 22 18 23 18 24 25 17 26 27 28 22 15 4 15 29 6 8 30 14 19 31 13 2 20 32 33 34 35 17 36 32 37 38 39 24 28 40 41 42 11 25 43 44 25 45 10 29 46 47 2 48 32 31 21 9 17 27 49 23 29 37 27 34 12 30 47 50 51 52 39 6 20 17 53 6 18 11 8 4 43 45 38 36 40 21 10 2 44 13 18 42 6 12 17 19 30 43 54 14 38 21 24 3 18 55 35 47 49 56 11 5 30 17 7 45 37 25 57 38 6 41 44 28 34 29 15 20 21 58 10 20 38 27 42 12 21 8 17 4 36 53 33 43 36 6 13 36 14 20 6 32 23 29 48 35 24 22 38 47 30 16 35 24 49 21 22 47 42 25 8 27 4 13 44 59 19 31 2 9 23 29 11 18 8 38 19 30 14 20 10 53 21 50 60 17 4 22 38 35 21 18 12 15 32 35 6 3 38 37 20 30 17 13 26 42 47 39 58 32 45 6 2 24 54 20 7 49 30 17 30 34 61 47 51 25 37 14 11 28 6 21 29 43 44 20 4 30 10 7 5 13 53 16 17 3 28 18 52 24 49 58 38 40 6 29 62 61 44 55 51 21 45 18 20 8 44 2 18 10 25 38 9 21 37 30 35 11 17 36 45 2 56 22 11 6 20 38 3 49 1 17 30 15 47 7 42 21 18 27 60 30 32 4 14 35 33 58 53 2,Z>J@NH.BT)K74U9 Q@WEF0QDQS&9^;04 >4:@H67W]K,EY5=*9 IYCA(S;+-"T&#R&8B :/3,LY]F.9^<D:C^= )63V'?(@E9O@QTH># 8AI+FB,RKQ"@)9W6# !7AFSZQ[*3<XTJ69N 8C&$A@-R;=:4EFGBE A^")FH9>IO5#I@KI7 E@YD:L*S0A36U*S<F 03"&H^>KRMW],.D:T QHAW67EBOFV%9>0A* FQ)4Y*@ZACE69K"3 (GY8@,S!EN<696=13 '&C7T;@F:#RE>6BNJ KOU9Z;Q?S<GA+@:P1 R['F8QEHR,QB&A.FC 6*T9I8,X0T@EAZ<29 643N"FQ^%6Y>7*5GO
FYI: no nulls at the end of my cipher jarlve_p20_5nulls_3skips. Leave it as it is.
First 3/5:
T H E Y H A D A H O U S E O F C R
Y S T A L P I L L A R S O N T H E
P L A N E T M A R S B Y T H E E D
G E O F A N E M P T Y S E A A N D
E V E R Y M O R N I N G Y O U C O
U L D S E E M R S K E A T I N G T
H E G O L D E N F R U I T S T H A
T G R E W F R O M T H E C R Y S T
A L W A L L S O R C L E A N I N G
T H E H O U S E W I T H H A N D F
U L S O F M A G N E T I C D U S T
W H I C H T A K I N G A L L D I R
T W I T H I T B L E W A W A Y O N
T H E H O T W I N D A F T E R N O
O N S W H E N T H E F O S S I L S
E A W A S W A R M A N D M O T I O
N L E S S A N D T H E W I N E T R
E E S S T O O D S T I F F I N T H
E Y A R D A N D T H E L I T T L E
D I S T A N T M A R T I A N B O N
Nulls: 46, 114, 148
Skips: 55, 186, 207, 267, 324 skipped symbols moved to the end of the message
53 62 44 20 13 58 22 54 1 55 59 61 56 57 40 14 36
15 16 10 23 50 31 17 60 32 13 21 33 24 31 25 61 22
37 2 32 14 62 26 15 53 3 40 17 63 11 20 46 61 17
50 27 28 51 61 17 52 4 51 62 1 38 18 47 50 42 15
1 23 43 8 54 24 61 1 51 52 48 55 25 33 17 3 62
15 31 61 32 40 19 22 23 40 24 49 50 56 12 4 5 44
54 38 34 13 12 18 33 58 35 54 27 63 57 14 61 1 42
2 37 11 26 34 15 43 35 16 46 51 50 1 4 55 61 37
5 37 42 39 56 1 31 2 34 46 48 63 36 41 42 17 21
30 6 27 57 46 12 11 10 35 24 33 47 44 37 49 19 34
48 61 38 28 33 39 23 35 53 50 54 1 43 4 55 29 30
46 53 9 26 13 36 14 12 15 5 23 57 24 46 59 48 7
37 16 58 25 31 53 54 20 61 1 19 38 17 27 13 55 52
49 62 50 21 2 28 14 17 22 29 3 4 19 40 12 61 19
32 28 16 41 53 17 62 57 53 9 3 23 9 33 61 54 51
34 27 19 28 1 40 39 24 43 29 52 46 36 3 11 31 4
13 51 40 37 26 54 38 18 56 14 5 59 39 53 62 29 39
58 12 62 47 28 53 17 37 54 6 8 25 17 37 9 20 25
50 39 10 51 29 41 39 32 26 55 56 33 42 14 11 12 43
57 1 53 21 18 54 46 38 40 52 41 47 13 51 62 2 48
Okay smokie. Thank you, that will be okay for now. The cipher I made will be perfect to improve the hill climber further. At 10000 HC iterations it only has a 2% solve rate.
I have the new program installed on my Acer, and am working on the 3/3. It solved the 2/2 in just a few minutes. I will also work on the spreadsheet in upcoming days. This work week is fairly busy. I am glad that you are having fun with your new creative endeavor. I think some of the most rewarding parts of cryptanalysis is the opportunity to get creative and that is what brings us back here. I am interested in seeing how you continue to refine the art of hill climbing the detection of null and skip positions. There are likely a lot of different possibilities. I also like the idea of being involved with a planned and organized attack plan. Mostly so far we have been exploring a lot of different ideas. But this time it is different and I am looking forward to the attack, when ready. Thanks for all of your work.
I got a good solution for the 3/3 with 13 restarts in about an hour, maybe a little longer. 10000 HC @ 50000.
Score: 23849.74 Ioc: 0.06265 Ngrams: 3770 PC-cycles: 2347 Key: Period(20) Nulls(71,113,193), Skips(123,161,203) EACHPLAYLREEARSEI THERLIGHTCADIUMOR HEAVYARMORHEAVIER ARMORSSUPPLYLARGE RACOUNTSOFARMOREN ERGYANDAMMUNITION DIFFERENTARMORTYP ESSUPPORTDIFFEREN THEAPONSANDEQUIPM ENTFOREXAMPLEONLY THESEABYASCORSUPP ORTSTHEHERBYCORTA RBUTONLYLIGHTARMO RSUPPORTSTHESNIPE RRIFLEWHENDAMAGEI SDEALTTOTHEPLAYER BYFALLINGORBEINGH URTBYAWEAPONARCOI SLOSTLOSSOFALLACO RRESULTWINTHEP
I think that the hill climber is ready to test up to a total of 6 nulls/skips. Whenever you release your spreadsheet we can decide to take off or not. For now I probably want to reserve my workstation for improving the hill climber rather than adding to the spreadsheet but I should be able to find some time here and there.
Solve rates for jarlve_p20_5nulls_3skips:
10k HC: 1.93%
20k HC: 4.20%
Started a 40k run right now.
Here is my best solve for jarlve_p20_5nulls_3skips, 76 restarts HC 10000. I can see bits and pieces of sentences enough to know it is a message. Do you have the actual positions for the nulls and skips available? I would like to compare.
Score: 21705.43 Ioc: 0.06891 Ngrams: 1289 PC-cycles: 6002 Key: Period(20) Nulls(43,118,137,194,246), Skips(70,173,231) VESINATPROSTITUTE ITWASNTMYFINESTHO URANDIASNOTHINGTH ATIMATULLASHAMEDO FNEALLTSEDTOEESOM ETHINGPHATWEMAYOR MAYNOTOELROUDOFIT SALSONOTSOSTONMET HEORGIDUEREDEMPLA NCEOFMCNEANDOFFIC IALOCAIERSRELATIO NSHIPTFTHATOFRICH ARDDEROANAGEOFTRA SHTHEEERSMCAFEEIS HAPPENMNDBECAUSEO FHATREDTOWARDSMEY ESSHEWASAPROSTITU TEGETNDERITPICTUR ESPAINTATHOUSANDW ORDSSELONTHEDAV
VESINATPROSTITUTE
ITWASNTMYFINESTHO
URANDIASNOTHINGTH
ATIMATULLASHAMEDO
FNEALLTSEDTOEESOM
ETHINGPHATWEMAYOR
MAYNOTOELROUDOFIT
SALSONOTSOSTONMET
HEORGIDUEREDEMPLA
NCEOFMCNEANDOFFIC
IALOCAIERSRELATIO
NSHIPTFTHATOFRICH
ARDDEROANAGEOFTRA
SHTHEEERSMCAFEEIS
HAPPENMNDBECAUSEO
FHATREDTOWARDSMEY
ESSHEWASAPROSTITU
TEGETNDERITPICTUR
ESPAINTATHOUSANDW
ORDSSELONTHEDAV
It looks to me like the best parts are at the top and bottom, for some reason.
EDIT: Do you mean 6/6?
Here is my best solve for jarlve_p20_5nulls_3skips, 76 restarts HC 10000. I can see bits and pieces of sentences enough to know it is a message. Do you have the actual positions for the nulls and skips available? I would like to compare.
Yes you have a partial solve, also clearly indicated by the ngrams and pc-cycles score. I did not keep the positions. The message starts with "Yes I was prostitute" and is from John McAfee’s twitter.
EDIT: Do you mean 6/6?
No, a total amount of 6.
These positions should be fairly accurate, it appears to be a near perfect decryption. I do remember stacking 2 nulls next to each other (130, 131).
Score: 24083.40 Ioc: 0.06647 Ngrams: 1800 PC-cycles: 6794 Key: Period(20) Nulls(43,130,131,195,247), Skips(72,169,231) YESIWASPROSTITUTE ITWASNTMYFINESTHO URANDITSNOTHINGTH ATIMATALLASHAMEDO FWEALLUSEDTOBESOM ETHINGTHATWEMAYOR MAYNOTBEPROUDOFIT SALSONOTLOSTONMET HEOBVIOUSRESEMBLA NCEOFMINEANDOFFIC IALLCAFEESRELATIO NSHIPTOTHATOFRICH ARDGEREAWAVEOFTRA SHTHEEMRSMCAFEEIS HAPPENINGBECAUSEO FHATREDTOWARDSMEY ESSHEWASAPROSTITU TEGETOVERITPICTUR ESPAINTATHOUSANDW ORDSBELOWTHEDAY
I am a little bit worried about a bias towards skips with higher nulls & skips settings in AZdecrypt. The bias is there because adding skips to a cipher typically increases the multiplicity more than removing nulls since each skip is a new symbol (64, 65, 66). This could be fixed by using the multiplicity weight setting, though I am not sure what value to use. I will do a run on a randomized 340 cipher to see how many nulls and skips are used on average. If that indicates that the bias exists figure out a multiplicity weight value that fixes it approximately. That will be more than good enough.
Nulls would be intentional filler, gibberish, or a shape or hidden additional message, etc. But skips would be a mistake. I think that there would probably be a lot more nulls than skips.
Nulls(43,118,137,194,246), Skips(70,173,231) partial solve
Nulls(43,130,131,195,247), Skips(72,169,231) perfect or near perfect solve
I see that the positions for the partial solve are the same or really close to the positions for the perfect solve, except for the two side by side nulls. Those are a lot farther off, one by 12 positions and one by 6 positions.
Here is the link to the spreadsheet. I made it so anyone can view, but only you and I can edit. I used your yahoo address, so not sure if that will work. Interesting to know, otherwise p.m. me a gmail address.
