Zodiac Discussion Forum

Cipher Solver Relea…
 
Notifications
Clear all

Cipher Solver Release — Project Zenith

12 Posts
5 Users
0 Reactions
2,258 Views
(@beldenge)
Posts: 48
Trusted Member
Topic starter
 

Hi All,

I’m not technically new to these forums, although I have been mostly a lurker. I just wanted to share a cipher solver application that I developed in Java. It’s a command line application built on top of Spring Boot.

I present to you project Zenith.
https://github.com/beldenge/Zenith

It’s successful in solving the Zodiac 408, and I realize there are a growing number of tools that can do the same. But for what it’s worth, this is completely free and open source and hopefully will appeal to those with Java experience.

I think a couple of things that make this project unique are:
1. The capability to run any number of "transformers" on the ciphertext prior to decipherment. As I think we might safely say at this point, if the Zodiac 340 is truly a homophonic substitution cipher, he must have shuffled the ciphertext in some manner. So I’ve provided a number of transformers out of the box, but more will be added (suggestions welcome).
2. The ability to swap in different or new implementations of various parts of the application, from the scoring algorithm, to the aforementioned transformers, to the optimization algorithm itself. Configuration of all of this can be done easily in a properties file.

Much more information is in the README, but I’m sure I’ve left out details so please don’t hesitate to ask. I do plan to maintain this project so feedback and pull requests are warmly welcomed.

The following is a direct link to the first release with pre-built artifacts.
https://github.com/beldenge/Zenith/releases/tag/1.0.0

http://projectzenith.net
https://github.com/beldenge/zenith

 
Posted : July 5, 2019 7:26 am
(@mr-lowe)
Posts: 1197
Noble Member
 

welcome to the vortex long time lurker. nice intro.. looking forward to using your code cracker.
cheers

 
Posted : July 5, 2019 1:00 pm
doranchak
(@doranchak)
Posts: 2614
Member Admin
 

Welcome! And thank you very much for sharing your solver! I have added a link on this list of software tools: http://zodiackillerciphers.com/wiki/ind … ware_Tools

I look forward to giving it a spin.

http://zodiackillerciphers.com

 
Posted : July 5, 2019 1:11 pm
Jarlve
(@jarlve)
Posts: 2547
Famed Member
 

Welcome beldenge and thanks for sharing your solver.

How does the scoring function look like? Using letter n-grams and something to keep the letter frequencies in check such as index of coincidence or entropy? What transformers (transposition?) does it currently have?

AZdecrypt

 
Posted : July 5, 2019 4:06 pm
(@beldenge)
Posts: 48
Trusted Member
Topic starter
 

Doranchak,

Thanks for the link! I’m a big admirer of your work!

Jarlve,

Yes you are spot on with the scoring function. It uses 5-gram probabilities and is penalized by the index of coincidence. I think I roughly based it off one of your comments actually:
viewtopic.php?f=81&t=4040&p=71336&hilit=scoring#p71291

I try to go into a little more depth in the README:
https://github.com/beldenge/Zenith/tree … nd-scoring

As far as transformers go, here is a link directly to the list of transformers that I’ve already implemented, and yes transposition is one of them.
https://github.com/beldenge/Zenith/tree … mentations

The transformers can also be chained and repeated, so it has the potential of creating some powerful combinations. For an example of the type of thing I was going for, here’s a chain of transformers I put together to "unwrap" the ciphertext with the assumption that it was transposed vertically with the key "paradice" and then horizontally with the key "slaves". I didn’t get anywhere with it, but I based that assumption off the "clue" in the Halloween card.

decipherment.transformers.list=TranspositionCipherTransformer(slaves),RotateClockwiseCipherTransformer,InvertHorizontallyCipherTransformer,TranspositionCipherTransformer(paradice),InvertHorizontallyCipherTransformer,RotateCounterClockwiseCipherTransformer,RemoveLastRowCipherTransformer

The non-intuitive part is that the transformers have to be in reverse order from however you hypothesize the cipher was created/mutated, as it’s essentially "unwrapping" the ciphertext from those mutations. And the transposition transformer is similarly doing a sort of reverse transposition in order to unwrap the transposition that was performed during encipherment. I hope that makes sense, and I want to add clarity to the README, so please let me know what parts are confusing.

Thanks to you all!

http://projectzenith.net
https://github.com/beldenge/zenith

 
Posted : July 5, 2019 7:51 pm
Jarlve
(@jarlve)
Posts: 2547
Famed Member
 

I am amazed at how complete your project is for a first release, you also have support for multi-threading!

The non-intuitive part is that the transformers have to be in reverse order from however you hypothesize the cipher was created/mutated, as it’s essentially "unwrapping" the ciphertext from those mutations. And the transposition transformer is similarly doing a sort of reverse transposition in order to unwrap the transposition that was performed during encipherment.

Indeed, an important realization.

Your README and explanations are very clear to me.

AZdecrypt

 
Posted : July 6, 2019 9:58 am
(@beldenge)
Posts: 48
Trusted Member
Topic starter
 

That’s a big complement coming from you Jarlve!

It might be a first release, but it has been a pet project of mine for over two years, and I was experimenting with other solutions for probably 5 years before that. So it definitely didn’t happen overnight. :)

In fact it was insight from a comment of yours that helped me make the breakthrough — once I penalized the score with the index of coincidence it was finally able to escape the local optima (most of the time).

I experimented a lot with deep learning networks (LSTM) and was shocked that none of those seem to work as compared with the ngram + IoC model.

http://projectzenith.net
https://github.com/beldenge/zenith

 
Posted : July 6, 2019 7:10 pm
(@largo)
Posts: 454
Honorable Member
 

Hi beldenge,

Welcome to the Forum!
Your solver is really amazing and the source code is very clean. Even unit tests are included, that’s great! One can see how much work and care you put into the project. Thank you for publishing your project as open source.
The more solvers with different approaches are available, the better. There are so many ciphers that are difficult to crack even for AZDecrypt. Perhaps in these cases the genetic algorithm proves to be the method of choice. I am looking forward to further releases.

Translated with www.DeepL.com/Translator

 
Posted : July 7, 2019 7:40 pm
(@beldenge)
Posts: 48
Trusted Member
Topic starter
 

Largo, I really appreciate the complements! I’m still working on the genetic algorithm optimizer. It’s not able to escape the local optima yet, but I suspect that’s due to the fitness function — I can’t use the same numeric output as the one I’m using for the standard hill climber. Thanks for taking a look at the project!

http://projectzenith.net
https://github.com/beldenge/zenith

 
Posted : July 7, 2019 7:49 pm
doranchak
(@doranchak)
Posts: 2614
Member Admin
 

Why did your reddit post get deleted?

http://zodiackillerciphers.com

 
Posted : July 15, 2019 8:15 pm
(@beldenge)
Posts: 48
Trusted Member
Topic starter
 

It looks to me like it’s still there. It’s kind of far down in the feed now, but that’s more or less expected. Thanks for looking out in any case!

By the way, do you have any other suggestions for places it would be good to share the project? I’ve only shared it here and Reddit thus far.

http://projectzenith.net
https://github.com/beldenge/zenith

 
Posted : July 16, 2019 5:43 am
doranchak
(@doranchak)
Posts: 2614
Member Admin
 

Oh! Weird – it looked earlier like it was removed. Glad it’s still there.

As for where else to post:

http://reddit.com/r/codes might be interested in it, although I generally only see posts about specific codes and ciphers rather than the tools for breaking them.

Readers at Klaus’ blog might be interested: http://scienceblogs.de/klausis-krypto-kolumne/

And Nick’s: http://ciphermysteries.com/

The American Cryptogram Association Discussion Group on Facebook might want to hear about it, too.

http://zodiackillerciphers.com

 
Posted : July 16, 2019 2:14 pm
Share: