No matter the dictionary or rule-based attacks sometimes the hashes still remain uncracked, and so begins our adventures in researching how to crack foreign character encoded hashes. This is the ultimate guide to cracking foreign character hashes using hashcat we wish we'd always had while tackling this challenge. If you would like a handy reference guide to cracking tool usage check out HASH CRACK: Password Cracking Manual on Amazon. Also if you're needing a resource to aid in making stronger passwords for your most sensitive accounts go take a look at the One-Time Grid: Random Password Book for random password creation and storage.
What is Hashcat you say? If you need a detailed introduction go visit the Hashcat.net site and learn about the most powerful open-source password recovery tool available. Essentially, Hashcat 3.0 is an advanced password recovery tool that leverages CPU or GPU resources to generate password attacks against 160+ hash types. This enables even the most meager of CPU's or GPU's, to generate 135 Million hashes per second, like our test Mac Mini. Monster password cracking rigs like the Brutalis offered by Sagitta can generate 350 Billion hashes per second, against Md4. There are plenty of guides on the Hashcat.net wiki and forum so we are not going to cover the basics of using hashcat. Instead we will be focusing exclusively on foreign character encoded passwords and why the topic deserves its own guide to deal with them.
Background on Foreign Characters
So most everyone in English speaking nations will be familiar with ASCII, since it's the original character encoding. ASCII uses single byte encodings to display various Latin/English character, and since all 256 bit combinations (0000 0000 - 1111 1111) were taken up for ASCII Latin/English characters, a new foreign character set was created, UTF-8. Why are we selecting UTF-8? Because it's the most widely used character encoding for Web applications/pages (at nearly 87%) and the techniques we describe for UTF-8 can be applied to any other character encodings.
UTF-8 character encoding is a one to four-byte encoding (0000 0000 - 0000 0000 0000 0000 0000 0000 0000 0000), encompassing ASCII's single byte (0000 0000) and therefore password guessing in hashcat has to account for this byte distinction. We account for this byte difference, when working through our examples for Cyrillic, by forcing hashcat to make password attempts in two-byte HEX. Before we move on let us look at some ASCII tables and UTF-8 HEX tables to understand this distinction more clearly.
ASCII HEX Table
Take note in the above table "Hx" column for the capital letter "A", it's represented in Hex as "41". This "41" is represented in single-byte Hex by the binary representation 0100 0001.
UTF-8 HEX Table
Take note of the third column in the UTF-8 table for the Cyrillic capital letter "A", which is represented in Hex as "d0 90". This is a two-byte Hex code for "A" in Cyrillic which converts to 1101 0000 1001 0000 in binary. Can you see the difference in the character encodings and why it's important to make this distinction in hashcat? ASCII A = 41 (0100 0001) | UTF-8 Cyrillic A = d0 90 (1101 0000 1001 0000). Looking closely at the Cyrillic A's hex you will notice it is split into two parts, d0 (base code) and 90 (character code). The base code spans from d0 - d4 (i.e. d0, d1, d2, d3, d4) and the possible character codes range from a0 - 9f (i.e. a0, a1, a2, a3 .....9c, 9d, 9e, 9f). So when you are formulating a brute-force password attack you need to take into account the full range in hex that Cyrillic is represented. Looking at THIS UTF-8 Hex Table you can see Cyrillic starts at d0 80 and ends at d4 af. Before you completely fall asleep let's move on to setting up your test environment and actual hashcat usage. Just remember this breakdown of Cyrillic can be applied to any other UTF-8 foreign character set.
Setup Testing Environment
Steps: 1) Installing foreign language input, 2) creating example passwords, 3) hash those passwords into MD5, 4) place hashes into text file, and 5) crack hashes with hashcat.
Let's setup a test environment so we can walk through a couple simple examples and demonstrate how to crack UTF-8 character encoding. First based on your OS, Windows, Mac, or flavor of Linux, install some of the most predominant foreign language keyboards input sources. Windows instructions <here>, Mac instructions <here>, or Kali Linux <here>.
Above we have added Russian, Arabic, and Simplified Chinese to our Mac keyboard preferences. This will enable us to easily switch between the different keyboard preferences based on our password hash attack.
In the above image we can see that we are able to easily switch between keyboard input languages from the top taskbar. This makes it much more convenient when frequently switching between examples. Windows and Linux should have a similar capability in their taskbars once multiple languages has been enabled.
Another handy application to have is HashMaker. HashMaker enables us to input foreign character passwords into a strings field and then quickly generate a number of hashing algorithm outputs, such as MD5 for our examples. Windows you can use HashTab and Linux you can use the built-in MD5sum from the command line.
Looking at the above image of HashMaker you can see we have input a simple key run of four Cyrillic characters 'фыва' and then we MD5 the string. We then take the MD5 hash and place it into a test hash file called "ru_charset.txt". We are starting with very simple one to four character Cyrillic passwords because any basic password cracking installation should be able to crack these hashes within 30 seconds to 30 minutes.
Above you can see we have created two text files. The first file contains two one-character passwords (lower and uppercase), two two-character passwords (lower and uppercase), two three-character passwords (lower and uppercase), and two four-character passwords (lower and uppercase). Pick any characters you would like just ensure they are simple for this first exercise. The next file you will notice that we have input the MD5 hashes into the "ru_charset.txt' file in the order they appear on the left. Save the hashfile "ru_charset.txt" somewhere you can remember for usage in the cracking phase. Next we will demonstrate the proper hashcat command line parameters for cracking these hashes with a bruteforce attack using custom character sets from the UTF-8 hex table.
Hashcat has extendability to include custom character sets, or charsets for short. Let's take advantage of this customization to pass our custom Cyrillic hex charset that we discussed at the beginning. Hashcat 3.0 has expanded parameters that allow up to four custom charsets at the command line, but we will only be creating two. Remember that UTF-8 Cyrillic in hex is essentially split into two parts, the base code and the character code, so we are going to represent all possible iterations of that range. Our "base code" custom charset for Cyrillic UTF-8 is d0d1d2d3d4 and our "character code" custom charset 808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf . This range of characters should cover all possible representative iterations of Cyrillic characters.
Now let's see this full hashcat command line to brute-force against our MD5 hashes of UTF-8 Cyrillic:
./hashcat --potfile-disable -m 0 -a 3 ../ru_charset.txt --hex-charset -1 d0d1d2d3d4 -2 808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf -i ?1?2?1?2?1?2?1?2
The above Hashcat parameters explained:
--pot-disable (for testing we do not want the cracked hashes to be added to hashcat's potfile)
-m 0 (hash-type settings, which is set to 0 for MD5 hashes)
-a 3 (attack mode setting to 3 for bruteforce)
../ru_charset.txt (path to our text file containing the test MD5 hashes)
--hex-charset (sets assume charset set is given in hex values)
-1 d0d1d2d3d4 (set our first charset base code)
-2 808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf (sets our character code range)
-i ?1?2?1?2?1?2?1?2 (sets hashcat to iterate from 1-4 character bruteforce attempts based on the mask of ?1?2?1?2?1?2?1?2)
Key thing to remember in the above settings is the mask of ?1?2?1?2?1?2?1?2 which is setup for a two-byte character and will only increment attack a four character password. What this is telling hashcat is to take values from my custom charset -1 and custom charset -2 and combine the values. For example, the first bruteforce attempt would likely be 'd080' (?1?2) based on current custom charsets. The next attempt would be "d081', and so on and so on, until finally hashcat has incremented through all possible variations, ending with the last two-byte four character UTF-8 hex encoding d4bcd4bdd4bed4bf (?1?2?1?2?1?2?1?2).
Above you can see hashcat starting the first character attack and successfully recovering the two one-character password hashes we placed into "ru_charset.txt". The "ATTENTION!" warning is telling us we have not given hashcat enough work to optimize its powerful parallelization capabilities but this is ok for now. If you want to learn more about how to give hashcat enough work then read <HERE>.
Above we can see hashcat has completed the brute-force attack recovering 8 out of 8 of our hashes, and taking only 1 minute 30 seconds to do so. One important thing to note is hashcat displaying the results in $HEX[d0a4d0abd092d090] format. You will need to the place "d0a4d0abd092d090' into an encoder/decoder to view the actual Cyrillic character results. Or you can enable in your command line options for hashcat "--outfile-autohex-disable" if you believe your terminal/environment will be able to handle the text encoding and display them properly. We prefer the $HEX[...] format because it can handle any variation of output and not mess up our representative results, i.e. less prone to errors or mistake on our part. Results using the --outfile-autohex-disable option are represented below:
Other Excellent Articles & Material
HASH CRACK: Password Cracking Manual v2.0 is an expanded reference guide for password recovery (cracking) methods, tools, and analysis techniques. A compilation of basic and advanced techniques to assist penetration testers and network security professionals evaluate their organization's posture. The Hash Crack manual contains syntax and examples for the most popular cracking and analysis tools and will save you hours of research looking up tool usage. It also includes basic cracking knowledge and methodologies every security professional should know when dealing with password attack capabilities. Hash Crack contains all the tables, commands, online resources, and more to complete your cracking security kit.
Below are other excellent articles on this topic of using Hashcat and highly recommend you give them a read.:
We just sent you an email. Please click the link in the email to confirm your subscription!