The nerd snipe ↩
If you've never heard the phrase "nerdsniped", then take 2 seconds to update your vocabulary by reading the following XKCD comic:
When I first started up the Steins;Gate game and the tutorial told me to open my phone and check my email, the first thing that stood out to me was that I already had an email. And... it looked kind of funny:
While you can't see the bottom of that email in one window, if you scroll the contents of it are the following:
/////////////////////// /////////////////////// ////////////7MU2Et9HuKf Pz4ifIT3CBSkeqxLsEnFVVj 26u7NieXQOcBzOxXl3bMbHH TzDSKhRtyJM8CXAqqwYJpsC AOBiWdcxZfQ9HKmHfREsSip KPwSdmUNHmtKGcFiudktOiO Y5KCFn9CORVY8sFcMF3RISl hoNvfrmjVgH41WhJDVOWuzv VdwlzOJjvLUI9vOqTzZIyd9 b1iDi3niJHDMnxMxLeLoW5F aWPgXjl2MS1vvoRpC54NE4F r7tmsgcZdAIq3KOXJsACNXn rUVGTWxfQZL3fp9QZm3oKyi PNN5TeRRzoJ9DBy2Hx3vRb4 Bxi4KAoi4Qu528ySaJrEppq 8n9ITveVE7gdQkgQuyPJDId 4o7b3SaAIfpFP2PNYFGPxrQ oULgxas7t7NKGEREXkMSCOn cf6QXaZc6mcw4E51PaJn47P h62pAnqB55sSXMiDXmQc4D2 BBUtJuS3h9iQbkLctfaAgh0 Yo7vSeFp3wEXA8EvDmI3tcd psHxA4yZSzM3CCScPLwhu3d OgHVLApyb9MIXaiI8FRwHxF ETLTUByT9MAvtNsygVDrLbp g2i1feALSCwK4teLw2CTBRf X5lwTHKzEhRtyB3xBWEAVrp 4BnEK6LmHuCkwV4KqyaG1hV dn5SRVkR53nUj61iLhFREcj aXTElE2WO5SdGGkaVlZMVzZ gu0yQVWisHvSvfsmegK8TYu 4O9XOzHSYdsQnhW6OQKcnZp uBuNrb4zx8IZUn8OUwWk0YI jNnucs1CJ04u4a9YdcCFKPO C3do5Ri9SHVzcYOUI00mmg9 GimMTxnUn4Lz3LU4Wzei0// /////////////////////// /////////////////////// /////// =?ISO-2022-JP?B?GyRCJDMkbCRDJEZDLyQrMnJGSSRHJC0kRiQ/JGokNyReJDkhKRsoQg==?=
Mysterious. Oddly enough, the game doesn't mention it at all, makes no note about it, no idle musing from the main character Okabe Rintaro, and no one ever brings it up either. He never seems to notice that it wasn't in our phone when we started the game, and then, when we shifted world lines after the initial setup for the plot, it suddenly appeared.
But. I was very curious, and today, after collecting 3 endings of the game so far, I decided to sit down and try to understand it. Here's my notes:
Nerd successfully sniped (me) ↩
Now, I've done plenty of web work in my career, and so seeing a few trailing == at
the end of some "mysterious" text immediately throw up my "Oh hey look, it's base64encoded" senses.
If I showed anyone this:
GyRCJDMkbCRDJEZDLyQrMnJGSSRHJC0kRiQ/JGokNyReJDkhKRsoQg==
They'd immediately toss it into a decoder. The problem of course, is that if you just blindly do this on something like base64decode.org then you'll get this trash:
$B$3$l$C$FC/$+2rFI$G$-$F$?$j$7$^$9!)(B
Because the defaults for the website are for utf-8. That said, if we look at the rest of the message
we can see that the "tags" =? ?= were wrapping more than just that. In fact,
there's pretty obvious ASCII text right there saying
ISO-2022-JP?B?
If we assume that the B stands for Base 64, then it stands to reason that the other encoding, ISO-2022-JP, is what this mysterious sequence of bytes is. And, if you were to swap the website's decode mode, you'll see the following text:
これって誰か解読できてたりします？ Kore tte dare ka kaidoku dekite tari shimasu? Can anyone decipher this?
The first line is the direct decode, while the next two are just a quick romanji and English translation.
That said. When I first was looking at this, I didn't actually notice the dropdown for character set on the website and thought I was out of luck and I'd have to figure out how to do the decoding myself. So, the majority of these last 5 hours was spent in Java world writing code like this:
final String hint = "GyRCJDMkbCRDJEZDLyQrMnJGSSRHJC0kRiQ/JGokNyReJDkhKRsoQg==";
System.out.println("HINT");
byte[] decoded = Base64.getDecoder().decode(hint);
for (byte rawByte : decoded) {
System.out.print(" ");
System.out.print(rawByte);
System.out.print(" ");
}
System.out.println(new String(decoded, Charset.forName("ISO-2022-JP")));
The last line, you'd expect to print out the Japanese characters, and yet, it doesn't. It prints out
????????????????? and no amount of fiddling with my terminal settings, changing fonts, or
anything else seemed to change that. So I was left looking at the raw bytes instead.
27 36 66 36 51 36 108 36 67 36 70 67 47 36 43 50 114 70 73 36 71 36 45 36 70 36 63 36 106 36 55 36 94 36 57 33 41 27 40 66
Now, given that I knew it was supposed to be ISO-2022-JP, I looked it up! RFC1468 defines the ISO-2022-JP standard. Which is pretty interesting. I haven't really gone deep into the world of character encodings before, but while I was reading this a few things immediately stood out that I'll highlight.
In the Steins;Gate game, they talk about the structure of the D-MAIL that's sent and the limitations thereof. They're only able to send 32 bytes of information to the past via D-MAIL, and it's delivered as 12 byte blocks. Or, as 3 emails with 12 bytes in it. They discuss how many words they can send and note that English text using ASCII can be sent a bit easier than Japanese because Japanese uses two bytes to encode while english only uses 1. So I was looking at things two bytes at a time.
27 36 if we look them up in the ASCII table are the ESC control code and a $.
Now check out this note from the RFC:
The text starts in ASCII, and switches to Japanese characters through an escape sequence. For example, the escape sequence ESC $ B (three bytes, hexadecimal values: 1B 24 42) indicates that the bytes following this escape sequence are Japanese characters, which are encoded in two bytes each. To switch back to ASCII, the escape sequence ESC ( B is used.
Emphasis mine. So, I've got the start of an escape sequence, but what's the third byte? 66 And in ASCII?
66 is a B. Perfect. So I know I've got an escape sequence. What kind? Looking at the RFC again, the format
being described actually allows more than one character set inside of it, determined by escape sequences:
Esc Seq Character Set ISOREG
ESC ( B ASCII 6
ESC ( J JIS X 0201-1976 ("Roman" set) 14
ESC $ @ JIS X 0208-1978 42
ESC $ B JIS X 0208-1983 87
A form of JIS (Japanese Industrial Standards) encoding it looks like! Further down in the RFC there's an important note for when you're reading in the different modes:
If there are JIS X 0208 characters on a line, there must be a switch to ASCII or to the "Roman" set of JIS X 0201 before the end of the line (i.e., before the CRLF). This means that the next line starts in the character set that was switched to before the end of the previous line.
So we'll always have a mix of character sets, and we should expect that the ending of a line should finish with the escape sequence to go to ASCII. Does our data follow that? The last three bytes of this single line message are:
27 40 66
Or in character form ESC ( B, which is the correct sequence! Great, so that's
a pretty strong reassurance that, yes, we do in fact have ISO-2022-JP data in
our hands. So each pair of bytes inbetween these escape sequences should be the encoding
of a Japanese Character!
So I focused my efforts onto the first pair: 36, 51 Unfortunately for me,
I wasn't really sure what to do with them or how to combine them right away. So, I
looked it up on Wikipedia
In order to represent code points, column/line numbers are used for one-byte codes and kuten numbers are used for two-byte codes. For a way to identify a character without depending on a code, character names are used.
Ok...
The double-byte codes are laid out in 94 numbered groups, each called a row (区, ku, lit. "section"). Every row contains 94 numbered codes, each called a cell (点, ten, lit. "point"). This makes a total of 8836 (94 x 94) possible code points (although not all are assigned, see below); these are laid out in the standard in a 94-line, 94-column code table
Well, it sounds like I probably want to find this table! If I just need to treat each byte as a pair for a row and column, then if I can figure that out we'll be great. After 30 minutes of googling for the table, repeatedly being taken to wikipedia pages with no actual links to the table itself and just descriptions and examples of each, I finally found myself on this page on unicode.org
And wouldn't you know it, I got pretty lucky because reading the comment section at the top supplied me with this information:
# Format: Four tab-separated columns # Column #1 is the shift-JIS code (in hex) # Column #2 is the JIS X 0208 code (in hex as 0xXXXX) # Column #3 is the Unicode (in hex as 0xXXXX) # Column #4 the Unicode name (follows a comment sign, '#') # The official names for Unicode characters U+4E00 # to U+9FA5, inclusive, is "CJK UNIFIED IDEOGRAPH-XXXX", # where XXXX is the code point. Including all these # names in this file increases its size substantially # and needlessly. The token "<CJK>" is used for the # name of these characters. If necessary, it can be # expanded algorithmically by a parser or editor. # # The entries are in JIS X 0208 order # # The following algorithms can be used to change the hex form # of JIS 0208 to other standard forms: # # To change hex to EUC form, add 0x8080 # To change hex to kuten form, first subtract 0x2020. Then # the high and low bytes correspond to the ku and ten of # the kuten form. For example, 0x2121 -> 0x0101 -> 0101; # 0x7426 -> 0x5406 -> 8406
I started off on the wrong foot here, getting confused about encoding or decoding.
I was thinking to myself, that I had a row and line potentially, so 36 and 51 needed to have their
higher and lower bits combined. 0x24 0x33 squished
together into 0x2433 and then have x2020 bitwise added
to get 0x4453 but... after beating my head against things for a while,
I realized that the data is already encoded isn't it?
Which means that the two bytes I have should already be in the order of the table. Which means that the first byte 0x24 is pointing to row 4 (0x24 - 0x20), and second byte 0x33 to columm 20 (0x33 - 0x20). The character こ. So all I truly have to do is the lookup in the unicode table site, no math required! To make my life easier, I cut the unicode file down a bit into just the information I cared about:
cut -d " " -f2,4- JIS0208.TXT
Then saved it to mappings.txt before running the following Java code:
final String hint = "GyRCJDMkbCRDJEZDLyQrMnJGSSRHJC0kRiQ/JGokNyReJDkhKRsoQg==";
byte[] decoded = Base64.getDecoder().decode(hint);
List<String> mappings = Files.readAllLines(Paths.get("mappings.txt"));
for (int i = 3; i < decoded.length - 4; i += 2) {
int row = i;
int col = i + 1;
for (String mapping : mappings) {
if (mapping.contains(String.format("0x%X%X", decoded[row], decoded[col]))) {
System.out.println(mapping);
}
}
}
And what did I get?
0x2433 # HIRAGANA LETTER KO 0x246C # HIRAGANA LETTER RE 0x2443 # HIRAGANA LETTER SMALL TU 0x2446 # HIRAGANA LETTER TE 0x432F # <CJK> 0x242B # HIRAGANA LETTER KA 0x3272 # <CJK> 0x4649 # <CJK> 0x2447 # HIRAGANA LETTER DE 0x242D # HIRAGANA LETTER KI 0x2446 # HIRAGANA LETTER TE 0x243F # HIRAGANA LETTER TA 0x246A # HIRAGANA LETTER RI 0x2437 # HIRAGANA LETTER SI 0x245E # HIRAGANA LETTER MA 0x2439 # HIRAGANA LETTER SU 0x2129 # FULLWIDTH QUESTION MARK
This is looking promising, but, what's that CJK thing?
Including all these names in this file increases its size substantially and needlessly. The token "<CJK>" is used for the name of these characters
Oh, I suppose that's... weird. I was hoping that unicode table would have everything in it without an extra hop, but it does look like we've only got 3, and we can look them up on wikipedia here. Except for the fact that these are in kuten form, so we need to convert the data:
0x432F -> 0x230F -> 0x23 0x0F -> row 35, col 15 -> 誰 0x3272 -> 0x1252 -> 0x12 0x52 -> row 18, col 82 -> 解 0x4649 -> 0x2629 -> 0x26 0x29 -> row 38, col 41 -> 読
And with that, we can take the list of mappings, the kanji, and write out the real sentence again in Japanese:
これって誰か解読できてたりします？
Which is just Rintaro asking us for help in decoding the other payload in the email.
The payload ↩
So, we've got a way to read the bytes out, and we did it based on the "tags" of =? ?= around the text. Looking at the rest of the email I can see we've got:
/////////////////////// /////////////////////// ////////////7MU2Et9HuKf Pz4ifIT3CBSkeqxLsEnFVVj ... C3do5Ri9SHVzcYOUI00mmg9 GimMTxnUn4Lz3LU4Wzei0// /////////////////////// /////////////////////// ///////
Which feels like perhaps the long string of //// could also be some form of
padding. But... If we count how many / on the front exist, versus how many are open
on the back, we get 58 and 55. Which doesn't really match up.
That said, this is an email sent to us from Rintaro, and he was paranoid enough to base64 encode the data once before it had to be converted to text for us to read via a character encoding lookup. So maybe it's also Base 64 encoded and then full of something else? So I gave it a try, assuming that the full message was one big string, and that there were no newlines:
decoded = Base64.getDecoder().decode(msg);
int i = 0;
for (byte rawByte : decoded) {
System.out.printf(" 0x%02X ", rawByte);
i++;
if (i > 23) {
System.out.println();
i = 0;
}
}
And surprisingly enough, it did actually decode! Printing this off with 23 characters per line gets us quite the... thing to look at. I mean, there's 927 characters here, an odd number.
0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFE 0xCC 0x53 0x61 0x2D 0xF4 0x7B 0x8A 0x7C 0xFC 0xF8 0x89 0xF2 0x13 0xDC 0x20 0x52 0x91 0xEA 0xB1 0x2E 0xC1 0x27 0x15 0x55 0x63 0xDB 0xAB 0xBB 0x36 0x27 0x97 0x40 0xE7 0x01 0xCC 0xEC 0x57 0x97 0x76 0xCC 0x6C 0x71 0xD3 0xCC 0x34 0x8A 0x85 0x1B 0x72 0x24 0xCF 0x02 0x5C 0x0A 0xAA 0xC1 0x82 0x69 0xB0 0x20 0x0E 0x06 0x25 0x9D 0x73 0x16 0x5F 0x43 0xD1 0xCA 0x98 0x77 0xD1 0x12 0xC4 0xA2 0xA4 0xA3 0xF0 0x49 0xD9 0x94 0x34 0x79 0xAD 0x28 0x67 0x05 0x8A 0xE7 0x64 0xB4 0xE8 0x8E 0x63 0x92 0x82 0x16 0x7F 0x42 0x39 0x15 0x58 0xF2 0xC1 0x5C 0x30 0x5D 0xD1 0x21 0x29 0x61 0xA0 0xDB 0xDF 0xAE 0x68 0xD5 0x80 0x7E 0x35 0x5A 0x12 0x43 0x54 0xE5 0xAE 0xCE 0xF5 0x5D 0xC2 0x5C 0xCE 0x26 0x3B 0xCB 0x50 0x8F 0x6F 0x3A 0xA4 0xF3 0x64 0x8C 0x9D 0xF5 0xBD 0x62 0x0E 0x2D 0xE7 0x88 0x91 0xC3 0x32 0x7C 0x4C 0xC4 0xB7 0x8B 0xA1 0x6E 0x45 0x69 0x63 0xE0 0x5E 0x39 0x76 0x31 0x2D 0x6F 0xBE 0x84 0x69 0x0B 0x9E 0x0D 0x13 0x81 0x6B 0xEE 0xD9 0xAC 0x81 0xC6 0x5D 0x00 0x8A 0xB7 0x28 0xE5 0xC9 0xB0 0x00 0x8D 0x5E 0x7A 0xD4 0x54 0x64 0xD6 0xC5 0xF4 0x19 0x2F 0x77 0xE9 0xF5 0x06 0x66 0xDE 0x82 0xB2 0x88 0xF3 0x4D 0xE5 0x37 0x91 0x47 0x3A 0x09 0xF4 0x30 0x72 0xD8 0x7C 0x77 0xBD 0x16 0xF8 0x07 0x18 0xB8 0x28 0x0A 0x22 0xE1 0x0B 0xB9 0xDB 0xCC 0x92 0x68 0x9A 0xC4 0xA6 0x9A 0xBC 0x9F 0xD2 0x13 0xBD 0xE5 0x44 0xEE 0x07 0x50 0x92 0x04 0x2E 0xC8 0xF2 0x43 0x21 0xDE 0x28 0xED 0xBD 0xD2 0x68 0x02 0x1F 0xA4 0x53 0xF6 0x3C 0xD6 0x05 0x18 0xFC 0x6B 0x42 0x85 0x0B 0x83 0x16 0xAC 0xEE 0xDE 0xCD 0x28 0x61 0x11 0x11 0x79 0x0C 0x48 0x23 0xA7 0x71 0xFE 0x90 0x5D 0xA6 0x5C 0xEA 0x67 0x30 0xE0 0x4E 0x75 0x3D 0xA2 0x67 0xE3 0xB3 0xE1 0xEB 0x6A 0x40 0x9E 0xA0 0x79 0xE6 0xC4 0x97 0x32 0x20 0xD7 0x99 0x07 0x38 0x0F 0x60 0x41 0x52 0xD2 0x6E 0x4B 0x78 0x7D 0x89 0x06 0xE4 0x2D 0xCB 0x5F 0x68 0x08 0x21 0xD1 0x8A 0x3B 0xBD 0x27 0x85 0xA7 0x7C 0x04 0x5C 0x0F 0x04 0xBC 0x39 0x88 0xDE 0xD7 0x1D 0xA6 0xC1 0xF1 0x03 0x8C 0x99 0x4B 0x33 0x37 0x08 0x24 0x9C 0x3C 0xBC 0x21 0xBB 0x77 0x4E 0x80 0x75 0x4B 0x02 0x9C 0x9B 0xF4 0xC2 0x17 0x6A 0x22 0x3C 0x15 0x1C 0x07 0xC4 0x51 0x13 0x2D 0x35 0x01 0xC9 0x3F 0x4C 0x02 0xFB 0x4D 0xB3 0x28 0x15 0x0E 0xB2 0xDB 0xA6 0x0D 0xA2 0xD5 0xF7 0x80 0x2D 0x20 0xB0 0x2B 0x8B 0x5E 0x2F 0x0D 0x82 0x4C 0x14 0x5F 0x5F 0x99 0x70 0x4C 0x72 0xB3 0x12 0x14 0x6D 0xC8 0x1D 0xF1 0x05 0x61 0x00 0x56 0xBA 0x78 0x06 0x71 0x0A 0xE8 0xB9 0x87 0xB8 0x29 0x30 0x57 0x82 0xAA 0xC9 0xA1 0xB5 0x85 0x57 0x67 0xE5 0x24 0x55 0x91 0x1E 0x77 0x9D 0x48 0xFA 0xD6 0x22 0xE1 0x15 0x11 0x1C 0x8D 0xA5 0xD3 0x12 0x51 0x36 0x58 0xEE 0x52 0x74 0x61 0xA4 0x69 0x59 0x59 0x31 0x5C 0xD9 0x82 0xED 0x32 0x41 0x55 0xA2 0xB0 0x7B 0xD2 0xBD 0xFB 0x26 0x7A 0x02 0xBC 0x4D 0x8B 0xB8 0x3B 0xD5 0xCE 0xCC 0x74 0x98 0x76 0xC4 0x27 0x85 0x6E 0x8E 0x40 0xA7 0x27 0x66 0x9B 0x81 0xB8 0xDA 0xDB 0xE3 0x3C 0x7C 0x21 0x95 0x27 0xF0 0xE5 0x30 0x5A 0x4D 0x18 0x22 0x33 0x67 0xB9 0xCB 0x35 0x08 0x9D 0x38 0xBB 0x86 0xBD 0x61 0xD7 0x02 0x14 0xA3 0xCE 0x0B 0x77 0x68 0xE5 0x18 0xBD 0x48 0x75 0x73 0x71 0x83 0x94 0x23 0x4D 0x26 0x9A 0x0F 0x46 0x8A 0x63 0x13 0xC6 0x75 0x27 0xE0 0xBC 0xF7 0x2D 0x4E 0x16 0xCD 0xE8 0xB4 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF
Which feels a bit odd since everything so far was even. Even though control codes resulted in an odd number of bytes, their symmetry made it add
up to even again afterwards. Out of curiousity, and returning to my previous hunch about the / being padding, I removed it and checked
to see if the data would decode:
0xEC 0xC5 0x36 0x12 0xDF 0x47 0xB8 0xA7 0xCF 0xCF 0x88 0x9F 0x21 0x3D 0xC2 0x05 0x29 0x1E 0xAB 0x12 0xEC 0x12 0x71 0x55 0x56 0x3D 0xBA 0xBB 0xB3 0x62 0x79 0x74 0x0E 0x70 0x1C 0xCE 0xC5 0x79 0x77 0x6C 0xC6 0xC7 0x1D 0x3C 0xC3 0x48 0xA8 0x51 0xB7 0x22 0x4C 0xF0 0x25 0xC0 0xAA 0xAC 0x18 0x26 0x9B 0x02 0x00 0xE0 0x62 0x59 0xD7 0x31 0x65 0xF4 0x3D 0x1C 0xA9 0x87 0x7D 0x11 0x2C 0x4A 0x2A 0x4A 0x3F 0x04 0x9D 0x99 0x43 0x47 0x9A 0xD2 0x86 0x70 0x58 0xAE 0x76 0x4B 0x4E 0x88 0xE6 0x39 0x28 0x21 0x67 0xF4 0x23 0x91 0x55 0x8F 0x2C 0x15 0xC3 0x05 0xDD 0x12 0x12 0x96 0x1A 0x0D 0xBD 0xFA 0xE6 0x8D 0x58 0x07 0xE3 0x55 0xA1 0x24 0x35 0x4E 0x5A 0xEC 0xEF 0x55 0xDC 0x25 0xCC 0xE2 0x63 0xBC 0xB5 0x08 0xF6 0xF3 0xAA 0x4F 0x36 0x48 0xC9 0xDF 0x5B 0xD6 0x20 0xE2 0xDE 0x78 0x89 0x1C 0x33 0x27 0xC4 0xCC 0x4B 0x78 0xBA 0x16 0xE4 0x56 0x96 0x3E 0x05 0xE3 0x97 0x63 0x12 0xD6 0xFB 0xE8 0x46 0x90 0xB9 0xE0 0xD1 0x38 0x16 0xBE 0xED 0x9A 0xC8 0x1C 0x65 0xD0 0x08 0xAB 0x72 0x8E 0x5C 0x9B 0x00 0x08 0xD5 0xE7 0xAD 0x45 0x46 0x4D 0x6C 0x5F 0x41 0x92 0xF7 0x7E 0x9F 0x50 0x66 0x6D 0xE8 0x2B 0x28 0x8F 0x34 0xDE 0x53 0x79 0x14 0x73 0xA0 0x9F 0x43 0x07 0x2D 0x87 0xC7 0x7B 0xD1 0x6F 0x80 0x71 0x8B 0x82 0x80 0xA2 0x2E 0x10 0xBB 0x9D 0xBC 0xC9 0x26 0x89 0xAC 0x4A 0x69 0xAB 0xC9 0xFD 0x21 0x3B 0xDE 0x54 0x4E 0xE0 0x75 0x09 0x20 0x42 0xEC 0x8F 0x24 0x32 0x1D 0xE2 0x8E 0xDB 0xDD 0x26 0x80 0x21 0xFA 0x45 0x3F 0x63 0xCD 0x60 0x51 0x8F 0xC6 0xB4 0x28 0x50 0xB8 0x31 0x6A 0xCE 0xED 0xEC 0xD2 0x86 0x11 0x11 0x17 0x90 0xC4 0x82 0x3A 0x77 0x1F 0xE9 0x05 0xDA 0x65 0xCE 0xA6 0x73 0x0E 0x04 0xE7 0x53 0xDA 0x26 0x7E 0x3B 0x3E 0x1E 0xB6 0xA4 0x09 0xEA 0x07 0x9E 0x6C 0x49 0x73 0x22 0x0D 0x79 0x90 0x73 0x80 0xF6 0x04 0x15 0x2D 0x26 0xE4 0xB7 0x87 0xD8 0x90 0x6E 0x42 0xDC 0xB5 0xF6 0x80 0x82 0x1D 0x18 0xA3 0xBB 0xD2 0x78 0x5A 0x77 0xC0 0x45 0xC0 0xF0 0x4B 0xC3 0x98 0x8D 0xED 0x71 0xDA 0x6C 0x1F 0x10 0x38 0xC9 0x94 0xB3 0x33 0x70 0x82 0x49 0xC3 0xCB 0xC2 0x1B 0xB7 0x74 0xE8 0x07 0x54 0xB0 0x29 0xC9 0xBF 0x4C 0x21 0x76 0xA2 0x23 0xC1 0x51 0xC0 0x7C 0x45 0x11 0x32 0xD3 0x50 0x1C 0x93 0xF4 0xC0 0x2F 0xB4 0xDB 0x32 0x81 0x50 0xEB 0x2D 0xBA 0x60 0xDA 0x2D 0x5F 0x78 0x02 0xD2 0x0B 0x02 0xB8 0xB5 0xE2 0xF0 0xD8 0x24 0xC1 0x45 0xF5 0xF9 0x97 0x04 0xC7 0x2B 0x31 0x21 0x46 0xDC 0x81 0xDF 0x10 0x56 0x10 0x05 0x6B 0xA7 0x80 0x67 0x10 0xAE 0x8B 0x98 0x7B 0x82 0x93 0x05 0x78 0x2A 0xAC 0x9A 0x1B 0x58 0x55 0x76 0x7E 0x52 0x45 0x59 0x11 0xE7 0x79 0xD4 0x8F 0xAD 0x62 0x2E 0x11 0x51 0x11 0xC8 0xDA 0x5D 0x31 0x25 0x13 0x65 0x8E 0xE5 0x27 0x46 0x1A 0x46 0x95 0x95 0x93 0x15 0xCD 0x98 0x2E 0xD3 0x24 0x15 0x5A 0x2B 0x07 0xBD 0x2B 0xDF 0xB2 0x67 0xA0 0x2B 0xC4 0xD8 0xBB 0x83 0xBD 0x5C 0xEC 0xC7 0x49 0x87 0x6C 0x42 0x78 0x56 0xE8 0xE4 0x0A 0x72 0x76 0x69 0xB8 0x1B 0x8D 0xAD 0xBE 0x33 0xC7 0xC2 0x19 0x52 0x7F 0x0E 0x53 0x05 0xA4 0xD1 0x82 0x23 0x36 0x7B 0x9C 0xB3 0x50 0x89 0xD3 0x8B 0xB8 0x6B 0xD6 0x1D 0x70 0x21 0x4A 0x3C 0xE0 0xB7 0x76 0x8E 0x51 0x8B 0xD4 0x87 0x57 0x37 0x18 0x39 0x42 0x34 0xD2 0x69 0xA0 0xF4 0x68 0xA6 0x31 0x3C 0x67 0x52 0x7E 0x0B 0xCF 0x72 0xD4 0xE1 0x6C 0xDE 0x8B
To my surprise, it did. And more importantly, we had an even number of pairs again. That said, trying to put this data through a direct conversion tool to ISO-2022-JP returned some glitchy nonsense.
ﾅ6ﾟGｸｧﾏﾏ!=ﾂ) ｫqUV=ｺｻｳbytp ﾎﾅywlﾆﾇ ｼﾃﾈｨﾑｷ｢ﾌ･ﾀｪｬｦ�bﾙﾗｱeｽ ｩ}ｬﾊｪﾊｿﾃﾇﾒpﾘｮvﾋﾎｹｨ｡g｣ﾕｬﾃﾝ
ｽﾘﾕ｡､ｵﾎﾚﾕﾜ･ﾌcｼｵｪﾏｶﾈﾉﾟﾛﾖ ﾞx ｳｧﾄﾌﾋxｺﾖｾcﾖﾆｹﾑｸｾﾈ eﾐｫrﾜ�ﾕｭﾅﾆﾍlﾟﾁ~ﾐfmｫｨｴﾞﾓysﾃｭﾇ{ﾑoq｢ｮｻｼﾉｦｬﾊiｫﾉ｡ｻﾞﾔﾎu ﾂ､ｲ ﾛﾝｦ｡ﾅｿcﾍ`ﾑﾆｴｨﾐｸｱjﾎﾒﾄｺw ﾚeﾎｦsﾓﾚｦ~ｻｾ ｶ､ lﾉs｢
ysｭｦｷﾘnﾂﾜｵ ｣ｻﾒxﾚwﾀﾅﾀﾋﾃqﾚl ｸﾉｳｳpﾉﾃﾋﾂｷtﾔｰｩﾉｿﾌ｡v｢｣ﾁﾑﾀ|ﾅｲﾓﾐ ﾀｯｴﾛｲﾐｭｺ`ﾚｭﾟxﾒ ｸｵﾘ､ﾁﾅﾇｫｱ｡ﾆﾜﾟﾖkｧgｮ{xｪｬﾘﾕv~ﾒﾅﾙyﾔｭbｮﾑﾈﾚﾝｱ･eｧﾆﾆﾍｮﾓ､ﾚｫｽｫﾟｲgｫﾄﾘｻｽﾜﾇﾉlﾂxﾖ
rviｸｭｾｳﾇﾂﾒﾓ､ﾑ｣ｶ{ｳﾐﾓｸkﾖ p｡ﾊｼｷvﾑﾔﾗｷｹﾂｴﾒihｦｱｼgﾒ~ ﾏrﾔlﾞ
Sure, there was a smattering of japanese characters mixed in, but nothing that looked like real Japanese at all. Out of curiousity, I dumped it out not as hex, but as decimal to see if I could observe any patterns in it:
No padding! Payload string decode as Base64, 12 per line
-20 -59 54 18 -33 71 -72 -89 -49 -49 -120 -97 33
61 -62 5 41 30 -85 18 -20 18 113 85 86 61
-70 -69 -77 98 121 116 14 112 28 -50 -59 121 119
108 -58 -57 29 60 -61 72 -88 81 -73 34 76 -16
37 -64 -86 -84 24 38 -101 2 0 -32 98 89 -41
49 101 -12 61 28 -87 -121 125 17 44 74 42 74
63 4 -99 -103 67 71 -102 -46 -122 112 88 -82 118
75 78 -120 -26 57 40 33 103 -12 35 -111 85 -113
44 21 -61 5 -35 18 18 -106 26 13 -67 -6 -26
-115 88 7 -29 85 -95 36 53 78 90 -20 -17 85
-36 37 -52 -30 99 -68 -75 8 -10 -13 -86 79 54
72 -55 -33 91 -42 32 -30 -34 120 -119 28 51 39
-60 -52 75 120 -70 22 -28 86 -106 62 5 -29 -105
99 18 -42 -5 -24 70 -112 -71 -32 -47 56 22 -66
-19 -102 -56 28 101 -48 8 -85 114 -114 92 -101 0
8 -43 -25 -83 69 70 77 108 95 65 -110 -9 126
-97 80 102 109 -24 43 40 -113 52 -34 83 121 20
115 -96 -97 67 7 45 -121 -57 123 -47 111 -128 113
-117 -126 -128 -94 46 16 -69 -99 -68 -55 38 -119 -84
74 105 -85 -55 -3 33 59 -34 84 78 -32 117 9
32 66 -20 -113 36 50 29 -30 -114 -37 -35 38 -128
33 -6 69 63 99 -51 96 81 -113 -58 -76 40 80
-72 49 106 -50 -19 -20 -46 -122 17 17 23 -112 -60
-126 58 119 31 -23 5 -38 101 -50 -90 115 14 4
-25 83 -38 38 126 59 62 30 -74 -92 9 -22 7
-98 108 73 115 34 13 121 -112 115 -128 -10 4 21
45 38 -28 -73 -121 -40 -112 110 66 -36 -75 -10 -128
-126 29 24 -93 -69 -46 120 90 119 -64 69 -64 -16
75 -61 -104 -115 -19 113 -38 108 31 16 56 -55 -108
-77 51 112 -126 73 -61 -53 -62 27 -73 116 -24 7
84 -80 41 -55 -65 76 33 118 -94 35 -63 81 -64
124 69 17 50 -45 80 28 -109 -12 -64 47 -76 -37
50 -127 80 -21 45 -70 96 -38 45 95 120 2 -46
11 2 -72 -75 -30 -16 -40 36 -63 69 -11 -7 -105
4 -57 43 49 33 70 -36 -127 -33 16 86 16 5
107 -89 -128 103 16 -82 -117 -104 123 -126 -109 5 120
42 -84 -102 27 88 85 118 126 82 69 89 17 -25
121 -44 -113 -83 98 46 17 81 17 -56 -38 93 49
37 19 101 -114 -27 39 70 26 70 -107 -107 -109 21
-51 -104 46 -45 36 21 90 43 7 -67 43 -33 -78
103 -96 43 -60 -40 -69 -125 -67 92 -20 -57 73 -121
108 66 120 86 -24 -28 10 114 118 105 -72 27 -115
-83 -66 51 -57 -62 25 82 127 14 83 5 -92 -47
-126 35 54 123 -100 -77 80 -119 -45 -117 -72 107 -42
29 112 33 74 60 -32 -73 118 -114 81 -117 -44 -121
87 55 24 57 66 52 -46 105 -96 -12 104 -90 49
60 103 82 126 11 -49 114 -44 -31 108 -34 -117
Payload string decode as Base64, 12 per line
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -2 -52 83 97 45 -12 123 -118 124
-4 -8 -119 -14 19 -36 32 82 -111 -22 -79 46 -63
39 21 85 99 -37 -85 -69 54 39 -105 64 -25 1
-52 -20 87 -105 118 -52 108 113 -45 -52 52 -118 -123
27 114 36 -49 2 92 10 -86 -63 -126 105 -80 32
14 6 37 -99 115 22 95 67 -47 -54 -104 119 -47
18 -60 -94 -92 -93 -16 73 -39 -108 52 121 -83 40
103 5 -118 -25 100 -76 -24 -114 99 -110 -126 22 127
66 57 21 88 -14 -63 92 48 93 -47 33 41 97
-96 -37 -33 -82 104 -43 -128 126 53 90 18 67 84
-27 -82 -50 -11 93 -62 92 -50 38 59 -53 80 -113
111 58 -92 -13 100 -116 -99 -11 -67 98 14 45 -25
-120 -111 -61 50 124 76 -60 -73 -117 -95 110 69 105
99 -32 94 57 118 49 45 111 -66 -124 105 11 -98
13 19 -127 107 -18 -39 -84 -127 -58 93 0 -118 -73
40 -27 -55 -80 0 -115 94 122 -44 84 100 -42 -59
-12 25 47 119 -23 -11 6 102 -34 -126 -78 -120 -13
77 -27 55 -111 71 58 9 -12 48 114 -40 124 119
-67 22 -8 7 24 -72 40 10 34 -31 11 -71 -37
-52 -110 104 -102 -60 -90 -102 -68 -97 -46 19 -67 -27
68 -18 7 80 -110 4 46 -56 -14 67 33 -34 40
-19 -67 -46 104 2 31 -92 83 -10 60 -42 5 24
-4 107 66 -123 11 -125 22 -84 -18 -34 -51 40 97
17 17 121 12 72 35 -89 113 -2 -112 93 -90 92
-22 103 48 -32 78 117 61 -94 103 -29 -77 -31 -21
106 64 -98 -96 121 -26 -60 -105 50 32 -41 -103 7
56 15 96 65 82 -46 110 75 120 125 -119 6 -28
45 -53 95 104 8 33 -47 -118 59 -67 39 -123 -89
124 4 92 15 4 -68 57 -120 -34 -41 29 -90 -63
-15 3 -116 -103 75 51 55 8 36 -100 60 -68 33
-69 119 78 -128 117 75 2 -100 -101 -12 -62 23 106
34 60 21 28 7 -60 81 19 45 53 1 -55 63
76 2 -5 77 -77 40 21 14 -78 -37 -90 13 -94
-43 -9 -128 45 32 -80 43 -117 94 47 13 -126 76
20 95 95 -103 112 76 114 -77 18 20 109 -56 29
-15 5 97 0 86 -70 120 6 113 10 -24 -71 -121
-72 41 48 87 -126 -86 -55 -95 -75 -123 87 103 -27
36 85 -111 30 119 -99 72 -6 -42 34 -31 21 17
28 -115 -91 -45 18 81 54 88 -18 82 116 97 -92
105 89 89 49 92 -39 -126 -19 50 65 85 -94 -80
123 -46 -67 -5 38 122 2 -68 77 -117 -72 59 -43
-50 -52 116 -104 118 -60 39 -123 110 -114 64 -89 39
102 -101 -127 -72 -38 -37 -29 60 124 33 -107 39 -16
-27 48 90 77 24 34 51 103 -71 -53 53 8 -99
56 -69 -122 -67 97 -41 2 20 -93 -50 11 119 104
-27 24 -67 72 117 115 113 -125 -108 35 77 38 -102
15 70 -118 99 19 -58 117 39 -32 -68 -9 45 78
22 -51 -24 -76 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
Negatives? That's interesting. Two thoughts came to mind:
- Could they be glitches or corruption that's intentional?
- What happens if I remove anything out of the range and try to decode it to JP again?
- Maybe the byte data isn't signed and I should interpret them as unsigned numbers?
- Maybe it's not actually base64 decoded?
Idea one I couldn't really do anything with, but idea one was something I could probably verify without actually coding anything. If the negative data is just garbage to be tossed out, then I should be able to spot some escape character sequences if this is supposed to be some form of iso-2022-jp. The ESC character is 27 in decimal, and while there are a few in the above dumps, none of them are followed by a $ or a (, aka 36 or 40.
So that can't be it. I gave the corruption idea a try with this code:
// using the msg with padding removed
byte rowByte;
byte colByte;
for (i = 0; i < decoded.length - 1; i++) {
if (decoded[i] < 128 && decoded[i] >= 0) {
rowByte = decoded[i];
for (int j = i + 1; j < decoded.length; j++ ) {
if (decoded[j] < 128 && decoded[j] >= 0) {
colByte = decoded[j];
System.out.println(String.format("0x%X%X", rowByte, colByte));
for (String mapping : mappings) {
if (mapping.contains(String.format("0x%X%X", rowByte, colByte))) {
System.out.println(mapping);
}
}
break;
}
}
}
}
And while this did actually find a few mappings, it, much like when I put the data through the base 64 decoder online directly into iso-2022-jp, didn't really make anything new or obviously interesting.
So, this wasn't really getting me anywhere. I was chatting with someone in L:Q's discord server (a source of quite a bit of fun nerdiness) and accidently nerdsniped them as well. They were doing a number of analysis on it and concluding that it probably wasn't base 64 encoded given the way the padding was working out. So, we turned towards more analytical thinking. Using CyberChef I started experimenting.
It's a pretty awesome site, with a lot of information and Shannon scale suggested that it wasn't likely to be compressed or english text. Though when I base 64 decoded it:
The entropy rating jumped quite a bit! This matched with the nerd-sniped discord member's conclusions that the text had no grammatical structure. Though I did find it interesting that when I just dumped things out to iso-2022-jp and fed it to DeepL or Google Translate that sometimes I did get vaguely steinsgate sounding things:
The dirty work and the bamboo shoots which are controlled by bamboo presses are a major source of revenue for the Organization.
I got that when I saved the base64 decoded iso-2022-jp data to a .txt file and tried to translate it as it. The fact that there's a mention of "the Organization" does sort of feel like maybe my idea about discarding certain bad bits was on the right track, but it could just as easily be the translation software hallucinating and making something up considering that each time I fed some decoded data down into it, it felt like it changed the "meaning" of it.
Assuming that the data isn't text, I wondered if maybe there was some file magic that indicated a file type it should be. For example, I could take the data and write it out to a .wav file, .jpg, or similar and see what happens. Or, maybe this was a save file for the game itself! If I could crack open my own save file and understand the structure of it, perhaps there'd be something in that that would provide some sort of clue.
Looking at the save file it's a .DAT, which doesn't really say much. And given that it's my save file and I haven't finished the game yet, it's probably unlikely to contain... well, I suppose if I'm thinking that maybe the data is somehow related to a flag or something in the game, that I'd be able to find similar pieces in it somewhere. The only problem is that DAT files are just sort of generic binary files.
I have no reason to think this, but I think that's a dead end. But one thing I do know about Steins;Gate is that there's a pretty heavy focus on the IBM 5100 computer as a focal point. And, specifically, the IBM 5100 is effectively used as a way to encrypt data by the ever-present "Organization" that Rintaro faces.
I'm not sure if I think that this thing is a program for it, but I did find an emulator for the 5100 that could potentially be used to try to fiddle around and find out. I also found someone, (also a fan of the serieS) who acquired a real 5100. I'm not sure if they're trying to pursue the same idea as I am here, but they've probably got a better shot than I do since they've got the reference manual and other things on hand to dig for clues with.
Anyway, back to our payload problem. One interesting thing that wikipedia notes is this:
At the same time IBM announced the IBM 5100, it also announced the IBM 5100 Communications Adapter, which allowed the 5100 to transmit data to and receive data from a remote system. It made the 5100 appear the same as an IBM 2741 Communications Terminal and was designed to be able to communicate with IBM 2741 compatible machines in start-stop mode using the EBCD (Extended Binary Coded Decimal) notation,[16] referred to as PTTC/EBCD in IBM 2741 documentation
If this payload was sent from an IBM 5100 then perhaps we should try to decode the data using EBCD? If I look around for PTTC/EBCD, I find myself looking at this page And, an interesting, very suspicious thing on this page is:
All-call addressing (transmitting to all stations on the line simultaneously) is accomplished by the transmission of a special two-character code (/)
This... this seems kind of interesting because if we look at our data, we've got a bunch of slashes at the start and stop of the payload. This implies that this message could have been potentially sent out to all IBM stations in "All-call addressing" mode as the reference docs describe.
So... if we know that the message was sent out by an IBM machine, and the game itself keeps talking about "the hidden feature" of the computer, then perhaps we should look into Z-code? This is something that's mentioned by a user named voidstar in answering a question about the IBM 5100
There's a hex dump here... I wonder if the data that we've dumped out so far matches anything and pops anything
useful out? Well, in the picture I can see 0xEC in Z code is 0xEB in EBCD. So, what's 0xEB? Assuming that it lines
up with the international format, then its an empty
character of . But if I look at the PTTC/EBCD
table I... am very lost.
These [PTTC/EBCD coding] refer to the positioning of the characters around the typeball and, therefore, the tilt/rotate codes that have to be applied to the mechanism to produce a given character
So... I don't really see how we could do anything with this information. I don't have a physical machine to do anything with a type ball, and I'm not sure I understand how I'd translate a value from potential Z code into that in the first place. Assumedly if I was on the right track, then I suppose we'd decode the non / characters as an instruction on what to have the machine do or type and then maybe it would produce something of value? But also, maybe it would just produce incomprehensible gibberish, much like all our attempts to convert the payload from Base 64 to other encodings.
Future work
I hate to give up. But I'm definitely grasping as straws right now and unsure about where to go next. Also, I haven't finished the game yet. I've got 3 endings (I need to get what I assume to be a Kurisu ending, and a true ending and... maybe like, a bad end or something? I don't know), but I imagine that I might find more clues if I keep playing.
I assume that this email will do something at some point in the game, so maybe the behavior it exhibits at that point will help inform me if it's even possible to figure it out, or if the MAGES developers are just messing with me and every other potential person trying to crack the code.
That said, it was a lot of fun to explore and learning about character encodings, write code to examine and fiddle with binary a bit, and to get excited quite a few times on seeing something interesting happening with the data, even if I was ultimately thwarted for the time being. If you start examing or exploring this, I'd love to know, especially if you start figuring something out or find out that there is meaning within this high-entropy payload in Rintaro's phone! Swing by the stream if you do!