[FIXED] Why does Fiddler response has weird unicode characters after decoding?

Issue

So I was debugging game with Fiddler, and found that the HTTP request is partially decoded and partially not decoded. The non-decoded part consists of incomprehensible unicodes:

POST https://example.com/something HTTP/1.1
Host: example.com
User-Agent: UnityPlayer/2020.3.32f1 (UnityWebRequest/1.0, libcurl/7.80.0-DEV)
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/octet-stream
x_acts: Duel.matching
atoken: 23128e425359819ac4253c93e72cbc944095812015c1ff83843f9f62ff1e
X-Unity-Version: 2020.3.32f1
Content-Length: 311

`   |  [  գF #   =-  a  = } E& W J &! Ġ    _  iU x_      I   cc  "`Hp!   B    x @ h~{"info":[{"n":"2","m":168,"params":{"rule":{"mode":2,"type":5}}}],"vae":"344142"}

I had the same problem in both Fiddler Classic and Fiddler Everywhere. I gave CA, and selected Decrypt option. It would be understandable if whole request is either completely decoded or completely encoded, but it is both. Is there something I'm missing? Thanks!

Solution

The content type of the request is set to application/octet-stream which means the request body data is binary data of any format (proprietary or standardized like PNG, ZIP, ...).

Binary data can contain plain text parts like in your example some JSON data. If you don't know the data format used for sending you can not decode the data, which means the data that Fiddler display to you are the real data that has been send. Therefore you should look at the data in hex mode and check if you can identify the format. A common binary encoding format is Google Protobuf, a decoding format that is not supported by Fiddler. If you have access to the game client which sends this data you could reverse engineer the executable and see what libraries it loads. May be one of the libraries is belongs to an encoding format. But of course the data format can be totally proprietary, then you would have to reverse engineer the executable in a tool like Ghidra, IDAPro, ... to understand what data is sent and how it is encoded.

Answered By - Robert

Answer Checked By - Robin (PHPFixing Admin)