⚠️ Some information, tools, or techniques discussed may have changed or evolved since the publishing of this article.

Originally published at https://antibot.blog/posts/1741549345619

Intro

The main reason for reverse engineering an anti-bot, which we did in part 1, is to create a gen. A gen is used to mass create valid cookies, in the case of grubhub, it's used as a header. To create a scale-safe gen for any anti-bot you'll typically need to do the following:

Typically a cookie gen will have an HTTP API that takes in params, most public APIs will try to make you input as little data as possible to make it more convenient to use for their large customer base and large number of supported sites/APKs. In this case, this is only for GrubHub and it's only for our own personal use. Therefore, the input in our API will be very different.

NOTE: I know I said this would be a 2 part series, however, that has changed. This will now be a 3-part series where part 3 will be making the scraper for device data. I made this decision because I didn't really want part 2 to be about reverse engineering, instead about applying the knowledge you gained in part 1. However, I will be linking the GitHub repo for those of you who don't really care about web reverse engineering/scraping.

Porting code

When I'm porting code, I don't tend to do it in any particular order. I'll be starting by porting the APPC code, this is going to give us the outputs for "PX257", "PX256", and "PX259". To begin, I'll create a struct used for output:

1type APPC struct {
2	PX257 int32
3	PX259 int64
4	PX256 string
5}

PX257 is of type int32 because when we look at the code behind the PX257 math function, we will notice it uses a type int. This is an important thing to remember when porting code between languages, depending on the language, it may handle certain stuff with numbers much differently. In this case, int in java reserves 32 bits, aka int32. In GoLang int reserves 64 bits, aka int64. Differences like this may seem small, however, when working with large numbers that go over the 32 bit limit, it's important to make sure you become familiar with the differences between languages.

As a reference, here is the code we're porting (without the math function):

1String string1 = cVar2.f82492a.f10774k;
2int a13 = aVar2.a(aVar2.a(aVar2.appcNumber3, aVar2.appcNumber4, aVar2.appcNumber1, aVar2.appcNumber6), aVar2.appcNumber5, aVar2.appcNumber2, aVar2.appcNumber6);
3byte[] bArr = new byte[4];
4try {
5    byte[] bytes = string1.getBytes(Charsets.UTF_8);
6    bArr = bytes;
7} catch (UnsupportedEncodingException unused) {
8}
9int i14 = (bArr.length >= 4 ? 0 : ByteBuffer.wrap(bArr).getInt()) ^ a13;
10Integer boxInt = Boxing.boxInt(i14);

Porting the math function from Java to Golang is basically just copy-pasting now that we know that, so this is what I have:

1func pow(i12, i13, i14, i15 int32) int32 {
2	i16 := i15 % 10
3	i17 := int32(0)
4	if i16 != 0 {
5		i17 = i14 % i16
6	} else {
7		i17 = i14 % 10
8	}
9	i18 := i12 * i12
10	i19 := i13 * i13
11	switch i17 {
12	case 0:
13		return i13 + i18
14	case 1:
15		break
16	case 2:
17		return i13 * i18
18	case 3:
19		return i13 ^ i12
20	case 4:
21		return i12 - i19
22	case 5:
23		i22 := i12 + 783
24		i12 = i22 * i22
25		break
26	case 6:
27		return i13 + (i12 ^ i13)
28	case 7:
29		return i18 - i19
30	case 8:
31		return i13 * i12
32	case 9:
33		return (i13 * i12) - i12
34	default:
35		return -1
36	}
37	return i12 + i19
38}

Now that we have that, we just need to do the final bitwise operation. If you remember from part 1, I said this could simplify down to "0 ^ mathOutputNumber". According to the decompiled output, this would be correct. This is the second biggest thing to remember when you're reverse engineering an APK, the decompiled code is not always correct. In this case, if we use an online java compiler to avoid having to actually setup a Java development environment, we can rewrite this code and see where the problem comes in.

Firstly, let's use a Frida hook to get the result of a math operation. The reason we use a Frida hook for this is because at this point we've only ported the code to Golang, we haven't ensured the output is correct yet, let's just kill 2 birds with 1 stone here. Here's the Frida hook we're using:

1Java.perform(function () {
2    let a = Java.use("ov0.a");
3    a["a"].implementation = function (i12, i13, i14, i15) {
4        console.log('a is called' + ', ' + 'i12: ' + i12 + ', ' + 'i13: ' + i13 + ', ' + 'i14: ' + i14 + ', ' + 'i15: ' + i15);
5        let ret = this.a(i12, i13, i14, i15);
6        console.log('a ret value is ' + ret);
7        return ret;
8    };
9})

Referring back to the code we're porting, we see the final number is actually an output of "Boxing.boxInt(int)". This is extremely useful, it means that to know what changes between "a13" and "i14", and between "i14" and "boxInt" aka the final output, we can just hook this method:

1Java.perform(function () {
2    let Boxing = Java.use("kotlin.coroutines.jvm.internal.Boxing");
3    Boxing["boxInt"].implementation = function (i12) {
4        console.log('boxInt is called' + ', ' + 'i12: ' + i12);
5        let ret = this.boxInt(i12);
6        console.log('boxInt ret value is ' + ret);
7        return ret;
8    };
9})

Now let's run the Frida script to both validate our math function and show you why our decompiled output is wrong. When we run Frida, we should see an output similar to this: If we plug in our list of inputs to the ported function, we see the output is correct, great news. Now after I wrote part 1 and started testing my assumption, I noticed the output when doing "0 ^ a13" was in fact wrong, it did not line up. I even wrote this Java code to make sure my assumption wasn't just wrong because of a difference between GoLang and Java:

1public static void main(String[] args) {
2    String string1 = "SM-G988N";
3    byte[] bArr = new byte[4];
4    byte[] bytes = string1.getBytes(Charset.forName("UTF-8"));
5    bArr = bytes;
6    System.out.println((bArr.length >= 4 ? 0 : 123) ^ 1459136714);
7}

As seen before, the output with the input "1459136714" should be "95782285", this code reports otherwise. The problem isn't how I ported the code or how I rewrote this code, this code is perfectly 1:1 with the decompiled output. Admittedly, this did cause me many hours of confusion, I knew the decompiled code didn't really look right, I just couldn't tell what was going on. I have had this sort of thing happen to me before but it's so rare that I didn't think about it, so there's a lesson to be learned here. If you ever come across something like this, you should make sure to not blindly trust the decompiler too much, look at the "Smali" code view instead. When we look at the Smali code view, we see bytecode. You likely won't know how to read this at first, it's not as hard as it may seem though. First lets locate where we want to be, use ctrl + f to search "UTF-8", this should bring you to here:

1.line 23
2    :try_start_165
3    sget-object v10, Lkotlin/text/Charsets;->UTF_8:Ljava/nio/charset/Charset;
4    invoke-virtual {v1, v10}, Ljava/lang/String;->getBytes(Ljava/nio/charset/Charset;)[B
5    move-result-object v1
6    const-string v10, "this as java.lang.String).getBytes(charset)"
7    invoke-static {v1, v10}, Lkotlin/jvm/internal/Intrinsics;->checkNotNullExpressionValue(Ljava/lang/Object;Ljava/lang/String;)V
8    :try_end_170
9    .catch Ljava/io/UnsupportedEncodingException; {:try_start_165 .. :try_end_170} :catch_171

This is our code that sets the value of bArr, looking at ".line 24" will show us the slight difference between the decompiled output and the Smali:

1.line 24
2    :catch_171
3    array-length v1, v9
4    if-ge v1, v5, :cond_176
5    const/4 v1, 0x0
6    goto :goto_17e
7    :cond_176
8    invoke-static {v9}, Ljava/nio/ByteBuffer;->wrap([B)Ljava/nio/ByteBuffer;
9    move-result-object v1
10    invoke-virtual {v1}, Ljava/nio/ByteBuffer;->getInt()I
11    move-result v1
12    :goto_17e
13    xor-int/2addr v1, v3

We can see the "array-length" instruction, this will set "v1" to the length of "v9", this is basically "bArr.length". Next, we see "if-ge", which means "if x is greater than or equal to y". We previously learned that v1 is "bArr.length", however, v5 is set previously as a compiler optimization. Scrolling up to ".line 22", we see "const/4 v5, 0x4", this means v5 is 0x4 aka 4. This translates to "bArr.length >= 4". Now the third input to this instruction is ":cond_176", this is a way of referring to another section of bytecode to be what the interpreter will do if the condition is true. Going one line under, we see what will happen if it's false. You should start to see the problem, before even looking at ":cond_176", let's rewrite what we know already:

1if(bArr.length >= 4){
2    :cond_176
3}else{
4    v1 = 0x0
5}

Looking at the decompiled output, we see this:

1bArr.length >= 4 ? 0 : ByteBuffer.wrap(bArr).getInt()

Notice that the cases are flipped. In the Smali it says "v1 = 0" in the case of false, in the decompiled output, it basically says "v1 = 0" in the case of true. The difference was so subtle that the first time I looked at the Smali, I didn't notice this difference, I had to re-read the Smali a few times to notice what was happening. Well, at least now we know what the code should really look like:

1int i14 = (bArr.length >= 4 ? ByteBuffer.wrap(bArr).getInt() : 0) ^ a13;

Let's go ahead and properly port it this time. First let's get the easy part out of the way, creating our byte array:

1bArr := []byte("SM-G988N")

Now we need to figure out how "getInt()" works, in part 1 I supplied a simple description, using this description, I started searching on google for a similar Golang internal function. After some time, I came across this stackoverflow post: https://stackoverflow.com/a/63519540. Here they mention using "binary.Read" with a uint32, his example uses uint32 but to make sure we're matching 1:1 with our ported code, I'll be using int32. Here's the ported Golang code I got:

1b := []byte("SM-G988N")
2buf := bytes.NewBuffer(b)
3res := int32(0)
4binary.Read(buf, binary.BigEndian, &res)
5fmt.Println(res)

This produces the correct number you get from "ByteBuffer.wrap(bArr).getInt()". Now we're done, let's just plug it all together and we can get our final output for PX257:

1mathOut := pow(pow(numbers[2], numbers[3], numbers[0], numbers[5]), numbers[4], numbers[1], numbers[5])
2bArr := []byte("SM-G988N")
3res := int32(0)
4if len(bArr) >= 4 {
5	res = binary.Read(bytes.NewBuffer(bArr), binary.BigEndian, &res)
6}
7PX257 := res ^ mathOut

Now I'm going to make a function to parse the incoming "APPC" instruction and return a populated struct.

1// file: ./px/appc.go
2// AppcInstruction is the PX257, PX259, and PX256 keys in the payload. Input is the APPC 2 instruction
3// ex: appc|2|1688967166574|42443399873c02eadc3ebd747567101f265e835563d18725f73ec88b7012eb8f,42443399873c02eadc3ebd747567101f265e835563d18725f73ec88b7012eb8f|374|3542|3311|1478|3717|352
4func AppcInstruction(appc string) *APPC {
5	parts := strings.Split(appc, "|")
6    // make sure it's APPC 2
7	if parts[1] != "2" || len(parts) != 10 {
8		return nil
9	}
10	output := &APPC{
11		PX256: parts[3],
12	}
13    // convert the string timeStamp value to an int64, if it errors, return nil
14	date, err := strconv.ParseInt(parts[2], 10, 64)
15	if err != nil {
16		return nil
17	}
18	output.PX259 = date
19    // remove the first 4 elements of the array, leaving only the numbers
20	parts = parts[4:]
21    // convert all numbers to int32's instead of strings
22	numbers := []int32{}
23	for _, str := range parts {
24        num, err := strconv.ParseInt(str, 10, 32)
25		if err != nil {
26			return nil
27		}
28		numbers = append(numbers, int32(num))
29	}
30	//aVar2.a(aVar2.a(aVar2.appcNumber3, aVar2.appcNumber4, aVar2.appcNumber1, aVar2.appcNumber6), aVar2.appcNumber5, aVar2.appcNumber2, aVar2.appcNumber6);
31	mathOut := pow(pow(numbers[2], numbers[3], numbers[0], numbers[5]), numbers[4], numbers[1], numbers[5])
32	bArr := []byte(p.D.Px320)
33	res := int32(0)
34	if len(bArr) >= 4 {
35		res = binary.Read(bytes.NewBuffer(bArr), binary.BigEndian, &res)
36	} 
37	output.PX257 = res ^ mathOut
38	return output
39}

Some avid GoLang users will notice that this is bad GoLang design, instead of the output being a populated struct, it should just return an error and populate the main payload struct in a file named "payload.go". I'll show you how that would look:

1// File: ./px/payload.go
2package px
3
4// Payload is our main px payload struct
5type Payload struct {
6	T string `json:"t"`
7	D struct {
8		Px1214 string  `json:"PX1214"`
9		Px91   int     `json:"PX91"`
10		Px92   int     `json:"PX92"`
11		// etc
12		Px326  string  `json:"PX326"`
13		Px327  string  `json:"PX327"`
14		Px328  string  `json:"PX328"`
15		Px259  *int64  `json:"PX259,omitempty"`
16		Px256  *string `json:"PX256,omitempty"`
17		Px257  *string `json:"PX257,omitempty"`
18		Px1208 string  `json:"PX1208"`
19	} `json:"d"`
20}

Now I'm going to edit the APPC function:

1// file: ./px/appc.go
2func (p *Payload) AppcInstruction(appc string) error {
3	//...
4	p.D.Px256 = &parts[3]
5	//...
6	p.D.Px259 = &date
7	//...
8	p.D.Px257 = &out
9	return nil
10}
11

Back on topic, porting code. Next up will be PX326, PX327, and PX328. In part 1, we saw what code was behind these keys and in general how it works, we didn't look at specifics though. It's important when making a gen that you get things absolutely 1:1, for example, the UUID passed into this code:

1public c(String uniqueString, String deviceModel) {
2    List split$default;
3    this.f10789a = uniqueString;
4    split$default = StringsKt__StringsKt.split$default((CharSequence) uniqueString, new String[]{"-"}, false, 0, 6, (Object) null);
5    Locale locale = Locale.ROOT;
6    String upperCase = ((String) split$default.get(0)).toUpperCase(locale);
7    String valueOf = String.valueOf(upperCase);
8    this.f10790b = valueOf;
9    iv0.c cVar = iv0.c.f46778a;
10    String upperCase2 = cVar.a(deviceModel + uniqueString + valueOf).toUpperCase(locale);
11    this.f10791c = upperCase2;
12}

is created with this code:

1long currentTimeMillis = System.currentTimeMillis();
2if (currentTimeMillis == iv0.f.f46782a) {
3    currentTimeMillis++;
4}
5iv0.f.f46782a = currentTimeMillis;
6String uuid = new UUID(currentTimeMillis, 1L).toString();

There are two things to note here, firstly, the UUID is created with a timestamp, this means it's using a UUIDv1. The most important thing we see though is that it's cacheing the timestamp used so that 2 UUIDs will never use the same exact timestamp.

Now let's port some of the code, up until we get to PX328. First I'm going to add a "cache" system of sorts so that our timestamp value can persist through the first and second payloads. This is both important for the UUID and important for PX323. Now I can go ahead and port the code for PX326 and PX327. First I create this mock Java code so I can see the timestamp used and the resulting UUID:

1public static void main(String[] args) {
2    long a = System.currentTimeMillis();
3    System.out.println(a);
4    System.out.println(new UUID(a, 1L).toString());
5}
6
7/*
8timestamp and output used in this example:
91690399484912
1000000189-93a9-2bf0-0000-000000000001
11*/

So now I figured there was no point in writing a UUID generator from scratch when I could just edit an existing package. Editing existing packages can be annoying, especially if they're more complicated than they need to be just to support a bunch of stuff we won't need. For this reason, I decided to fork github.com/satori/go.uuid which is a UUID generation package that looks less feature-rich than most of the other packages I came across.

Now we're working off this base code:

1// file: generator.go
2// NewV1 returns UUID based on current timestamp and MAC address.
3func (g *rfc4122Generator) NewV1() (UUID, error) {
4	u := UUID{}
5
6	timeNow, clockSeq, err := g.getClockSequence()
7	if err != nil {
8		return Nil, err
9	}
10	binary.BigEndian.PutUint32(u[0:], uint32(timeNow))
11	binary.BigEndian.PutUint16(u[4:], uint16(timeNow>>32))
12	binary.BigEndian.PutUint16(u[6:], uint16(timeNow>>48))
13	binary.BigEndian.PutUint16(u[8:], clockSeq)
14
15	hardwareAddr, err := g.getHardwareAddr()
16	if err != nil {
17		return Nil, err
18	}
19	copy(u[10:], hardwareAddr)
20
21	u.SetVersion(V1)
22	u.SetVariant(VariantRFC4122)
23
24	return u, nil
25}

The first thing I did was override "timeNow" to be "1690399484912", I have also overridden "clockSeq" to be "0", this is just so I could remove that line of code entirely, it turned out that "0" was the correct thing to use at the end of it but that was just by chance, we aren't there yet. Now when generating a UUID we see this output:

1UUIDv1: 93a92bf0-0189-1000-8000-04421aef6061
2

Let's compare this to the Java output we're looking for:

100000189-93a9-2bf0-0000-000000000001
2

The first thing I notice is that "04421aef6061" should be "000000000001". Looking at the Java code, we see this looks a lot like it's just setting those bytes to our second input, which is hardcoded as "1L". To match this, I hard coded "hardwareAddr" to "hardwareAddr := []byte{0, 0, 0, 0, 0, 1}". Next I noticed "8000" should be "0000". Looking at the code, I can see:

1// file: generator.go
2u.SetVariant(VariantRFC4122)
3
4// file: uuid.go
5// UUID layout variants.
6const (
7	VariantNCS byte = iota
8	VariantRFC4122
9	VariantMicrosoft
10	VariantFuture
11)
12
13// Variant returns UUID layout variant.
14func (u UUID) Variant() byte {
15	switch {
16	case (u[8] >> 7) == 0x00:
17		return VariantNCS
18	case (u[8] >> 6) == 0x02:
19		return VariantRFC4122
20	case (u[8] >> 5) == 0x06:
21		return VariantMicrosoft
22	case (u[8] >> 5) == 0x07:
23		fallthrough
24	default:
25		return VariantFuture
26	}
27}

I can see that if I want that 8 to be a 0, all I should have to do is use "VariantNCS". Now we have this code:

1// file: generator.go
2// NewV1 returns UUID based on current timestamp and MAC address.
3func (g *rfc4122Generator) NewV1() (UUID, error) {
4	u := UUID{}
5	timeNow := uint64(1690399484912)
6	clockSeq := uint16(0)
7	binary.BigEndian.PutUint32(u[0:], uint32(timeNow))
8	binary.BigEndian.PutUint16(u[4:], uint16(timeNow>>32))
9	binary.BigEndian.PutUint16(u[6:], uint16(timeNow>>48))
10	binary.BigEndian.PutUint16(u[8:], clockSeq)
11
12	hardwareAddr := []byte{0, 0, 0, 0, 0, 1}
13	copy(u[10:], hardwareAddr)
14
15	u.SetVersion(V1)
16	u.SetVariant(VariantNCS)
17
18	return u, nil
19}

Our Golang output should be:

1UUIDv1: 93a92bf0-0189-2000-0000-000000000001
2
3Java output:
400000189-93a9-2bf0-0000-000000000001
5

Which is still far from our Java output, once you look closer though, you should see some similarities. "00000189" looks like the second section of our Golang output except with 4 0's at the start, for the sake of not wanting to look into the spec super closely, we can gen a bunch of UUIDs and see that they always have those first 4 0's. I also noticed that the Java output's second and third sections are both in our Golang output's first section. To see what I mean, look at it like this:

193a92bf0 => 
293a9 2bf0
393a9-2bf0
4

Now you should see it, if we just take the first section and move the first 4 bytes to the second section and the second 4 bytes to the third section, we have it! So I made up this code, you'll see I made the function able to take in a timeStamp instead of hard coding:

1// file: generator.go
2// NewV1 returns UUID based on current timestamp and MAC address.
3func (g *rfc4122Generator) NewV1(timeNow uint64) (UUID, error) {
4	u := UUID{}
5
6	clockSeq := uint16(0)
7	binary.BigEndian.PutUint16(u[0:], uint16(0))
8	binary.BigEndian.PutUint16(u[2:], uint16(timeNow>>32))
9	binary.BigEndian.PutUint32(u[4:8], uint32(timeNow))
10	binary.BigEndian.PutUint16(u[8:], clockSeq)
11
12	hardwareAddr := []byte{0, 0, 0, 0, 0, 1}
13	copy(u[10:], hardwareAddr)
14
15	u.SetVariant(VariantNCS)
16
17	return u, nil
18}

Notice that I removed "u.SetVersion(V1)", this is because it would override one of our bytes, we don't want this. It may seem like we do but after you try a few different Java outputs, you notice that byte is never hard coded. This code does generate a valid UUID for this PX SDK version though.

To add this custom fork to our project we will be using the go.mod replace method:

1module github.com/obfio/px-grubhub-mobile
2
3go 1.18
4
5require github.com/satori/go.uuid v1.2.0 // indirect
6
7replace github.com/satori/go.uuid => ./uuid

Now we can write this code to use it:

1// file: ./px/uuid.go
2// UUIDSection generates the keys PX326, PX327, and PX328.
3func (p *Payload) UUIDSection() error {
4	if p.Cache.TimeStamp == p.Cache.PrevUUIDTime {
5		p.Cache.TimeStamp++
6		p.Cache.PrevUUIDTime++
7	}
8	// ! generate UUIDv1
9	p.D.Px326 = uuid.Must(uuid.NewV1(p.Cache.TimeStamp))
10
11	return nil
12}

Adding "PX327" is just this code:

1p.D.Px327 = strings.ToUpper(p.D.Px326[0:8])

To add "PX328" we will need to do more porting. We need to look into this "cVar.a" method:

1public final String a(String input) {
2    MessageDigest messageDigest = MessageDigest.getInstance("SHA-1");
3	byte[] bytes = input.getBytes(Charsets.UTF_8);
4    byte[] bytes2 = messageDigest.digest(bytes);
5    StringBuilder sb2 = new StringBuilder(bytes2.length * 2);
6    for (byte b12 : bytes2) {
7        sb2.append("0123456789abcdef".charAt((b12 >> 4) & 15));
8        sb2.append("0123456789abcdef".charAt(b12 & 15));
9    }
10    String sb3 = sb2.toString();
11    return sb3;
12}

The first part is just SHA1 hashing the input. The second part looks like it's just hex encoding the SHA1 hash, if you didn't recognize that at first don't feel bad, the more you do this the more you'll be able to recognize stuff like that without having to do any work. Anyways, here's the final product:

1// file: ./px/uuid.go
2// UUIDSection generates the keys PX326, PX327, and PX328.
3func (p *Payload) UUIDSection() error {
4	if p.Cache.TimeStamp == p.Cache.PrevUUIDTime {
5		p.Cache.TimeStamp++
6		p.Cache.PrevUUIDTime++
7	}
8	// ! generate UUIDv1
9	p.D.Px326 = uuid.Must(uuid.NewV1(p.Cache.TimeStamp))
10	p.D.Px327 = strings.ToUpper(p.D.Px326[0:8])
11	p.D.Px328 = strings.ToUpper(px328(p.D.Px320 + p.D.Px326 + p.D.Px327))
12	return nil
13}
14
15func px328(str string) string {
16	h := sha1.New()
17	h.Write([]byte(str))
18	return hex.EncodeToString(h.Sum(nil))
19}

With this, we now have those 3 values done! That marks the end of this section!

Getting Device Data

Let's first see what exactly we need to collect. When looking at "PX1214" aka "android_id", we can see after some research how it's generated:

1String androidId = Long.toHexString(new SecureRandom().nextLong());

So it converts a random 64bit number to a hex string, we won't need to collect this then. Next, we have the "displayMetrics.widthPixels" and "displayMetrics.heightPixels", they could be validating this to a database of known phone models so we will need to collect this. The same goes for "Build.MODEL", "Build.MANUFACTURER", and "Build.DEVICE". We can apply this same reason to the following as well:

1context.getPackageManager().hasSystemFeature("android.hardware.location.gps")
2context.getPackageManager().hasSystemFeature("android.hardware.sensor.gyroscope")
3context.getPackageManager().hasSystemFeature("android.hardware.sensor.accelerometer")
4context.getPackageManager().hasSystemFeature("android.hardware.ethernet")
5context.getPackageManager().hasSystemFeature("android.hardware.touchscreen")
6context.getPackageManager().hasSystemFeature("android.hardware.nfc")
7context.getPackageManager().hasSystemFeature("android.hardware.wifi")

After some time of looking, I came across a website that we can scrape to get all of this information. This site is called PhoneMore, it has information on hundreds of different phones and includes all of the information we are looking for. As explained above, I won't be covering the actual process of scraping in this post, however, the GitHub repo for that is in the references section.

There are still three things data-related that we need to figure out though. Ethernet, Build.VERSION.SDK_INT, and System.getProperty("os.version"). Admittedly, I don't know anything about how Ethernet works or how mobile devices handle it. Due to this lack of knowledge, I asked around in the botting community. There wasn't anyone that knew anything concrete, however, a user named "Luma" suggested that the ethernet system feature would only return true if an adapter was actually plugged in. I decided to just go with this and hard code ethernet as false. If anyone reading this blog has more information on this, feel free to contact me and let me know because I could not find any valuable information online about this topic. Anyways, We still have Build.VERSION.SDK_INT to do, I didn't really know what this number meant so I started doing some research. I googled "android SDK versions" first and got lucky, turns out, that number refers to the API level for the Android version installed on the device. Thankfully, the device data website we scrape from supplies a default Android version that comes installed on the phone so we won't need to worry about randomly choosing an Android version and accidentally choosing one that doesn't work with the selected phone's specs. The best way I could figure to do this was to just make a JSON file named "levels.json" with Android versions as the selectors and the API levels as the values. An example can be seen below:

1// file: levels.json
2{
3    "10": 29,
4    "11": 30,
5    "12": 31,
6    "13": 33
7}

I couldn't really be bothered to parse floating numbers since only 1 of them (12.1) is actually relevant so I'm only supporting whole number versions. Now that we have that, we just have System.getProperty("os.version"). This also took some research but eventually, I ended up here: source.android.com/docs/core/architecture/kernel/android-common. From what this says, it appears for android 10, I can use any version from 4.19.0 to 4.19.292. For Android 11, I should be able to use anything between 5.4.0 and 5.4.254. I won't list out the ranges for every version, however, the reason I'm saying this is because the docs say that you can use any official kernel.org release. We can see here that they provide charts for the kernel versions supported for each Android version: Then when you go to kernel.org, you will see this: You can see the most recent versions, so you can use that for your highest version and 0 for the lowest version. This isn't exactly a perfect solution but it's better than using a static value.

Finally, we have what we need!

Filling In The Payload

For simplicity, we will go from top to bottom. First we need to parse our device data and levels that we made in the last section. For this we just need to make a struct, a map, and use json.Unmarshal.

So first, create our struct:

1// file: ./px/payload.go
2// Device is an instance of scraped device data
3type Device struct {    
4	Model         string `json:"model"`        
5	Manufacturer  string `json:"manufacturer"` 
6	Device        string `json:"device"`       
7	Width         int    `json:"width"`       
8	Height        int    `json:"height"`       
9	GPS           bool   `json:"gps"`          
10	Gyro          bool   `json:"gyro"`         
11	Accelerometer bool   `json:"accelerometer"` 
12	Ethernet      bool   `json:"ethernet"`      
13	TouchScreen   bool   `json:"touchScreen"`   
14	NFC           bool   `json:"nfc"`          
15	WiFi          bool   `json:"wifi"`          
16	AndroidVersion int `json:"androidVersion"`
17}

Then in our main file, we're going to load our levels and devices:

1var(
2	devices = []*px.Device{}
3	levels = make(map[string]int)
4)
5
6func init() {
7	b, err := os.ReadFile("devices.json")
8	if err != nil {
9		panic(err)
10	}
11	err = json.Unmarshal(b, &devices)
12	if err != nil {
13		panic(err)
14	}
15	b, err = os.ReadFile("levels.json")
16	if err != nil {
17		panic(err)
18	}
19	err = json.Unmarshal(b, &levels)
20	if err != nil {
21		panic(err)
22	}
23}

Now that we have those two parsed and ready to be used, let's add a method to our Payload struct to populate all of the other values. This method will need to take a Device object and a level as an int. For convenience, in case of updates in the future, we will also have inputs for PX340, PX342, PX341, PX348, and PX1159. These values would be input to the API when requesting a cookie/header, now let's talk about the func. This func is just filling in data, remember, we already have this documented. For the most part, we're just writing 1 line of code for these values. I will type out explanations for the stuff I think needs explaining with comments, here's the code I wrote:

1// file: ./px/payload.go
2func randomHex(n int) string {
3	b := make([]byte, n)
4	if _, err := rand.Read(b); err != nil {
5		return ""
6	}
7	return hex.EncodeToString(b)
8}
9
10func getkernelVersion(l int) string {
11	out := ""
12	switch l {
13	case 29:
14		out = fmt.Sprintf("4.19.%v", mrand.Intn(292))
15	case 30:
16		out = fmt.Sprintf("5.4.%v", mrand.Intn(254))
17	case 31:
18		out = fmt.Sprintf("5.10.%v", mrand.Intn(192))
19	case 33:
20		out = fmt.Sprintf("5.15.%v", mrand.Intn(128))
21
22	}
23	return out
24}
25
26func getRandomTemp() float64 {
27	return -40.0 + mrand.Float64()*100.0
28}
29
30func getRandomVoltage() float64 {
31	return 1.0 + mrand.Float64()*5.0
32}
33
34func (p *Payload) getBatteryStatus() string {
35	if p.D.Px418 > 40.0 {
36		return "overheat"
37	}
38	if p.D.Px418 < -20.0 {
39		return "cold"
40	}
41	if p.D.Px420 > 4.205 {
42		return "over voltage"
43	}
44	return "good"
45}
46
47func (p *Payload) getChargingPortStatus() string {
48	if p.D.Px420 > 1.0 {
49		return "USB"
50	}
51	return ""
52}
53
54func (p *Payload) getChargingStatus() string {
55	if p.D.Px415 > 100 {
56		return "full"
57	}
58	if p.D.Px413 != "" {
59		return "charging"
60	}
61	return "not charging"
62}
63
64// Populate fills in all the values we don't need to write a lot of code for
65func (p *Payload) Populate(d *Device, l int, sdkVer, appVer, appName, packName string, isInstantApp bool) {
66	p.T = "PX315"
67	// remember, we saw this was Long.toHexString(new SecureRandom().nextLong());
68	// We're going to just generate a random 16 long hex string for this
69	p.D.Px1214 = randomHex(8)
70	p.D.Px91 = d.Width
71	p.D.Px92 = d.Height
72	p.D.Px316 = true
73	p.D.Px318 = fmt.Sprint(l)
74	p.D.Px319 = getkernelVersion(l)
75	p.D.Px320 = strings.ToUpper(d.Model)
76	p.D.Px339 = d.Manufacturer
77	p.D.Px321 = d.Device
78	// remember, this timestamp is also what's used to make the UUID so we must add to the cache.
79	p.Cache.TimeStamp = time.Now().UnixMilli()
80	p.D.Px323 = p.Cache.TimeStamp / 1000
81	p.D.Px322 = "Android"
82	p.D.Px337 = d.GPS
83	p.D.Px336 = d.Gyro
84	p.D.Px335 = d.Accelerometer
85	// remember I said I don't know how ethernet works in phones so I'm just assuming if the level is >= 28, ethernet is true.
86	p.D.Px334 = l >= 28
87	p.D.Px333 = d.TouchScreen
88	p.D.Px331 = d.NFC
89	p.D.Px332 = d.WiFi
90	// These next 2 are basically checking if our device is rooted, most android users wont root their device so lets keep this false.
91	p.D.Px421 = "false"
92	p.D.Px442 = "false"
93	// we have 3 options for this, one of them is NA though so lets not use that one.
94	p.D.Px317 = []string{"WiFi", "Mobile"}[mrand.Intn(2)]
95	// gonna use a few different carriers here, doubt they'd just ban all T-Mobile users but ya never know
96	p.D.Px344 = []string{"T-Mobile", "Vodafone", "Msg2Send", "Mobitel", "Cequens", "Vodacom", "MTN", "Meteor", "Android", "Movistar", "Swisscom", "Orange", "Unite", "Oxygen8", "Txtlocal", "TextOver", "Virgin-Mobile", "Aircel", "AT&T", "Cellcom", "BellSouth", "Cleartalk", "Cricket", "DTC", "nTelos", "Esendex", "Kajeet", "LongLines", "MetroPCS", "Nextech", "SMS4Free", "Solavei", "Southernlinc", "Sprint", "Teleflip", "Unicel", "Viaero", "UTBox"}[mrand.Intn(38)]
97	// I'll use a static language here because they might cross-check it to the IP and I'm not sure about that yet
98	p.D.Px347 = "[en_US]"
99	// this is how many bars you have at the moment, the scraper makes sure that every device supports 5G so we can choose anything from Unknown - 5G
100	p.D.Px343 = []string{"Unknown", "2G", "3G", "4G", "5G"}[mrand.Intn(5)]
101	// this is your battery level, I'll make this between 20 and 100
102	p.D.Px415 = 20 + mrand.Intn(80)
103	// there's many options here, however, we will need to make sure whatever we choose lines up with our voltage/temp options.
104	// for this reason, we will be setting voltage and temp first.
105	// to know what's cold, what's hot, or what's high voltage, we will need to do some research into batterys.
106	// after some googling, I see that it's dependant on the battery to tell android that it's too hot, too cold, etc.
107	// so we don't have to be percise, just a general number will work here and for this number I'll choose anything above 40C as being too hot with the max being 60C
108	// It seems anything below -20C is considered too cold for a battery so that will be our mentric for that, with a low of -40C
109	// as for voltage, I saw the number 4.205 mentioned on some random android fourm so I'll just use that, anything above that will be considered too high and the max will be 6.0, lowest will be 1.0
110	p.D.Px418 = getRandomTemp()
111	p.D.Px420 = getRandomVoltage()
112	// we aren't sure if we should prioritize a the temp or voltage so we'll just prioritize the temp
113	p.D.Px413 = p.getBatteryStatus()
114	// for the charging port status, we're gonna make it only set it to either USB or "", this is because they may validate the model we choose has wireless charging.
115	// we will only set it to charging if the voltage is > 3.5
116	p.D.Px416 = p.getChargingPortStatus()
117	// I don't know enough about batteries to confidently use the discharging option and I wont be using the empty string option because it should never be triggered.
118	// Full will only be used if Px415 == 100
119	// charging will only be used if Px413 != ""
120	// else, not charging
121	p.D.Px414 = p.getChargingStatus()
122	// I had someone reach out after I posted part 1 and tell me that this value was actually a mistake on the part of PX and was changed in more recent SDK versions.
123	// the reason I bring this up is because it means the value is easy to mess up and I've seen people messs it up before when open-sourcing their APis
124	// it's important you make everything after the `@` hex characters and 7 characters.
125	// Also note bv0.a, this is the class and method that the payload building code is running in but this may not be the same between every APK using this SDK version so it's important you validate you have the correct calss and method name there.
126	p.D.Px419 = "bv0.a@" + randomHex(4)[:7]
127	p.D.Px340 = sdkVer
128	p.D.Px342 = appVer
129	p.D.Px341 = appName
130	p.D.Px348 = packName
131	p.D.Px1159 = isInstantApp
132	p.D.Px330 = "new_session"
133	p.D.Px1208 = "[]"
134}

Now let's finally test this! In main.go I'll write this to generate a test payload:

1func main() {
2	p := &px.Payload{}
3	d := devices[rand.Intn(len(devices))]
4	p.Populate(d, levels[fmt.Sprint(d.AndroidVersion)], "v2.2.3", "2023.25", "\"Grubhub\"", "com.grubhub.android", false)
5	p.UUIDSection()
6	b, _ := json.MarshalIndent(p, "", "	")
7	os.WriteFile("./payloads/1/req_test_gen.json", b, 0666)
8}

The payload looks great, besides the voltage and temperature values, I realized I forgot to trim the floating points. To fix that I went ahead and did this:

1func round(num float64) int {
2	return int(num + math.Copysign(0.5, num))
3}
4
5func toFixed(num float64, precision int) float64 {
6	output := math.Pow(10, float64(precision))
7	return float64(round(num*output)) / output
8}
9
10func getRandomTemp() float64 {
11	return toFixed(-40.0+mrand.Float64()*100.0, 1)
12}
13
14func getRandomVoltage() float64 {
15	return toFixed(1.0+mrand.Float64()*5.0, 3)
16}

Now the payload looks perfect, remember though, this is just the first request. Time to test our APPC code. To do this, I'll add this code to the main file:

1func main() {
2	p := &px.Payload{}
3	d := devices[rand.Intn(len(devices))]
4	p.Populate(d, levels[fmt.Sprint(d.AndroidVersion)], "v2.2.3", "2023.25", "\"Grubhub\"", "com.grubhub.android", false)
5	p.UUIDSection()
6	b, _ := json.MarshalIndent(p, "", "	")
7	os.WriteFile("./payloads/1/req_test_gen.json", b, 0666)
8	p.AppcInstruction("appc|2|1693271866748|dfb1719bf2d23204db8e8826fc5155fc4bb5f38ddaabc733bd746f273f836f5a,dfb1719bf2d23204db8e8826fc5155fc4bb5f38ddaabc733bd746f273f836f5a|1148|1837|546|2561|3788|1507")
9	b, _ = json.MarshalIndent(p, "", "	")
10	os.WriteFile("./payloads/2/req_test_gen.json", b, 0666)
11}

When looking at the second output, I realize I forgot to set t to PX329, I'll add that to the APPC.go code since that code only runs on the second payload build. Now both of our payloads look perfect! We do still need to make the final payload though, which isn't a JSON object, it's a www-form, thankfully Golang has an std-lib for this.

HTTP Requests

Now that we're ready to make the HTTP requests required to get our _px2 cookie/header, we need to touch on the very complicated topic of TLS fingerprinting. I am not qualified enough to speak extensively on the topic, however, I'll do my best in this article.

When talking about TLS fingerprinting, there's generally 3 sections, HTTP1 (ja3), HTTP2, and HTTP3. I don't know anything about HTTP3 and thankfully we don't need to for this project, HTTP3 is a fairly new standard anyways so it's not a very commonly used fingerprinting technique. The other two are industry standard though, if you're not prepared to deal with them, you are likely to fail. I said previously that I'm not qualified to speak about the topic extensively so I'll do a quick overview of both.

When making an HTTPS request you open a TLS connection, sending a HELLO handshake packet. In this packet, you send various data to tell the server a lot of different stuff it needs to know in order to more effectively communicate. For example, in both HTTP1 (H1) and HTTP2 (H2), you will send a list of cipher suites in this packet. These suites are used to tell the server what ciphers the browser supports, this is imparative because let's say the browser you're using is outdated and doesn't support TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 for some ungodly reason, the server knows to either use another cipher or to just simply deny the handshake instead of trying to send a cipher to the client that it cannot understand. There's much more data sent in this handshake but it's important that what you gather from this explanation is that the data that's sent could theoretically be used to validate if the useragent you're using matches the information sent in your handshake. This is what is done in H1 and H2 fingerprinting. They're just using data they know SHOULD be present in your TLS cipher suites, TLS exnteions, TLS curves, etc. in order to see if it matches what features are expected from your browser (chrome, firefox, safari, etc.) and, more importantly, specific version. If you'd like to see not only good resources on TLS fingerprinting but also TLS fingerprinting itself, you can visit tls.peet.ws, peet is a very smart member of the SneakerDev discord community who has open-sourced a lot of information on TLS fingerprinting!

After reading that simplified explanation of TLS, you may be wondering, how does that apply to us? I specifically mentioned browsers like chrome, firefox, and safari? If we're not using a browser to make requests and there's no browser user-agent to check TLS on, then what do we have to worry about? Well that's what's so great about mobile reverse engineering! It's made so much easier when apps don't use webview for their anti-bots. In this case, we do still have to match some TLS though, if you remember from part 1, requests are being made through okhttp3 and since a handshake does have to happen that means they can reliably fingerprint the okhttp3 handshake. Thankfully since okhttp3 is an open-source library and just mimicks the http standard, pretty much any TLS library will have a perfect implementation of the okhttp3 TLS fingerprint already added. I know this may be a little bit anti-climatic, some of you may have been hoping I would show you how to actually copy the TLS fingerprint for yourself, however, if I ever decide to do that, it will be in another article.

Now that I got that out of the way, that's get started! For the TLS client, I'll be using a custom client posted on GitHub by bogdanfinn, a member of the SneakerDev discord community. How you choose to design your project is purely personal preference, however, I usually like to make a file named "client.go" with a func usually called something like "makeClient" that takes in a proxy and returns a custom struct so I can add methods that can implement the HTTP client and anything else I need. That may be a bit complicated to explain so here's the code:

1// file: ./px/client.go
2package px
3
4import (
5	"errors"
6	"strings"
7
8	// used for header formatter below
9	http "github.com/bogdanfinn/fhttp"
10
11	tlsclient "github.com/bogdanfinn/tls-client"
12)
13
14// Client - holds our PX payload struct, proxy, device, and HTTP client
15type Client struct {
16	PXID string
17	PXPayload    *Payload
18	Proxy        string // returned to API
19	DeviceData   *Device
20	HTTPClient   tlsclient.HttpClient
21	SDKVersion   string
22	AppVersion   string
23	AppName      string
24	PackageName  string
25	IsInstantApp bool
26}
27
28// MakeClient - makes a *Client struct given a proxy and device as well as app/px info
29func MakeClient(PXID, proxy, sdkVer, appVer, appName, packName string, isInstantApp bool, d *Device) (*Client, error) {
30	//if proxy == "" {
31		//return nil, errors.New("no proxy input")
32	//}
33	opts := []tlsclient.HttpClientOption{
34		tlsclient.WithTimeoutSeconds(30),
35		//tlsclient.WithProxyUrl(proxy),
36	}
37	switch d.AndroidVersion {
38	case 10:
39		opts = append(opts, tlsclient.WithClientProfile(tlsclient.Okhttp4Android10))
40		break
41	case 11:
42		opts = append(opts, tlsclient.WithClientProfile(tlsclient.Okhttp4Android11))
43		break
44	case 12:
45		opts = append(opts, tlsclient.WithClientProfile(tlsclient.Okhttp4Android12))
46		break
47	case 13:
48		opts = append(opts, tlsclient.WithClientProfile(tlsclient.Okhttp4Android13))
49		break
50	}
51	h, err := tlsclient.NewHttpClient(nil, opts...)
52	if err != nil {
53		return nil, err
54	}
55	return &Client{
56		PXID: PXID,
57		HTTPClient:   h,
58		Proxy:        proxy,
59		DeviceData:   d,
60		PXPayload:    &Payload{},
61		SDKVersion:   sdkVer,
62		AppVersion:   appVer,
63		AppName:      appName,
64		PackageName:  packName,
65		IsInstantApp: isInstantApp,
66	}, nil
67}

You can see I added an input for device data, this is so I can match the Android version with the TLS, I'm not sure if this is actually required for PX but it's better safe than sorry. I also added all the extra stuff we needed like SDK version, app version, etc. for convenience. Now let's code in the requests, to do this, we have to open our proxy interceptor again so we can see the headers, URL, www-form, etc. I won't be going over that but I will be writing all the HTTP requests in a file named "http.go". So first we need to do the "px-conf" request, I'm not entirely sure if this request is required to pass on scale but we will code it in for now, when we confirm our API works, we can remove the request and see if it still works.

Now that we're looking into headers it's important I talk about header order, sometimes anti-bots will detect APIs because they aren't sending their headers in a specific order. If you inspect the PX requests, you can notice that GrubHub is injecting headers into the requests. It wouldn't make sense for header order to be a detection vector here because of this factor, therefore, I won't worry about it. Even though I won't end up doing it, I do want to mention that when you're looking at the response from a proxy intercept, you shouldn't inherently trust that the headers are in order. A lot of proxy interceptors change header orders or even add/remove headers. Since I won't be using pseudo header order, I'm going to be writing a simple helper function for parsing headers into an http.Header:

1// file: ./px/client.go
2// FormatHeaders turns a string of headers seperated by `|` into a http.Header map
3func (c *Client) FormatHeaders(h string) http.Header {
4	headers := http.Header{}
5	for _, header := range strings.Split(h, "|") {
6		parts := strings.Split(header, ": ")
7		headers.Set(parts[0], parts[1])
8	}
9	return headers
10}

The conf request has two parts, request body and response body. For the request body I won't bother making a struct, it's really just because I'm lazy but in this case there's no real advantage to making a proper struct for the request body. I will be making a struct for the response body though, in part 1 I didn't ever look into what the config was used for so we still don't know, however, in case it's useful, I will be making a struct in a file named "structs.go":

1// file: ./px/structs.go
2// PXConfig is the response from the PX Config request
3type PXConfig struct {
4	Enabled bool `json:"enabled"`
5	IPv6 bool `json:"ipv6"`
6}

Now we can code the px-conf request:

1// file: ./px/http.go
2import (
3	"encoding/json"
4	"fmt"
5	"io"
6	"strings"
7
8	http "github.com/bogdanfinn/fhttp"
9)
10
11// GetConfig does the px-conf request and sets c.Config to a *Config struct
12func (c *Client) GetConfig() error {
13	body := fmt.Sprintf(`{"device_os_version":"%v","device_os_name":"Android","device_model":"%s","sdk_version":"%s","app_version":"%s","app_id":"%s"}`, c.DeviceData.AndroidVersion, strings.ToUpper(c.DeviceData.Model), c.SDKVersion, c.AppVersion, c.PXID)
14	req, err := http.NewRequest("POST", "https://px-conf.perimeterx.net/api/v1/mobile", strings.NewReader(body))
15	if err != nil {
16		return err
17	}
18	req.Header = c.FormatHeaders(`Host: px-conf.perimeterx.net|Accept-Charset: UTF-8|Accept: */*|Content-Type: application/json|Connection: close`)
19	req.Header.Set("User-Agent", "PerimeterX Android SDK/"+c.SDKVersion[1:])
20	resp, err := c.HTTPClient.Do(req)
21	if err != nil {
22		return err
23	}
24	defer resp.Body.Close()
25	if resp.StatusCode != 200 {
26		return fmt.Errorf("http resp code: %v", resp.StatusCode)
27	}
28	b, err := io.ReadAll(resp.Body)
29	if err != nil {
30		return err
31	}
32	f := &Config{}
33	err = json.Unmarshal(b, &f)
34	if err != nil {
35		return err
36	}
37	c.Config = f
38	return nil
39}

As detailed in the method's description, I have updated the Client struct to add *Config* so we can keep everything in the Client struct incase it's needed later. Let's go ahead and test this by writing this code in the main file:

1// file: main.go
2func main() {
3	c, err := px.MakeClient("PXO97ybH4J", "", "v2.2.3", "2023.25", "\"Grubhub\"", "com.grubhub.android", false, devices[rand.Intn(len(devices))])
4	if err != nil {
5		panic(err)
6	}
7	err = c.GetConfig()
8	if err != nil {
9		panic(err)
10	}
11	fmt.Printf("%+v\n", c.Config)
12}

After running, you should see this output:

1&{Enabled:true IPv6:false}

This means the request worked, it also matches the response we see in our proxy interceptor! Now we can get on to the 2 harder requests. For the sake of simplicity, I'm going to make both requests into 2 different methods even though they're being made to the same URL and technically could be made into the same method. For the request payload of the second request, we're going to be using json.Marshal then base64.StdEncoding.EncodeToString to base64 encode the payload. To generate the UUID we will be following the UUIDv4 standard, we're also going to be adding "UUIDv4" to the Client struct. We can use the UUID package we modified earlier to generate UUIDv4's for us!

We already went over the other values in part 1, as for how we will build a request body using www-form content-type, we will use "net/url" in the Golang stdlib. I'm also making a new struct called "Instructions" for the response body for requests two and three. Here's the code I made for making the second request:

1// file: ./px/http.go
2// GetInstructions - first PX request that gets instructions, mainly the APPC instruction used for the second request
3func (c *Client) GetInstructions() (*Instructions, error) {
4	p, err := json.Marshal([]*Payload{c.PXPayload})
5	if err != nil {
6		return nil, err
7	}
8	body := url.Values{}
9	body.Set("payload", base64.StdEncoding.EncodeToString(p))
10	body.Set("uuid", c.UUIDv4)
11	body.Set("appId", c.PXID)
12	body.Set("tag", "mobile")
13	body.Set("ftag", "22")
14	req, err := http.NewRequest("POST", fmt.Sprintf("https://collector-%s.perimeterx.net/api/v1/collector/mobile", c.PXID), strings.NewReader(body.Encode()))
15	if err != nil {
16		return nil, err
17	}
18	req.Header = c.FormatHeaders(`Accept-Charset: UTF-8|Accept: */*|Content-Type: application/x-www-form-urlencoded; charset=utf-8`)
19	req.Header.Set("User-Agent", "PerimeterX Android SDK/"+c.SDKVersion[1:])
20	resp, err := c.HTTPClient.Do(req)
21	if err != nil {
22		return nil, err
23	}
24	defer resp.Body.Close()
25	if resp.StatusCode != 200 {
26		return nil, fmt.Errorf("http resp code: %v", resp.StatusCode)
27	}
28	b, err := io.ReadAll(resp.Body)
29	if err != nil {
30		return nil, err
31	}
32	f := &Instructions{}
33	err = json.Unmarshal(b, &f)
34	if err != nil {
35		return nil, err
36	}
37	return f, nil
38}

You'll notice that I marshal'd an array, this is because if you look at the payload, it's actually an array not just 1 JSON object, this was an easier way to implement that. Now we can test this method by implementing some of our previous payload testing code and adding some new code:

1// file: main.go
2func main() {
3	//...
4	c.PXPayload.Populate(c.DeviceData, levels[fmt.Sprint(c.DeviceData.AndroidVersion)], c.SDKVersion, c.AppVersion, c.AppName, c.PackageName, c.IsInstantApp)
5	err = c.PXPayload.UUIDSection()
6	if err != nil {
7		panic(err)
8	}
9	instructions, err := c.GetInstructions()
10	if err != nil {
11		panic(err)
12	}
13	fmt.Printf("%+v\n", instructions)
14}

You should see an output something like this:

1&{Enabled:true IPv6:false}
2&{Do:[sid|xxxxx-46bd-11ee-aef7-xxxxxxx vid|xxxxxxx-46bd-11ee-aef7-xxxxxxxxxx|31536000|false appc|1|1693349156834|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|xxxxxxxxxx-46bd-11ee-8b38-xxxxxxxxxxxxxxxxxxx appc|2|1693349156834|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|123123|123123|123123|123123|123123|123123]}

If it's not, it may be possible that your IP is flagged or this post is outdated, likely the second. We're very close to being done at this point! If you remember from before, the second payload uses the "sid", "vid", and "appc" instructions. I'll go ahead and add "SID" and "VID" to the Client struct for easy use. Next I'm going to write some code to handle instructions so we can set sid, vid, and handle appc correctly. Here's what I wrote:

1// file: main.go
2for _, instruction := range instructions.Do {
3	parts := strings.Split(instruction, "|")
4	if len(parts) < 1 {
5		continue
6	}
7	if parts[0] == "sid" {
8		c.SID = parts[1]
9		continue
10	}
11	if parts[0] == "vid" {
12		c.VID = parts[1]
13		continue
14	}
15	if parts[0] == "appc" && parts[1] == "2" {
16		c.PXPayload.AppcInstruction(instruction)
17		continue
18	}
19}

The rest of what we need to do is mainly just copy-pasting to do the last request, we really only need to add "sid" and "vid" since our helper function handles everything else! I'll name the method "GetCookie", here's the code:

1// file: ./px/http.go
2// GetCookie - last PX request, this request should return the instructions that give you the `bake` aka _px2 cookie/header
3func (c *Client) GetCookie() (*Instructions, error) {
4	p, err := json.Marshal([]*Payload{c.PXPayload})
5	if err != nil {
6		return nil, err
7	}
8	body := url.Values{}
9	body.Set("payload", base64.StdEncoding.EncodeToString(p))
10	body.Set("uuid", c.UUIDv4)
11	body.Set("appId", c.PXID)
12	body.Set("tag", "mobile")
13	body.Set("ftag", "22")
14	body.Set("sid", c.SID)
15	body.Set("vid", c.VID)
16	req, err := http.NewRequest("POST", fmt.Sprintf("https://collector-%s.perimeterx.net/api/v1/collector/mobile", c.PXID), strings.NewReader(body.Encode()))
17	if err != nil {
18		return nil, err
19	}
20	req.Header = c.FormatHeaders(`Accept-Charset: UTF-8|Accept: */*|Content-Type: application/x-www-form-urlencoded; charset=utf-8`)
21	req.Header.Set("User-Agent", "PerimeterX Android SDK/"+c.SDKVersion[1:])
22	resp, err := c.HTTPClient.Do(req)
23	if err != nil {
24		return nil, err
25	}
26	defer resp.Body.Close()
27	if resp.StatusCode != 200 {
28		return nil, fmt.Errorf("http resp code: %v", resp.StatusCode)
29	}
30	b, err := io.ReadAll(resp.Body)
31	if err != nil {
32		return nil, err
33	}
34	f := &Instructions{}
35	err = json.Unmarshal(b, &f)
36	if err != nil {
37		return nil, err
38	}
39	return f, nil
40}

Then to test this method, we can pretty much paste code too:

1// file: main.go
2func main() {
3	//...
4	for _, instruction := range instructions.Do {
5		parts := strings.Split(instruction, "|")
6		if len(parts) < 1 {
7			continue
8		}
9		if parts[0] == "sid" {
10			c.SID = parts[1]
11			continue
12		}
13		if parts[0] == "vid" {
14			c.VID = parts[1]
15			continue
16		}
17		if parts[0] == "appc" && parts[1] == "2" {
18			c.PXPayload.AppcInstruction(instruction)
19			continue
20		}
21	}
22	instructions, err = c.GetCookie()
23	if err != nil {
24		panic(err)
25	}
26	fmt.Printf("%+v\n", instructions)
27}

Your response should be something like this:

1&{Enabled:true IPv6:false}
2&{Do:[sid|xxxxx-46bd-11ee-aef7-xxxxxxx vid|xxxxxxx-46bd-11ee-aef7-xxxxxxxxxx|31536000|false appc|1|1693349156834|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|xxxxxxxxxx-46bd-11ee-8b38-xxxxxxxxxxxxxxxxxxx appc|2|1693349156834|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|123123|123123|123123|123123|123123|123123]}
3&{Do:[bake|_px2|500|eyJ1IjoiMzFlxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMjM0MGQ5NTIxYTA5NDZkMjhlZWMyZmY3NDk1In0=|true|500]}

This is great! We are successfully generating cookies/headers! Don't get too happy though, our job isn't done, typically when you're working on "good" anti-bots, this is the point where the job starts to get difficult.

Tricks Of Anti-Bots

Anti-bots are known to do stuff that just doesn't make logical sense, for example, you have just generated 2 payloads that appear perfect, you confirmed your TLS is perfect, and you're testing on your home IP so IP quality won't be an issue, and when you go to generate a cookie, the anti-bot gives you back a response that appears that you passed. From this point, you start making an API so you can use these cookies, only to realize that these cookies were never actually "good" cookies. This is a very common thing that anti-bots are known to do, giving you a response that appears valid until code another external tool to test your cookies. It's very inconvenient.

Another thing you'll often see is biometrics fingerprinting, a big one for this is Akamai 2.0 (Web) and Akamai BMP (mobile SDK). For the explanation, I'll be referring to how it's done on the web, I'm less familiar with how biometrics fingerprinting is done on mobile devices. Biometrics fingerprinting on the web can be done a few different ways but the main way is by tracking your cursor when you're on a page, to stay on topic, PX has a web anti-bot and their web anti-bot has biometrics fingerprinting. Their biometrics tracks where your cursor moves and what HTML elements it goes over while it's moving, this makes generating good biometrics data more annoying because ideally you would be required to scrape all HTML elements on the page, their x,y locations, and account for that when generating your cursor movement data.

The last thing I'll cover in this section is how anti-bots actually process the data you send them. I can't speak for every anti-bot but a majority of anti-bots use some sort of machine learning (ML) model to process the data and try to tell if data is bot data or human data, Akamai shows that this works very well with biometrics data if your ML model is well trained. A poorly trained ML model can be very bad for both a "bad actor" (technically us lol) and real users if they're not careful, PX shows us that very well. PX has been known to ban timezones, browsers, and pretty much entirely lock a website behind their captcha during "drops", aka any time there's a large influx of users. You'd think that this makes PX a good solution against bots, but this isn't true though, PX is very bad at stopping bots and detecting bot data, a lot of the fields in the PX web and SDK payloads will let you input pretty much anything and you won't get flagged any more than usual. That may make you wonder why I put so much effort into making sure everything in the payloads was as correct as possible, that was just so people could learn, you could probably leave half the payload blank and still pass. Anyways, this concept of using ML to detect bot data is why I added a section for collecting phone data to randomize what we send to PX.

Simple API

Here we're going to setup a simple local host API, I'll be using "github.com/gin-gonic/gin" for this, if you have a framework you like better, feel free to use it. I'll be making a new file named "bake.go" in the "main" package, it's named "bake" because that's what our API's path will be called. If you were paying attention, "bake" is what the cookie/header instruction is called.

Firstly we will write the http server code, assuming our router in "bake.go" will be called "bake":

1// file: main.go
2func main() {
3	router := gin.New()
4	router.Use(gin.Recovery())
5	px := router.Group("px")
6	px.POST("bake", bake)
7	s := &http.Server{
8		Addr:           "127.0.0.1:7356",
9		Handler:        router,
10		ReadTimeout:    15 * time.Second,
11		WriteTimeout:   15 * time.Second,
12		MaxHeaderBytes: 1 << 20,
13	}
14	err := s.ListenAndServe()
15	if err != nil {
16		log.Fatalf("httpServe: %s", err)
17	}
18}

To contact our API, we would go to "127.0.0.1:7356/px/bake", using a POST request. Now we will code the router, first we need to make the struct that defines what data is going to be passed into the API. I'll be including everything that would change between apps and a proxy input:

1// file: bake.go
2type input struct {
3	PXID         string `json:"pxid"`
4	Proxy        string `json:"proxy"`
5	SDKVersion   string `json:"sdkVersion"`
6	AppVersion   string `json:"appVersion"`
7	AppName      string `json:"appName"`
8	PackageName  string `json:"packageName"`
9	IsInstantApp bool   `json:"isInstantApp"`
10}

Now we can make our router, most of this is just copying our testing code and changing how we handle errors:

1// file: bake.go
2type output struct {
3	Cookie string `json:"cookie"`
4	Error  string `json:"error"`
5}
6
7func bake(c *gin.Context) {
8	o := &output{}
9	b, err := io.ReadAll(c.Request.Body)
10	if err != nil {
11		o.Error = err.Error()
12		c.AbortWithStatusJSON(500, o)
13		return
14	}
15	i := &input{}
16	err = json.Unmarshal(b, &i)
17	if err != nil {
18		o.Error = err.Error()
19		c.AbortWithStatusJSON(500, o)
20		return
21	}
22	client, err := px.MakeClient(i.PXID, i.Proxy, i.SDKVersion, i.AppVersion, i.AppName, i.PackageName, i.IsInstantApp, devices[rand.Intn(len(devices))])
23	if err != nil {
24		o.Error = err.Error()
25		c.AbortWithStatusJSON(500, o)
26		return
27	}
28	err = client.GetConfig()
29	if err != nil {
30		o.Error = err.Error()
31		c.AbortWithStatusJSON(500, o)
32		return
33	}
34	client.PXPayload.Populate(client.DeviceData, levels[fmt.Sprint(client.DeviceData.AndroidVersion)], client.SDKVersion, client.AppVersion, client.AppName, client.PackageName, client.IsInstantApp)
35	err = client.PXPayload.UUIDSection()
36	if err != nil {
37		o.Error = err.Error()
38		c.AbortWithStatusJSON(500, o)
39		return
40	}
41	instructions, err := client.GetInstructions()
42	if err != nil {
43		o.Error = err.Error()
44		c.AbortWithStatusJSON(500, o)
45		return
46	}
47	for _, instruction := range instructions.Do {
48		parts := strings.Split(instruction, "|")
49		if len(parts) < 1 {
50			continue
51		}
52		if parts[0] == "sid" {
53			client.SID = parts[1]
54			continue
55		}
56		if parts[0] == "vid" {
57			client.VID = parts[1]
58			continue
59		}
60		if parts[0] == "appc" && parts[1] == "2" {
61			client.PXPayload.AppcInstruction(instruction)
62			continue
63		}
64	}
65	instructions, err = client.GetCookie()
66	if err != nil {
67		o.Error = err.Error()
68		c.AbortWithStatusJSON(500, o)
69		return
70	}
71	for _, instruction := range instructions.Do {
72		parts := strings.Split(instruction, "|")
73		if len(parts) < 1 {
74			continue
75		}
76		if parts[0] == "bake" {
77			o.Cookie = parts[3]
78		}
79	}
80	if o.Cookie == "" {
81		o.Error = "unable to get cookie"
82		c.AbortWithStatusJSON(500, o)
83		return
84	}
85	c.JSON(200, o)
86}

Now we can use our proxy interceptor "Burp" to test our API. Go into the "Repeater" tab, make a new session and paste this:

1POST /px/bake HTTP/1.1
2Host: 127.0.0.1:7356
3Accept: */*
4Content-Type: application/json
5Content-Length: 188
6
7{
8	"pxid": "PXO97ybH4J",
9	"proxy": "",
10	"sdkVersion": "v2.2.3",
11	"appVersion": "2023.25",
12	"appName": "\"Grubhub\"",
13	"packageName": "com.grubhub.android",
14	"isInstantApp": false
15}

The API should return a cookie! Now we need to go back into "./px/client.go" and uncomment the stuff I left commented so we could test the requests without a proxy, that should be these lines:

1// file: ./px/client.go
2if proxy == "" {
3	return nil, errors.New("no proxy input")
4}
5opts := []tlsclient.HttpClientOption{
6	tlsclient.WithTimeoutSeconds(30),
7	tlsclient.WithProxyUrl(proxy),
8}

Testing Our Cookies

If I were making a serious API that I was trying to sell to people or make work for multiple apps/versions, I would make a tool that uses this API to gen cookies and test them. I won't be doing this here, I find it easier to just add some temporary code for the sake of the article that tests the cookies for us using GrubHub's login API. We've already unpinned GrubHub's requests in part 1 so we should be able to just attempt a login with our interceptor open and see how it all works! For now, let's assume we can ignore the tracker headers, meaning "Traceparent" for example. We can see our "cookie" is set via a header named "X-Px-Authorization", because it's px2, the header is set to "2:cookie". When we look at the payload, we see we do have a few values we have to reverse engineer which is very unfortunate. Lets look into "client_id" , "device_id", and "device_public_key". After some time of looking into client_id, I've realized that it does seem to be static, that's great news! After more time, I noticed "device_id" is not static and not being sent by the server to the client, this means the client is generating a UUIDv4 and using it here. Now we just need to figure out device_public_key. I don't want this post to be heavy with reverse engineering so I'll keep this light, I looked up where "device_public_key" was located, tracked where it was set, and then ended up here:

1public Void mo3771j() {
2    return null;
3}
4public final <T> T mo3757y(SerialDescriptor descriptor, int i, InterfaceC35575b<T> deserializer, T t) {
5    return (deserializer.getDescriptor().mo41b() || mo3795D()) ? (T) m6141H(deserializer, t) : (T) mo3771j();
6}

I figured, if that method "mo3771j" returns "null", then maybe I should try to set "device_public_key" to "null" and just see what happens. Surprisingly it ended up working. I didn't realize this before but we do still need one more thing, this is the "Authorization" header, after looking more closely, it looks like this header is returned by the server after requesting to "https://api-gtm.grubhub.com/auth/anon". When testing cookies, we will have to request this first, then request to the login endpoint. Here's the request code:

1// file: ./px/http.go
2func (c *Client) getAuth() (*GrubHubAuth, error) {
3	req, err := http.NewRequest("POST", "https://api-gtm.grubhub.com/auth/anon", strings.NewReader(`{"brand":"GRUBHUB","client_id":"ghandroid_Ujtwar5s9e3RYiSNV31X41y2hsK6Kh1Uv7JDrkpS","scope":"anonymous"}`))
4	if err != nil {
5		return nil, err
6	}
7	req.Header = c.FormatHeaders(`Host: api-gtm.grubhub.com|Vary: Accept-Encoding|Accept: */*|X-Px-Authorization: 1|Content-Type: application/json; charset=utf-8`)
8	req.Header.Set("User-Agent", fmt.Sprintf("Grubhub/2023.25 (%s; Android %v)", c.DeviceData.Model, c.DeviceData.AndroidVersion))
9	req.Header.Set("X-Gh-Features", fmt.Sprintf("0=phone;1=grubhub 2023.25.0;2=Android %v;", c.DeviceData.AndroidVersion))
10	resp, err := c.HTTPClient.Do(req)
11	if err != nil {
12		return nil, err
13	}
14	defer resp.Body.Close()
15	if resp.StatusCode != 200 {
16		return nil, fmt.Errorf("http resp code: %v", resp.StatusCode)
17	}
18	b, err := io.ReadAll(resp.Body)
19	if err != nil {
20		return nil, err
21	}
22	f := &GrubHubAuth{}
23	err = json.Unmarshal(b, &f)
24	if err != nil {
25		return nil, err
26	}
27	return f, nil
28}
29
30var (
31	i      = 0
32	locker sync.RWMutex
33)
34
35func (c *Client) logDebug(debug *LoginDebug) {
36	b, _ := json.MarshalIndent(debug, "", "	")
37	locker.Lock()
38	os.WriteFile(fmt.Sprintf("./tests/%v.json", i), b, 0666)
39	i++
40	locker.Unlock()
41}
42
43// TestCookie uses the grubhub login endpoint to test if our PX cookies pass or not
44func (c *Client) TestCookie(cookie, email, password string) error {
45	debug := &LoginDebug{}
46	auth, err := c.getAuth()
47	if err != nil {
48		debug.Error = err.Error()
49		c.logDebug(debug)
50		return err
51	}
52	if auth.Session.AuthToken == "" {
53		debug.Error = "no auth token found"
54		c.logDebug(debug)
55		return errors.New("no auth token found")
56	}
57	body := fmt.Sprintf(`{"brand":"GRUBHUB","client_id":"ghandroid_Ujtwar5s9e3RYiSNV31X41y2hsK6Kh1Uv7JDrkpS","email":"%s","password":"%s","exclusive_session":false,"scope":"diner","device_id":"%s","device_public_key":null,"metadata_map":null}`, email, password, uuid.Must(uuid.NewV4()))
58	req, err := http.NewRequest("POST", "https://api-gtm.grubhub.com/auth/login", strings.NewReader(body))
59	if err != nil {
60		debug.Error = err.Error()
61		c.logDebug(debug)
62		return err
63	}
64	req.Header = c.FormatHeaders(`Host: api-gtm.grubhub.com|Vary: Accept-Encoding|Accept: */*|Content-Type: application/json; charset=utf-8`)
65	req.Header.Set("User-Agent", fmt.Sprintf("Grubhub/2023.25 (%s; Android %v)", c.DeviceData.Model, c.DeviceData.AndroidVersion))
66	req.Header.Set("X-Gh-Features", fmt.Sprintf("0=phone;1=grubhub 2023.25.0;2=Android %v;", c.DeviceData.AndroidVersion))
67	req.Header.Set("X-Px-Authorization", "2:"+cookie)
68	req.Header.Set("Authorization", "Bearer "+auth.Session.AuthToken)
69	resp, err := c.HTTPClient.Do(req)
70	if err != nil {
71		debug.Error = err.Error()
72		c.logDebug(debug)
73		return err
74	}
75	defer resp.Body.Close()
76	debug.StatusCode = resp.StatusCode
77	b, err := io.ReadAll(resp.Body)
78	if err != nil {
79		debug.Error = err.Error()
80		c.logDebug(debug)
81		return err
82	}
83	debug.Body = string(b)
84	debug.CookieUsed = cookie
85	c.logDebug(debug)
86	return nil
87}

Then the struct I made for the response from "/auth/anon" and a debug output for our login request:

1// file: ./px/structs.go
2// GrubHubAuth is the response from /auth/anon when trying to get the Authorization header for the login endpoint
3type GrubHubAuth struct {
4	Session struct {
5		AuthToken string `json:"access_token"`
6	} `json:"session_handle"`
7}
8
9// LoginDebug holds info used to debug if our cookies are working on the login endpoint or not
10type LoginDebug struct {
11	StatusCode int    `json:"statusCode"`
12	Body       string `json:"body"`
13	CookieUsed string `json:"cookieUsed"`
14	Error      string `json:"error"`
15}

The debug logging is saving JSON files to a folder named "tests", we're using a mutex so that we can have an incrementing file name, I'm sure there's another way to do that but it's the easiest way I could think of and this is just for debug anyways. To add this cookie testing code to the API, we need to add this code to the router:

1// file: bake.go
2err = client.TestCookie(o.Cookie, randStr(20) + "@gmail.com", randStr(10) + "3463456!")
3if err != nil {
4	o.Error = err.Error()
5	c.AbortWithStatusJSON(500, o)
6	return
7}
8
9func randStr(n int) string {
10	b := make([]byte, n)
11	for i := range b {
12		b[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"[rand.Intn(52)]
13	}
14	return string(b)
15}

Here's the part that can be kinda tricky though if you have no expirence and don't want to spend money, to properly test if your cookie generation works, you need to use rotating residential proxies. These don't cost that much, I get mine for 4$ per GB from iproyal.com, however, iproyal.com is known as a pretty low-quality proxy service. When doing scale tests, it's usually best to use high-quality IPs, IPs with high trust scores, those can cost a lot more, I'm not going to use those.

Now that the API is setup and I have my proxies, I'm going to use my proxy intercept to do a few tests, just to make sure my proxies are working and the logins are working first. I'm seeing only 401's which matches what I'm seeing normally, no one wants to spam a button 100 times, luckily, burp has something called "intruder". Using this, you can make it spam requests on multiple threads. If you can't be bothered though, you can just spam the button, even when the button is gray, clicking it will send another request, sending 100 requests only takes a few seconds if you click quickly. Anyways, after you send 100 or so requests, you should have enough samples to see what a bad response and a normal response is. A normal response should be 401, unauthorized, meaning your login creds were wrong. A bad response is 403, forbidden, meaning you aren't allowed to attempt to login, being blocked by PX. On a 401, you should see that you get no response in the body section, on a 403, you should see a response similar to this:

1{
2	"action": "captcha", 
3	"uuid": "xxxx-xxxx-xxxx-xxxx-xxxx", 
4	"vid": "xxxx-xxxx-xxxx-xxxx-xxxx", 
5	"appId": "PXO97ybH4J", 
6	"page": "very long string",
7	"collectorUrl": "https://collector-PXO97ybH4J.perimeterx.net"
8}

Taking a long look at our responses, we do see a lot of captcha responses, this is odd considering our payloads are fine and in-general there isn't much that could really stop us from scaling here.

Troubleshooting

This section of the article isn't really as edited as the rest of it, therefore not as good for beginners in my opinion. It should better show how I personally think when I'm troubleshooting something like this though because this section wasn't really edited at all, it's been left basically exactly as I originally wrote it.

When getting to this section, I won't be able to present concrete numbers, your "detection" aka captcha rates may vary wildly from mine. When looking at my detection rates, I was seeing about 50%, this number does seem quite high, even for PX. If you do remember earlier, I mentioned that PX is known for heavily relying on their captcha, I'm going to expand on that thought a little more here. The PX SDK has been known to be relatively easy, compared to the web protection, and since most websites will have mobile apps (grubhub, walmart, etc.) and the PX SDK is easier to reverse engineer than the web version, it's only logical that a large botting problem came to the PX SDK. This forced PX to just use webview and present a captcha when any slight thing looks wrong, it's very sensitive. This makes our job of troubleshooting a lot harder, are we just feeling the effects of a company that can't detect bots correctly? Did we do something wrong in our payload? Is our TLS wrong? It's hard to tell, this is why I said previously that this is where the job gets difficult.

From this point, a lot of what I'll be saying will purely be based on my experience, I won't be able to guide you to how I came to have these ideas. Firstly, I thought to print the text being sent to PX to console, the final form payloads after they're encoded. I noticed two problems instantly, firstly, the Golang stdlib "url.Values" doesn't persist key order. The second thing I noticed was that PX wasn't using URL encoding, they used text encoding, this means that "=" shouldn't become "%3D", whereas it is in our payloads. I brought this up in the SneakerDev discord and quickly got a response from a user named "justhyped", he showed me a package he made that persists key order when creating forms! From now on, we will use this package, it should be as easy as doing "go get github.com/justhyped/OrderedForm" and replacing "url.Values{}" with "new(OrderedForm.OrderedForm)", then replacing "Encode()" with "URLEncode()". Some of you may be confused, why are we using "URLEncode" when I just said PX isn't using URL encoding? To be honest, it's because I'm lazy, the only difference I really saw was in the "%3D" so I figured I would just use "strings.ReplaceAll(body.URLEncode(), "%3D", "=)" to try and fix that problem. From here, I'm going to add a new field to the debug struct, this will be for holding our payloads, and we will also add this same field to the Client struct temporarily.

1// file: ./px/client.go
2type Client struct {
3	// ...
4	Payloads     []string
5}
6// file: ./px/structs.go
7type LoginDebug struct {
8	// ...
9	Payloads   []string `json:"payloads"`
10}
11// file: ./px/http.go
12// example of new encoding:
13import "github.com/justhyped/OrderedForm"
14body := new(OrderedForm.OrderedForm)
15body.Set("payload", base64.StdEncoding.EncodeToString([]byte("hi")))
16body.Set("uuid", c.UUIDv4)
17body.Set("appId", c.PXID)
18body.Set("tag", "mobile")
19body.Set("ftag", "22")
20finalResult := strings.ReplaceAll(body.URLEncode(), "%3D", "=")
21c.Payloads = append(c.Payloads, finalResult)
22req, err := http.NewRequest("POST", fmt.Sprintf("https://collector-%s.perimeterx.net/api/v1/collector/mobile", c.PXID), strings.NewReader(finalResult))

Now we can do another test, this time I'll only do 50, to save myself some proxy bandwidth. After looking at the payloads, it appears now we need to change "\u0026" to "&", if I was doing a serious project, I would take the time to code my own form encoder, it's just not that serious here. Now we have this ungodly line of code:

1// file: ./px/http.go
2finalResult := strings.ReplaceAll(strings.ReplaceAll(body.URLEncode(), "%3D", "="), "\\u0026", "&")

You should truly never code like this in a serious environment, however, I wanted to make this blog realistic to how I work. When I'm debugging, I don't write good code, I will write code like this all the time when I'm debugging. Regardless, we now should have everything cleaned up, let's run another small test. At this point, I was confused, the \u0026 had stayed, I didn't really think this was the issue anyways but it was strange. I went ahead and tested my code on go.dev/play, a convenient sandbox to run Golang code, and it seemed to be working fine. I figured this must mean that there was a gap in my understanding somewhere and I figured it was in how JSON works. I knew as long as what was being sent to the server was correct, it didn't really matter if it was wrong in our debug logging, so I went ahead and proxy intercepted the API itself. To do this I changed the proxy I was using to match the one I had set in Burp, I also had to add this to my HTTP Client config:

1// file: ./px/client.go
2tlsclient.WithInsecureSkipVerify(),

This allows the client to execute a request with an insecure certificate, without it, you would get this error:

1Post "https://px-conf.perimeterx.net/api/v1/mobile": x509: certificate signed by unknown authority

After looking at our requests, we see it looks all good! My next thought was to look at the headers though. I copied the headers from my request into one file and the headers from a real PX request into another file. I then used pane view in Visual Studio Code so I could see them at the same time, I got rid of the headers I ignored previously, every header looked fine, except one. When doing this, I realized a very minor detail, my host header has a fully uppercase PXID whereas the official request does not. I'm unsure why the host header is uppercase because if you look at the URL, it's fully lowercase, very odd but either way I'll go ahead and lowercase that. We can then go ahead and test again for a lower detection rate. Nothing changed, I got a new idea though, I was going to compare payloads that passed to payloads that failed. When I did this, I only compared 3 passing payloads to 3 failing payloads, so not a large sample size. I did see one thing though, it looked like all of the passing payloads had 3 or more letters in the "PX419" value's random "hex" string, and all of the ones that failed either had 1 letter or 0 letters. This is not a large enough sample size for me to think this change is actually going to work but it's worth a shot. We will do this just by changing the "randomHex" func we made earlier:

1// file: ./px/payload.go
2const hexChars = "abcdef0123456789"
3
4func randomHex(n int, isPx419 bool) string {
5	o := []byte{}
6	numCount := 0
7	for i := 0; i < n; i++ {
8		num := mrand.Intn(16)
9		if isPx419 && numCount == 4 && num > 5 {
10			i--
11			continue
12		}
13		if num > 5 {
14			numCount++
15		}
16		o = append(o, hexChars[num])
17	}
18	return string(o)
19}

There may be a better way to do this but this works for now. There's just one more thing I want to fix before I do another test, while doing the last test, I noticed that I was getting a lot of failed attempts at getting cookies, our debug logging isn't showing us what's happening there, I added the debug logging to all PX requests and it showed me what was happening. It looks like the requests are returning empty instructions, like this:

1{"do":[]}

Sometimes it's only on the first request, then it returns instructions on the second request instead of a cookie, and then sometimes it returns blank instructions again on the second request. What I'm going to do is see how the app handles it. I'm going to use the proxy intercept to intercept the first request and change the response to be the empty body we see. I'll mimic what that does. After looking, it does appear like it is regenerating the payload and resending the requests, which means we will need to handle "PX345" and "PX351" which shouldn't be too bad. Logically, the voltage, charge, temp, kernel version, etc. of the phone shouldn't really change between payloads, therefore we need to modify our existing payload code a bit:

1// file: ./px/payload.go
2// ! First, we need to have a flag that sees if we've already populated the payload before
3populated := p.Cache.StartTime != 0
4if !populated {
5// here we're doing things we can't do again, like set our kernel version for example
6} else {
7	p.D.Px345++
8	p.D.Px351 += time.Now().UnixMilli() - p.Cache.StartTime
9}
10// here we're going about building the payload like normal, screen width/height, timestap, device, etc.

Then modify our router:

1// file: bake.go
2if len(instructions.Do) < 4 {
3	for i := 0; i < 3; i++ {
4		client.PXPayload.Populate(client.DeviceData, levels[fmt.Sprint(client.DeviceData.AndroidVersion)], client.SDKVersion, client.AppVersion, client.AppName, client.PackageName, client.IsInstantApp)
5		err = client.PXPayload.UUIDSection()
6		if err != nil {
7			o.Error = err.Error()
8			c.AbortWithStatusJSON(500, o)
9			return
10		}
11		instructions, err = client.GetInstructions()
12		if err != nil {
13			o.Error = err.Error()
14			c.AbortWithStatusJSON(500, o)
15			return
16		}
17		if len(instructions.Do) == 4 {
18			break
19		}
20	}
21	if len(instructions.Do) != 4 {
22		o.Error = "unable to get instructions"
23		c.AbortWithStatusJSON(500, o)
24		return
25	}
26}

If this request fails after these three tries, it wont make the other failed request, since there would be no reason to, and if it does work, all payloads generated should end up in our debug log! Let's do a small-scale test. Well, my solution for the empty instructions problem looks to have worked, it does seem like the detection rate is the exact same though. At this point, I'm left to believe that there's a few things at play here that aren't really in my control, an inherent weakness I was discussing earlier. Firstly, IP quality could be a factor, I did a test on low threads to see if lock rates would be similar, but they weren't, which leads me to think IP quality is probably not the issue. The fact that I can make 1000 working cookies on low threads but can't on high threads tells me that what's probably happening is I'm just hitting the normal PX captcha, when PX has an influx of users (or bots), everyone trying to use an app starts randomly getting captchas. I wasn't really sure if this was the problem and didn't want to come to this conclusion without being sure, to test this, I went ahead and started a scale test on high threads, getting about a 50-60% detection rate, I then proceeded to open GrubHub and just try to login. Unsurprisingly it did give me a captcha, I then shut down the app, cleared the storage, and repeated the steps, this time I wasn't given a captcha. I didn't solve the captcha it gave me previously so it's not like it whitelisted my IP or something like that, I'm fairly certain what I think is happening is really what's happening.

Conclusion

At this point, the article has covered everything mobile-related in the best way I could. This isn't where you have to stop though, you could solve the captcha, this would involve researching deobfuscation, javascript, webview, etc. That would be the next step in this API though. As for what my next project will be, I'm not entirely sure, I do have some ideas but nothing is set in stone. If you have any ideas for what you would want to see, let me know and I hope you enjoyed the read!

I also wanted to put some further stuff in at the end here now that the article is over. I wanted to keep all of my previous knowledge out of this article as much as possible to make it as beginner-friendly as possible. I do have a lot of experience with PX though, there are also a few more anti-bot related tricks that do apply to PX that I didn't really cover. One of which is that most anti-bots will actually let you pass on a low scale with literally no cookie, so on GrubHub, you can do a login attempt every like 300ms with no PX header and you won't get blocked, it's only once you start getting into high scale that you start feeling the anti-bot. The reason I still used a low-scale test to rule out IP quality is that, though I didn't mention it in the article, I did a few test login attempts on different IPs of different degrees of quality and found that if an IP had too low of a trust score, you would still get blocked even on just 1 login attempt. This is normal for every anti-bot I've worked on.

References

GitHub for this project: github.com/obfio/px-grubhub-mobile PhoneMore Scraper: github.com/obfio/phonemore-scraper Golang: go.dev HTTP Client: github.com/bogdanfinn/tls-client HTTP TLS: github.com/bogdanfinn/utls HTTP fhttp: github.com/bogdanfinn/fhttp PhoneMore: phonemore.com/phones/ Gin: github.com/gin-gonic/gin Ordered Forms: github.com/justhyped/OrderedForm

PerimeterX SDK: Part 2, generator time!

The main reason for reverse engineering an anti-bot, which we did in part 1, is to create a gen. A gen is used to mass create valid cookies, in the case of grubhub, it's used as a header. To create a...