Certificate rejected arbitrarily after showing "System Trust Enabled"

[noxface-case3]

I get to this screen after I run install with adb:

This is the output window for HTTP toolkit

Emulator is user dev build with Magisk installed as root and working fine.

My device is an emulator on Android 12 x86_64 running on Pixel 6 6.4(1080x2400 420 dpi)(no playstore)

It works sometimes(1 in 10 maybe) and then all of a sudden "Certificate rejected". I have verified that my certificate is present in the /system/etc/security/cacerts but for some reason it does not show in the Trusted Certificates section. I tried to run the script manually with burp suite and it got installed in the folder but not showing in the Trusted Certificates on the device.

The format of the certificate is PEM converted to (.0 android) and I observed that the certificate created by us only has certificate begin and end but the certificates that are working example(Starfield Class 2 Certification Authority) has a lot more standard data about the certificate with SHA1 Fingerprint.

Can someone from the team help me in debugging this.

[pimterry]

Does this apply to all apps, or does it work in some apps but not in others?

Is it always the same requests/hostnames that fail?

What Magisk version are you using? Do you have any Magisk modules installed, or anything else unusual that might affect this?

If you create a non-playstore emulator without Magisk you'll have root access via ADB anyway - in that case, does it work correctly? That case (using vanilla emulators directly) is quite heavily tested and I've never seen it fail. It could be that a recent change in how Magisk works has caused this...

Can you please:

• Stop and then cold boot the emulator
• Run adb logcat -T1
• Launch the ADB-based interception from HTTP Toolkit
• Reproduce the issue
• Share both the output from the logcat command and your HTTP Toolkit logs (from Help -> "View HTTP Toolkit logs" in the menu).

[noxface-case3]

Thank you @pimterry for your early reply.

1. It applies to a variety of hosts and requests. Does not have a pattern.
2. Magisk settings in image
3. Safety net global fix module alongside magisk
4. If I create without magisk, I have root access and everything needs to be working

--- Launching HTTP Toolkit desktop v1.14.10 ---
INFO: Initialising UI (version fe91a322e7391b0c60f37b82461674469b5cfa97)
INFO: Account store initialized
INFO: UI store initialized
INFO: Proxy settings loaded
INFO: Server initialization failed TypeError: Failed to fetch
INFO: API store initialized
Config checked in 8 ms
Certificates setup in 8 ms
Standalone server started in 7 ms
Server started in 17 ms
Total startup took 40 ms
(node:6249) Warning: Using insecure HTTP parsing
(Use `node --trace-warnings ...` to show where the warning was created)
Mock session started, http on port 8000, webrtc enabled
Docker not available: connect ENOENT /var/run/docker.sock
INFO: Server started
Webextension installed at /var/folders/0f/k1vpkxb97x713r_349l7p1q40000gp/T/httptoolkit-webextension
INFO: Config loaded
INFO: Server started on port 8000
INFO: Proxy store initialized (server version 1.14.10)
INFO: Rules store initialized
INFO: Events store initialized
INFO: Send store initialized
INFO: Interceptor store initialized
INFO: App started, rendering
httptoolkit-server: Updating CLI... fetching manifest
httptoolkit-server: Updating CLI... already on latest version: 1.14.10
Browser cache updated
INFO: Service worker loaded
httptoolkit-server: Updating CLI... fetching manifest
httptoolkit-server: Updating CLI... already on latest version: 1.14.10
Android command [ 'sh', '/data/local/tmp/htk-root-test.sh' ] returned `shell`
Android command [ 'su', '-c', 'sh /data/local/tmp/htk-root-test.sh' ] returned `root`
Android command [ 'su', '-c', 'sh', '/data/local/tmp/htk-root-test.sh' ] returned `root`
Android command [ 'su', 'root', 'sh', '/data/local/tmp/htk-root-test.sh' ] threw Timeout for ADB command su,root,sh,/data/local/tmp/htk-root-test.sh
Error: Timeout for ADB command su,root,sh,/data/local/tmp/htk-root-test.sh
    at /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:19931
    at runNextTicks (node:internal/process/task_queues:60:5)
    at listOnTimeout (node:internal/timers:540:9)
    at process.processTimers (node:internal/timers:514:7)
    at async Promise.all (index 3)
    at async e.getRootCommand (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:21810)
    at async e.AndroidAdbInterceptor.injectSystemCertIfPossible (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:30737)
    at async e.AndroidAdbInterceptor.activate (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:28565)
    at async e.ApiModel.activateInterceptor (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:890:743387)
    at async /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:3509
    at async /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:2198
Android command [ 'su', 'root', 'sh /data/local/tmp/htk-root-test.sh' ] threw Timeout for ADB command su,root,sh /data/local/tmp/htk-root-test.sh
Error: Timeout for ADB command su,root,sh /data/local/tmp/htk-root-test.sh
    at /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:19931
    at async Promise.all (index 4)
    at async e.getRootCommand (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:21810)
    at async e.AndroidAdbInterceptor.injectSystemCertIfPossible (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:30737)
    at async e.AndroidAdbInterceptor.activate (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:28565)
    at async e.ApiModel.activateInterceptor (/Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:890:743387)
    at async /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:3509
    at async /Applications/HTTP Toolkit.app/Contents/Resources/httptoolkit-server/bundle/index.js:944:2198
Cert already installed, nothing to do
Android command [ 'rm', '-f', '/data/local/tmp/htk-root-test.sh' ] returned ``
Android command [ 'su', '-c', 'sh /data/local/tmp/htk-set-chrome-flags.sh' ] returned `Chrome flags script completed`
Chrome flags script completed

Android command [ 'su', '-c', 'am force-stop com.android.chrome' ] returned ``
Android Chrome flags set
App not installed, installing...
Streaming local APK
INFO: Previous server version was 1.14.10
App installed successfully

03-12 01:46:31.673  7138  7220 D tech.httptoolkit.android.vpn.socket.SocketNIODataService: Registered selector successfully
03-12 01:46:31.673  7138  7220 D tech.httptoolkit.android.vpn.SessionHandler: Send SYN-ACK to client
03-12 01:46:31.689  7138  7220 D tech.httptoolkit.android.vpn.SessionHandler: FIN from vpn client, will ack it.
03-12 01:46:31.690  7138  7220 D tech.httptoolkit.android.vpn.SessionManager: closed session -> TCP|169.254.61.43:56748->192.168.0.104:8000
03-12 01:46:31.690  7138  7220 D tech.httptoolkit.android.vpn.SessionHandler: ACK to client's FIN and close session => 192.168.0.104:8000-169.254.61.43:56748
03-12 01:46:32.076  6397  7451 D PipelineWatcher: onInputBufferReleased: frameIndex not found (7364); ignored
03-12 01:46:32.263  6397  7457 D BufferPoolAccessor2.0: bufferpool2 0x73ad12952168 : 4(32768 size) total buffers - 4(32768 size) used buffers - 1/6 (recycle/alloc) - 78/14512 (fetch/transfer)
03-12 01:46:33.512  6397  7451 D PipelineWatcher: onInputBufferReleased: frameIndex not found (7289); ignored
03-12 01:46:34.223  6397  7473 D BufferPoolAccessor2.0: bufferpool2 0x73ad12965998 : 4(8388608 size) total buffers - 4(8388608 size) used buffers - 0/4 (recycle/alloc) - 66/7306 (fetch/transfer)
03-12 01:46:34.677   425   545 I AudioFlinger: BUFFER TIMEOUT: remove(66) from active list on thread 0x73aad966aa70
03-12 01:46:34.882   497  7478 D BufferPoolAccessor2.0: bufferpool2 0x7f2f860193d8 : 6(24576 size) total buffers - 1(4096 size) used buffers - 14353/14359 (recycle/alloc) - 110/14358 (fetch/transfer)
03-12 01:46:34.892   425   545 D AudioFlinger: mixer(0x73aad966aa70) throttle end: throttle time(1)
03-12 01:46:34.962  6397  7467 W AudioTrack: restartIfDisabled(28): releaseBuffer() track 0x73ad42b160b0 disabled due to previous underrun, restarting
03-12 01:46:34.966  6397  7477 I CCodecConfig: query failed after returning 19 values (BAD_INDEX)
03-12 01:46:34.966  6397  7477 W Codec2Client: query -- param skipped: index = 1342179345.
03-12 01:46:34.967  6397  7477 W Codec2Client: query -- param skipped: index = 2415921170.
03-12 01:46:34.967  6397  7477 W Codec2Client: query -- param skipped: index = 1610614798.
03-12 01:46:35.306  6397  7473 W MediaCodec: mapFormat: no mediaType information
03-12 01:46:36.294   425   545 D AudioFlinger: mixer(0x73aad966aa70) throttle end: throttle time(13)
03-12 01:46:36.564  6397  7443 D BufferPoolAccessor2.0: bufferpool2 0x73ad12945a88 : 4(8388608 size) total buffers - 4(8388608 size) used buffers - 0/4 (recycle/alloc) - 48/7473 (fetch/transfer)
03-12 01:46:36.625   497  7459 D BufferPoolAccessor2.0: bufferpool2 0x7f2f86013068 : 6(24576 size) total buffers - 1(4096 size) used buffers - 14701/14710 (recycle/alloc) - 129/14709 (fetch/transfer)
03-12 01:46:36.735  6397  7477 D BufferPoolAccessor2.0: bufferpool2 0x73ad129b20e8 : 4(32768 size) total buffers - 4(32768 size) used buffers - 1/6 (recycle/alloc) - 81/14462 (fetch/transfer)
03-12 01:46:36.802   425   545 D AudioFlinger: mixer(0x73aad966aa70) throttle end: throttle time(3)
03-12 01:46:37.218   425   545 D AudioFlinger: mixer(0x73aad966aa70) throttle end: throttle time(3)
03-12 01:46:37.688   425   545 I AudioFlinger: BUFFER TIMEOUT: remove(65) from active list on thread 0x73aad966aa70
03-12 01:46:37.797  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#491:1D-Input.Impl[N]] codec released a buffer owned by client (index 0)
03-12 01:46:37.803  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#491:1D-Input.Impl[N]] codec released a buffer owned by client (index 2)
03-12 01:46:38.022  6397  7395 W AudioTrack: restartIfDisabled(27): releaseBuffer() track 0x73ad42ae3c00 disabled due to previous underrun, restarting
03-12 01:46:38.038  6397  7457 I CCodecConfig: query failed after returning 19 values (BAD_INDEX)
03-12 01:46:38.038  6397  7457 W Codec2Client: query -- param skipped: index = 1342179345.
03-12 01:46:38.038  6397  7457 W Codec2Client: query -- param skipped: index = 2415921170.
03-12 01:46:38.039  6397  7457 W Codec2Client: query -- param skipped: index = 1610614798.
03-12 01:46:38.041  6397  7457 D BufferPoolAccessor2.0: bufferpool2 0x73ad12952168 : 4(32768 size) total buffers - 4(32768 size) used buffers - 1/6 (recycle/alloc) - 78/14722 (fetch/transfer)
03-12 01:46:38.337  6397  7443 W MediaCodec: mapFormat: no mediaType information
03-12 01:46:38.599  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#460:1D-Input.Impl[N]] codec released a buffer owned by client (index 0)
03-12 01:46:38.842  6397  7451 D PipelineWatcher: onInputBufferReleased: frameIndex not found (7416); ignored
03-12 01:46:38.931   425   545 D AudioFlinger: mixer(0x73aad966aa70) throttle end: throttle time(22)
03-12 01:46:39.098  6397  7451 D CCodecBuffers: [c2.goldfish.h264.decoder#811:1D-Input.Impl[N]] codec released a buffer owned by client (index 0)
03-12 01:46:39.817  6397  7473 D BufferPoolAccessor2.0: bufferpool2 0x73ad12965998 : 4(8388608 size) total buffers - 4(8388608 size) used buffers - 0/4 (recycle/alloc) - 66/7441 (fetch/transfer)
03-12 01:46:40.013   497  7478 D BufferPoolAccessor2.0: bufferpool2 0x7f2f860193d8 : 6(24576 size) total buffers - 1(4096 size) used buffers - 14607/14613 (recycle/alloc) - 114/14612 (fetch/transfer)
03-12 01:46:40.608  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#460:1D-Input.Impl[N]] codec released a buffer owned by client (index 1)
03-12 01:46:41.243  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#491:1D-Input.Impl[N]] codec released a buffer owned by client (index 0)
03-12 01:46:41.611  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#491:1D-Input.Impl[N]] codec released a buffer owned by client (index 2)
03-12 01:46:41.706  7138  7220 D tech.httptoolkit.android.vpn.transport.tcp.TCPPacketFactory: Set Initial Sequence number: 1500277287
03-12 01:46:41.716  7138  7220 D tech.httptoolkit.android.vpn.SessionManager: created new SocketChannel for TCP|169.254.61.43:58594->192.168.0.104:8000
03-12 01:46:41.717  6397  7443 D BufferPoolAccessor2.0: bufferpool2 0x73ad12945a88 : 4(8388608 size) total buffers - 4(8388608 size) used buffers - 0/4 (recycle/alloc) - 50/7579 (fetch/transfer)
03-12 01:46:41.727  7138  7220 D tech.httptoolkit.android.vpn.SessionManager: Protected new SocketChannel
03-12 01:46:41.727  7138  7220 D tech.httptoolkit.android.vpn.SessionManager: Initiate connecting to remote tcp server: /192.168.0.104:8000
03-12 01:46:41.733  7138  7220 I tech.httptoolkit.android.vpn.socket.SocketNIODataService: Registering new session: Session (TCP|169.254.61.43:58594->192.168.0.104:8000)
03-12 01:46:41.734  7138  7220 D tech.httptoolkit.android.vpn.socket.SocketNIODataService: Registered selector successfully
03-12 01:46:41.735  7138  7220 D tech.httptoolkit.android.vpn.SessionHandler: Send SYN-ACK to client
03-12 01:46:41.739  6397  7451 D CCodecBuffers: [c2.android.aac.decoder#491:1D-Input.Impl[N]] codec released a buffer owned by client (index 1)
03-12 01:46:41.758  7138  7221 D tech.httptoolkit.android.vpn.socket.SocketChannelReader: End of data from remote server, will send FIN to client
03-12 01:46:41.758  7138  7221 D tech.httptoolkit.android.vpn.socket.SocketChannelReader: send FIN to: Session (TCP|169.254.61.43:58594->192.168.0.104:8000)
03-12 01:46:41.759  7138  7221 D tech.httptoolkit.android.vpn.socket.SocketChannelReader: removing aborted connection -> Session (TCP|169.254.61.43:58594->192.168.0.104:8000)

I hope this helps.

[pimterry]

This is really interesting, and I've done some digging but I haven't been able to work out what could cause this... There's nothing obviously wrong in the logs there, but this clearly isn't working correctly.

Can you share full detailed steps to set up an equivalent emulator, so I can see this for myself?

E.g.

1. Open Android Studio (version X)
2. Create an emulator with OS version Y and launch it
3. Root the emulator by installing Z from https://... and running some-command

As specific as possible to make sure that the environment exactly matches would be great.

If you can share enough info that I can set up an identical emulator myself, from there it should be much easier to work out what's going on, and get a proper working fix for you.