Failure to send POST requests over SSL with TinyGSM

[reusables-official]

Hello, I am observing repeated failures trying to send a post request via SSLClient using my SIMCOM7600G modem with TinyGSM.

The most common and consistent error message:

[318168][E][ssl__client.cpp:401] init_tcp_connection(): Connection to server failed, is the signal good, server available at this address and timeout sufficient? reusables.hyperether.com:443
[319175][E][ssl__client.cpp:46] _handle_error(): [start_ssl_client():371]: (-2) BIGNUM - An error occurred while reading from or writing to a file
[319177][E][SSLClient.cpp:242] connect(): start_ssl_client failed: 0

I have tried to set headers to adjust the time out like so:

client.connectionKeepAlive();
client.setHttpWaitForDataDelay(5000);
client.sendHeader("Keep-Alive", "timeout=1000, max=5000");
int err = client.post(API_ENDPOINTS.container, "application/json", body);

where body is a relatively small json payload:

{
  "mode": "return-containers",
  "data": {
    "containerIds": [
      "64367339246bf8633c782348"
    ],
    "locationId": "6621a0099cf68973c58f68bf"
  }
}

Here is my setup code:

#define TINY_GSM_MODEM_SIM7600
#define TINY_GSM_RX_BUFFER 2048

#include <TinyGsmClient.h>
#include <ThingerTinyGSM.h>

HardwareSerial gsmSerial(1);
TinyGsm modem(gsmSerial);
TinyGsmClient gsmClient(modem, 0);

SSLClient secure_layer_lte(&gsmClient);
HttpClient lte_ssl_client = HttpClient(secure_layer_lte, API_HOST, PORT);

void lteInit() {
  gsmSerial.begin(BAUD_RATE, SERIAL_8N1, LTE_RX, LTE_TX, false);
  secure_layer_lte.setCACert(api_root_ca);
  modem.init();
  modem.setNetworkMode(38);
  modem.sendAT("AT+CMNB=1");
  modem.gprsConnect(APN);
};

Then I use a RTOS task to monitor the connection and reconnect as needed:

void lteConnect(void * parameter) {
  lteInit();
  for (;;) {
    if (modem.isNetworkConnected()) {
      if (modem.isGprsConnected()) {
        vTaskDelay(20000 / portTICK_PERIOD_MS);
      } else {
        modem.gprsConnect(APN);
      };
    } else {
      bool networkConnected = modem.waitForNetwork();
    };
    vTaskDelay(5000 / portTICK_PERIOD_MS);
  };
  vTaskDelete(NULL);
};

What confuses me the most is that sometimes the request will go through and I'll get a 200 back as if nothing is wrong, but the vast majority of the time, I get the error pasted above.

I have run a test on signal quality to assess, and I will get a mix of 99 back as a value and anything from 28-31 at other points. Running on an interval of one second might produce output like: [99, 31, 31, 31, 99, 27, 99, 99, 99, 28] which also confuses me. My understanding is that a value of 99 for signal quality from TinyGSM means it could not retrieve the value, 31 is the best quality possible and that a value of 0 would mean there was no reception at all.

Any ideas what I'm doing wrong or inconsistently? I know the error is being thrown by this library, but very possible some other culprit is to blame here. I have the same functionality working perfectly over WiFi, where I provide the WiFi client to the "client.post" function I pasted above, also with SSL also via this library.

Very puzzled. Any thoughts greatly appreciated. I commented on a recent PR for this library that I thought might address the issue but it has not been resolved in my case, I am on version 1.2.0.

Platform.ini

[env:esp32-s3-devkitc-1]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
monitor_speed = 115200
monitor_filters = esp32_exception_decoder
lib_deps = 
	bblanchon/[email protected]
	vshymanskyy/[email protected]
	makuna/[email protected]
	arduino-libraries/[email protected]
	digitaldragon/[email protected]
	thinger-io/[email protected]
	bblanchon/[email protected]

[reusables-official]

Spent the day trying to tweak things, running examples from this repository. Still seeing totally variable results, some calls succeed, most fail. I've tried interacting with different servers like the vsh.pp.ua from tiny GSM repo to the same effect. Using a client without SSL seems to have the same effect so, really, I don't think the issue I'm seeing is rooted in this repository.

[kgmuzungu]

I see something similar on a SIMCom A7670E modem with a ESP32. If I restart the modem, e.g. every 30s, to send a bunch of IoT data, then it works fine. But if I try to keep the connection alive, some HTTPS POST work, but mostly not.

The worst of the sslClient error is that it blocks until, my guess, some underlying TCP/IP stack function times out. That takes some 30sec. It would be great being able to set a shorter timeout! The response code I get then for my HTTPS request is -3... probably an error code from TinyGSM.

@reusables-official did you get any further?
@reusables-official one guess is that the modem doesnt get enough current, hence for a send burst it can drag more than 2A. That could result in a voltage drop at the feeding voltage below specs level. I dont have a oscilloscope at hand currently to confirm that.

[reusables-official]

@kgmuzungu Interesting theory. We moved on from using this SSL Client and instead were using a fork of the TinyGSM library that had "native" SSL support. I think there is a link to it in the issues of TinyGSM repository.

We also eventually stopped doing that and offloaded all of our networking on to our IoT cloud provider (thinger.io) and generally find this much more reliable.

[kgmuzungu]

Thank you for the hints. Native SSL is not working with all firmware versions of the SIMComA7670E.... and I havnt found how to upgrade/change the firmware.

I am still looking for possiblities to get SSLClient working with A7670E 🙏