PEM/DER certificate and key related functions in the public mongoose.h API

[ChristianSchulte]

Hi,

I am searching for help on how to use the function

void mg_hmac_sha256(uint8_t dst[32], uint8_t *key, size_t keysz, uint8_t *data, size_t datasz);

Is there a way to load the needed key and keysz data from an EC PRIVATE KEY pem certificate using only the mongoose API, or am I heading into the wrong direction with this? All crypto related functions implementing MG_TLS_BUILTIN seem to be declared static. My use case is creating a JWT Authorization: Bearer HTTP header. Currently I see no other way of loading the private key from a PEM file using a 3rd party library like OpenSSL. Mainly what mg_tls_init does when setting up certificates and keys in struct tls_data but using the public mongoose.h API. Any pointers highly appreciated.

Thanks,
Christian

[scaprile]

tls_builtin is not a general purpose library but TLS 1.3 support for Mongoose. Some functions are not static, but don't have a documented public API because we didn't intended them to be user callable.

mg_hmac_sha256() is in the SHA256 module and is not publicly documented.
It is called from other modules, where an HMAC is needed, and there you can see how it is called. I don't know what else I could add to that.
The way to load stuff from a key and/or a cert is the way it is done inside tls_builtin functions, we never intended to have another use for keys or certs than being initialized by tls_init at connect time, to be handled during the TLS handshake by the handshake functions.

Given the current "trend" on super security for everything, we might see the need to create an API for other types of HTTP authentication... maybe. As long as your case can be done with the building blocks we have, I can try to guide you. I guess some of the current stuff has to be moved to a function that can be called from within its current place and this new authentication helper.
I'm working on how that can be done with minimum impact. Let me know exactly what you need so we can do our side. I'm not familiar with JWT so be explicit.
Are you doing this for device login ?

[scaprile]

Added a key loading helper and minimal JWT infrastructure with a mock example in branch https://github.com/cesanta/mongoose/tree/jwt
Let me know if you can work from there and want to add your work as a PR by signing the CLA, we can reuse this branch for the PR

[ChristianSchulte]

Seems all functionality is already there. Having a way to load keys and certificates over a stable API seems to be the only "missing" link to make use of the internal uecc as is without having to write a bunch of boilerplate code. I haven't had the time to read the sources in detail but am having a working demo showing what I am heading for.

The internal uecc has this comment A correct RNG function is set by default when...

Mongoose does not seem to follow that, so I added an additional step during initialization to set that up like:

static int uecc_random(uint8_t *dest, unsigned size) {
  if (!mg_random(dest, size))
    fatal("%s", "No PRNG");
  return 1;
}

mg_boot_timestamp_ms = (uint64_t)(epoch_ms - mg_ms);
mg_log_set(MG_LL_NONE); // NONE, ERROR, INFO, DEBUG, VERBOSE
mg_uecc_set_rng(uecc_random);

For an easy way to test things I removed the 'static' qualifier from function

int mg_parse_pem(const struct mg_str pem, const struct mg_str label, struct mg_str *der);

to have a way to load an EC private key from a PEM file quickly. So loading the key currently looks as ugly like

  if (mg_parse_pem(cnf_key, mg_str("EC PRIVATE KEY"), &key) != 0) {
    // No EC PRIVATE KEY;
    return -1;
  }

  // MG_TLS_BUILTIN: mg_tls_init()
  // expect ASN.1 SEQUENCE=[INTEGER=1, BITSTRING of 32 bytes, ...]
  // 30 nn 02 01 01 04 20 [key]
  if (key.len < (2 + 5 + 32) || key.buf[0] != 0x30 ||
      (key.buf[1] & 0x80) != 0 ||
      memcmp(key.buf + 2, "\x02\x01\x01\x04\x20", 5) != 0) {
    // Invalid EC PRIVATE KEY ASN.1
    return -1;
  }

The final JWT can then be created like

  mg_sha256(sha256, (uint8_t *)buf, r);

  r = mg_uecc_sign(ec_key, sha256, 32, (uint8_t *)buf, mg_uecc_secp256r1());

using a base64 wrapper function for URI encoding

static size_t mg_base64_encode_uri(const unsigned char *restrict p,
                                   size_t n, char *restrict buf,
                                   size_t len) {
  const size_t b64_len = mg_base64_encode(p, n, buf, len);

  if (b64_len == 0)
    return 0;

  char *restrict c = buf;

  while (*c) {
    switch (*c) {
    case '=':
      *c = '\0';
      return c - buf;
    case '+':
      *c = '-';
      break;
    case '/':
      *c = '_';
      break;
    }
    c++;
  }

  return b64_len;
}

Having a way to just load a key from whatever format is provided (like in mg_tls_init) really seems the only thing needed. Not sure if this is of general interest, though.

[scaprile]

Did you actually check what I provided in that branch ?

[scaprile]

Ehm... looks like I was too distracted and forgot to commit before pushing... now that branch has been updated with the actual contents...
It provides the necessary rng stuff, you don't need to do anything unless you want to, and it shows some "standard" JWT stuff. Not your specific signing, but you can still do it; and there's of course support for you to get your key. Let me know.
https://github.com/cesanta/mongoose/compare/jwt?expand=1

[ChristianSchulte]

When I looked at that branch it was in sync with master without any changes. I now had time to look at it. You did all of that in just a couple of hours? Wow. Some notes.

Function jwt_header_payload hard codes the header values to arg and typ which is a sane default but would make it impossible for me to use the jwt_xy functions due to needing additional header values like kid.

RFC7515 - Section 4.1.4

Looking at that section others may need other values as well. Adding a struct mg_str header argument in front of payload to functions mg_jwt_sign_hs256 and mg_jwt_sign_es256 would do it, but has the risk of allowing to pass invalid headers like "alg":"ES256"to function mg_jwt_sign_hs256. Simple and pragamtic, though.

[scaprile]

Yeah, my fault, sorry about that.
Thanks, we'll check that, looks like it is aligned to usual Mongoose usage.
Stay tuned for a next post here

[scaprile]

@ChristianSchulte What about now ? More in the classic Mongoose style of things, closer to how TLS and MQTT (and even HTTP) work, I think.

[ChristianSchulte]

Looks great and aligns nicely with mg_tls_opts. I just replaced a former function creating the jwt message based on OpenSSL to one based on this API. It reduces the lines of code from 168 to 15 including error handling. Neat. Well done. Thanks a lot.

[scaprile]

Cool! Could you comment on your usage scenario, if possible, so our marketing folks can have some new stuff to work on ? Thank you.

[ChristianSchulte]

I give it a try. The usage scenario is quite simple on a first look - then became more and more complex over time and lastly a maintenance nightmare. Login to a REST API using JWT. Consume a stream of samples over a websocket to process in real time. First versions were using curl. That soon turned out to be - albeit simple in use - too unflexible when it comes to detecting connection stalls, or things as simple as reconnecting on stall or timeout. No support for websockets. When the server moved from a simple key/value auth scheme to JWT, we first threw in libjwt but the number of 3rd party dependencies grew over time. Just add another license to the LICENSES.txt file soon was no longer an option. Sadly most of the libraries we looked at themselves depend on other things like JSON parsers, SSL libraries, etc. All with theire own licensing and API. When searching for a turn-key ready one-stop solution providing all the needed use cases with a stable and unified API allowing to just not care about the target platform to compile for, Mongoose looked very promising. The documentation on the website made migration as straightforward as can be. Mongoose turned out to be the go to solution for us. Builds without warnings on the target platforms involved (Linux, Mac, Windows) using the native OS compilers like MSVC or Apple clang. Provides a sane API abstracting away all portability things like POSIX vs. Windows vs. whatsoever in a clean and intuitive design. Knows about certain OS level features like epoll on Linux. None of the options we looked at (libevent, libjwt, libwebsocket, libcurl) provided the correct abstractions with a clean API. They were either too simple not allowing fine grained control over the full connection lifecycle, too bloated by still being too unflexible, or just not portable. Mongoose, with the new jwt functions added, allows us to remove all dependencies on all other 3rd party libraries. With these jwt functions and the builtin TLS support, the application even no longer needs to link to -lssl and not even to -lcrypto anymore. Drop two files into the project - compile - test - ship - and be done with it. Not only does it spare lots of developer time. It also takes away that maintenance burdon when you have to keep various inter dependent libraries in sync in a compatible way. Something along those lines.

[scaprile]

Wow, that's quite cool and considerate, thank you !
In my defense, actually, I meant to ask for the application itself, a brief overview on why JWT Bearer Authentication. Thanks a lot, this is great, they'll love it !
BTW, the branch has turned into a PR, #3642, and its being reviewed, possibly making it into 7.23

[scaprile]

Is it OK for this to be added to our Testimonials page (https://mongoose.ws/testimonials/)? If so, please drop us a message on support at cesanta.com.
Thank you again !