First step towards implementing #JWT in #swad done, just committed a good 1000 LOC and now my #poser lib can do #JSON 😎
https://github.com/Zirias/poser/commit/7f1772e85c869d544f8a12099ed6545e163dc163

Seems a first step is almost done, adding #JSON support to my #poser lib. This could be the foundation for #JWT support in #swad. 😎

Need to do more thorough testing I guess, but at least the two example documents from #rfc8259 work fine ... the test tool does a full #deserialization / #serialization roundtrip (with specific internal representations of the data types supported by JSON).

edit: Look at the "Longitude" value of the second object in the second example 😏 I only noticed myself right now, but of course that's the desired behavior.

Now that #swad 0.7 is released, it's time to prepare a new release of #poser, my own lib supporting #services on #POSIX systems, following a #reactor with #threadpool design.

During development of swad, I moved poser from using strictly only POSIX APIs (with the scalability limits of e.g. #select) to auto-detected support for #kqueue, #epoll, #eventports, #signalfd and #timerfd (so now it could, in theory(!), "compete" with e.g. libevent). I also fixed quite some hidden bugs, and added more base functionality, like a #dictionary using nested hashtables internally, or #async tasks mimicking the async/await pattern known from e.g, #csharp. I also deprecated two features, the periodic and global "service tick" (superseded by individual timers) and the "resolve hosts" property of a "connection" (superseded by a separate resolve class).

I'll have to decide on a few things, e.g. whether I'll remove the deprecated stuff immediately and bump the major version of the "posercore" lib. I guess I'll do just that. I'd also like to add all the web-specific stuff (http 1.0/1.1 server) that's currently part of the swad code as a "poserweb" lib. This would get a major version of 0, indicating a generally unstable API/ABI as of now....

And then, I'd have to decide where certain utility classes belong to. The rate limiter is probably useful for things other than web, so it should probably go to core. What about url encoding/decoding, for example? 🤔

Stay tuned, something will come here, maybe helping you to write a nice service in plain #C 😎:

https://github.com/Zirias/poser

The next release of #swad will probably bring not a single new feature, but focus on improvements, especially regarding #performance. Support for using #kqueue (#FreeBSD et al) to handle #signals is a part of it (which is done and works). Still unsure whether I'll also add support for #Linux' #signalfd. Using kqueue also as a better backend for #timers is on the list.

Another hopefully quite relevant change is here:

https://github.com/Zirias/poser/commit/798f23547295f89fa0c751f0e707c3474b5c689c

In short, so far my #poser lib was always awaiting readiness notification (from kqueue, or #epoll on Linux, or select/poll for other platforms) before doing any read or write on a socket. This is the ideal approach for reads, because in the common case, a socket is NOT ready for reading ... our kernel must have received something from the remote end first. But for writes, it's not so ideal. The common case is that a socket IS ready to write (because there's space left in the kernel's send buffers). So, just try it, and only register for notifications if it ever fails, makes more sense. Avoids pointless waiting and pointless events, and e.g. with epoll, even unnecessary syscalls. 😉

I'm trying to add "genric" #signal handling to #poser. Ultimate goal is to provide a way for #swad to handle #SIGHUP, although signal handling must be done in poser's main event loop (signals are only ever unblocked while waiting for file descriptor events).

Okay, I could just add explicit handling for SIGHUP. But a generic solution would be nicer. Just for example, a consumer might be interested in #SIGINFO which doesn't even exist on all platforms ... 🤔

Now, #POSIX specs basically just say signal constants are "integer values". Not too helpful here. Is it safe to assume an upper bound for signal numbers on "real world" OS implementations, e.g. 64 like on #Linux? Should I check #NSIG and, if not defined, just define it to 64? 🙈

#C #coding #question

I finally eliminated the need for a dedicated #thread controlling the pam helper #process in #swad. 🥳

The building block that was still missing from #poser was a way to await some async I/O task performed on the main thread from a worker thread. So I added a class to allow exactly that. The naive implementation just signals the main thread to carry out the requested task and then waits on a #semaphore for completion, which of course blocks the worker thread.

Turns out we can actually do better, reaching similar functionality like e.g. #async / #await in C#: Release the worker thread to do other jobs while waiting. The key to this is user context switching support like offered by #POSIX-1.2001 #getcontext and friends. Unfortunately it was deprecated in POSIX-1.2008 without an obvious replacement (the docs basically say "use threads", which doesn't work for my scenario), but still lots of systems provide it, e.g. #FreeBSD, #NetBSD, #Linux (with #glibc) ...

The posercore lib now offers both implementations, prefering to use user context switching if available. It comes at a price: Every thread job now needs its private stack space (I allocated 64kiB there for now), and of course the switching takes some time as well, but that's very likely better than leaving a task idle waiting. And there's a restriction, resuming must still happen on the same thread that called the "await", so if this thread is currently busy, we have to wait a little bit longer. I still think it's a very nice solution. 😎

In any case, the code for the PAM credential checker module looks much cleaner now (the await "magic" happens on line 174):
https://github.com/Zirias/swad/blob/57eefe93cdad0df55ebede4bd877d22e7be1a7f8/src/bin/swad/cred/pamchecker.c

#C #coding

On a #coding mission to improve my #poser lib 😎.

In the current implementation of #swad, I don't really like that I need an extra thread, just to control a child #process. A first piece to add to poser is generic "child process support", which I'm testing right now. I realized I could reuse my #Connection class, which was built for #sockets, but works just as well with #pipes 🙃

TODO now is mostly testing. See screenshots for some mimimal testing code and its output ... would you like this kind of interface? 🤔

#C #programming

I revisited that, AGAIN. Getting #random data in #poser now has yet another fallback, in case we don't have #arc4random and we also don't have #getrandom: read from /dev/random and/or /dev/urandom, "old style" 🙈. Still better to try this before resorting to a simple little #xorshift.

In the best case — arc4random found — this is still all the code of PSC_Random_bytes() 😆:

arc4random_buf(buf, count);
return count;

https://zirias.github.io/poser/api/latest/class_p_s_c___random.html

#C #coding

More #poser improvements:

* Use arc4random() if available, avoids excessive syscalls just to get high-quality random data
* Add a "resolver" to do #reverse#DNS lookups in a batch, remove the reverse lookup stuff from the connection which was often useless anyways, when a short-lived connection was deleted before resolving could finish 🙈

As a result, #swad can now reliably log requests with reverse lookups enabled 🥳

#C #coding

Still working on #swad, and currently very busy with improving quality, most of the actual work done inside my #poser library.

After finally supporting #kqueue and #epoll, I now integrated #xxhash to completely replace my previous stupid and naive hashing. I also added a more involved #dictionary class as an alternative to the already existing #hashtable. While the hashtable's size must be pre-configured and collissions are only ever resolved by storing linked lists, the new dictionary dynamically nests multiple hashtables (using different bits of a single hash value). I hope to achieve acceptable scaling while maintaining also acceptable memory overhead that way ...

#swad already uses both container classes as appropriate.

Next I'll probably revisit poser's #threadpool. I think I could replace #pthread condition variables by "simple" #semaphores, which should also reduce overhead ...

https://github.com/Zirias/swad

#c #coding