Heads up to everyone, I made a small release of lua-http yesterday.

lua-http is a library that hopes to provide all HTTP operations you should ever need. In addition to all the low-level parsing and serialising operations you might expect, lua-http provides high level interfaces for writing your own HTTP server or performing HTTP requests.

This was a minor release 2 years on from 0.3 that fixes a few easy bugs and improves Lua 5.4 support. Notable inclusions in this release:

• Support multiple elliptic curves under OpenSSL 1.1.1+ (#150)
• Improve support for Lua 5.4 (not longer require bit library to be installed) (#180)
• Ignore delayed RST_STREAM frames in HTTP 2 (#145)

Links:

• Release page: https://github.com/daurnimator/lua-http/releases/tag/v0.4
• Homepage: https://daurnimator.github.io/lua-http/
• Documentation: https://daurnimator.github.io/lua-http/0.4

I’m happy to announce the second release of lua-http.

lua-http is a library that hopes to provide all HTTP operations you should ever need. In addition to all the low-level parsing and serialising operations you might expect, lua-http provides high level interfaces for writing your own HTTP server or performing HTTP requests.

This is mostly a bugfix release. Included in this release:

• Remove broken http.server .client_timeout option (replaced with .connection_setup_timeout)
• Fix http1 pipelining locks
• Miscellaneous http2 fixes
• HTTP 2 streams no longer have to be used in order of creation
• No longer raise decode errors in hpack module
• Fix hpack:lookup_index() to treat static entries without values as empty string
• Fix HTTP 1 client in locales with non-“.” decimal separator
• Add h1_stream.max_header_lines property to prevent infinite list of headers
• New .bind option for requests and http.client module

Links:

• Release page: https://github.com/daurnimator/lua-http/releases/tag/v0.2
• Homepage: https://daurnimator.github.io/lua-http/
• Documentation: https://daurnimator.github.io/lua-http/0.2

I’m happy to announce the first release of mmdblua.

mmdblua is a lua library to read MaxMind’s Geolocation database format. This database is the most common source of Geo-IP data; you’ll probably want to download a database from Maxmind.

The mmdblua library was something I originally wrote back during the 2013 lua workshop as a quick side project, but as I’ve recently found out there are quite a few people and projects using it. Recognising this, I did a bit of code cleanup and figured I should tag a release. I’m sorry but I haven’t had time to write documentation for the library yet, so it won’t be super friendly for new users.

Links:

• This release: https://github.com/daurnimator/mmdblua/releases/tag/v0.1
• Homepage: https://github.com/daurnimator/mmdblua
• On luarocks: https://luarocks.org/modules/daurnimator/mmdblua
• Free GeoLite2 database: https://dev.maxmind.com/geoip/geoip2/geolite2/

I’m happy to announce the first release of lua-http!

lua-http is a library that hopes to provide all HTTP operations you should ever need. In addition to all the low-level parsing and serialising operations you might expect, lua-http provides high level interfaces for writing your own HTTP server or performing HTTP requests. I’d endeavoured to make behaviour as configurable as possible yet have sensible defaults.

Features:

• Support for HTTP versions 1, 1.1 and 2 (yes! HTTP/2 support)
• Provides both client and server APIs
• Friendly request API with sensible defaults for security
• All operations are fully non-blocking and can be managed with cqueues
• Support for WebSockets (client and server), including ping/pong, binary data transfer and TLS encryption.
• Transport Layer Security (TLS) - lua-http supports HTTPS and WSS via luaossl.
• luasocket compatibility API if you’re looking to use lua-http with older projects.

Check the manual to learn the API, or read through and run the samples in the ‘examples/’ directory in the source tree.

This release has been a long time in the making! For an unreleased library it already has over a hundred stars on Github, and has had contributions from 5 others. I’d like to give a special thanks to @RussellHaley, @ChickenNuggers, @leafo, @vavrusa and @torhve for being early users and providing feedback, as well as @wahern for dealing with all my cqueues and luaossl feature requests.

Links:

• Release page: https://github.com/daurnimator/lua-http/releases/tag/v0.1
• Homepage: https://github.com/daurnimator/lua-http
• Documentation: https://daurnimator.github.io/lua-http/
• Bug tracker: https://github.com/daurnimator/lua-http/issues
• On luarocks: https://luarocks.org/modules/daurnimator/http

I’m happy to announce the 0.4 release of lpeg_patterns.

lpeg_patterns is a collection of patterns I’ve written for various widely used formats. Current sub-modules are: IPv4, IPv6, email addresses, http, language, phone numbers, uri.

This release includes:

• Reduce memory usage by refactoring IPv6 pattern Also fixes lpeg 0.10 compatability
• http: case-normalise cookie attribute names
• http: fix captures of Via header
• http: fixes some whitespace rules
• uri: fix missing case-normalisation for percent encoded characters in hostnames
• uri: export IP_literal and sub_delims patterns
• Lots more. check the git log for the details

Project homepage

lpeg_patterns is available via luarocks

This is a project of mine that has been sitting without a release for a long time now. With luapgsql recently getting the last fixes I was waiting for, I figured it’s time to do a proper release.

cqueues-pgsql is a library that wraps luapgsql to be non-blocking when in a cqueues managed thread. The API should be the same as luapgsql; if it diverges, file a bug with me :)

Homepage: https://github.com/daurnimator/cqueues-pgsql
Available on luarocks: https://luarocks.org/modules/daurnimator/cqueues-pgsql/

Q. What is luapgsql?

A. Bindings to libpq, i.e. a postgresql client.
Homepage: https://github.com/arcapos/luapgsql/

Q. What is cqueues?

A. A library for cooperative threading in lua.
Homepage: http://25thandclement.com/~william/projects/cqueues.html
Github: https://github.com/wahern/cqueues

Q. When do I want to use cqueues-pgsql?

A. When you want to talk to postgres without blocking your entire application/thread.

Q. So I can only use it in a cqueues managed thread/application?

A. No! if used outside of a cqueues managed thread, cqueues-pgsql will appear to operate in a blocking manner. i.e. it will look and behave just like luapgsql.

Q. Show me an example!

A. That’s not a question, but okay:

$ lua
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio
> require"cqueues_pgsql".connectdb"":exec("SELECT 42"):getvalue(1,1)
42

I’m happy to announce the 0.3 release of lpeg_patterns.

lpeg_patterns is a collection of patterns I’ve written for various widely used formats. Current sub-modules are: IPv4, IPv6, email addresses, http, language, phone numbers, uri.

This release includes:

• New http module
  • Includes parsers for almost every HTTP header.
  • Expect this API to be unstable, it’s a significant amount of new code
• New language module that parses language codes e.g. zh-Hans-CN
• New email.mailbox pattern (name + email like: Bob <bob@smith.com>)
• New uri.absolute_uri pattern that does not allow fragments
• Expose some previously internal uri patterns
• Fix: Don’t percent decode in URIs when it could introduce ambiguity (thanks @torhve)
• • Lots more. check the git log for the details

Homepage: https://github.com/daurnimator/lpeg_patterns

It’s available via luarocks: https://luarocks.org/modules/daurnimator/lpeg_patterns

It seems that every dev environment/framework wants to be in charge of your program’s main loop.

Prompted by a question of how to add an optional non-blocking GUI to an OSX networking daemon, I thought I would investigate how you can wrestle back control of your main loop from Apple.

First some understanding of how Apple’s recommended main loop fits together, there are four relevant layers (in order of what uses what):

• NSRunLoop: the main loop class used by Cocoa etc.
• CFRunLoop: part of Core Foundation
• libdispatch: also known as “Grand Central Dispatch”
• mach ports: the kernel objects used for communication

In the dispatch_get_main_queue man page you can find:

Cocoa applications need not call dispatch_main(). Blocks submitted to the main queue will be executed as part of the “common modes” of the application’s main NSRunLoop or CFRunLoop. However, blocks submitted to the main queue in applications using dispatch_main() are not guaranteed to execute on the main thread.

From this you can infer that CFRunLoop is getting notified of events, and has a way to control processing them.

This brings us to my key discovery: the undocumented functions _dispatch_get_main_queue_port_4CF() and _dispatch_main_queue_callback_4CF(). These functions are used to retrieve and process messages from the mach port that receives events scheduled with libdispatch. These functions are part of an undistributed header dispatch/private.h, so you’ll need to declare them yourself.

Now with access to the underlying mach port we have a kernel primitive that can be used to drive your own main loop. You could build a main loop that simply uses mach_msg() to wait for messages from libdispatch. However on unix systems you usually want a file descriptor to use with poll() or select(). Fortunately, there is another primitive to help us out here: kqueue() using EVFILT_MACHPORT.

Annoyingly, OSX doesn’t support using a mach port directly with EVFILT_MACHPORT, only a mach port set (a fact only mentioned in the sys/event.h header!). So you need to wrap the mach port in a mach port set, and hand that to kevent.

I did some work on a demo/example over here: https://gist.github.com/daurnimator/8cc2ef09ad72a5577b66f34957559e47

The above research and inferences were made without access to a mac. Thanks to Peter van Dijk for testing the demo/example.

I saw https://github.com/brunoos/luasec/issues/72#issuecomment-205328635 and couldn’t resist writing the linked code to use lua-http instead.

Read more