Victus Spiritus

home

Dynamic scripting with static speed, the best of both worlds

28 Jan 2011

Many dream of success. A rare few focus on grand visions to reshape the future. Me, I fantasize about ideal software structures*.

Just a few days back I dreamt of middleware that would take completely dynamic objects and map required information into static structures which are connected to blazing fast compiled pipelines. In other words, flexible routing tables that bind dynamic to static objects, in tandem with interpreted scripts which call compiled code. This is something that I not only think is cool, but can readily apply. This form of middleware opens the door to scripting languages calling compiled c++ libraries at my day job. If you do this type of thing, I'd love to hear from you.

Highly desirable forms lead to blindspots

While this idea was baking in the background of my noodle, I scanned the following tweet from Andraz Tori (CTO of Zemanta):
[blackbirdpie url='http://twitter.com/andraz/status/30571012311482368']
Protocol buffers transfer encoded data into binary streams. Not only are dynamic calls to thinly wrapped compiled code happening, they're common place. In the HackerNews post Andraz referred to, several folks commented on their own development of rapid protocol buffers callable from Python. So the I/O layer was already well developed.

*Cough* Node.js Yo

Speaking of dynamic scripting language bindings to compiled c++ code, what about node.js? I've read a good deal of documentation on node, and toyed around with servers. One of my favorite high level descriptions of node is a post by Denton Gentry, node.js from 30000 feet:

Node.js Overall Structure

The JavaScript implementation in node.js is Google's V8. As mentioned in an earlier article, V8 compiles the source JavaScript directly to machine code the first time it is executed. There is no intermediate bytecode format and no JS interpreter. In addition to V8, node.js relies on libev for its event loop, libeio for asynchronous I/O, and c-ares for asynchronous DNS support. Like everything else in the known universe, it relies on OpenSSL for cryptography and SSL/TLS support.

A standard library in JavaScript is supplied. This provides access to the underlying C++ implementation, and also has helpful bits like a URL parser and a REPL shell for easy experimentation. One thing it does not provide is the DOM. Node.js is not a browser, there is no HTML document to interact with.

There are likely dozens of examples of bindings for dynamic scripting languages to static structures and executable pipelines. For my own personal usage it's just a matter of selecting the right tools for the job after a little more research. I'm leaning towards ruby, javascript or python calling c++ dynamic libraries with narrow exposed interfaces.

Notes:
*= Ideal software structure posts

Probe, loose ideas for a programming language