jobd: an alternative to Gearman, created primarily for PHP applications

I'm a core backend developer and sysadmin in some large PHP project. In this project, we have a couple dozens of servers, and we need to launch many kinds of background and foreground tasks on them. And to do that right, we need a task queue daemon. Something like Gearman.

Actually, for many years, we used to use Gearman.

To be honest, we weren't happy with it.

We had to have a giant wrapper around its API to make it suit our needs. (As just one example of many, Gearman doesn't have built-in support for logging failed tasks to be investigated and relaunched in future).

We were experiencing mysterious problems with its PHP module all the time. 100% CPU usage for nothing (was fixed later in upstream). MySQL connection losses, once in a week or so. PHP-to-gearmand connection losses. You name it.

Finally I gave up on it and decided to write my own task queue daemon, specifically designed for our needs. A few days later, first somewhat working version of jobd appeared.

Few days more, and we started testing it in production and quickly migrated all our tasks from Gearman to jobd. It' simple, stable, efficient and fast.

It's written in Node.JS, the fact that I'm not very proud of, because I'm not a big fan of JavaScript. (I'm just tired of it. I used to work as a frontend developer at vk.com for 6+ years and that time made me realize that I much more enjoy reverse-engineering some driver or hacking coreboot than writing JavaScript.)

But still, I admit that Node.JS is great for prototyping and writing network services quickly. Maybe some day I'll rewrite jobd in C++ or Go, but until then, its Node.JS implementation is working very well on our servers and I can't complain.


jobd – jobd and its documentation

PHP client

PHP example of how it can be used

If you have any comments, contact me by email.
powered by OpenBSD
© ch1p 2021