Asked

Best way to queue or thread CPU intensive tasks so Node server doesn't crash?

I wrote code that forks and queues up and stops child processes but the processes themselves use up quite a bit of CPU.

Is there a library that's particularly helpful with building these types of async/cron-like Node services? Wasn't finding anything in my search.

Should I consider offloading that to another server or is it just time to up the CPU of my virtual machine?

Making things. Making them happen. 👨🏻‍💻☕️

Although I have never used it or anything, I've heard about forking child processes. I think this talk covers it - www.youtube.com/watch?v=K8spO…

Not sure if it'll help though. You should also try asking in the subreddit /r/nodejs

Developer, writing at https://codewithhugo.com

You should try using a queue eg. Bull queue.
You can run the workers on the same server or somewhere else (it's backed by redis).
Not sure if it 100% fits your use case since as far as I understand you seem to want to wait for the job to complete before sending a response back or something? In which case you should wait on the output of the queued job (that writes to a db or something)

Making things. Making them happen. 👨🏻‍💻☕️

Bull looks like exactly what I need. I don't have redis setup for my project but shouldn't be too tough. Thanks dude!!! 🙌 Will let you know how it goes.

Developer, writing at https://codewithhugo.com

There's cool UIs to monitor the queues as well, I've got an example of a working setup at github.com/HugoDF/express-bul… (post explaining what's going on at codewithhugo.com/bring-redux-…).

Bull is the only queue with a modern interface (ie Promises) that I found when digging around.

Let me know if you get stuck anywhere.