Offloading work using Django Channels

Creating custom channels to distribute work between workers

Posted by Agustín Bartó 1 year, 2 months ago Comments

Continuing our coverage Django Channels, this time we’ll show you how to create a custom channel and offloading work to a specific consumer.

The idea is to isolate resource intensive work (writing to the database, generating thumbnails, you name it) to a specific consumer.

The application

We’ll be using the same shared canvas application presented in in a previous blogpost where we showed you how to add custom authentication to a channel.

This time we not only want to distribute the messages with all the users connected to a specific Group, we want to save those messages to our database. Interacting with a database could be slow, so we want to isolate this process into a function that can be run by any of the available workers.

We’ll need to create a new Channel and route the messages received to a consumer that’s going to deal with recording it.

The code for the application is available on GitHub.

The new channel

Creating a new channel is quite simple. All we need to do is add it to our routing declaration:

# routing.py

from shared_canvas.consumers import (
    websocket_connect, websocket_disconnect, websocket_receive,
    save_message
)

channel_routing = {
    ...
    'save_message': save_message  # send everything received on the
                                  # 'save_message' channel to the
                                  # save consumers.save_message consumer
}

Now we have to write the consumer responsible for handling the messages:

# consumers.py

# Connected to save_message
def save_message(message):
    logger.debug('save_message. message = %s', message)
    Message.objects.create(**message.content)

Simple right? But how do we send messages to this new consumer? We’ll do it from websocket_receive consumer (which distributes the messages received among all the members of the Group):

# consumers.py

# Connected to websocket.receive
@jwt_message_text_field
def websocket_receive(message):
    logger.debug('websocket_receive. message = %s', message)

    Group('shared_canvas').send({
        'text': message.content['text']
    })
    # Send the channel contents alongside the user id so the appropriate
    # consumer can handle the actual saving.
    Channel('save_message').send({
        'user_id': message.user.id,
        'text': message.content['text']
    })

Done. That’s all it takes. Aren’t channels great?

Conclusion

Although the concept of channels isn’t new, its integration into Django is going to open up a whole new world of possibilities. During our coverage, we’ve seen use cases that required a lot of coding and external tools, which are now made dead simple. What’s truly exciting is that this is only the beginning. Who know what we’ll be able to accomplish once the community starts using Channels in unexpected ways?

Vagrant

A Vagrant configuration file is included if you want to test the solutions.

Feedback

As usual, I welcome comments, suggestions and pull requests.


Previous / Next posts


Comments