Reading Data from @socket.io/redis-emitter without Using a Socket.io Client

Real-time communication has become a crucial aspect of many applications. Socket.io is a popular library that facilitates real-time communication between clients and servers using WebSockets. However, when dealing with distributed systems and scaling applications, using a Redis server as a message broker can greatly enhance performance and reliability.

One essential component in this setup is @socket.io/redis-emitter, which allows events to be emitted to a Redis server. However, there's a challenge when it comes to reading the messages from the Redis server directly without using a socket.io client. The messages are encoded in a way that is not immediately compatible with JavaScript strings, requiring a careful decoding process.

In this article, we'll solve the problem of reading data from @socket.io/redis-emitter without a socket.io client.

The Problem

When emitting events to a Redis server using @socket.io/redis-emitter, the messages are encoded before being published.

The problem is that the messages are encoded in a way that is not compatible with strings. So, if you want to read the messages from the Redis server without using a socket.io client, you will have to decode them first yourself.

Digging through the node modules, we see in the package:

const msg = this.broadcastOptions.parser.encode([UID, packet, opts]);
...
this.redisClient.publish(channel, msg);

Where the parser used in the package is typically set to notepack.io. If you try to simply read the messages using a Redis Client:

import { Redis } from 'ioredis';
 
let io = new Redis('redis://localhost:6379');
 
io.on('connect', () => {
  console.log('RedisIO Client Connected');
});
 
io.psubscribe('*', (err) => {
  if (err) {
    console.error(err);
  }
});
 
io.on('pmessage', (pattern, channel, message) => {
  console.log(`Received ${message} from ${channel}`);
});

Your logs will show that you are receiving the messages, but they are not readable:

Solution

The root of the problem is that, even though the Redis protocol is binary safe, JavaScript Strings are not. If we do not specify to our Redis Client reading these messages that the messages are binary data, there is an intermediate JS String that is not necessarily binary safe; it will try to decode them as strings, which will result in the above.

Below is the adjusted code to correctly read the messages:

import { Redis } from "ioredis";
import { decode } from "notepack.io";
 
let io = new Redis("redis://localhost:6379");
 
io.on("connect", () => {
  console.log("RedisIO Client Connected");
});
 
io.psubscribe("*", (err) => {
  if (err) {
    console.error(err);
  }
});
 
io.on("pmessageBuffer", (pattern, channel, message) => {
  let parsedMessage = parseRedisEmitterMessage(message);
  console.log(parsedMessage);
});
 
let parseRedisEmitterMessage = (message: Buffer) => {
  try {
    let parsedMessage = decode(message);
    return parsedMessage[1].data[1].value;
  } catch (err) {
    console.log(err);
  }
};

By making use of pmessageBuffer instead of pmessage, we ensure that the messages are treated as binary data. The decode function from the notepack.io package then handles the decoding process, resulting in readable and usable data.

Now we can finally read the dummy data we were sending through!

Conclusion

Integrating real-time communication through @socket.io/redis-emitter and Redis provides a powerful way to scale applications and build robust distributed systems. However, when dealing with encoded messages and direct message retrieval, it's crucial to understand how to decode the messages correctly.

In this article, we explored the challenge of reading data from @socket.io/redis-emitter without a socket.io client and presented a clean and effective solution using the notepack.io package. The key insight is using pmessageBuffer instead of pmessage to handle binary data properly, ensuring that the Redis protocol's binary safety is preserved throughout the decoding process.

This solution enables direct message retrieval from Redis while maintaining readable and structured data, making it easier to build scalable real-time applications without being tied to socket.io clients.