What the heck is Meteor DDP?

I was using Meteor for my messenger project. I was so curious about the real time connection. I wanted to know how exactly this mechanism works. In this post, I will go through the DDP Specification, an overview of WebSocket, and a simple demo about how to subscribe a publication of Rocket.Chat (containing a DDP server) from an external webpage.

At a glance, I knew that Meteor invented a protocol called DDP which uses for handling real time connection. So then, what is DDP?

"DDP (Distributed Data Protocol) is the stateful WebSocket protocol that Meteor uses to communicate between the client and the server." [1]

All right! Why does DDP matter?

"DDP is a standard way to solve the biggest problem facing client-side JavaScript developers: querying a server-side database, sending the results down to the client, and then pushing changes to the client whenever anything changes in the database". [2]

In order to understand deeply the protocol, I decided to explore its specification, and then took a look at its implementation.

The Specification

The official specification is easily found here. It is quite simple! I tried to draw a diagram (sequence-liked) to visualize its flow.

The Implementation

As its definition, DDP is a WebSocket protocol. Hence, I needed to understand WebSocket protocol as well. A normative way to learn a protocol is to read it RFC. The WebSocket Protocol is specified under RFC-6455. After that, I followed to implement a WebSocket server with Node.js. I was able to understand how WebSocket protocol works.

Then, I took a look into some implementations of DDP available on the net. Here was an example:

- DDP client

- DDP server

I tried to simplify an implementation of DDP by writing a pseudo code as below:

	// DDP Server
	// Use an implementation of WebSocket Server such as SockJS
	ddpServer.socket = WebSocketServer
	ddpServer.socket.onData = (msg) => {
	switch(msg):
	case 'connect':
	check whether WebSocket connection
	socket.send({msg: 'connected', ...})
	case 'method':
	result = methodsMap.get(msg.methodID).invoke()
	socket.send({'result': ...})
	case 'sub':
	publicationsMap.get(msg.subId).push(new Subscription)
	socket.send({msg: 'ready', ...})
	case ...
	}
	ddpServer.publish = (name) => {
	publicationsMap.push(name)
	}
	dppServer.methods = (names) => {
	methodsMap.push(names)
	}
	// DDP Client
	// Simply use a WebSocket client to open connection
	// and send messages to the WebSocket Server
	ddpClient.ws = WebSocket
	ddpClient.connect = () => {
	ws.on('open', () => ws.send(EJSON({msg: 'connect', ...})))
	}
	ddpClient.call = () => {
	ws.send(EJSON({msg: 'method', ...}))
	}
	ddpClient.subscribe = () => {
	ws.send(EJSON({msg: 'sub', ...}))
	}

view raw meteor-ddp-pseudo.js hosted with ❤ by GitHub

Practice

Assignment: “Obtaining Active Users of Rocket.Chat instance for a webpage in real time”

As my understanding so far, I needed to use WebSocket with DDP, so that I could:

- Maintain real time connection via WebSocket

- Communicate with Meteor via DDP

With node-ddp-client, I could make a connection with DDP server of Meteor, but the implementation was only in server side. I decided to implement a similar stuff, but in client side, which can be executed in browsers.

Check out my project here

------------

Reference:

[1]. https://github.com/meteor/meteor/tree/devel/packages/ddp

[2]. https://blog.meteor.com/introducing-ddp-6b40c6aff27d?gi=65983e20e448

[3]. https://hackernoon.com/implementing-a-websocket-server-with-node-js-d9b78ec5ffa8

[4]. https://github.com/hharnisc/node-ddp-client