Drupal Detachment : The Theory - Part 1/3

Over the last couple of months I have been struggling with an idea that involved using Drupal as the content management system and the web service for an application that I am currently building in iOS and Android.  The reason I am struggling with this idea is because I am using Drupal 7 and Drupal 7 was not intended to serve data to a web service, but rather to serve web pages.  Based upon my knowledge of Drupal 8 this will all change and I feel that Drupal 8 will be a perfect fit for a project like this, but at the time I am writing this Drupal 8 is still in beta with 87 critical issues remaining.  So where does this leave me?  Well, I felt that I had a couple of choices, number one; dive into Drupal 8 and try to work through the issues as they come up, number two; abandon Drupal in search of another solution, or number three; try and explore my options that I still have with Drupal 7.   After considering my options I thought that I would at least look into option three before moving on to option one.  I thought that someone out there in the community must have run into this situation before, and I was right - I actually did not have to look very far.  It turns out that Larry Garfield from Palantir, right down the road in Chicago, had written an article about this very idea already entitled Decoupling Drupal In Larry's article he described a project Palantir had done with Ooyala in which they had separated out three specific tools that were involved in their web application.  Each one of these tools had a specific role in either managing or serving the data.  Larry's article really sparked an idea with me and suddenly I was pretty confident that I could achieve a solution like the one I was looking for with not just Drupal 7, but also MongoDB and Node.js.   

DRUPAL
 

So where does Drupal fall into this equation?  Well, in my opinion, Drupal manages content really well, so why not try and use Drupal for managing my content instead of serving data.  Specifically, I will use Drupal to create my content structures, manage my users, and manage my node articles.  Drupal basically sits at the top of my web architecture and everything written to Drupal through the CMS or through a web service is then written to MongoDB to be served by Node.js on my read server.  The majority or the data that needs to be requested in my web service is read-only and since Drupal is pretty inefficient at serving web service routes, why not move all of the read-only requests to another server.  Drupal will however expose some write web service routes for creating users and creating articles, but these requests will be far less than the ones hitting Node.js to read data.

MONGO DB

So what is MongoDB and how is it used in my architecture?  Well for starters, MongoDB is a document based NoSQL database that stores data as JSON in flat files instead of a table like in a standard RDBMS.  MongoDB is fast, lightweight, and works really nicely with PHP and JavaScript.  In my architecture I am using MongoDB as a secondary data store that holds only my users and articles nodes to be read and served to the read-only routes in Node.js.  Every time an article or user is updated, created, or deleted in Drupal a custom hook is triggered and the data is also mapped and sent to MongoDB to also be updated.  Therefore by the time the data is in MongoDB nothing more has to happen with it, all Node.js has to do is read it and serve it.  

So you may be asking yourself, why MongoDB?  Why not just tap right into MySQL?  This is a good question and one I pondered for awhile before deciding on MongoDB.  There were two deciding reasons that really made me choose MongoDB instead of MySQL.  Number one is the data in MySQL is specifically designed for use with Drupal.  There are many extra fields that exist on a node object that I would not need to be served to a web service route.  If I setup a Node.js application and read data straight form MySQL I would have had to perform a lot of data manipulation in a non-PHP environment to get that data the way I wanted it to be served in our web service, doing this would have majorly impacted the performance of the web service.  In writing data from Drupal to MongoDB I still need to perform some data mapping before the data goes into MongoDB, but at least at this point I have access to Drupal functions that can help me out along the way.  

Number two, MongoDB and Node.js work really nicely together.  Using Mongoose in Node.js, I could create my data models and then very simply query data in MongoDB from Node.js and serve it directly on my web service.  Performing this operation is extremely fast an lightweight, I could not believe it when I seen the load times coming through at 30ms and 50ms.

Node.js

So what is Node.js and how specifically is it used in my architecture?  First of all, Node.js is a scalable networking platform built on top of Chrome's JavaScript runtime that allows users to build applications using server side JavaScript.  Think of C#, Python, or PHP development, but instead you are writing your server side code with JavaScript.  So how is Node.js used in my architecture?  Node.js is used as the web service platform in my architecture - serving read-only request.  The write request in my architecture will be handled strictly by Drupal, but will only make up a small percentage of the total amount of requests.  The read-only requests will greatly outnumber the write requests so I needed my own engine to serve this content and Node.js is what I decided upon.  I ended up choosing Node.js for a couple of reasons, one of them being is that Node.js is very lightweight and very fast.  At the end of the day, all I really need from Node.js is to take the data in MongoDB and serve it on a requested route, so it made sense to choose something that is lightweight and fast to perform this action.  Another reason I chose Node.js is because it works really nicely with MongoDB and extracting data from MongoDB using Mongoose is extremely simple and fast.  Choosing Node.js was not something that came right away.  I was not convinced that Node.js could get me anything that PHP, Python, or Java could not with a little tweaking, so we will see how this decision holds up with time and I will let you know what I find out. 

Code Example

So now that I have explained my theory in using Drupal as part of this detachment architecture, now I will demonstrate my code.  In the next part to this blog post I will demonstrate how to get MongoDB setup and working with Drupal, then in the last part I will demonstrate how to get Node.js working with MongoDB on an isolated server.

Please let me know if you have any thoughts, questions, or concerns.