Drupal Detachment : Drupal and MongoDB - Part 2/3

Submitted by matt_eaton on Thu, 01/08/2015 - 08:14 PM

Tags

MongoDB

In part one of this post I explained the need to have a content management system that was detached from the web service. In explaining this need I also described how I am going to use Drupal for the CMS, MongoDB for the secondary data store, and Node.js as the web service platform. Now it is time time for the fun part, setting up our integration! In this post I will demonstrate how to take your existing Drupal CMS and integrate it with MongoDB to take the first step in building out data for your web service. There is a couple of prerequisites first before jumping right in. Number one, you will need access to a Linux server. In this post I will be using a CentOS server. Number two, you need an existing Drupal site or a newly booted up Drupal site installed and running on your Linux server. Number three, you will need a user that has access to run commands on your server as a root user, using sudo for example.

OK, first of all, let me preface this by saying that I am using a RedHat/CentOS/Fedora server, specifically a CentOS 6.5 server, so I will not be using systemd, and I will be installing MongoDB with YUM. I do not have anything against systemd, this was just the path I choose when I started the project. OK, so the first thing you need to do is make YUM aware of the MongoDB repository. Add the repo file out at /etc/yum.repos.d/mongod.repo. I user nano as my default editor, but you could use vi/vim if you like.

# create the repo file
$ sudo nano /etc/yum.repos.d/mongodb.repo
 
# contents of the file
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1
 
# here is the 32bit alteration of the repo file
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686/

Now that YUM is aware of the MongoDB repo, use YUM to install the latest stable version of MongoDB.

# install MongoDB
$ sudo yum install -y mongodb-org

Once MongoDB has been pulled down from the repo we must configure SELinux to work nicely with MongoDB. There are a couple of options that are available, one of them is disabling SELinux or another is to set SELinux to permissive, but I personally do not recommend disabling SELinux or setting it to permissive because I think it has security implications. Instead I will use semanage to notify SELinux of MongoDB’s default ports.

# make sure semanage is aware of mongodb's default ports
$ sudo /usr/sbin/semanage port -a -t mongod_port_t -p tcp 27017

Next I will allow input, new, and established connections on the IP of our server that is hosting our Node.js application for the MongoDB default port of 27017.
After that we will allow established output connections to the application IP for the MongoDB port.

$ sudo iptables -A INPUT -s <node.js IP> -p tcp --destination-port 27017 -m state --state NEW,ESTABLISHED -j ACCEPT
$ sudo iptables -A OUTPUT -d <node.js IP> -p tcp --source-port 27017 -m state --state ESTABLISHED -j ACCEPT
 
# make sure and save your iptable rules so they are not lost if your server reboots
$ sudo /sbin/service iptables save

Next set MongoDB to start when the server boots up.

# make sure MongoDB starts on boot
$ sudo chkconfig mongod on

Next we need to setup the PHP MongoDB driver so your PHP that is running on your Drupal site can talk to MongoDB. Depending upon your server image you may need a couple of extra development packages before installing the MongoDB PHP driver. If you do not have the cpp or gcc packages installed, it is a good chance you need to group install the development package.

# group install the dev tools
$ sudo yum groupinstall "Development tools"
 
# make sure the PHP PECL package is installed if you did not install it with Drupal
$ sudo yum install php-pecl

Now you should be good to install the MongoDB PHP driver.

# Install the PHP driver
$ sudo pecl install mongo

Next, you need to make sure PHP is aware of this extension. So in the directory, /etc/php.d/, create the file mongo.ini, and add the following to it:

# create file
$ sudo vi /etc/php.d/mongo.ini
 
# add this line as the contents and save the file
extension=mongo.so

Now you must restart your web server. In my case, Apache.

# restart Apache
$ sudo service httpd restart

At this point you should now be able to communicate with MongoDB through PHP, which is what we will be doing in Drupal using a Drupal library. First, before we do that, we must perform some configuration with MongoDB. Open the MongoDB configuration file found at /etc/mongod.conf.

# Set bind_ip to listen for all interfaces
bind_ip = 0.0.0.0
 
# Turn on security to make sure no non-authenticated requests are let through to MongoDB
auth = true

Save the file and restart the MongoDB daemon.

# Restart MongoDB
$ sudo service mongod restart

Now lets connect to the MongoDB shell and setup a root user for our MongoDB database.

# From the terminal, open the MongoDB shell
$ mongo
 
#MongoDB shell version: 2.6.6
#connecting to: 127.0.0.1:27017/test
 
// create our superuser
> use admin
> db.createUser(
    {
      user: "superuser",
      pwd: "myrootpassword",
      roles: [ "root" ]
    }
)
 
> exit
# bye

Exit the MongoDB shell once the superuser is created and log in as the superuser.
Now using the superuser we will create a database for our Drupal data and create a user for that database that can read and write from Drupal.

# Connect to MongoDB using the superuser
$ mongo --port 27017 -u superuser -p myrootpassword --authenticationDatabase admin
#MongoDB shell version: 2.6.6
#connecting to: 127.0.0.1:27017/admin
 
//create Drupal database
> use drupalData
//create a user for drupalData with the readWrite, dbAdmin and userAdmin roles combined in //dbOwner.
> db.createUser(
    {
      user: "drupalUser",
      pwd: "drupalPassword",
      roles: [
         { role: "dbOwner", db: "drupalData" }
      ]
    }
)

At this point we are ready to integrate Drupal and MongoDB. Remember, our goal for MongoDB is to be updated with data every time a node or a user entity is saved, updated, or deleted in Drupal. This will keep the data in MongoDB in-sync with Drupal. To start our integration we will need to create a very simple Drupal Library. I am using a Drupal Library because the PHP that connects to MongoDB is not native to Drupal and I wanted to draw a line in the sand between external code and Drupal code. In your Drupal site, under /sites/all create the folder for libraries and then inside your libraries folder create the mongoDB folder. The code we will use to create our library is very simple and very brief, the whole library will consist of an info file and a basic PHP class.

; mongoDB.libraries.info
 
name = MongoDB
machine name = mongoDB
description = MongoDB CRUD wrapper
version = 2.6.5
files[php][] = MongoGlobal.php

<?php
/**
 * @file MongoGlobal Class file so Drupal can communicate with MongoDB
 *
 */
class MongoGlobal
{
  //the db host
  private $mongo_db_host = '';
  //the db name
  private $mongo_db_name = '';
  //the db user on the database
  private $mongo_user = '';
  //the db password for the above user on the database
  private $mongo_password = '';
  //the db object
  private $db;
 
  /**
   * The constructor to make our connection
   **/
  function __construct($host, $db_name, $user, $password) {
   	//make sure the connection is using a username and password
	if(!empty($user) && !empty($password)){
		$this->mongo_db_host = $host;
		$this->mongo_db_name = $db_name;
		$this->mongo_user = $user;
		$this->mongo_password = $password;
 
		$m = new MongoClient("mongodb://$this->mongo_user:$this->mongo_password@$this->mongo_db_host:27017/$this->mongo_db_name");
	//otherwise try and connect using the localhost connectio
	}else{
		$this->mongo_db_host = $host;
		$this->mongo_db_name = $db_name;
		$m = new MongoClient("mongodb://$this->mongo_db_host");
	}
	//make sure your database connection is available to the rest of the class
	$this->db = $m->drupal_uk_mongo;
  }
 
  /**
   * A function to insert on or many article nodes
   **/
  public function insert_one_or_many_nodes($nodes, $type){
  	//get the count on the nodes coming in from Drupal
	$count = count($nodes);
	$i = 0;
	//make sure we have nodes to insert
	if($count > 0){
	   //assign the article collection
	   $nodes_collection = $this->db->article;
 
		//try and save the node data
		foreach ($nodes as $key => $value) {
			try{
				$nodes_collection->save($value);
				$i++;
			}catch(Exception $e){
				return FALSE;
			}
		}
		if($i == $count){
			return TRUE;
		}else{
			return FALSE;
		}
	}else{
		return FALSE;
	}
  }
 
  /**
   *  Function that deletes article nodes
   **/
  public function delete_existing_node($nid, $type){
	//assign the article collection
	$nodes_collection = $this->db->article;
 
	//try and delete a document.
	try{
		$nodes_collection->remove(array("nid" => $nid), array("justOne" => true));
	}catch(Exception $e){
		return FALSE;
	}
	return TRUE;
  }
 
  /**
   *  Function that specifically updates existing or adds new article nodes
   **/
  public function add_or_update_node($data, $nid, $type){
	//assign the article collection
	$nodes_collection = $this->db->article;
 
	//try and update a document.
	//if the document does not exist then a new one is created using the upsert flag
	try{
		$nodes_collection->update(array("nid" => $nid), $data, array("upsert" => true));
	}catch(Exception $e){
		return FALSE;
	}
	return TRUE;
  }
 
  /**
   *  Function that specifically updates existing or adds new nodes
   **/
  public function add_or_update_user($data, $uid){
	// Get the users collection
	$nodes_collection = $this->db->users;
 
	//try and update a document.
	//if the document does not exist then a new one is created using the upsert flag
	try{
		$nodes_collection->update(array("uid" => $uid), $data, array("upsert" => true));
	}catch(Exception $e){
		return FALSE;
	}
	return TRUE;
  }
 
}

Now that we have our Library up and running, it is time to create a Drupal module that hooks into the CRUD routines of our user and article node entities so we can map that data and send it off to our newly created library to be added to MongoDB.

; mongo_export_data.info
 
name = Mongo Export Data
description = This module exports data to MongoDB when the CRUD routines are executed in Drupal.
package = My Site Custom Modules
core = 7.x
php = 5.3
files[] = mongo_export_data.module
 
dependencies[] = libraries (2.x)

<?php
/**
 * @file Module file to export data to MongoDB
 *
 */
 
/*----------------------------------------------
     Node Related Functions
----------------------------------------------*/
  /**
   *  @Implements hook_node_delete()
   *  This takes place before the Drupal transaction
   *  
   **/
  function mongo_export_data_node_delete($node){
    //get the info for our MongoDB library
    $info = libraries_load('mongoDB');
    //here we make sure that the mongo db library is loaded
    if($info['loaded']) {
	  try{
	       //delete data in mongo
	       $m = new MongoGlobal('localhost', 'drupalData', 'drupalUser', 'drupalPassword');
	       //make the call to our library to delete our node in MongoDB
	       $flag = $m->delete_existing_node($node->nid, $node->type);
	  }catch(Exception $e){
		  //log the failure 
	  }
	  //log the successful 
    }	
  }
 
  /**
   *  @Implements hook_node_update()
   *  
   **/
  function mongo_export_data_node_update($node){
       //get the info for our MongoDB library
       $info = libraries_load('mongoDB');
       //here we make sure that the mongo db library is loaded
       if($info['loaded']) {
	  //*** Important ***//
	  //Map out our node data here to be the structure it needs to be served in Node.js
 
	  //make sure we are dealing with a set of data that is not empty
	  if(count($node_data) > 0){
	      try{
		  //update data in mongo
	          $m = new MongoGlobal('localhost', 'drupalData', 'drupalUser', 'drupalPassword');
		  $flag = $m->add_or_update_node($node_data, $node->nid, $node->type);
	       }catch(Exception $e){
		  //log the failure 
	       }
	       //log the successful 
	  }
      }
  }
 
  /**
   *  @Implements hook_node_insert()
   *  
   **/
  function mongo_export_data_node_insert($node){
     //get the info for our MongoDB library
    $info = libraries_load('mongoDB');
 
     //here we make sure that the mongo db library is loaded
    if($info['loaded']) {
 
      //*** Important ***//
      //Map out our node data here to be the structure it needs to be served in Node.js
 
      //make sure we are dealing with a set of data that is not empty
      if(count($node_data) > 0){
    	    try{
    		//update data in mongo
	        $m = new MongoGlobal('localhost', 'drupalData', 'drupalUser', 'drupalPassword');
	    	$flag = $m->add_or_update_node($node_data, $node->nid, $node->type);
	    }catch(Exception $e){
	        //log the failure 
    	    }
    	    //log the successful 
      }
    }
  }
 
  /*----------------------------------------------
     User Related Functions
  ----------------------------------------------*/
 
  /**
   *  @Implements hook_user_insert()
   *  This function detects when a new user is being created
   **/
  function mongo_export_data_user_insert(&$edit, $account, $category){
 	//make sure we are dealing with a brand new user
 	if($edit['is_new']){
	 	//the mongoDB libraries reference
	 	$info = libraries_load('mongoDB'); 
 
	 	//here we make sure that the mongo db library is loaded
	    if($info['loaded']) {
	    	//send the data to be updated
    		_mongo_export_data_map_user_and_update($user_array);
	    }
 	}
  }
 
  /**
   *  @Implements hook_node_update()
   *  This function detects when a user is being updated
  **/
  function mongo_export_data_user_update(&$edit, $account, $category){
 	//make sure we are dealing with a brand new user
 	if($edit['is_new']){
	 	//the mongoDB libraries reference
	 	$info = libraries_load('mongoDB'); 
 
	 	//here we make sure that the mongo db library is loaded
	    if($info['loaded']) {
	    	//send the data to be updated
    		_mongo_export_data_map_user_and_update($user_array);
	    }
 	}
  }
 
  function _mongo_export_data_map_user_and_update($user_array){
 
        //*** access the correct user data ***//
 
        //*** map the user data ***//
  	$user_data = array(
	   /* map data for user */
 	);
 
	//make sure we are dealing with a set of data that is not empty
	if(count($user_data) > 0){
    	   //insert or updates user data
	    try{
		$m = new MongoGlobal('localhost', 'drupalData', 'drupalUser', 'drupalPassword');
	    	$flag = $m->add_or_update_user($user_data, $uid);
	    }catch(Exception $e){
	     	//log the failure 
            }
            //log the successful 
        }
  }
 
  /*----------------------------------------------
     Mongo Library Related Functions
  ----------------------------------------------*/
  /**
   * Implements hook_admin().
   */
  function mongo_export_data_admin() {
    $info = libraries_load('mongoDB');
    if (!$info['loaded']) {
      form_set_error('', t('The Mongo DB Library could not be loaded.'));
    }
  }
 
  /**
   * Implements hook_libraries_info().
   */
  function mongo_export_data_libraries_info() {
    return array(
      'mongoDB' => array(
        'name' => 'Mongo DB CRUD wrapper',
        'vendor url' => 'http://mongodb.org/',
        'download url' => 'http://www.mongodb.org/downloads',
      ),
      'files' => array(
          'php' => array(
            'MongoGlobal.php',
          ),
       ),
    );
  }

At this point if you have filled in the rest of the code above with code that is specific to your node articles and users entities then you should be able to push data to MongoDB from Drupal based upon the create, update, and delete functions covered above.The next and last step to this process is setting up Node.js to read and serve this data from a remote server. Let me know if you have any questions, thoughts, or concerns from this post as I covered a lot! I would also like to hear of any edits or mistakes I need to make based upon the code or setup above. Thanks!

Member for

3 years 9 months

Matt Eaton

Long time mobile team lead with a love for network engineering, security, IoT, oss, writing, wireless, and mobile. Avid runner and determined health nut living in the greater Chicagoland area.

Twitter

Github