Docker Container Linking - MongoDB and NodeJS

In this post, I’ll use Docker to perform the following tasks:

• Setting up a MongoDB installation using the official docker image.
• Setting up a NodeJS installation using the official docker image.
• Connecting the MongoDB container to the NodeJS container using docker container linking.

The whole process should take only a few minutes if you have a fast internet connection. I hope that this example will demonstrate that docker container linking is quite easy and practical, an impression you might not share if you wade through the official Docker documentation.

Retrieving the MongoDB and NodeJS images

For simplicity, we’ll use the official images of MongoDB and NodeJS in this example. First off, we need to retrieve the images from Docker hub:

docker pull mongo
docker pull node

These commands will download a few hundred MBs worth of data, so it might take a little while.

Starting a MongoDB instance

For a more complete example of running MongoDB in a container, take a look at my previous post. For this exercise, we’ll simply run MongoDB as container named myMongoDB:

docker run -d --name myMongoDB mongo

Running that command should simply return the hash of the running container as illustrated. We didn’t specify the port mapping for our MongoDB container, container linking will take care of that as you’ll see shortly.

Next, let’s start up our NodeJS container.

Starting a NodeJS container

docker run --link=myMongoDB:mongodb -it node /bin/bash

That should bring up a bash shell inside your newly created NodeJS container. We specified --link=myMongoDB:mongodb to instruct Docker to use the container named myMongoDB as a linked container, and name it mongodb inside our NodeJS container.

Linking a container simply extracts the runtime information of your MongoDB container (its IP address, the exposed ports) and exposes that information into your new container. To see the effects of linking, run these commands in your NodeJS bash shell:

cat /etc/hosts
env | grep MONGODB

As you can see, linking the container created an entry in your container’s hosts file and a bunch of environment variables starting with MONGODB.

For our purposes, we’re mostly interested in the following 2 entries:

MONGODB_PORT_27017_TCP_ADDR=172.17.0.1
MONGODB_PORT_27017_TCP_PORT=27017

Using these 2 variables should be enough for us to connect to MongoDB. Let’s do that next.

Connecting to the linked MongoDB with mongoskin

I’m using Mongoskin to connect to MongoDB within Node. The concepts are the same for any other client.

First off, we need to install Mongoskin:

mkdir /data
cd /data
npm install mongoskin

Then, we use Mongoskin to connect to our MongoDB database:

node

var mongo = require('mongoskin');
var db = mongo.db('mongodb://' + process.env.MONGODB_PORT_27017_TCP_ADDR + ':' + process.env.MONGODB_PORT_27017_TCP_PORT + '/mydb');

Your db variable should now point to a functional MongoDB installation.

In this example, we’re running both MongoDB and NodeJS on the same host using docker container linking. Running your containers on different hosts require more work, with a lot of competing solutions. Personally, I’m learning towards Ansible for the combination of ease-of-use and flexibility, a topic I hope to address in a future post.

Running MongoDB as a Docker container

A main strength of MongoDB is arguably its ease-of-use. You can complete its installation and have your first database up-and-running in minutes.

With docker, this process can become even easier. In this post, we’ll install and create a MongoDB database in a docker container in just a few simple commands.

All you need to get started is an installation of docker. The commands in this post assume that you’re running Ubuntu.

The official MongoDB container

MongoDB conveniently provides us with an official container. To try it out:

mkdir ~/data
sudo docker run -d -p 27017:27017 -v ~/data:/data/db mongo

I’ll explain the syntax of the docker run command in a bit. At this point, you should have a MongoDB instance listening on port 27017. Its data is stored in ~/data directory of the docker host.

Now, let’s populate the fresh database with some data.

Connecting to your MongoDB container

# Install the MongoDB client
sudo apt-get install mongodb-clients

# Change mydb to the name of your DB
mongo localhost/mydb

After that, you’ll get into a MongoDB prompt, like this:

We want to store some cities in our database:

db.createCollection('cities')
db.cities.insert({ name: 'New York', country: 'USA' })
db.cities.insert({ name: 'Paris', country: 'France' })
db.cities.find()

You should see the entries that you created:

So far, getting data into our MongoDB container has proven to be easy. How about taking data out of the container?

Migrating data to a new installation

Let’s start a new MongoDB container, this time running on port 37017 instead of the default 27017:

# Copy the data from the previous container
sudo cp -r ~/data ~/data_clone

# Start another MongoDB container
sudo docker run -d -p 37017:27017 -v ~/data_clone:/data/db mongo

We can query the data in the new container in the same way:

Stopping your MongoDB containers

To stop our runnings MongoDB queries, you run sudo docker ps to see the list of the running containers and sudo docker stop HASH to stop each container:

Alternatively, you can also execute sudo docker stop $(sudo docker ps -q) as a shortcut to stop all running containers.

DOCKER RUN command syntax

The docker run -d -p 27017:27107 -v ~/data:/data/db mongo does 3 main things:

• -d tells docker to run the container as a daemon, which is the mode that’ll you want to use for server containers.
• -p 27017:27107 maps the port 27017 of the container to the port 27017 of the host. The syntax is -p HOST_PORT:CONTAINER_PORT.
• -v ~/data:/data/db maps the /data/db directory of the container to the ~/data directory on the host. This is called a data volume, the principal mechanism to import and export data with your docker container.

How do you know which port(s) the container expose and which directory the container uses to store data? What I did was simply reading the Dockerfile. Look for the VOLUME command to know what directory the container uses for its data. The EXPOSE command tells you which network port(s) the container listens on.

Learning Tips

I recently took a short online course on Coursera: Learning how to learn. The roughly 20 hours I put into this course might well be the best investment of my time to date. In this post, I’ll present three concepts that have reshaped my way of thinking about our brain.

Focused and diffuse thinking modes

The focused thinking mode should not be a stranger to anyone. When someone says “I’m working hard on a problem”, they usually refer to this mode of thinking.

The focused thinkind mode helps us examine the problems with the tools we know best: critical thinking and analysis, logical reasoning and what have you. The focused mode works well for the problems that we know how to solve, e.g. problems that don’t necessarily present a learning opportunity.

In the course, Dr. Oakley suggests that there’s another mode of thinking: the diffuse mode. You enter this mode of thinking when you relax and let your mind wander without constraints.

In this diffuse mode of thinking, your mind broadens its horizon and taps into the full power of your memory. This mode of thinking is less disciplined, but powerful nevertheless. Personally, my more creative solutions to tough programming problems often arise from the diffuse thinking mode; such as after a good rest, or a while walking and thinking about the problem absent-mindedly.

The trick to better problem solving is a good balance between the two modes of thinking. Day dreaming all the time without a good grasp of the problem usually doesn’t lead anywhere. On the contrary, doggedly trying to solve a problem only in the focused mode does not utilise all the capabilities of your brain. Try to alternate between the two modes and find a method that works for you.

Procrastination

Everyone procrastinates to a certain degrees. There are always unpleasant tasks that you don’t want to start, like taking out the trash or studying a particular subject. There are things you know you should do, but lack the courage to begin.

Procrastination is a natural defense mechanism of your brain against unpleasant activities. If left unchecked, however, procrastination can negatively impact your life in many ways. Understanding and knowing how to deal with procrastination is an essential skill for more effective learning.

The mechanism behind procrastination is surprisingly simple. The area that triggers procrastination is the same brain region that detects pain. In order to avoid that painful feeling of starting a difficult task, your brain comes up with a simple tactic: do something easier, and get an immediate reward out of it. From this point of view, procrastination shares many traits with addiction.

There are four components to procrastination:

1. The cue
2. The routine
3. The reward
4. The belief

If you wish to know more about these components, I invite you to view the lectures by Dr. Oakley who did an excellent job at explaining the concept in an easy-to-follow manner.

I do want to touch on item #4 of procrastination: the belief. The procrastination behavior can only work reliably if it’s sustained by an underlying belief. If you’re telling yourself, consciously or not, that “I’m no good at starting new things” or “I’ll never get through this chapter of the book”, you help to fuel the belief that sustains the behavior.

There are a few simple but effective techniques to help you combat procrastination:

• Know that the pain you experience when starting something is only temporary. In fact, as you get into the flow of learning something new, the whole process can become quite enjoyable.
• Do not focus on the product, or the outcome, of a task. The product of a task is what triggers the pain of procrastination. Focus instead on the process, or how to get there.
• The Pomodoro technique is a helpful method to get you started. The core idea is to fix a 25 minutes block of time exclusively for a task. You must stop after the 25 minutes interval and give yourself a break or some other reward.

Memory

There are two main types of memory: working memory and long term memory. You can think of working memory as slots that store things you can recall immediately. We only have around four of these slots in our brain. On the other hand, long term memory is similar to a warehouse of your brain where you store past events, concepts and skills. Unlike working memory, the capacity of long term memory is immense; there doesn’t seem to be a fixed limit for our long term memory.

The tricky part about long term memory is that we do not get to choose what goes into your long term memory. Your brain selects what to put into your long term memory automatically, we only have limited control over this process. Personally, I see learning as little more than an elaborate process to conceptualize and store some useful information in the long term memory.

Your memory, just like your brain, is a living thing. Newly retained information must be used and recalled or it will fade away. Old information does not stay still, but it’s being continuously re-evaluated and even possibly rewritten by your brain!

In order to effectively store the things you learn into long term memory, you can try some of the following techniques:

• Recalling: the simple fact of trying to retrieve the information you have learned could increase the chance of ingraining it in your memory.
• Spaced repetition: spreading the learning activity over the longer period of time creates more concrete chunks of memory than intensive cramming. Cramming might work to store a lot of information in a short time, but that memory is in no guarantee to persist.
• Sleep: when you sleep, your brain organizes and stores new information into long term memory. Sleep also cleans away the toxin produced by brain activities to restore the brain to a fresh state.
• Deliberate practice: for a particularly slippery subject, you could put extra effort in it by doing extra exercises, researching more information to read on the subject, discussing the subject with others. The extra effort will give your brain a better chance to digest and record the information.
• Interleaving: instead of studying one subject sequentially, mixing things up could in fact be more productive!

Learning to learn

Learning is hard; effective learning is probably the most rewarding skill you might have in your repertoire. I might already have had some vague notions about the concepts and techniques before taking the Coursera course. But having them presented in such a clear and concise way really helped to put everything into perspective. Many thanks to Dr. Barbara Oakley and Dr. Terrence Sejnowski for putting up this small but tremendously useful missing manual for the brain.