Epidemiology & Technology

MongoDB on Ubuntu18.04 from the MongoDb website

Vivek Gupta·April 27, 2020·0 comments

Commands to set up MongoDb 4.2 LINK

The mongodb package provided by Ubuntu is not maintained by MongoDB Inc. and conflicts with the official mongodb-org package. If you have already installed the mongodb package on your Ubuntu system, you must first uninstall the mongodb package before proceeding with these instructions.

wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -

echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list

sudo apt-get update
sudo apt-get install -y mongodb-org=4.2.6 mongodb-org-server=4.2.6 mongodb-org-shell=4.2.6 mongodb-org-mongos=4.2.6 mongodb-org-tools=4.2.6Code language: PHP (php)

OS Configurations

Set Currently installed versions in Hold to avoid accidental apt-upgrades

echo "mongodb-org hold" | sudo dpkg --set-selections
echo "mongodb-org-server hold" | sudo dpkg --set-selections
echo "mongodb-org-shell hold" | sudo dpkg --set-selections
echo "mongodb-org-mongos hold" | sudo dpkg --set-selections
echo "mongodb-org-tools hold" | sudo dpkg --set-selectionsCode language: PHP (php)

Modify The DB Directory

By default, MongoDB runs using the mongodb user account. One must also modify the permission to the data directory to give this user access to these directories.

sudo mkdir /mnt/data/mongo
sudo chown -R mongodb:mongodb /mnt/data/mongo

sudo nano /etc/mongod.conf

# Where and how to store data.
storage:
  dbPath: /mnt/data/mongoCode language: PHP (php)

MongoDb Service

Ubuntu uses systemd. 

sudo systemctl daemon-reload
sudo systemctl start mongod.service
sudo systemctl status mongod.service 
sudo systemctl enable --now mongodCode language: CSS (css)

UNIX ulimit Settings

Reference LINK

  • -f (file size): unlimited
  • -t (cpu time): unlimited
  • -v (virtual memory): unlimited
  • -l (locked-in-memory size): unlimited
  • -n (open files): 64000
  • -m (memory size): unlimited
  • -u (processes/threads): 64000
 cat /lib/systemd/system/mongod.service

Look for the lines:

# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=falseCode language: PHP (php)

Edit them as per the ulimit recommendations

[Service]
# Other directives omitted
# (file size)
LimitFSIZE=infinity
# (cpu time)
LimitCPU=infinity
# (virtual memory size)
LimitAS=infinity
# (locked-in-memory size)
LimitMEMLOCK=infinity
# (open files)
LimitNOFILE=64000
# (processes/threads)
LimitNPROC=64000Code language: PHP (php)

Each systemd limit directive sets both the “hard” and “soft” limits to the value specified.

After changing limit stanzas, ensure that the changes take effect by restarting the application services, using the following form:

systemctl restart mongod

Firewall status

sudo ufw status
sudo ufw allow 27017 comment 'MongoDB'
sudo ufw reload
sudo ufw status numbered
sudo ufw show added
sudo ufw show listening
sudo ufw status

sudo netstat -tunelp | grep 27017Code language: JavaScript (javascript)

Swappiness

Check and ensure that Swappiness is NOT ‘0’. LINK

cat /proc/sys/vm/swappiness
sudo sysctl -w vm.swappiness=2
cat /proc/sys/vm/swappiness

NUMA Architecture

sudo apt install numactl
numactl --hardware


OUTPUT in MY CASE
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 40 41 42 43 44 45 46 47 48 49
node 0 size: 64318 MB
node 0 free: 63079 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19 50 51 52 53 54 55 56 57 58 59
node 1 size: 64508 MB
node 1 free: 60103 MB
node 2 cpus: 20 21 22 23 24 25 26 27 28 29 60 61 62 63 64 65 66 67 68 69
node 2 size: 64508 MB
node 2 free: 62211 MB
node 3 cpus: 30 31 32 33 34 35 36 37 38 39 70 71 72 73 74 75 76 77 78 79
node 3 size: 64485 MB
node 3 free: 63123 MB
node distances:
node   0   1   2   3 
  0:  10  21  21  21 
  1:  21  10  21  21 
  2:  21  21  10  21 
  3:  21  21  21  10 
Code language: PHP (php)

MongoDB is not NUMA-aware, and because of this, MongoDB can allocate memory unevenly, leading to the swap issue even with memory available.

To solve this issue the mongod process can use the interleaved-mode (fair memory allocation on all the nodes) 

# cat /etc/systemd/system/multi-user.target.wants/mongod.service

# ExecStart=/usr/bin/mongod --config /etc/mongod.conf

ExecStart=/usr/bin/numactl --interleave=all /usr/bin/mongod --config /etc/mongod.conf

sudo systemctl daemon-reload
sudo systemctl stop mongod
sudo systemctl start mongod
sudo numastat -p $(pidof mongod)


mongo
# YOU SHOULD NOT GET NUMACTL ERROR
Code language: PHP (php)

SECURITY AND AUTHENTCIATION

Checklist Link

Mongo by default listens only on localhost and is run without authentciation

mongo

We want to make Mongo work only with authentication (link) and enable access control

Admin User

Craeate a SuperAdmin user

use admin

db.createUser(
  {
    user: "myUserAdmin",
    pwd: passwordPrompt(), // or cleartext password
    roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
  }
)

# ADD A STRONG PASSWORD - REMEMBER IT !!!

sudo systemctl stop mongod
sudo systemctl status mongod


sudo nano /etc/mongod.conf 

#security:
security:
    authorization: enabled
# SAVE

sudo systemctl start mongod
sudo systemctl status mongod
Code language: PHP (php)

You may get an error on Status. Do some digging around LINK

sudo cat /var/log/mongodb/mongod.log 
# Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted      Fatal Assertion  at src/mongo/transport/transport_layer_asio.cpp

ls -ls /tmp/mongodb-27017.sock
# SOCK File is owned by the current user and not by root !!!
  
sudo rm -rf /tmp/mongodb-27017.sock

sudo systemctl daemon-reload
sudo systemctl start mongod
sudo systemctl status mongod

ls -lsah /tmp/mongodb-27017.sock
## 0 srwx------ 1 mongodb mongodb 0 Apr 27 12:08 /tmp/mongodb-27017.sockCode language: PHP (php)

Connect to the local instance

mongo --port 27017 -u "myUserAdmin" --authenticationDatabase "admin" -p


# Enter the password created in the last instanceCode language: PHP (php)

Salted Challenge Response Authentication Mechanism (SCRAM) – Default in 4.2

The following operation creates a user named appAdmin in the admin database and gives the user readWrite access to the config database, which lets the user change certain settings for sharded clusters, such as to the balancer settin

The following operation creates a user named restricted in the admin database. This user may only authenticate if connecting from IP address 192.0.2.0 to IP address 198.51.100.0.

mongo

# Mongo shell started
use admin

db.createUser(
   {
     user: "restricted",
     pwd: passwordPrompt(),      // Or  "<cleartext password>"
     roles: [ { role: "readWrite", db: "reporting" } ],
     authenticationRestrictions: [ {
        clientSource: ["192.0.2.0"],
        serverAddress: ["198.51.100.0"]
     } ]
   }
)Code language: PHP (php)

The following operation creates a user with only SCRAM-SHA-256 credentials.

use reporting

db.createUser(
   {
     user: "reportUser256",
     pwd: passwordPrompt(),   // Or  "<cleartext password>"
     roles: [ { role: "readWrite", db: "reporting" } ],
     mechanisms: [ "SCRAM-SHA-256" ]
   }
)Code language: CSS (css)

Vivek Gupta

Related posts