Wednesday, April 13, 2016

YCSB installation and using as Client to the MongoDB cluster / Standalone



1.       Create 2VMs and install RHEL
2.       Add static IP ( Private IP in our case) to these 2VMs
3.       Assign hostname to 2VMs in /etc/sysconfig/network file
4.       Map 2VMs static IP and Hostname details in the /etc/hosts file.
5.       Disable the firewall (iptables)  /etc/sysconfig/iptables

OpenJDK uninstall and Oracle JDK installation
Uninstall OpenJDK and install Oracle complete JDK , preinstalled OpenJDK does not have compiler installed by default.

Java installed version
root@2daygeek [~]# java -version
java version "1.7.0_72"
Java(TM) SE Runtime Environment (build 1.7.0_72-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)

Java Installed method
root@2daygeek [~]# rpm -qa | grep java
java-1.7.0-openjdk-1.7.0.71-2.5.3.2.el6_6.x86_64

 Verifying JAVA installed
root@2daygeek [~]# rpm -qa | grep jdk
jdk1.8.0_25-1.8.0_25-fcs.x86_64

Remove/Uninstall Openjdk
root@2daygeek [~]# rpm -e (grep output)
root@2daygeek [~]# rpm -e java-1.7.0-openjdk-1.7.0.71-2.5.3.2.el6_6.x86_64
root@2daygeek [~]# java -version
bash: /usr/bin/java: No such file or directory

Download Oracle java jdk rpm and run, this case have installed the latest version of JAVA 8 available during the time testing.
rpm –Uvh jdk-8u73-linux-x64.rpm

Check Java installed and add JAVA_HOME and bin path into environment variables.

Install Maven
apache-maven-3.1.1-bin.tar.gz
sudo tar xzf apache-maven-*-bin.tar.gz -C /usr/local
cd /usr/local
sudo ln -s apache-maven-* maven
sudo vi /etc/profile.d/maven.sh

Add the following to maven.sh
export M2_HOME=/usr/local/maven
export PATH=${M2_HOME}/bin:${PATH}

Reload bash and test mvn
bash
mvn –version

Install YCSB and Setup
Download the latest version of YCSB with MongoDB binder only, no need to download whole YCSB package with all DB binders.

Extract the tar file and place in the home directory (or any place where you maintain the software repository)
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.7.0/ycsb-0.7.0.tar.gz
tar xfvz ycsb-0.7.0.tar.gz
cd ycsb-0.7.0

Try below commands if it runs successfully then you’re good to proceed if not then necessary python argparse package have to be installed. (RHEL 6 deprecated this this package)

Tar –zxvf argparse-1.4.0.tar.gz
Cd argparse-1.4.0
Python setup.py install

Test YCSB runs successfully
$ ./bin/ycsb shell basic
> help
Commands:
  read key [field1 field2 ...] - Read a record
  scan key recordcount [field1 field2 ...] - Scan starting at key
  insert key name1=value1 [name2=value2 ...] - Insert a new record
  update key name1=value1 [name2=value2 ...] - Update a record
  delete key - Delete a record
  table [tablename] - Get or [set] the name of the table
  quit - Quit

Run YCSB on the MongoDB cluster
Start mongd instance on all the nodes of Recplicaset.

Data Load command
./bin/ycsb load mongodb-async -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0

To run with the synchronous driver from MongoDB:
./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0

Load more data using more threads
./bin/ycsb load mongodb-async -s -P workloads/workloada -p recordcount=10000000 -threads 16 mongodb.url=mongodb://localhost:27017/ycsb?w=0
·         load‘ flag indicates that this is a load run.
·         s‘ flag prints status at 10 sec intervals
·         recordcount‘ is set to 10 million.
·         threads‘ sets the number of client threads to 16.

Run async driver mongodb with operation count same as data load row count and thread 2
./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0 -p operationcount=10000000 -threads 2
 

Running the Tests on different workloads.
·         Workload A: Update heavy workload: 50/50% Mix of Reads/Writes
·         Workload B: Read mostly workload: 95/5% Mix of Reads/Writes
·         Workload C: Read only: 100% reads
·         Workload D: Read latest workload: More traffic on recent inserts
·         Workload E: Short ranges: Short range based queries
·         Workload F: Read-modify-write: Read, modify and update existing records

Loading 94Gb data Record count 8000000
[root@ycsb1 ycsb]# ./bin/ycsb load mongodb-async -s -P workloads/workloada -p recordcount=80000000 -threads 16 -p mongodb.url=mongodb://node1.rs05.mongodb:27017/ycsb?w=0
: Count=116769, Max=539135, Min=5, Avg=344.97, 90=25, 99=8511, 99.9=33919, 99.99=256639]
[OVERALL], RunTime(ms), 2266923.0
[OVERALL], Throughput(ops/sec), 35290.126748901486
[CLEANUP], Operations, 16.0
[CLEANUP], AverageLatency(us), 95.5625
[CLEANUP], MinLatency(us), 1.0
[CLEANUP], MaxLatency(us), 1462.0
[CLEANUP], 95thPercentileLatency(us), 11.0
[CLEANUP], 99thPercentileLatency(us), 1462.0
[INSERT], Operations, 8.0E7
[INSERT], AverageLatency(us), 449.2443936
[INSERT], MinLatency(us), 5.0
[INSERT], MaxLatency(us), 1.6097279E7
[INSERT], 95thPercentileLatency(us), 1672.0
[INSERT], 99thPercentileLatency(us), 5351.0
[INSERT], Return=OK, 80000000








No comments:

Post a Comment