Another MySQL DBA: source

Showing posts with label source. Show all posts

Monday, April 7, 2014

WebScaleSQL Installation solved... part 2

This is a follow up to the : WebScaleSQL Installation attempted... part 1

So being a typical geek, developer, idiot or whatever.. I failed to RTFM or in this case the FAQ.

So once I paid attention and didn't just rush into it, I installed WebScaleSQL correctly.
The biggest problem was that I was using the Linux OS (Oracle Linux VM) which is just too old and the Oracle repositories did little to help get it upgraded. While I wanted to start with something that everyone could use easily, that just was not it.

I downloaded Fedora 20. You can see that they already use GCC 4.8.2.
So the point is simple enough, if you want to use new technology, update your OS as well.

If you really want to upgrade your GCC and stay on your current OS these hyperlinks might be useful:

Below are the steps I took to get it installed. Once the dependencies are in place it is a basic source install.


#cd /usr/local/

#yum -y install gcc git readline-devel  gcc-c++ bison-devel bison cmake ncurses-devel

# gcc -v

gcc version 4.8.2 20131212 (Red Hat 4.8.2-7) (GCC) 



#git clone https://github.com/webscalesql/webscalesql-5.6.git

#ln -s webscalesql-5.6 mysql 

#groupadd mysql

#useradd -r -g mysql mysql

#cd mysql/ 

#cmake . -DENABLE_DOWNLOADS=1

-- Successfully downloaded http://googlemock.googlecode.com/files/gmock-1.6.0.zip

-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/src/webscalesql-5.6


#make

Scanning dependencies of target INFO_BIN
[  0%] Built target INFO_BIN
Scanning dependencies of target INFO_SRC
[  0%] Built target INFO_SRC
Scanning dependencies of target abi_check
[  0%] Built target abi_check
Scanning dependencies of target zlib
[  1%] Building C object zlib/CMakeFiles/zlib.dir/adler32.c.o
 .....

[100%] Building CXX object mysql-test/lib/My/SafeProcess/CMakeFiles/my_safe_process.dir/safe_process.cc.o
Linking CXX executable my_safe_process
[100%] Built target my_safe_process

#make install

#chmod +x scripts/mysql_install_db
#yum -y install perl-Data-Dumper-Names

#./scripts/mysql_install_db --user=mysql 

#chown -R mysql data

#./bin/mysqld_safe &

# ./bin/mysql



mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.6.17    |
+-----------+


mysql> show variables like '%read_only%';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| innodb_read_only | OFF   |
| read_only        | OFF   |
| super_read_only  | OFF   |
| tx_read_only     | OFF   |
+------------------+-------+


cd mysql-test ; perl mysql-test-run.pl

OK now that we got it up and running... we can explore it...

Thursday, March 27, 2014

WebScaleSQL Installation attempted... part 1

Well if you have not see the news yet, then by all means follow this link and read about WebScaleSQL. Many thanks to the team of engineers from Facebook, Twitter and Google for their hard work on this. I am curious to see how this develops more in the future, once I get it to work.

Here is what I encountered so far. I wanted this to be a successful blog post, instead it will be part one until I get it running.

I figured I would put up a little how to install WebScaleSQL for those of you new to Github any of this. It takes feedback from the community after all, so the community needs to be able to install and try it out.

For this example I took the pre-built Oracle Linux Virtual Box image. I had one already but it is available here : http://www.oracle.com/technetwork/server-storage/linux/downloads/vm-for-hol-1896500.html.

It should be obvious that WebScaleSQL is not intended for use on a VirtualBox in production, but this is just a demo. I assumed that it meant that people do not have an excuse to try it out since anyone can download and use this with VirtualBox. I did have to enable the additional disk that come with the image so you have enough space. To format, partition and mount the disks all can be done via disk utility. The finished product for my example is below.

 

[root@oraclelinux6 oracle]# ls -al /media/ 

drwx------  3 oracle oracle 4096 Mar 27 16:19 webscaledisk
drwx------  3 oracle oracle 4096 Mar 27 16:21 workspace

[root@oraclelinux6 oracle]# chmod 777 /media/workspace/
[root@oraclelinux6 oracle]# chmod 777 /media/webscaledisk/

The code for WebScaleSQL is available on Github. The zip file can be downloaded here. I choose to check out the code via the github code repository methods. If others need help with the SSH keys see the Gitbhub reference page.

The git package is not available on Oracle Linux but that is quickly solved if you plan to use the Github version.

 

[oracle@oraclelinux6 ~]$ sudo bash

[root@oraclelinux6 oracle]# yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel 

[root@oraclelinux6 oracle]# wget http://git-core.googlecode.com/files/git-1.7.10.1.tar.gz 

[root@oraclelinux6 oracle]# tar -xvzf git-1.7.10.1.tar.gz

[root@oraclelinux6 oracle]# cd git-1.7.10.1

[root@oraclelinux6 git-1.7.10.1]# make prefix=/usr/local all

[root@oraclelinux6 git-1.7.10.1]# make prefix=/usr/local install

[root@oraclelinux6 git-1.7.10.1]# exit 

$ git --version
git version 1.7.10.1

Once git and the ssh-keys are ready you can download:



[oracle@oraclelinux6 ~]$ cd /media/workspace/
[oracle@oraclelinux6 workspace]$ git clone https://github.com/webscalesql/webscalesql-5.6.git
Cloning into 'webscalesql-5.6'...
remote: Counting objects: 28637, done.
remote: Compressing objects: 100% (10926/10926), done.
remote: Total 28637 (delta 17074), reused 28419 (delta 16936)
Receiving objects: 100% (28637/28637), 48.01 MiB | 796 KiB/s, done.
Resolving deltas: 100% (17074/17074), done.

OK so however you decided to get the code... Lets us now try to get it installed. To be clear this is should nothing new, this is a basic MySQL source install but it is just using the WebScaleSQL package. It basically follows the same procedure as I did here before.

I needed a few dependancies on this clean system first. You might find you need some as well, it all depends on your system and the result of the cmake command below.

 

[root@oraclelinux6 workspace]# yum -y install ncurses-devel   readline-devel cmake gcc gcc-c++ bison-devel bison 

[root@oraclelinux6 workspace]# wget http://googlemock.googlecode.com/files/gmock-1.6.0.zip

[root@oraclelinux6 workspace]# unzip gmock-1.6.0.zip

[root@oraclelinux6 workspace]# cd gmock-1.6.0 

[root@oraclelinux6 gmock-1.6.0]# ./configure

[root@oraclelinux6 gmock-1.6.0]# make

[root@oraclelinux6 gmock-1.6.0]# make check

.... 

=====================================================
1 of 4 tests failed
Please report to googletestframework@googlegroups.com
=====================================================
make[2]: Leaving directory `/media/workspace/gmock-1.6.0/gtest'
make[1]: Leaving directory `/media/workspace/gmock-1.6.0/gtest'

OK so the next phase (part 2) should be the following once I get the google code to work...

Btw the denable_downloads should download and install the gmock as well but it also fails.

 

[root@oraclelinux6 workspace]# groupadd mysql
[root@oraclelinux6 workspace]# useradd -r -g mysql mysql
[root@oraclelinux6 workspace]# ln -s /media/workspace/webscalesql-5.6/ /usr/local/mysql
[root@oraclelinux6 workspace]# chown -R mysql:mysql webscalesql-5.6/

[root@oraclelinux6 workspace]# cd webscalesql-5.6/ 

[root@oraclelinux6 webscalesql-5.6]# cmake -DENABLE_DOWNLOADS=1 .

make

make install 

./scripts/mysql_install_db --user=mysql

chown -R root .

chown -R mysql data

So not ready just yet.. but I will get back with you all once I get it working. I have faith I will.

Saturday, May 4, 2013

Building from Source MySQL :: MariaDB :: Percona

It is possible to run more than one MySQL server on the same server. At times people might like to install another version of a database on the same hardware for testing purposes, as well as evaluations.

Installing the databases from source and custom installations for each is easier than it might sound to some. I would suggest to review MySQL Sandbox first though, because it allows for evaluations and testing to be done very quickly and easily. However, installing from source worked out better for me when I did some comparisons. Below is the process I used. I will be looking to build out future blog posts with these databases once I tune then configurations.

http://dev.mysql.com/doc/refman/5.6/en/installing-source-distribution.html

This is the default information from mysql.com. I already had MySQL installed so I did not execute the following but I wanted this here for reference. You can compare these steps to the MySQL, MariaDB and Percona source installations below too see how I updated the default steps in order to get all three versions of the database running on the same box. (No production value here was done just to test the process.)

# Preconfiguration setup
shell> groupadd mysql
shell> useradd -r -g mysql mysql

# Beginning of source-build specific instructions
shell> tar zxvf mysql-VERSION.tar.gz
shell> cd mysql-VERSION
shell> cmake .
shell> make
shell> make install
# End of source-build specific instructions

# Postinstallation setup
shell> cd /usr/local/mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data

# Next command is optional
shell> cp support-files/my-medium.cnf /etc/my.cnf
shell> bin/mysqld_safe --user=mysql &

# Next command is optional
shell> cp support-files/mysql.server /etc/init.d/mysql.server

If you prefer to use the mysql.server script for start and stop make sure to review and edit accordingly.

 
# Preconfiguration setup

shell> groupadd oracle_mysql

shell> useradd -r -g

oracle_mysql

 oracle_mysql

# Beginning of source-build specific instructions
shell> tar zxvf MariaDB-VERSION.tar.gz
shell> cd MySQL-VERSION
shell> cmake .
shell> make
shell> make install DESTDIR="/usr/local/

oracle_mysql-tmp"

# End of source-build specific instructions

I do not like the results

-- Installing: /usr/local/oracle_mysql-tmp/usr/local/mysql/

If DESTDIR is should install into that location not start with user under that location. This is a MySQL original issue as it does this with all versions of MySQL.

# Fix the odd/bug setup
shell> cd /usr/local/

oracle_mysql-tmp

shell> mv usr/local/mysql/ ../oracle_mysql ;

shell> cd ../; # rm -Rf oracle_mysql-tmp

# Postinstallation setup

shell> cd /usr/local/oracle_mysql


shell> chown -R

oracle_mysql .

shell> chgrp -R oracle_mysql .

# Next command is optional
shell> cp support-files/my-small.cnf /etc/

oracle_mysql.cnf

shell> vi /etc/oracle_mysql.cnf

port = 3309

socket = /tmp/oracle_mysql.sock

shell> scripts/mysql_install_db --defaults-file=/etc/oracle_mysql.cnf --basedir=/usr/local/oracle_mysql --skip-name-resolve --datadir=/var/lib/oracle_mysql --user=oracle_mysql

shell> chown -R oracle_mysql /var/lib/oracle_mysql/*


shell> # bin/mysqld_safe --defaults-file=/etc/oracle_mysql.cnf --user=oracle_mysql  --datadir=/var/lib/oracle_mysql/ --port=3309 &

shell> # bin/mysql --port=3309 --socket=/tmp/oracle_mysql.sock

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.5.31 Source distribution

# Preconfiguration setup
shell> groupadd mariadb
shell> useradd -r -g mariadb mariadb

# Beginning of source-build specific instructions
shell> tar zxvf MariaDB-VERSION.tar.gz
shell> cd MariaDB-VERSION
shell> cmake .
shell> make
shell> make install DESTDIR="/usr/local/mariadb-tmp"
# End of source-build specific instructions

I do not like the results

-- Installing: /usr/local/mariadb-tmp/usr/local/mysql/

If DESTDIR is should install into that location not start with user under that location. This is a MySQL original issue as it does this with all versions of MySQL.

# Fix the odd/bug setup
shell> cd /usr/local/mariadb-tmp

shell> mv usr/local/mysql/ ../mariadb ;

shell> cd ../; # rm -Rf mariadb-tmp

# Postinstallation setup

shell> cd /usr/local/mariadb
shell> chown -R mariadb .

shell> chgrp -R mariadb .

# Next command is optional
shell> cp support-files/my-small.cnf /etc/mariadb.cnf

shell> vi /etc/mariadb.cnf

port = 3308

socket = /tmp/mariadb.sock

shell> scripts/mysql_install_db --defaults-file=/etc/mariadb.cnf --basedir=/usr/local/mariadb --skip-name-resolve --datadir=/var/lib/mariadb --user=mariadb

shell> chown -R mariadb /var/lib/mariadb/*


shell> # bin/mysqld_safe --defaults-file=/etc/mariadb.cnf --user=mariadb  --datadir=/var/lib/mariadb/ --port=3308 &

shell> # bin/mysql --port=3308 --socket=/tmp/mariadb.sock

Welcome to the MariaDB monitor. Commands end with ; or \g.

Your MariaDB connection id is 1

Server version: 5.5.30-MariaDB Source distribution

# Preconfiguration setup
shell> groupadd percona
shell> useradd -r -g percona percona

# Beginning of source-build specific instructions
shell> tar zxvf Percona-Server-VERSION.tar.gz
shell> cd Percona-Server-VERSION
shell> cmake .
shell> make
shell> make install DESTDIR="/usr/local/percona-tmp"
# End of source-build specific instructions

I do not like the results

-- Installing: /usr/local/percona-tmp/usr/local/mysql/

If DESTDIR is should install into that location not start with user under that location. This is a MySQL original issue as it does this with all versions of MySQL.

# Fix the odd/bug setup
shell> cd /usr/local/percona-tmp

shell> mv usr/local/mysql/ ../percona ;

shell> cd ../; # rm -Rf percona-tmp

# Next command is optional
shell> cp support-files/my-small.cnf /etc/percona.cnf

shell> vi /etc/percona.cnf

port = 3307

socket = /tmp/percona.sock



# Postinstallation setup

shell> cd /usr/local/percona
shell> chown -R percona .

shell> chgrp -R percona .




shell> scripts/mysql_install_db --defaults-file=/etc/percona.cnf --basedir=/usr/local/percona --skip-name-resolve --datadir=/var/lib/percona --user=percona 

shell> chown -R percona /var/lib/percona/*

shell> # bin/mysqld_safe --defaults-file=/etc/percona.cnf --user=percona  --datadir=/var/lib/percona/ --port=3307 











shell> # bin/mysql --port=3307 --socket=/tmp/percona.sock 

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.5.30 Source distribution




Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Now I have access to all 3 flavors or MySQL.

For easy client access I added this to my .bashrc file:

alias percona='/usr/local/percona/bin/mysql --port=3307 --socket=/tmp/percona.sock'
alias oracle_mysql='/usr/local/oracle_mysql/bin/mysql --port=3309 --socket=/tmp/oracle_mysql.sock'
alias maria='/usr/local/mariadb/bin/mysql --port=3308 --socket=/tmp/mariadb.sock'

[Warning] .....because the user was set to 'mysql' earlier on the command line


shell> scripts/mysql_install_db --basedir=/usr/local/demouser  --datadir=/var/lib/demodb --user=demouser --ldata=/var/lib/demodb 



Installing MySQL system tables...



[Warning] Ignoring user change to 'demouser' because the user was set to 'mysql' earlier on the command line



Installation of system tables failed!

This is an error that makes you look back to the command you just entered and begin to question yourself. The error is not as it appears. If you are installing MySQL onto a server that already has an installation or once did it is likely that a my.cnf file is currently in place. This file could be very valid for the active server and it is ok to be left alone in those cases. The fix for this error is to override the defaults and enable the MySQL server to pull from a different file that includes the username you prefer. You can change the username in the file but it is likely that you are looking to test and do other things, so below is an example of how to get the error above up and running again.

shell> cp support-files/my-small.cnf /etc/demodb.cnf

shell> vi /etc/demodb.cnf

port = 3307



       socket         = /tmp/demodb.sock




       user            = demouser 

       pid_file       = /var/lib/demodb/demodb.pid





shell> scripts/mysql_install_db --defaults-file=/etc/demodb.cnf --basedir=/usr/local/demouser  --datadir=/var/lib/demodb --user=demouser --ldata=/var/lib/demodb 





Installing MySQL system tables...

OK

Filling help tables...

OK

Again, if a my.cnf file currently exists the mysql_install_db will try to use that file. Often in that file a username is set which led to the error "the user was set to 'mysql' earlier on the command line"

shell> chown -R demouser /var/lib/demodb/*

shell> # bin/mysqld_safe --defaults-file=/etc/demouser.cnf --user= demouser  --datadir=/var/lib/demodb/ --port=3307


# Next command is optional
shell> cp support-files/mysql.server /etc/init.d/demodb.server

shell> # bin/mysql --port=3307 --socket=/tmp/demodb.sock