Sunday, June 30, 2013

MySQL 5.6 on Solaris Using a Solaris PKG

The MySQL.com site does a good job with the documentation on installing MySQL on Solaris. The hyperlinks can be found below.

I create this post as a reference and an example of how it done as well.
(I prefer the bash shell but that is just me... )

bash-3.2# groupadd mysql
bash-3.2# useradd -g mysql mysql
bash-3.2#  pkgadd -d mysql-5.6.12-solaris*.pkg

The following packages are available:
  1  mysql     MySQL Community Server (GPL)
              5.6.12

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: all


Now you can take a little break. Solaris takes a little bit ot process and install here.


 Processing package instance <mysql> from </export/home/mysql-5.6.12-solaris10*.pkg>

MySQL Community Server (GPL) 5.6.12
Copyright (c) 2000, , Oracle and/or its affiliates. All rights reserved.
Use is subject to license terms.
Using </opt/mysql> as the package base directory.
## Processing package information.
## Processing system information.
   1 package pathname is already properly installed.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

The selected base directory </opt/mysql> must exist before
installation is attempted.

Do you want this directory created now [y,n,?,q] y
Using </opt/mysql> as the package base directory.
## Processing package information.
## Processing system information.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of <mysql> [y,n,?] y


Installing MySQL Community Server (GPL) as <mysql>
.....


Again let it do some work...


......2013-06-30 17:40:12 1117 [Note] InnoDB: Starting shutdown...
2013-06-30 17:40:12 1117 [Note] InnoDB: Shutdown completed; log sequence number 1625987

A RANDOM PASSWORD HAS BEEN SET FOR THE MySQL root USER !
You will find that password in '/.mysql_secret'.

You must change that password on your first connect,
no other statement but 'SET PASSWORD' will be accepted.
See the manual for the semantics of the 'password expired' flag.

Also, the account for the anonymous user has been removed.

In addition, you can run:

  /opt/mysql/mysql/bin/mysql_secure_installation

which will also give you the option of removing the test database.
This is strongly recommended for production servers.

See the manual for more instructions.

Please report any problems with the /opt/mysql/mysql/bin/mysqlbug script!

The latest information about MySQL is available on the web at

  http://www.mysql.com

Support MySQL by buying support/licenses at http://shop.mysql.com

New default config file was created as /opt/mysql/mysql/my.cnf and
will be used by default by the server when you start it.
You may edit this file to change server settings


Installation of <mysql> was successful.



So now following the instructins per dev.mysql.com 

bash-3.2# ln /etc/init.d/mysql /etc/rc3.d/S91mysql
bash-3.2# ln /etc/init.d/mysql /etc/rc0.d/K02mysql


So now if I want to use the database I must get past the root password that was dynamically created 1st. I personally like this feature.


#  /etc/init.d/mysql start
Starting MySQL
........ SUCCESS!
bash-3.2# ls -al /.mysql_secret
-rw-------   1 root     other         96 Jun 30 17:40 /.mysql_secret
# more /.mysql_secret
# The random password set for the root user at Sun Jun 30 17:40:07 2013 (local time): c29IHIZ4


So let us log in and change the password then we are ready to get started. Frankly you have no other choice.


bash-3.2# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.12

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.


mysql> show grants for 'root'@'localhost';
ERROR 1820 (HY000): You must SET PASSWORD before executing this statement

mysql> SET PASSWORD FOR  'root'@'localhost' = PASSWORD('somepassword');
Query OK, 0 rows affected (0.04 sec)


And that is it. We are now good and ready to go...


mysql> exit
Bye
bash-3.2# rm -f /.mysql_secret
bash-3.2# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.12 MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.6.12    |
+-----------+
1 row in set (0.00 sec)



References || Helpful Links:









Saturday, June 15, 2013

Backup and Recovery Script for MySQL using Percona's innobackup and Xtrabackup

So Percona has the widely used backup tool Xtrabackup and they realize that everyone often uses this tool in a script of some kind. The have a page that talks about this:
http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/scripting_backups_xbk.html?id=percona-xtrabackup:xtrabackup:scripting

Since I recently gave an example of how to use the backup in a previous post. I figured I might as well write a script that shows how to script the backup process. Plus it has been years since I wrote in Python so I wanted to get a little practice as well.

So the introduction to the code is below but I have placed the script on github.
It needs more testing but feel free to check the codebase out and update and edit.

Once the code is tested more I can update examples but I wanted to be open with this project from the start.

Since it is in the early stages I would recommend using the --showcommands=1 option so you can see what the code plans to do and maybe try those commands. Obviously it should not be used on a production system yet.


First an introduction to it:

# ./backup_restore.py --help
Usage: backup_restore.py  --process=[fullbackup,incremental,prepare,restore] --help --version --showcommands=1

This program enables you to backup full and incremental backups then prepare
and restore them using Percona's Xtrabackup

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --process=PROCESS     What would you like to do --process=
                        [fullbackup,incremental,prepare,restore]
  --debug=DEBUG         TURN DEBUG ON 1 OR OFF 0 OR VERBOSE 3
  --showcommands=SHOWCOMMANDS
                        Shows the commands instead of executing them except
                        for the restore section because we go through that
                        step by step
  --backup_root_directory=BACKUP_ROOT_DIRECTORY
                        THE ROOT DIRECTORY OF ALL YOUR BACKUPS, You can set
                        DEFAULT at start of the script
  --percona_xtrabackup_location=PERCONA_XTRABACKUP_LOCATION
                        THE LOCATION OF YOUR xtrabackup FILE, You can set
                        DEFAULT at start of the script
  --datadir=DATADIR     MYSQL DATA DIR LOCATION, You can set DEFAULT at start
                        of the script
  --username=DB_USERNAME
                        MySQL Username, You can set DEFAULT at start of the
                        script
  --password=DB_PASSWORD
                        MySQL Password, You can set DEFAULT at start of the
                        script
  --default_file=DEFAULT_FILE
                        MySQL my.cnf file location, You can set DEFAULT at
                        start of the script
  --options=PERCONA_OPTIONS
                        Additional Options for innobackupex




Thursday, June 13, 2013

max_binlog_cache_size

As you evaluate your database performance and stability it is very likely that you will start to review your variables.

At a glance the typical first reaction to the variables below is.. WAIT something is wrong my box does not have that much RAM or even disk space to meet that MAX limits listed below....

MariaDB [(none)]> select @@max_write_lock_count, @@max_binlog_cache_size, @@max_seeks_for_key, @@myisam_max_sort_file_size\G
*************************** 1. row ***************************
     @@max_write_lock_count: 4294967295                -- 4GB
    @@max_binlog_cache_size: 1844674407370954752      --1.6EB
        @@max_seeks_for_key: 429496729                 -- 4GB
@@myisam_max_sort_file_size: 9223372036853727232        --8EB


You are not alone in concerns with these variables as a few bugs have been listed about these variables over the years. Below are just a few of some legacy ones.


"MySQL currently cannot work with binary log positions greater than 4GB."
Keep in mind that these are just the DEFAULT and MAX settings. You can adjust them to make you feel more comfortable.

MariaDB [(none)]> SET GLOBAL max_binlog_cache_size = 4294967296;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SELECT @@max_binlog_cache_size;
+-------------------------+
| @@max_binlog_cache_size |
+-------------------------+
|              4294967296 |  -- 4GB
+-------------------------+
1 row in set (0.00 sec)


Why you would want to... That is an entirely different topic. This is just the upper limit allowed and transactions get split into 4GB anyway. "The maximum recommended value is 4GB", so you can update it if you so choose too.

Read more about your options with this in the MySQL Documentation:
http://dev.mysql.com/doc/refman/5.5/en/replication-options-binary-log.html#sysvar_max_binlog_cache_size

Wednesday, June 12, 2013

MariaDB 10.0.3 Alpha install on Fedora 17 x86_64

MariaDB 10.0.3 Alpha was just released.
So for those of you that recall my previous MariaDB 5.5 install post, I decided to see how it works with 10.0.3.

I like some of the features that are going into the MariaDB and Percona releases. Even if you are a big supporter of MySQL, when features are available in these releases that are not being back ported into MySQL releases DBAs have to review their options and make a choice.

So the install....

As I said before from the previous post I have this installed. So I will just to an upgrade first.

[root@Fedora64 10]# rpm -qa | grep maria
mariadb-5.5.31-1.fc17.x86_64
mariadb-server-5.5.31-1.fc17.x86_64
mariadb-libs-5.5.31-1.fc17.x86_64
mariadb-bench-5.5.31-1.fc17.x86_64
mariadb-devel-5.5.31-1.fc17.x86_64

So  packages conflicted at the start.

MariaDB-10.0.3-fedora17-x86_64-client.rpm
MariaDB-10.0.3-fedora17-x86_64-common.rpm
MariaDB-10.0.3-fedora17-x86_64-compat.rpm
MariaDB-10.0.3-fedora17-x86_64-connect-engine.rpm
MariaDB-10.0.3-fedora17-x86_64-devel.rpm
MariaDB-10.0.3-fedora17-x86_64-server.rpm
MariaDB-10.0.3-fedora17-x86_64-shared.rpm
MariaDB-10.0.3-fedora17-x86_64-test.rpm
 

[root@Fedora64 10]# rpm -Uhv *.rpm
warning: MariaDB-10.0.3-fedora17-x86_64-client.rpm: Header V3 DSA/SHA1 Signature, key ID 1bb943db: NOKEY
error: Failed dependencies:
    libodbc.so.2()(64bit) is needed by MariaDB-connect-engine-10.0.3-1.x86_64
    MySQL-devel conflicts with (installed) mariadb-devel-5.5.31-1.fc17.x86_64 


MariaDB-server-10.0.3-1.x86_64 conflicts with file from package mariadb-server-5.5.31-1.fc17.x86_64 
[root@Fedora64 10]# 


So this is just a Virtualbox instance, for demo and evaluation, so I just removed all that I could and had to uninstall.  I was hopeful that an upgrade would work but this is Alpha code still.
ie:

[root@Fedora64 10]# rpm -e mariadb mariadb-server mariadb-bench
[root@Fedora64 10]# rpm -e mariadb-libs perl-DBD-MySQL percona-xtrabackup


So now that the past is cleared out... 

[root@Fedora64 10]# rpm -ihv *.rpm
Preparing...                ########################################### [100%]
   1:MariaDB-common         ########################################### [ 11%]
   2:MariaDB-server         ########################################### [ 22%]
   3:MariaDB-cassandra-engin########################################### [ 33%]
   4:MariaDB-client         ########################################### [ 44%]
   5:MariaDB-devel          ########################################### [ 56%]
   6:MariaDB-shared         ########################################### [ 67%]
   7:MariaDB-test           ########################################### [ 78%]
   8:MariaDB-compat         ########################################### [ 89%]
   9:galera                 ########################################### [100%]


If you recall the past post, I got the init.d script this time..

[root@Fedora64 10]# /etc/init.d/mysql start
Starting MySQL..... SUCCESS!
[root@Fedora64 10]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 10.0.3-MariaDB MariaDB Server


If they are not terrible on performance then I do not see why these are not on by default:

vi /etc/my.cnf
[mysqld]

userstat=1
# http://www.percona.com/doc/percona-server/5.5/diagnostics/user_stats.html?id=percona-server:features:userstatv2 
# https://kb.askmonty.org/en/user-statistics/  
feedback=ON
# https://kb.askmonty.org/en/user-feedback-plugin/
MariaDB [(none)]> show variables like '%feedback%';
+--------------------------+------------------------------------------+
| Variable_name            | Value                                    |
+--------------------------+------------------------------------------+
| feedback_send_retry_wait | 60                                       |
| feedback_send_timeout    | 60                                       |
...
| feedback_url             | https://mariadb.org/feedback_plugin/post |
| feedback_user_info       |                                          |
+--------------------------+------------------------------------------+

MariaDB [(none)]> show variables like '%userstat%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| userstat      | ON    |
+---------------+-------+




Issues I found 30seconds after install...:

MariaDB [(none)]> show variables;
ERROR 1946 (HY000): Failed to load replication slave GTID position from table mysql.gtid_slave_pos 


That is it so far... I have it installed and can review now...

UPDATE:
I have submitted this as a bug. The MariaDB team got right back to me and pointed out that I failed to run mysql_upgrade and restart. That fixed the Error listed above. Still variables feels like it should show everything it has but this is a valid fix and mistake on my part. Thank you to the MariaDB team.


To learn more: