Friday, September 6, 2013

MySQL access and replication blocked by secure_auth

ERROR 2049 (HY000): Connection using old (pre-4.1.1) authentication protocol refused (client option 'secure_auth' enabled)

If you have tried to connect to a MySQL database and you see this error then you need to have valid 41byte hash password.  If you are unsure which you have execute the SQL below. If you have 16 character passwords they are older passwords.

select Password from mysql.user;

The following is how I solved this as part of a migration from MySQL 5.0 to MySQL 5.6.

The MySQL 5.0 server had a mixture of the older pre 4.1 passwords and valid 41byte passwords.  Because the MySQL 5.0 server had some accounts with the older passwords I decided to not dump the MySQL table as part of setting up replication. I did dump all of the databases except the mysql database. This allow ensured that I would keep the valid MySQL 5.6 table enhancements.

The MySQL 5.6 server installed easily and was up and I loaded the dump data.  Part of the migration was to use replication while they evaluated the new database. While on the MySQL 5.6 server I tested the replication user account. The response I got was the error at the top of this page. Replication will not run of course without a valid user account. This is why the error logs was giving me this error:
[ERROR] Slave I/O: error connecting to master '<user>@<hostname>:3306' - retry-time: 10  retries: 68, Error_code: 2049

A quick review of the account on the MySQL 5.0 server showed that the new account was established with the pre 4.1 password.  So I needed to upgrade the account to a valid 41 byte password.

The following query showed that they did indeed have old passwords enabled. So I have to disable that and update the user account again to set the password as a valid 41 byte hash.

>SELECT @@session.old_passwords, @@global.old_passwords;
+-------------------------+------------------------+
| @@session.old_passwords | @@global.old_passwords |
+-------------------------+------------------------+
|                       1 |                      1 |
+-------------------------+------------------------+
1 row in set (0.00 sec)


>SET @@session.old_passwords = 0;
Query OK, 0 rows affected (0.00 sec)

>GRANT REPLICATION SLAVE ON *.* TO '<user>'@'<ip_address>' IDENTIFIED BY '<Password>';
Query OK, 0 rows affected (0.00 sec)

A check of the password showed the password as the 41byte password now. I was this able to connect to the primary server from the secondary server and avoid the secure_auth error. replication connected easily and problem was solved.

Going forward I needed to get the MySQL 5.0 users accounts onto the MySQL 5.6 server. ( since I skipped them as part of building the secondary server. )

The client needed to set the grants again for each user regardless of valid password or not.
So I instructed them to execute the following sql. I could have done this but I would need to know all of their passwords and that was not needed.

For each user in their system. You do not have to do the root user because you already have a valid root account on the 5.6 system.

>SET @@session.old_passwords = 0;
>show grants for '<User>'@'<Host>';
To gather the sql needed for each user run the following :
SELECT CONCAT("SHOW GRANTS FOR '",User,"'@'",Host,"';") as sql_command from mysql.user;

For each result given execute the "show grants" statement and then execute the statement given.
The statements should be similar to the following:

GRANT USAGE ON *.* TO 'bob'@'%.example.org' IDENTIFIED BY 'cleartext password';

Replication then created and populated the MySQL table on the MySQL 5.6 server.

More can be found here:
http://dev.mysql.com/doc/refman/5.6/en/password-hashing.html

Also as an update --

Check out http://dev.mysql.com/doc/refman/5.6/en/account-upgrades.html

Monday, September 2, 2013

MySQL Optimization Tip - thread_cache_size

Recently I encountered a MySQL database that easily was running with 300 to 600 rows in the processlist. The Max connections was set easily more than twice this amount as well. This was a set up that I just do not agree with. I was called in because it also proved itself not to be working out very well. So here are a few of my thoughts on the process I uncovered.

  1. In my opinion the majority of MySQL databases in use will not need a max connections or 1500 or more. The more connections you allow the more overhead you bring to your server. Use your connections efficiently. 
  2. Second, Understand the % of threads_created versus the connections used. You might consider this the threads created hit rate.  BTW.. This is not new  information, this is information that has been understood in the community for some time. I do not pretend to present this in any other way than trying to help others. So do the following to understand your current %
    •  show status like 'Threads_created';
      •  set @Threads_created=< result from query above>;
    • show status like 'Connections';
      •  set @Connections=< result from query above>; 
    • select 100 - (( @Threads_created / @Connections ) * 100) as "Threads_created % of Connections"\G



So if your execute the process above what is your percentage ? You want this to be as close to 100 as possible. So for example the server I recently encountered had a % under 10%.  So how do you fix this and raise your %?


 The variable thread_cache_size has a default of 0. If you start to notice your process grow and but queries are not blocked by deadlocks and etc, then you should check your "Threads_created % of Connections" as mentioned above. It is likely that your % will be low. You can raise the % and drastically improve performance of your database by finding the sweet spot that fit your server environment. The thread_cache_size can be changed in a live environment. So this allows you to set the variable then monitor the status value of the "Threads_created" (see above to get value). If this continues to increment in value then continue to raise the thread_cache_size. Typically I prefer to raise the value by 25 at a time for a few then move to 500 at a time. I often check the "Threads_created % of Connections" and the 'Threads_created'. Once you get close to the sweet spot you will notice the % to gain and the processlist to begin to drop in rows. Typically one more adjustment of the thread_cache_size will get you into the sweet spot.

Every server and environment is different.
Some servers might be 98% with a thread_cache_size of 50 while others have a 98% with the thread_cache_size set to 15000. The max is 16384.

So if nothing else... find out what your percentage is first then look into making adjustments.

Wednesday, August 14, 2013

MySQL Workbench 6.0 on CentOS 6.4

I am not really a GUI DBA.  They have their place and can do nice things but I just do not use them much.  You can tell based on what I have upgraded. 

# cat /etc/*-release
CentOS release 6.4 (Final)

# rpm -qa | grep MySQL
MySQL-server-5.6.13-1.el6.x86_64
MySQL-devel-5.6.13-1.el6.x86_64
MySQL-shared-compat-5.6.13-1.el6.x86_64
perl-DBD-MySQL-4.013-3.el6.x86_64
MySQL-python-1.2.3-0.3.c1.1.el6.x86_64
MySQL-client-5.6.13-1.el6.x86_64
MySQL-shared-5.6.13-1.el6.x86_64
MySQL-test-5.6.13-1.el6.x86_6

# rpm -qa | grep mysql
php-mysql-5.3.3-23.el6_4.x86_64
mysql-workbench-gpl-5.2.47-1el6.x86_64


The mysql-workbench does happen to be the latest version in the distro but the latest version is mysql-workbench-community-6.0.6-1.el6 . So I was curious after I saw the latest MySQL workbench blog post : http://mysqlworkbench.org/2013/08/mysql-workbench-6-0-6-ga-released/

So time to upgrade.... 
First, mysql-workbench-community is dependent on libtinyxml so you have to make sure that is installed or gets installed when you upgrade/install.

Others have dealt with this before :
So after I looked and found the tinyxml package...

 yum install tinyxml-2.6.1-1.el6.x86_64.rpm mysql-workbench-community-6.0.6-1.el6.x86_64.rpm

Installed:
  mysql-workbench-community.x86_64 0:6.0.6-1.el6                             tinyxml.x86_64 0:2.6.1-1.el6                           

Replaced:
  mysql-workbench-gpl.x86_64 0:5.2.47-1el6                                                                                          

Complete!


Ok ... so it starts up and has a new look.
Being a shell geek I go to the shell first. Of course it complains that mysql-utilities are not installed. It can take you to the download page but might as well just go ahead of time because you might need to install the Connector-Python as well. So basically... go get these packages and install as well...
mysql-connector-python-1.0.12-1.el6.noarch.rpm
mysql-utilities-1.3.4-1.el6.noarch.rpm
# yum install mysql-*.rpm

So... all of that and I have a GUI that opens a terminal window :)

The following MySQL Utilities are available:
Launching console ...

Utility           Description                                             
----------------  ---------------------------------------------------------
mysqlauditadmin   audit log maintenance utility                           
mysqlauditgrep    audit log search utility                                
mysqldbcompare    compare databases for consistency                       
mysqldbcopy       copy databases from one server to another               
mysqldbexport     export metadata and data from databases                 
mysqldbimport     import metadata and data from files                     
mysqldiff         compare object definitions among objects where the      
                  difference is how db1.obj1 differs from db2.obj2        
mysqldiskusage    show disk usage for databases                           
mysqlfailover     automatic replication health monitoring and failover    
mysqlfrm          show CREATE TABLE from .frm files                       
mysqlindexcheck   check for duplicate or redundant indexes                
mysqlmetagrep     search metadata                                         
mysqlprocgrep     search process information                              
mysqlreplicate    establish replication with a master                     
mysqlrpladmin     administration utility for MySQL replication            
mysqlrplcheck     check replication                                       
mysqlrplshow      show slaves attached to a master                        
mysqlserverclone  start another instance of a running server              
mysqlserverinfo   show server information                                 
mysqluserclone    clone a MySQL user account to one or more new users
 


So moving beyond that I started to look around.
The Administration has a nice new look with some decent graph information easily available for you.  

 The sakila model does look very nice though. The modeller is a handy feature in the GUI I do admit.

Well that is about all I did with the new MySQL workbench but I am upgraded now :) 



Friday, August 9, 2013

Create a Slave ( secondary) server with Percona Xtrabackup



So first you might just save yourself some time and read the Percona example for this:
http://www.percona.com/doc/percona-xtrabackup/2.1/howtos/setting_up_replication.html

But just in case here is an example based on a real situation.

PRIMARY SERVER

# innobackupex /tmp/  <---- this is whatever directory you want to store the backup in. This is a very basic no fluff hot backup.

InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
.........
130809 14:40:11  innobackupex: Connection to database server closed
130809 14:40:11  innobackupex: completed OK!

Make sure you see the xtrabackup_binlog_info file. If you do not you will not easily have the position and log information. You will have to dig into the binlogs based on time and etc. Which is more work than needed.

 innobackupex --apply-log /tmp/<Timestamp Directory Here>

Now up to you. You can rsync the directory to the slave or tar[gzip] then scp to the slave. Regardless of method to move to slave, you have a hotbackup created and ready to go.


SECONDARY SERVER

# /etc/init.d/mysql stop
mv /var/lib/mysql  /var/lib/mysql_ORIG

However you moved the file from the master to the slave, put the contents into the datadir folder, assumed for example: /var/lib/mysql .

# chown -R mysql:mysql mysql
 /etc/init.d/mysql start
Starting MySQL...                                          [  OK  ]

Now in your slave MySQL server, you can set the replication user information easily.

CHANGE MASTER TO
MASTER_HOST='<MASTER_HOST>',
MASTER_USER='<MASTER_USER>',
MASTER_PASSWORD='<MASTER_PASSWORD>',
MASTER_CONNECT_RETRY = 10 ;

Get the log and position from the xtrabackup file.

# more xtrabackup_binlog_info
<BinLog info> <POSITION INFO>

CHANGE MASTER TO MASTER_LOG_FILE='<BinLog info>', MASTER_LOG_POS=<POSITION INFO>;

Start slave;


That is it in a nutshell. For more information review the Percona url given at the start. 

Monday, July 22, 2013

MySQL Sample Databases

I saw a post on the forums.mysql.com site about the sample databases and I thought it might be worth a post to give a quick overview to them for others.

The Sample databases can be found here:  http://dev.mysql.com/doc/index-other.html
You can load these databases via the MySQL command line:

$ tar -vxf sakila-db.tar.gz
$cd sakila-db
$ mysql -u root -p < sakila-schema.sql
Enter password:
$ mysql -u root -p < sakila-data.sql
Enter password:

$ gzip -d world_innodb.sql.gz
$ mysql -u root -p -e "DROP SCHEMA IF EXISTS world";
Enter password:
$ mysql -u root -p -e "CREATE SCHEMA world";
Enter password:
$ mysql -u root -p world < world_innodb.sql
Enter password:

You get the idea. Sakila example database has the Drop SCHEMA and CREATE SCHEMA commands in the file so no need to do that step for that SCHEMA.

You can also use MySQL Workbench to load this data.
  • Create a connection handle to you database. 
  • Use this newly created connection handle to set up a Server Administration instance. 
  • Double click on your new instance.
  • Under Data Export / Restore you should see a Data import.
  • Import from a Self-Contained File
    • File path will be the location of your sakila-schema.sql then repeat for sakila-data.sql
    • You can select a schema or create a new one in the case of world.
    • Select start import and then you will be on the Import Progress view.
You now have access to the sample databases in your database. 

$ mysql -u root -p
> use sakila
> show tables;
> select * from actor limit 3;
+----------+------------+-----------+---------------------+
| actor_id | first_name | last_name | last_update         |
+----------+------------+-----------+---------------------+
|        1 | PENELOPE   | GUINESS   | 2006-02-15 04:34:33 |
|        2 | NICK       | WAHLBERG  | 2006-02-15 04:34:33 |
|        3 | ED         | CHASE     | 2006-02-15 04:34:33 |
+----------+------------+-----------+---------------------+
Via workbench :
  • Close the admin tab
  • Select your connection handle under the SQL Development
  • You can either just type in select * from actor limit 3; and hit the lighting bolt.
  • Or you can type parts of the command and double click on table name or column names to have it populate the names for you. Then select the lighting bolt. 
Now you have data to start playing and learning with.

If you want to add tables to this you can use the MySQL command line or SQL Development and right click on "Tables" under the Schema of your choice and "create new table"