Larry's Stuff

Dropping a MySQL trigger gets blocked by your own lock While documenting the process of using triggers to smooth staggered deployments, I found what looks like a bug in MySQL. I wanted to lock the affected table for write in order to perform both the alter table and drop trigger atomically. It appears that the dropping of the trigger will wait until the write lock is released even though the write lock is held by your own session. The end result is that the drop trigger statement hangs indefinitely and you have to hit ctrl-c. The other weirdness is that the hung drop trigger statement does not show up in show processlist but instead shows up as sleeping.

I've tested this on 5.0.44sp1, 5.0.45, and 5.0.51a.

Using the previous trigger example you can hang your session...

test @ monster 12:17 > lock tables tbl1 write;
Query OK, 0 rows affected (0.00 sec)

test @ monster 12:18 > drop trigger birth_update_trig;
ERROR 2013 (HY000): Lost connection to MySQL server during query
test @ monster 12:20 > 


As you can see, after a couple of minutes I logged in on a different term and killed the above thread. Here is what the other session looked like...

test @ monster 12:20 > show processlist;
+------+--------+-----------------+------+---------+------+-------+------------------+
| Id   | User   | Host            | db   | Command | Time | State | Info             |
+------+--------+-----------------+------+---------+------+-------+------------------+
| 1348 | larryd | localhost:54746 | test | Sleep   |   27 |       | NULL             | 
| 1349 | larryd | localhost:54747 | test | Query   |    0 | NULL  | show processlist | 
+------+--------+-----------------+------+---------+------+-------+------------------+
2 rows in set (0.01 sec)

test @ monster 12:20 > kill 1348;
Query OK, 0 rows affected (0.00 sec)

test @ monster 12:20 > 
How to use a trigger to keep different columns up to date in MySQL Recently an internal customer asked for an example on how to keep some columns updated in MySQL. They currently have three columns birth_day, birth_month and birth_year which they would like to consolidate down to one birth_date column. They want to be able to add the new column and deploy their new code without having a huge impact on the site.

I came up with these triggers...

CREATE TABLE if not exists `tbl1` (
  `name` varchar(64) NOT NULL default '',
  `birth_year` smallint(4) unsigned NOT NULL default '0',
  `birth_month` tinyint(2) unsigned ZEROFILL NOT NULL default '0',
  `birth_day` tinyint(2) unsigned ZEROFILL NOT NULL default '0',
  `birth_date` date NOT NULL default '0000-00-00'
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


DELIMITER |

CREATE TRIGGER birth_update_trig BEFORE UPDATE ON tbl1
FOR EACH ROW BEGIN
SET NEW.birth_date = IF(NEW.birth_date != OLD.birth_date, NEW.birth_date, CONCAT(LPAD(NEW.birth_year, 4, '0'), '-', LPAD(NEW.birth_month, 2, '0'), '-', LPAD(NEW.birth_day, 2, '0')));

SET NEW.birth_day = IF(NEW.birth_day != OLD.birth_day, NEW.birth_day, DAY(NEW.birth_date));
SET NEW.birth_month = IF(NEW.birth_month != OLD.birth_month, NEW.birth_month, MONTH(NEW.birth_date));
SET NEW.birth_year = IF(NEW.birth_year != OLD.birth_year, NEW.birth_year, YEAR(NEW.birth_date));

END;
|

CREATE TRIGGER birth_insert_trig BEFORE INSERT ON tbl1
FOR EACH ROW BEGIN

SET NEW.birth_date = IF(NEW.birth_date != '0000-00-00', NEW.birth_date, CONCAT(LPAD(NEW.birth_year, 4, '0'), '-', LPAD(NEW.birth_month, 2, '0'), '-', LPAD(NEW.birth_day, 2, '0')));

SET NEW.birth_day = IF(NEW.birth_day != 0, NEW.birth_day, DAY(NEW.birth_date));
SET NEW.birth_month = IF(NEW.birth_month != 0, NEW.birth_month, MONTH(NEW.birth_date));
SET NEW.birth_year = IF(NEW.birth_year != 0, NEW.birth_year, YEAR(NEW.birth_date));

END;
|

DELIMITER ;


Using this, we should be able to add the new column and keep it populated while waiting for the new code. If there is some small window where some app servers are updated with the new code while others are not, it shouldn't pose a problem.
Nintendo DS AutoSync (using udev under linux) I've recently bought an r4ds slot-1 adapter for the Nintendo DS.

The only problem I've ran into so far is that it doesn't support sdhc yet. This means that you're limited to 2GB microSD cards. You could try to find a non sdhc 4GB card but I looked and it didn't seem like a trivial task and I wasn't positive that it would even work anyways. There are also rumors that the r4ds will get sdhc support in a future kernel update, though some question if it is even possible.

For now I have to live with a 2GB limit on storage for all Backups/Music/Movies. I ended up making a little script that will automagically sync the 2GB card with a subset of data that I keep updated with hardlinks. I also have this script backup the *.SAV files from the card so I have a history of different save points that I could optionally go back to if needed.

Here is the script thus far...

#!/bin/bash

USERNAME=$1
BASEPATH="/storage/Games/NDS/${USERNAME}/"

LOGFILE="${BASEPATH}/sync.log"

SAVESPATH="Saves"

GAMESPATH="Games"
MUSICPATH="Music"
PHOTOPATH="Photo"
VIDEOPATH="Video"

DEVICE="/dev/${USERNAME}_nds"
MOUNTPATH="${BASEPATH}/mount/"

function main() {
        # Do some sanity checks
        CheckSanity

        # Mount the drive
        MountDrive

        # Sync the saves
        SyncSaves

        # Sync the media
        SyncPath ${GAMESPATH}
        SyncPath ${MUSICPATH}
        SyncPath ${PHOTOPATH}
        SyncPath ${VIDEOPATH}

        # Unmount the drive
        UnmountDrive

        log "----- Finished Successfully -----"
        exit 0
}

function log() {
        echo "$@" >> ${LOGFILE}
}

function die() {
        log `id`
        log "$@"

        if [ "${MOUNTED}" == "1" ]; then
                UnmountDrive
        fi

        log "----- Finished With Error -----"
        exit 1
}

function CheckSanity() {
        [ "${USERNAME}" != "" ] || die "Empty username passed in"
        [ -d ${BASEPATH} ] || die "Invalid base dir [ ${BASEPATH} ]"

        # Verify mount path exists
        [ -d ${MOUNTPATH} ] || mkdir -p ${MOUNTPATH} 2>/dev/null
        [ -d ${MOUNTPATH} ] || die "Unable to create mount path [ ${MOUNTPATH} ]"
}

function MountDrive() {
        log "Mounting drive..."
        mount ${DEVICE} ${MOUNTPATH} || die "Unable to mount device [ ${DEVICE} ] to mountpath [ ${MOUNTPATH} ]"
        log "done."
}

function UnmountDrive() {
        log "Unmounting drive..."
        umount ${MOUNTPATH} || die "Unable to unmount [ ${MOUNTPATH} ]"
        log "done."
}

function SyncSaves() {
        if [ "${SAVESPATH}" == "" ]; then
                log "Skipping [ Saves ]... SLAVESPATH is not defined!"
                return
        fi

        if [ "${GAMESPATH}" == "" ]; then
                log "Skipping [ ${SAVESPATH} ]... GAMESPATH is not defined!"
                return
        fi

        SAVECOUNT=`ls -al ${MOUNTPATH}/${GAMESPATH}/*.SAV 2>/dev/null |wc -l`
        if [ ${SAVECOUNT} -lt 1 ]; then
                log "Skipping [ ${SAVESPATH} ]... No saves found on card"
                return
        fi

        NOWSAVESPATH="${BASEPATH}/${SAVESPATH}/`date +%Y-%m-%d:%H:%M:%S`/"
        mkdir -p ${NOWSAVESPATH} 2>/dev/null

        if [ ! -d ${NOWSAVESPATH} ]; then
                log "Unable to create new saves dir [ ${NOWSAVESPATH} ]"
                return
        fi

        rsync ${MOUNTPATH}/${GAMESPATH}/*.SAV ${NOWSAVESPATH} >> ${LOGFILE} 2>&1

        if [ "$?" == "0" ]; then
                log "Successfully backed up ${SAVECOUNT} save(s) to [ ${NOWSAVESPATH} ]"
        else
                log "Failure while backing up savegames to [ ${NOWSAVESPATH} ]"
        fi

        # Set permissions
        if id ${USERNAME} >/dev/null 2>&1; then
                chown -R ${USERNAME} ${NOWSAVESPATH} || log "Unable to chown save games [ ${NOWSAVESPATH} to [ ${USERNAME} ]"
        fi
}

function SyncPath() {
        TYPENAME="$1"

        if [ "${TYPENAME}" == "" ]; then
                log "Empty Typename"
                return
        fi

        MYPATH="${BASEPATH}/${TYPENAME}/"

        if [ ! -d ${MYPATH} ]; then
                log "Skipping [ ${TYPENAME} ]... Source directory not found"
                return
        fi

        log ""
        log "\\/ \\/ \\/     Syncing [ ${TYPENAME} ]      \\/ \\/ \\/"
        log ""

        rsync -r -v --ignore-existing --delete-before --exclude=*.SAV --exclude=*.sav ${MYPATH} ${MOUNTPATH}/${TYPENAME}/ >> ${LOGFILE} 2>&1
        log ""
        log "/\\ /\\ /\\ Finished Syncing [ ${TYPENAME} ] /\\ /\\ /\\"
        log ""
}

main


If someone else wanted to use this, they would probably want to change BASEPATH to a proper location for their machine. BASEPATH is meant to contain a local copy of the subset of data you want to copy to the microSD card. I just put mine on my /storage partition so I could benefit by using hardlinks.

The next piece needed to make this all work is to write a udev rule to fire off the script when the proper usb card reader is inserted into the server. For this, you need the vendor and product ids for your card reader. You get these numbers by running lsusb. I ran lsusb before and after inserting the card reader and ran a diff against each output...

root@monster:~# lsusb >d1
root@monster:~# # insert card reader
root@monster:~# lsusb >d2
root@monster:~# diff d1 d2
5a6
> Bus 001 Device 049: ID 05e3:0715 Genesys Logic, Inc. 
root@monster:~# 


The output above shows that my card reader's vendor id is 05e3 while the product id is 0715. We'll need these values for the udev rule below...

root@monster:~# cat /etc/udev/rules.d/85-nds.rules 
# ACTION           - Match when plugging in card reader
# KERNEL           - Pattern that your card reader gets mounted as (/dev/sdb1)
# SYSFS{idVendor}  - VendorId of your card reader (get with lsusb)
# SYSFS{idProduct} - ProductId of your card reader (get with lsusb)
# SYMLINK          - Create a symlink to /dev/username_nds (script expects this)
# RUN              - Location of script (make sure to pass in the username)

ACTION=="add", KERNEL=="sd*1", SYSFS{idVendor}=="0781", SYSFS{idProduct}=="a7a8", SYMLINK+="wife_nds", RUN+="/usr/local/bin/nds_mount_hook.sh wife"
ACTION=="add", KERNEL=="sd*1", SYSFS{idVendor}=="05e3", SYSFS{idProduct}=="0715", SYMLINK+="shizno_nds", RUN+="/usr/local/bin/nds_mount_hook.sh shizno"
root@monster:~# 


All I had to do was drop the above text into /etc/udev/rules.d/85-user_nds.rules and presto. Ubuntu has udev installed by default and I didn't even need to restart it to pick up changes, even while testing.
How to tell if two files live on the same partition
diff -q <(df /path/to/file1) <(df /path/to/file2)
MySQL Pager I know this is probably old news to most people, but I just found out about it. You can use the pager command in mysql to pipe the resultset through a filtering app like so...

test@localhost> select * from foo;
+------+--------------------------+
| Id   | bar                      |
+------+--------------------------+
|    9 | test test test           |
|    9 | test test test           |
|    3 | foo bar                  |
|   99 | yoo zoo                  |
|    5 | yikes                    |
| 5555 | this is a test from code |
| 5555 | this is a test from code |
+------+--------------------------+
7 rows in set (0.00 sec)

test@localhost> pager grep 'oo'
PAGER set to 'grep 'oo''
test@localhost> select * from foo;
|    3 | foo bar                  |
|   99 | yoo zoo                  |
7 rows in set (0.00 sec)


Here are some other good usages that I could think of off the top of my head...

pagination:   pager less
edit result:  pager vim -
pastebin:     pager pastebinit
save to file: pager cat >filename

Here is a cheezy script that I made that will hilight up to 14 rows in a resultset a different color each...


#!/bin/bash

BOLDSTART=`echo -e "\e[1m"`
BOLDEND=`echo -e "\e[m"`

COLORSTART=`echo -e "\033["`

HILITESTART=`echo -e "\033[30m"`
HILITEEND=`echo -e "\033[39m\033[22m"`

COL="| [^|]* "

    sed                                       "s/^\(| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}31m\2${HILITEEND}${BOLDEND} |/g" 
    | sed                               "s/^\(${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}32m\2${HILITEEND}${BOLDEND} |/g" 
    | sed                         "s/^\(${COL}${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}33m\2${HILITEEND}${BOLDEND} |/g" 
    | sed                   "s/^\(${COL}${COL}${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}34m\2${HILITEEND}${BOLDEND} |/g" 
    | sed             "s/^\(${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}35m\2${HILITEEND}${BOLDEND} |/g" 
    | sed       "s/^\(${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}36m\2${HILITEEND}${BOLDEND} |/g" 
    | sed "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${BOLDSTART}${COLORSTART}37m\2${HILITEEND}${BOLDEND} |/g" 
    | sed                                     "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}31m\2${HILITEEND} |/g" 
    | sed                               "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}32m\2${HILITEEND} |/g" 
    | sed                         "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}33m\2${HILITEEND} |/g" 
    | sed                   "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}34m\2${HILITEEND} |/g" 
    | sed             "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}35m\2${HILITEEND} |/g" 
    | sed       "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}36m\2${HILITEEND} |/g" 
    | sed "s/^\(${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}${COL}| \)\([^|]*\) |/\1${COLORSTART}37m\2${HILITEEND} |/g"

I'm not sure I like it yet, but it does make it handy for resultsets that wrap lines. If I do end up liking it, I will probably add this to my .my.cnf file...

[mysql]
pager = /path/to/sqlhilite.sh
Cloning a table If you would like to clone a table in mysql, there are several ways to do it. One method that I see a lot of people using is to create the table via the contents of a select statement.

test@localhost> show create table old\G
*************************** 1. row ***************************
       Table: old
Create Table: CREATE TABLE `old` (
  `Id` int(11) NOT NULL auto_increment,
  `bar` varchar(16) NOT NULL,
  PRIMARY KEY  (`Id`),
  KEY `barIdx` (`bar`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8


test@localhost> create table new select * from old;


The problem here is that we just lost our indexes and charset...

test@localhost> show create table new\G
*************************** 1. row ***************************
       Table: new
Create Table: CREATE TABLE `new` (
  `Id` int(11) NOT NULL default '0',
  `bar` varchar(16) character set utf8 NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1


A better method would be to use two separate statements to create and then insert...

CREATE TABLE new LIKE old;
INSERT INTO new SELECT * FROM foo;


This has the advantage of not dropping your indexes and charset on the floor.

test@localhost> show create table new\G
*************************** 1. row ***************************
       Table: new
Create Table: CREATE TABLE `new` (
  `Id` int(11) NOT NULL auto_increment,
  `bar` varchar(16) NOT NULL,
  PRIMARY KEY  (`Id`),
  KEY `barIdx` (`bar`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8


Be warned however, it is still not perfect. It will lose any foreign key constraints that you may have should your table be using the InnoDB storage engine.

I used mysql version 5.0.38 for all of the examples pasted above.

test@localhost> show variables like 'version';
+---------------+----------------------------+
| Variable_name | Value                      |
+---------------+----------------------------+
| version       | 5.0.38-Ubuntu_0ubuntu1-log |
+---------------+----------------------------+

RSS Feed Yesterday I wrote an rss feed for the photo section of this site. I wanted my rss feed to automatically show up in IE or FF via the little orange rss icon. I went to a site that exhibited this behavior and viewed source. It was just as easy as you'd think, but I wanted to write it down here for future reference.

<link rel="alternate" type="application/rss+xml"
     title="ldamman.com - MMS Photos" href="/?a=mmsphoto&amp;out=rss" />
nsupdate I'm now using nsupdate to manage my domains. I'm posting this as a reminder for the next time my ip changes. Who knows, it might be useful to someone.

shizno@monster:~$ nsupdate -k /path/to/funkykeyfile.key
> update add shizno.com 86400 A 83.99.122.23
> update add www.shizno.com 86400 A 83.99.122.23
> update add shizno.com 86400 MX 10 shizno.com
> send
shizno@monster:~$ dig @ns.linuxrocket.net shizno.com AXFR -k /path/to/funkykeyfile.private
Welcome To This Blog This is the first post on this blog. I will try to keep things interesting from here on out.

Copyright ©2005-2007 BuffSoft. All rights reserved.