My Random Blog

Make MySQL and GFS play nice

2007-06-14T17:24:09Z

So you want to use a GFS shared storage option and have mysql run its databases on that shared storage? Well I had this problem at work and it took a bit to figure out since there wasn't much documentation on it but it works. It is slower then just using it over ext3. I have never tested it on a mysql cluster so I am unsure of the speeds compared to that So the first thing you want to do is edit your /etc/my.cnf file in the [mysqld] section you want to add this skip-innodb Then in the /etc/init.d/mysql script you want to add --external-locking to the following line so it looks like

/usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --external-locking --pid-file="$mypidfile" --log-error="$errlogfile" >/dev/null 2>&1 &

That is pretty much it.. Now if you want to connect to localhost you need to set a new sock file so in [mysqld] I do something like this socket=/var/lib/sock/mysql.sock and in the [client] section ( create one if one isn't in the cnf file) socket=/var/lib/sock/mysql.sock Then

mkdir /var/lib/sock/ chown mysql:mysql /var/lib/sock/

mysql should now run over gfs!

Webapp from scratch the right way part 1

2007-03-19T17:40:20Z

So I am writing this series to help others that might stumble upon this entry as a starting point for building a good web application from the start. Let me tell you that there is nothing worse then building up a web application just to figure out you need to make a change that requires a change in 8 other sections. So a bit of background. I am the lead developer for Photoblog. Before doing Photoblog, I had around 5-6 years of PHP experience. I made many mistakes and have learned a lot from them. So please use this and the others I write to learn from my mistakes. So my first and foremost advice is to create a wrapper class around the database driver. If you were one of the poor souls that didn't do this and wanted to move from php's mysql to their mysqli class then you had to change a ton of mysql_* commands. So that is where the benefits of a wrapper class comes in. When you need to make some changes, you just do it in the wrapper class. Basically to make a simple mysql query this is what I do

$DB = new Database;
$DB->connect();
$DB->query("SELECT * FROM table WHERE id=1 LIMIT 1");
$DB->fetch_array();

Do you see the benefits to that? You might at first think that it is overkill to do this but its really not. You can do so much with a wrapper class. If you want to run an explain on each query or find the time to complete each query you just add it in the query() function in the wrapper class. Doing this will save you time in the end. I use to write sql queries like this

mysql_query("query here") or die(mysql_error());

So now I just do

$DB->query("query here");

In that query() function I can create my own error page if sql errors out. For Photoblog what I do is if the user is an admin it shows the SQL query and the error. If its a normal user it will email the admins the SQL error and just show a error page that the user can understand and say a email has been sent out. So as you can see there are major benefits to creating a wrapper class for mysql. This is just the tip of the iceberg of what you can do with a wrapper class.

Calculate Distance In Mysql with Latitude and Longitude

2007-03-10T03:25:25Z

So you have a whole table full of members or places with latitude and longitude's associated with them. Just replace the $lat and $lon with the center point you want to find distances from. You can also change the distance<=10 to a number you want to search from. This will limit your results to all results that are under 10 miles from the starting point

SELECT ((ACOS(SIN($lat * PI() / 180) * SIN(lat * PI() / 180) + COS($lat * PI() / 180) * COS(lat * PI() / 180) * COS(($lon - lon) * PI() / 180)) * 180 / PI()) * 60 * 1.1515) AS distance FROM members HAVING distance<='10' ORDER BY distance ASC

Bash Patch to log to mySQL

2007-03-01T22:04:28Z

So at work we wanted to create a centralized syslog server. I had a great idea of why don't we log all user commands run on all servers into mysql. I knew about the bash-paranoia patch so that gave me the idea of using that as my basis for my mysql patch. This should apply to both 3.0 and 3.1 even with all the all the latest security patches included. So first lets download the paranoia patch

wget http://e133.enemy.cx/xSH-paranoia/download/bash-paranoia.patch

Using that and compiling bash with ---enable-paranoia will allow you to log all user commands to syslog. So now lets download my patch now

wget http://zcentric.com/bash-mysql.patch

Now if you want to apply it to bash 3.1 I will give a little howto on how to do that. All in 1 step You want to create the following table in a mysql database

CREATE TABLE `logs` ( `rowid` mediumint(8) NOT NULL auto_increment `host` varchar(100) character set utf8 collate utf8_unicode_ci NOT NULL default '', `user` varchar(100) character set utf8 collate utf8_unicode_ci NOT NULL default '', `as_user` varchar(100) character set utf8 collate utf8_unicode_ci NOT NULL default '', `ip` varchar(100) character set utf8 collate utf8_unicode_ci NOT NULL default '', `ut_line` varchar(100) character set utf8 collate utf8_unicode_ci NOT NULL default '', `command` text character set utf8 collate utf8_unicode_ci NOT NULL, `ts` timestamp NOT NULL default CURRENT_TIMESTAMP, PRIMARY KEY (`rowid`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

Now that mysql table is made lets install bash and patch it and all that good stuff

wget http://ftp.gnu.org/gnu/bash/bash-3.1.tar.gz
tar zxfv bash-3.1.tar.gz
cd bash-3.1
wget http://e133.enemy.cx/xSH-paranoia/download/bash-paranoia.patch
wget http://zcentric.com/bash-mysql.patch
patch -p0 < bash-paranoia.patch
patch -p1 < bash-mysql.patch
autoconf
./configure --enable-paranoia --prefix=/usr
make

Now the config file to let bash know where the mysql server is. Now if the mysql server is down bash shouldn't crash or anything. So you have to create a file /etc/bash.conf and use the following lines.

SERVER=192.168.0.10
USER=username
PASS=password
DB=dbName

Now you should be able to like run /path/to/bash-3.1/bash and it should bring you to a new shell that is now the bash version with mysql. You should now be able to type commands it it will log to mysql! You can then run make install if you wish to install

Get topleft position of a element in javascript

2007-02-15T17:28:17Z

I am doing this as a first part of my series on how to make a Fotonotes system like Flickr uses but from scratch. Ok well almost from scratch. The reason I want to redo this is mainly because Fotonotes imports the note data in an XML form inside the jpeg image. Now this is good for a site like Flickr that only shows one image but I want it to be able to handle 20 images on a page. So my idea was to put everything into a database and then just have the notes be hidden until a user hovers over a image. Then just use javascript to show those images. I am thinking this would be easier on the server since it doesn't have to reload the image each time to show the data. I could be wrong though :) The first thing you want to do is is download dragresize. This pretty much creates the re-sizable boxes so you can select an area you want to add a note to. I pretty much took the example that comes with it and I found this thread that had the perfect solution to find a x/y coordinate of a element on a page in javascript. Check out the second post for the function I am using. Right now it does just what I want. The issue I am running into with this test is that firefox does not move the div to the upper left corner until you actually click on it. IE6 and opera seem to behave how I want. I will have to look into this issue and update. Example of this in action! The example has some bounding issues, but I just think that it is due to it being right at the top. Also its a layout ripped from Photoblog since I need for this to work on that site :)

Great Examples of mySQL Queries

2007-02-01T21:02:03Z

So this is just a quick on to show a nice website I found about great example of queries in mySQL v5 http://www.artfulsoftware.com/queries.php

Remove Leading 0's in Bash

2007-01-25T16:11:23Z

So you have a var you want to remove leading 0's on.. well do the follwing

mzupan@mzupan-desktop:~$ var=00014
mzupan@mzupan-desktop:~$ let var="10#$var"
mzupan@mzupan-desktop:~$ echo $var
14
mzupan@mzupan-desktop:~$

Make the HP dv2120us webcam work in Linux

2007-01-16T19:41:26Z

So here is a little quick little howto on how to get the dv2120us webcam working in Linux. I am using the 2.6.20-rc5 kernel, which at the time of writing this is the latest release candidate kernel. It was not picked up in Linux but a lspci produced the following 03:09.2 System peripheral: Ricoh Co Ltd Unknown device 0843 (rev 01) I think that is it.. I am not sure if it sees it as a usb or pci device or not. The module uses usbcore so its beyond me. So to get it working I ran the following commands

svn checkout http://svn.berlios.de/svnroot/repos/linux-uvc/
cd linux-uvc/linux-uvc/trunk/
make
make install
modprobe uvcvideo

You should see something like this in your dmesg once you load it

[ 1022.224000] usbcore: deregistering interface driver uvcvideo
[ 1045.764000] uvcvideo: Found UVC 1.00 device USB 2.0 Camera (0c45:62c0)
[ 1045.764000] usbcore: registered new interface driver uvcvideo
[ 1045.764000] USB Video Class driver (v0.1.0)

Also on a side not it sees it as a v4l2 device and not a v4l device

Slideshow for Photoblog

2007-01-11T19:24:10Z

So I have been working on a nice little slideshow application that will be used for Photoblog. Its a little bit of html and editing some java code to make it work. I am looking to make this work a big better so it will be easily implemented into any webpage. So first here are two examples of how you can use it Showing most recent post http://www.zcentric.com/examples/slideshow_front.html Show a certain post by date http://www.zcentric.com/examples/slideshow_date.html You can see the different things you can change in the one javascript line for now.. This will probably change but for now it works http://www.photoblog.com/slideshow/js/slideshow.js?user=mikezupan&type=large&timer=6&date=2006-12-3 now you can see some things user : that will be your username type : this is the size of the images you want to show.. the options can be large/medium/small timer : this is the time in seconds you want it to cycle each picture. It will default to 3 so you can't put anything less. date : this is optional but it needs to be in the yyyy-mm-dd format. If you leave it out it will show your most recent post.

ARP Poison A Network

2006-12-17T17:37:10Z

So this little video I made will describe on how to use Cain & Abel to ARP poison a wireless network. It will then show you how you can get all the passwords from clients on the network. So watch and learn. This is one of the rare times I will do something like this in windows. This is the first part in the series and I hope to do more advanced stuff. Learn to ARP poison Here is a Xvid of the video to for the people that don't have flash8 installed Learn to ARP poison (xvid version)

Get the Broadcom BCM4311 to work in Linux

2006-12-12T21:54:39Z

So I was pulling my hair for a bit on this. Now to start off the driver for this card is still under major development along with the whole wireless subsystem in the Linux kernel so this could become outdated soon. So first lets download the kernel. I am gonna use kernel 2.6.18.1 since it just seems to work. There are a few upstream patches for .19 but they locked up my cpu.

cd /usr/src
wget http://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2
wget ftp://lwfinger.dynalias.org/patches/combined_2.6.20.patch

Now lets extract the kernel

tar xfj linux-2.6.20.tar.bz2
ln -s linux-2.6.20 linux

Ok now lets apply the patches

cd linux
patch -p1 < ../combined_2.6.20.patch

Now you want to compile your kernel.. I will not go into this.. there are plenty of howtos that describe it. Now you have to install the bcm43xx-fwcutter package Now you want to get the driver. You have to use v3 of it.. I will post the one I used. From what I know it shouldn't matter if you have a 64 or 32 bit system. I could be wrong though. You want to run cutter as root

cd /tmp
wget http://www.zcentric.com/brcmDrv340rc100a.zip
unzip brcmDrv340rc100a.zip
bcm43xx-fwcutter -w /lib/firmware /tmp/bcmwl5.sys

Once the kernel is installed and you rebooted you should be able to boot into the new kernel and the light for the wireless should turn on which means the radio is on. The following command should return APs

iwlist scan ethX

If for some reason it doesn't work.. try to extract the firmware into the kernel version

mkdir /lib/firmware/2.6.20
bcm43xx-fwcutter -w /lib/firmware/2.6.20 /tmp/bcmwl5.sys

Calling for Super Dodgeball On the Wii

2006-11-27T23:47:28Z

So I've played the Wii for almost a week and I love it. I can't begin to explain how fun the Wii Sports game is and it gets even better with a crowd. So that got me thinking about what old games, I would love to see re-made in Wii form. One of the games I loved playing as a kid was Super Dodgeball for the NES. Its a perfect game to get a face lift and use the Wii for. I mean what person hasn't played dodgeball at sometime in their life. So I am calling for any developing studio out there to PLEASE make a game as fun as the NES version was for the Wii. I am thinking full online play too. Why not right? I mean you have everything built in to make a great game. Throwing and dodging. You could use the nunchuck to move your guy around. I am including some pictures to remember this game in all its glory.

Install NoSteam Counterstrike 1.6 server

2006-11-20T14:38:08Z

So I was a long time Counterstrike player who broke the habit around 3 years ago. A few weeks ago people at my office were saying how we should setup a gaming server and Counterstrike got mentioned. We decided to play 1.6 since none of us have the video cards to play CS:S. So I have setup a server in the 1.3 days for Counterstrike and it was easy to get around connecting in using an invalid CD key. You just removed the auth servers from the cfg file. I forget the filename but no one plays 1.3 anyway. So this little howto will get you to setup a LAN type environment and not worry about having people use steam to auth. You can find on various places a NoSteam version of Counterstrike. This is great for office environments or if you are setting up a LAN event. You could do this and place the server on the Internet but there are so many cheaters they almost destroy the experience. So first lets download all the needed files.

wget http://www.steampowered.com/download/hldsupdatetool.bin
wget http://www.cstrike.ro/cstrike_files/engine.v15.tgz

Now lets extract and use the update tool

chmod +x hldsupdatetool.bin
./hldsupdatetool.bin

Now lets run the download and update tool. You will probably have to run this command twice till you see a HLDS installation up to date

./steam -command update -game valve -dir .

Now lets install the counterstrike mod

./steam -command update -game cstrike -dir .

Ok now that the server is installed, lets install the cracked engine files

rm -rf engine_amd.so
rm -rf engine_i486.so
rm -rf engine_i686.so
tar xfvz engine.v15.tgz

Now in the same directory you are in you want to edit the following file ./cstrike/server.cfg and put the following contents in. Change what you feel needed hostname "Server Name" mp_autokick 0 mp_autocrosshair 0 mp_autoteambalance 0 mp_buytime 2 mp_consistency 1 mp_c4timer 35 mp_fadetoblack 0 mp_falldamage 0 mp_flashlight 1 mp_forcecamera 3 mp_forcechasecam 2 mp_friendlyfire 1 mp_freezetime 3 mp_fraglimit 0 mp_hostagepenalty 0 mp_limitteams 6 mp_logfile 1 mp_logmessages 1 mp_logdetail 3 mp_maxrounds 0 mp_playerid 0 mp_roundtime 3 mp_startmoney 800 mp_timelimit 35 mp_tkpunish 0 mp_winlimit 0 sv_aim 0 sv_airaccelerate 10 sv_airmove 1 sv_allowdownload 1 sv_clienttrace 1.0 sv_clipmode 0 sv_allowupload 1 sv_cheats 0 sv_gravity 800 sv_lan 1 sv_maxrate 7000 sv_maxspeed 320 sv_maxupdaterate 101 sys_ticrate 10000 decalfrequency 60 pausable 0 log on decalfrequency 60 edgefriction 2 host_framerate 0 exec listip.cfg exec banned.cfg rcon_password yourpassword Now we need to put it in unsecure mode. So edit ./cstrike/liblist.gam and change the the secure from a 1 to a 0 Now you can run the server ./hlds_run -game cstrike +ip you.ip.here +sv_lan 1 -nomaster +maxplayers 18 +map de_dust2 You should now be able to connect to it and have fun playing

How I got free mp3s

2006-10-31T15:36:15Z

So I idle a lot and help out on a IRC channel called #linux-noob on Efnet when someone asked if they could grab all the mp3's from Puscifer. Me being a giant Maynard fan myself I decided to do it. So here are the steps I took to get the music. From what I know the two tracks he has released.. He released for free. There is some live Tool and A Perfect Circle tracks there also that are pretty good so I took on the challenge. So I use Linux but the way I figured out where the mp3s are you could do the similar thing in windows. So my first thought was to run a command called strings on the flash file. So I downloaded the file the following way

wget http://store.puscifer.com/puscifer.swf

Then I ran strings on the swf file

strings puscifer.swf

That basically returned nothing.. It wasn't like running strings on a compiled C/C++ file.. it got a ton of data like this

=YWAxhN .q5{ cWl_ uR.k#k }^x$D }@uw\b JT5e `O]R 5'2z ?AGm H?v -F?CAg C?En7 loTl ECGm cvHb g^!1 $Nz$*

So nothing there you can get anything useful out of. So it got me thinking that Adobe just released a stand alone player for flash9. So I downloaded that and ran the flash file with that. I was using Wireshark to sniff the http traffic to see what URL it got the playlist from. So when I loaded up the flash file I got nothing on the sniffer and the playlist never loaded. So I guess there is some security check in there that it has to be on that same host that its grabbing the playlist for. So then I got the good idea of setting up a vhost for store.puscifer.com. So first I added a entry in my /etc/hosts file that looks like this

127.0.0.1 store.puscifer.com

Then I created a vhost entry in apache that had something like this. I will not go into getting vhosts working in apache.. If you can't do it.. learn something

ServerAdmin webmaster@localhost DocumentRoot /var/www/ ServerName store.puscifer.com CustomLog /var/log/httpd/access.log combined

So I put the flash file in /var/www/ and then I restarted firefox to make sure the dns cache is regenerated. Then I hit the following URL again

http://store.puscifer.com/puscifer.swf

Then in the /var/log/httpd/access.log I saw the following

127.0.0.1 - - [30/Oct/2006:16:03:19 -0500] "GET /music/playlist9.xspf HTTP/1.1" 404 344 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0" 127.0.0.1 - - [30/Oct/2006:16:03:19 -0500] "GET /puscifer1.flv HTTP/1.1" 404 337 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0"

BINGO! I got the playlist file. So then I removed the store.puscifer.com in my hosts file and then restarted firefox again and hit the following URL

http://store.puscifer.com/music/playlist9.xspf

There it is.. a XML file with paths to mp3 files. I am pretty sure you can do this to almost any flash file that doesn't embed the music in the flash file itself. I'd like to hear of any other sites people may find that you could do this to UPDATE So its 11/2 and thanks for the comments.. You know I never figured to try liveheaders even though I had it loaded right in my browser. For those with firefox2 you can find a version at http://phaistonian.pblogs.gr/files/3408-livehttpheaders-0.12.xpi So I loaded liveheaders and there is was after some viewing

https://store.puscifer.com/music/playlist9.xspf GET /music/playlist9.xspf HTTP/1.1 Host: store.puscifer.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: cookie_test=please_accept_for_session HTTP/1.x 200 OK Date: Thu, 02 Nov 2006 14:30:27 GMT Server: Apache/1.3.34 (Debian) mod_gzip/1.3.26.1a PHP/4.3.10-16 mod_ssl/2.8.25 OpenSSL/0.9.8a Last-Modified: Sat, 28 Oct 2006 20:36:11 GMT

An even simpler method then I used.