Foorum v0.07

05 October 2007


Foorum, a forum system based on Catalyst + DBIx::Class + Template.

I just spent days of my golden week holiday to write codes for Foorum.

new features:
* job queue: TheSchwartz. to resize profile photo and send scheduled email. it would ease mod_perl to load the whole Foorum.
* use jquery.ui Tabs to beauty most of my pages.
* remove RSS.pm and use template to create RSS feed.

u can download the source code from http://fayland.googlecode.com/files/Foorum.v007.tar.gz

If u want to join me to write this forum system, plz send me an email. Thanks.

use TheSchwartz job queue to handle Image-Resize For Catalyst App.

04 October 2007


TheSchwartz - reliable job queue. original developped by Brad and used in LiveJournal.

My main aim is to remove 'Image::Magick' out of my Catalyst App since Image::Magick takes a lot of memories for every process under mod_perl. so we need a non-stop cron script to monitor and resize photos. and I picked TheSchwartz up for my Foorum Catalyst App.

Here comes the instructions:

1, create db 'theschwartz' by using schema: http://search.cpan.org/src/BRADFITZ/TheSchwartz-1.04/doc/schema.sql

2, write the main module Foorum::TheSchwartz::Worker::ResizeProfilePhoto use base TheSchwartz::Worker.
package Foorum::TheSchwartz::Worker::ResizeProfilePhoto;

use TheSchwartz::Job;
use base qw( TheSchwartz::Worker );
use Foorum::ExternalUtils qw/schema/;
use File::Spec;
use Image::Magick;
use Cwd qw/abs_path/;
use File::Copy;

my (undef, $path) = File::Spec->splitpath(__FILE__);

sub work {
my $class = shift;
my TheSchwartz::Job $job = shift;

my @args = $job->arg;

my $schema = schema();

# get upload from db
my $upload_id = shift @args;
if ($upload_id !~ /^\d+$/) {
return $job->failed("Wrong upload_id: $upload_id");
}
my $upload = $schema->resultset('Upload')->find( { upload_id => $upload_id } );
unless ($upload) {
return $job->failed("No upload for $upload_id");
}

# get file dir
my $directory_1 = int( $upload_id / 3200 / 3200 );
my $directory_2 = int( $upload_id / 3200 );
my $file = abs_path("$path/../../../../root/upload/$directory_1/$directory_2/" . $upload->filename);

# resize photo
my $p = new Image::Magick;
$p->Read($file);
$p->Scale(geometry=>'120x120');
$p->Sharpen(geometry=>'0.0x1.0');
$p->Set(quality=>'75');

my ($width, $height, $size) = $p->Get('width', 'height', 'filesize');

my $tmp_file = $file . '.tmp';
$p->Write($tmp_file);

move($tmp_file, $file);

# update db
$schema->resultset('UserProfilePhoto')->search( {
type => 'upload',
value => $upload_id,
} )->update( {
width => $width,
height => $height,
} );
($size) = ( $size =~ /^(\d+\.?\d{0,1})/ ); # float(6,1)
$upload->update( { filesize => $size } );

$job->completed();
}

sub max_retries { 3 };

1;


3, write a TheSchwartz_worker.pl and run it. it will monitor the "job" table in "theschwartz" database non-stop.
package Foorum::ExternalUtils;
# ... etc.
use TheSchwartz;
sub theschwartz {

$config = config() unless ($config);

my $theschwartz = TheSchwartz->new(
databases => [ {
dsn => $config->{theschwartz_dsn}, # dbi:mysql:theschwartz
user => $config->{dsn_user},
pass => $config->{dsn_pwd},
} ],
verbose => 1,
);
return $theschwartz;
}
use Foorum::ExternalUtils qw/theschwartz/;
use Foorum::TheSchwartz::Worker::ResizeProfilePhoto;

my $client = theschwartz();
$client->can_do('Foorum::TheSchwartz::Worker::ResizeProfilePhoto');
$client->work();


4, at last, we need insert data into 'job' table then let TheSchwartz_worker.pl to handle it.
package Foorum::Controller::Profile;
use Foorum::ExternalUtils qw/theschwartz/;
sub xxx : Local {
# ... etc.
my $client = theschwartz();
$client->insert('Foorum::TheSchwartz::Worker::ResizeProfilePhoto', $new_upload_id);
}


That's ALL.

The idea behind TheSchwartz is not so complicate.

  • the "theschwartz" database contains 5 tables:

    • 'error' for recording error for every failed job. like in Foorum::TheSchwartz::Worker::ResizeProfilePhoto, $job->failed("No upload for $upload_id");

    • 'funcmap' has two columns: on is funcid and the other is funcname. It's pretty simple. when we call "$client->can_do('Foorum::TheSchwartz::Worker::ResizeProfilePhoto');" in TheSchwartz_worker.pl, it will create a new row ($auto_increasing_func_id, 'Foorum::TheSchwartz::Worker::ResizeProfilePhoto') if there hasn't

    • 'job' is the most important table here. TheSchwartz_worker.pl monitor this table every 5 secs. and if $client->insert one job, it will work_once() immediately.

    • other two tables: exitstatus and notes


  • TheSchwartz_worker.pl can tell what jobs it can handle by using "$client->can_do('Foorum::TheSchwartz::Worker::ResizeProfilePhoto');". add this line means that TheSchwartz_worker.pl can do the jobs which the "funcid" in "job" table is mapping with 'Foorum::TheSchwartz::Worker::ResizeProfilePhoto'.

  • TheSchwartz_worker.pl is a non-stop script to monitor the "job" table every 5 secs. if there are jobs, it will work_once() one by one.

  • $client->insert('Foorum::TheSchwartz::Worker::ResizeProfilePhoto', $new_upload_id); it inserts one row in 'job' table then let worker.pl to handle it. 'Foorum::TheSchwartz::Worker::ResizeProfilePhoto' is the funcname, and $new_upload_id is the args.
    in package Foorum::TheSchwartz::Worker::ResizeProfilePhoto; we need use base TheSchwartz::Worker and write a sub work.
    sub work {
    my $class = shift;
    my TheSchwartz::Job $job = shift;

    my @args = $job->arg;

    # do something

    $job->completed();
    }

  • If u want another worker:
    just write package Foorum::TheSchwartz::Worker::Another
    then in TheSchwartz_worker.pl use it and add $client->can_do('Foorum::TheSchwartz::Worker::Another');
    then in pl or pm. $client->insert('Foorum::TheSchwartz::Worker::Another', @args);



That's ALL. @Enjoy;

two ways to write test cases for Catalyst App

25 September 2007


one is using Test::WWW::Mechanize::Catalyst. as recommended in Catalyst::Manual::Tutorial::Testing.

the other way is using MyApp to get $c. like here:
http://lists.scsys.co.uk/pipermail/catalyst/2007-April/012926.html

my $controller = MyApp->controller('MyController');
my $c = MyApp->prepare();

# Monkey with $c to set up a fake context (set req->uri, or params)

my $result = $controller->method_to_test($c, @args);


Enjoy!

BIG NEWS!

09 August 2007


My wife is pregnant. I'll be a DADDY!

share slides

08 August 2007



Powerful jQuery for SubmitOnce

01 August 2007


jQuery is pretty powerful. It's pretty easy to write a jquery js without change any html code.

I just wrote a small js to finish one task: disable the "submit" button when submit the form and let Ctrl+Enter submit the form like QQ message.

code is pretty simple:
$(document.forms).each( function(theform) {

// disabled the Submit and Reset when submit a form
// to avoid duplicate submit
$(theform).submit( function() {
$('input:submit').attr( { disabled : 'disabled' } );
$('input:reset').attr( { disabled : 'disabled' } );
} );

// Press Ctrl+Enter to submit the form. like QQ.
$(theform).keypress( function(evt) {
var x = evt.keyCode;
var q = evt.ctrlKey;

if (q && ( x == 13 || x == 10)) {
theform.submit();
}
} );

} );


If we are not using jQuery, we need change every <form in html files.(add onsubmit="javascript:SubmitOnce()" like).

jQuery operate on every form without changing any html files. that's amazing!

enjoy();

Javascript Minfier

27 July 2007


As Yahoo! suggested, minified js would improve your site performance.
http://developer.yahoo.com/performance/rules.html#minify

Luckily, Perl has JavaScript::Minifier.
but be careful, cpan will fail, u need download from web then perl Makefile.PL.

I wrote a script to minify all js files under a dir.

@Enjoy.

Nerver/Ever do this: my $foo = $bar if ($cond);

20 July 2007


well, I met this a few days ago.
my $foo = $bar if ($cond);
makes a lot of trouble for me.

I view that this is talked in Catalyst maillist.
http://lists.scsys.co.uk/pipermail/catalyst/2007-July/thread.html#14484

And just for your attention, be careful!

example from mst:
sub foo { my $foo if 0; $foo++; }
print foo(), foo(), foo();


guess what it is?

DO NEVER try code like this, instead try my $foo = ($cond) ? $bar : undef;

@Enjoy;

Catalyst debug trick

20 July 2007


well, we always meet this when we develop a Catalyst application:
* we want -Debug and StackTrace or DBIC::Schema::Profiler when we develop the Catalyst App.
* we don't want -Debug like in the production server because it costs much.

Sometimes u may want a config name like debug_mode to control whether we want or not.
Here comes my solution:
__PACKAGE__->setup();

__PACKAGE__->log->levels('error', 'fatal'); # for real server
if( __PACKAGE__->config->{debug_mode} ) {

__PACKAGE__->log->enable('debug', 'info', 'warn'); # for developer server
{
# these code are copied from Catalyst.pm setup_log
no strict 'refs';
my $class = __PACKAGE__;
*{"$class\::debug"} = sub { 1 };
}

my @extra_plugins = qw/ StackTrace DBIC::Schema::Profiler /;
__PACKAGE__->setup_plugins( [ @extra_plugins ] );
}
Full code please check Foorum.pm

now u can set debug_mode: 1 in _local.yml when u develop and forget it in production server.

Explanation:
* why we write code after __PACKAGE__->setup(); is because we want __PACKAGE__->config->{debug_mode}. config is setup in ->setup().
* *{"$class\::debug"} = sub { 1 }; means $c->debug is on. code from Catalyst.pm
* __PACKAGE__->setup_plugins( [ @extra_plugins ] ); will setup extra_plugins whenever u want.

@Enjoy;

Perl ssh `` tip

18 July 2007


my $result = `ssh $IP 'ls -ls /root;exit'`;
print "result = $result";