sphinx search with varchar primary key

08 April 2012

usually when you index the mysql data into sphinx, you'll use id int/bigint for the primary key. but it's broken for me on http://findmjob.com/, we use uuid everywhere for the primary key.

here is the solution for it. use @id := @id + 1 for the indexer, and use sql_field_string to get the real id when matched. sample code below:

sql_query_pre = SET NAMES utf8
sql_query_pre = SET @id := 1;
sql_query = \
    SELECT @id := @id + 1 AS tid, id, title, description, location, contact, inserted_at FROM job ORDER BY inserted_at DESC LIMIT 10000

sql_field_string = id

it requires latest sphinx to support sql_field_string. and the latest CPAN module too.

and Perl code for it will be normal like before. and instead you use ->{doc}, you need use the attribtues ->{id} like

my @jobids = map { $_->{id} } @{$ret->{matches}};

so now we have sphinx search supports in my new site, eg: http://findmjob.com/search/Perl.html?q=Perl


DBIx::Class with Moose has

04 April 2012

well, I don't know how to name it.

DBIx::Class is one of my favorite modules. for its structure, with DBIx::Class you can make all your code very well organized and clean. writing code with DBIx::Class means you can use it in framework like Catalyst or Mojo or Dancer, and you can use it in any perl script (cron usually). for me, DBIx::Class is the right model, TT2 is the right template, and framework is just for URL dispatch.

Moose is another module I like. Role, and clean OO.

by using DBIx::Class::Schema::Loader make_schema_at, with use_moose => 1 on (eg: https://github.com/fayland/findmjob.com/blob/master/script/make_schema_at.pl), we can generate very clean Result module with Moose.

sometimes you want make assessor based on the table column. like for each job in table, we all have a URL which is based on the id and title in the table, it's so related to those two fields, so we'd better to put it in the Result.
with DBIx::Class, it's very simple. (sample code)

package FindmJob::Schema::Result::Job;


use FindmJob::Utils 'seo_title';
has 'url' => ( is => 'ro', isa => 'Str', lazy_build => 1 );
sub _build_url {
    my ($self) = @_;

    return "/job/" . $self->id . "/" . seo_title($self->title) . ".html";

after that, we can always call $job->url after we ->search for ->find it. very neat. live demo like: 

have fun. Thanks


02 April 2012

maybe to earn some money (not for fun this time), I decided to write a new website http://findmjob.com/

I sit front of my computer and coded it for 2 days and here is it. it's out.

it's very simple and without much stuff yet. and the final goal is undecided. but there it is. I'm very pleased to see it in public.

since there is no reason to keep it private, I opened source it in github: https://github.com/fayland/findmjob.com

for programmer, it's very simple to write website. but it's very hard to make it a success. so suggestions are welcome.


better pagination url design in Dancer

02 April 2012

usually People do param for pager like ?page=1 or ?p=1, it maybe not that good for search engine because they may not go scrape inside. so we may come out a solution with /page=1/ or /p=1/ or even /p.1/ etc.

in Dancer, it's very tricky to do add pagination regex in all URLs. and thank God, we have 'forward' and with code like below, it becomes very simple and easy to use.

Perl code:
get qr'.*?/p\.(\d+).*?' => sub {
    my $uri = request->uri;
    $uri =~ s'/p\.(\d+)'';
    var page => $1;
    $uri =~ s/\/$//;
    forward $uri;

get '/' => sub {
    my $p = vars->{page} || 1; $p = 1 unless $p =~ /^\d+$/;

we use regex to get the page stuff, then remove it from the request uri then using forward to do a internal request.

have fun.


30 March 2012

Github is moving on with their API that "We will terminate API v1 and API v2 in 1 month on May 1st, 2012.".

I know lots of people are preferring that they want token instead of writing user/pass in the config or code.
and now we have the choice with create access_token with code instead of web flow.

it's time to kick Net::GitHub default to V3 now. so here comes the beta release. and tests/patches are welcome.

Tips to create access_token with script:

use Net::GitHub;
my $gh = Net::GitHub->new( login => 'fayland', pass => 'secret' );
my $oauth = $gh->oauth;
my $o = $oauth->create_authorization( {
    scopes => ['user', 'public_repo', 'repo', 'gist'], # just ['public_repo']
    note   => 'test purpose',
} );
print $o->{token};

record the token down, and later on we can always use that token without writing down user/pass.

my $github = Net::GitHub->new(
    access_token => $token, # from above



04 December 2011

When you really go coding, you'll meet lots of issues. then you'll write solution for them. that's straight.

Today I have another CPAN module Plack::Middleware::FileWrap out to fit my demand: I'll have lots of plain HTML files, they'll share the same header/footer and I don't want to use stupid iframe.

under Plack, it just means wrap $res->[2] with file content or strings. 

so here comes Plack::Middleware::FileWrap, very simple if you looked at the source code.

then I used it in the KinderGarden project, with snippets:

    mount '/static/docs/' => builder {
        enable 'FileWrap', headers => ["$root/static/docs/header.html"], footers => ["$root/static/docs/footer.html"];
        Plack::App::File->new( root => "$root/static/docs" )->to_app;

now all the files under static/docs will wrapped with header.html and footer.html (header.html/footer.html itself too!)

for some advanced example, like if you just want to apply to html files, then you can code something like:

    mount '/static/docs/' => builder {
        enable_if { $_[0]->{PATH_INFO} =~ /\.html/ } 'FileWrap',
            headers => ["$root/static/docs/header.html"], footers => ["$root/static/docs/footer.html"];
        Plack::App::File->new( root => "$root/static/docs" )->to_app;
I didn't put the Content-Type check b/c I think it's better to be handled with enable_if.

Enjoy. Thanks

git submodule

03 December 2011

When you include another open source in your own project, it's usually pretty hard to keep it up to date. it becomes even more harder if you have some modification on it.

recently I have bootstrap in my KinderGarden project, it's very easy to use git submodule to handle it.

kindergarden> git submodule add https://github.com/twitter/bootstrap.git static/bootstrap
kindergarden> git add .gitmodules static/bootstrap
kindergarden> git commit -a -m "remote bootstrap"
kindergarden> git push
kindergarden> git submodule init

in delopy or other machine:

kindergarden$ git submodule init
kindergarden$ git submodule update

pretty easy and simple, and the logic behind is simple too. reference as http://help.github.com/submodules/ or http://progit.org/book/ch6-6.html

BTW, I added Live.com OAuth2 supports to KinderGarden.

Enjoy. Thanks

2011 CN Perl Advent

01 December 2011

Hi, it's time for advent again!



27 November 2011

as talked yesterday, I get it uploaded into github. well, under PerlChina. https://github.com/PerlChina/kindergarden

I really want to draw more people to add more features and fix more bugs.

and I added one new feature with Mojolicious which is http://kindergarden.fayland.org/app/whereilive

feel free to view the source and let me know what you think!



26 November 2011

I'm writing some toy once again with Plack and Dancer (and Mojo later).

this time, I'm playing Plack::Middleware::OAuth and Dancer::Template::Xslate a bit. the website is http://kindergarden.fayland.org/ and I'll open source it if someone is interested. for now, it's just 'Login with ...' OAuth and nothing else. (layout is built with twitter bootstrap)

the Dancer::Template::Xslate has some bugs and I tried to submit few commits through github to fix it. (at least it's working for me now)

here is a tip to add function like gravatar into Xslate within Dancer.

problem as Text::Xslate supports function param when ->new but Dancer YAML config can't have Perl code inside. and it's very tricky or hard to fix the engine 'template' b/c we can never modify it. it has 'my $_engine;' inside code and you can't modify it at all.

after a while, I find a good solution with the module param of the Text::Xslate. it's very neat. sample code as below:

# config.yml
template: xslate
    syntax: 'TTerse'
    extension: 'tt'
      - 'layout/header.tt'
      - 'layout/footer.tt'
      - KinderGardenX::Text::Xslate::Bridge::KinderGarden

# KinderGardenX::Text::Xslate::Bridge::KinderGarden
package KinderGardenX::Text::Xslate::Bridge::KinderGarden;

use strict;
use warnings;
use parent qw(Text::Xslate::Bridge);

use Gravatar::URL;

my %funtion_methods = (
    gravatar_url => \&gravatar_url,

    function => \%funtion_methods,


# template

    <img src="[% gravatar_url( email => user.email, size => 30) %]" />
    <img src="[% gravatar_url( email => user.email, size => 50) %]" />
    <img src="[% gravatar_url( email => user.email) %]" />

Note there is always more than one way to do it.